Epoxy composites filled with different amounts of rubber particles- and aggregate-free nanoparticles are fabricated to examine the synergistic effect of multiphase particles on the mechanical properties of composites. In this work, crack propagation (growth) is simulated using the Extended Finite Element Method in Abaqus, and the results of displacement, stress, and crack propagation are compared with the experimental results. The main aim of this study is to evaluate the performance of the finite element method, the number of elements required in predicting results, and the ability of Extended Finite Element Method to predict crack propagation behavior. This study examines the required number of elements in crack propagation displacement and computational grid independence and also compares the numerical and experimental results obtained for a nanoparticle-reinforced epoxy polymer in 2D in Abaqus. Particle separation and crack propagation, absorbing energy during tensile simulation, can play a key role in surface hardening and strengthening. The simulation confirms the nanocomposite interface weakening and strengthening mechanisms and demonstrates that Extended Finite Element Method is an effective method to simulate the mechanical behavior of nanocomposites.