Shear behavior of granular soil with fines is investigated using the discrete element method (DEM) and particle arrangements and inter-particle contacts during shear are examined. The DEM simulation reveals that fine particles play a vital role in the overall response of granular soil to shearing. The occurrence of liquefaction and temporary reduction of strength is ascribed mainly to the loss of support from the fine particle contacts (S-S) and fine particle-to-large particle contacts (S-L) as a consequence of the removal of fine particles from the load-carrying skeleton. The dilative strain-hardening response following the strain-softening response is associated with the migration of fine particles back into the load-carrying skeleton, which is thought to enhance the stiffness of the soil skeleton. During shear, the unit normal vector of the large particle-to-large particle (L-L) contact has the strongest fabric anisotropy, and the S-S contact unit normal vector possesses the weakest anisotropy, suggesting that the large particles play a dominant role in carrying the shear load. It is also found that, during shear, fine particles are prone to rolling at contacts while the large particles are prone to sliding, mainly at the S-L and L-L contacts.