A solution of probabilistic FEM for elastic-plastic materials is presented based on the incremental theory of plasticity and a modified initial stress method. The formulations are deduced through a direct differentiation scheme. Partial differentiation of displacement, stress and the performance function can be iteratively performed with the computation of the mean values of displacement and stress. The presented method enjoys the efficiency of both the perturbation method and the finite difference method, but avoids the approximation during the partial differentiation calculation. In order to improve the efficiency, the adjoint vector method is introduced to calculate the differentiation of stress and displacement with respect to random variables. In addition, a time-saving computational method for reliability index of elastic-plastic materials is suggested based upon the advanced First Order Second Moment (FOSM) and by the usage of Taylor expansion for displacement. The suggested method is also applicable to 3-D cases.