In order to verify the validity of finite element numerical simulation method for asphalt mixture, which consists of aggregates, mastic (where mastic is a kind of fine mixture composed of asphalt binder mixed with fines and fine aggregates) and air voids, based on three-dimensional (3D) heterogeneous specimen, X-ray computerized tomography (X-ray CT) was used to scan the asphalt specimens to obtain the real internal microstrnctures of asphalt mixture. CT images were reconstructed to build up 3D digital specimen, and the viscoelastic properties of mastic were described with Burgers model The uniaxial creep numerical simulations of three different levels of aggregate gradation were conducted. The simulation results agree well with the experimental results. The numerical simulation of asphalt mixture incorporated with real 3D microstructure based on finite element method is a promising application to conduct research of asphalt concrete. Additionally, this method can increase the mechanistic understanding of global viscoelastic properties of asphalt mixtures by linking the real 3D microstructure.