The extinction curve, which depends on the composition and size distribution of interstellar dust grains, is one of the most important factors for clarifying the nature of the objects from the observed spectral energy distributions. In order to understand how the extinction curve changes in the course of galaxy evolution, we construct the evolution model of grain size distribution, taking into account various processes of dust production and dust processing. We show that, in the early stage of galaxy evolution, where relatively large grains are supplied by stellar sources such as core-collapse supernovae, the grain size distribution is biased to large radius (0.1-1 um), leading to a flat extinction curve. As the galaxies evolve, the size distribution is dominated by small (<0.01 um) grains that are enhanced through shattering and growth of grains in the interstellar medium, making the extinction curve steeper. We also discuss how the extinction curve in our Galaxy can be explained in the context of our dust evolution model.