The few extant reports of longitudinal white matter (WM) changes in healthy aging, using diffusion tensor imaging (DTI), reveal substantial differences in change across brain regions and DTI indices. According to the “last-in-first-out” hypothesis of brain aging late-developing WM tracts may be particularly vulnerable to advanced age. To test this hypothesis we compared age-related changes in association, commissural and projection WM fiber regions using a skeletonized, region of interest DTI approach. Using linear mixed effect models, we evaluated the influences of age and vascular risk at baseline on seven-year changes in three indices of WM integrity and organization (axial diffusivity, AD, radial diffusivity, RD, and fractional anisotropy, FA) in healthy middle-aged and older adults (mean age = 65.4, SD = 9.0 years). Association fibers showed the most pronounced declines over time. Advanced age was associated with greater longitudinal changes in RD and FA, independent of fiber type. Furthermore, older age was associated with longitudinal RD increases in late-developing, but not early-developing projection fibers. These findings demonstrate the increased vulnerability of later developing WM regions and support the “last-in-first-out” hypothesis of brain aging.