Background: The terminal duct lobular unit (TDLU) is the most dynamic structure in the human breast and the putative site of origin of human breast cancer. Although stromal cells contribute to a specialized microenvironment in many organs, this component remains largely understudied in the human breast. We here demonstrate the impact on epithelium of two lineages of breast stromal fibroblasts, one of which accumulates in the TDLU while the other resides outside the TDLU in the interlobular stroma. Methods: The two lineages are prospectively isolated by FACS based on different expression levels of CD105 and CD26. The characteristics of the two fibroblast lineages are assessed by immunocytochemical staining and gene expression analysis. The differentiation capacity of the two fibroblast populations is determined by exposure to specific differentiating conditions followed by analysis of adipogenic and osteogenic differentiation. To test whether the two fibroblast lineages are functionally imprinted by their site of origin, single cell sorted CD271low/MUC1high normal breast luminal epithelial cells are plated on fibroblast feeders for the observation of morphological development. Epithelial structure formation and polarization is shown by immunofluorescence and digitalized quantification of immunoperoxidase stained cultures. Results: Lobular fibroblasts are CD105high/CD26low while interlobular fibroblasts are CD105low/CD26high. Once isolated the two lineages remain phenotypically stable and functionally distinct in culture. Lobular fibroblasts have properties in common with bone marrow derived mesenchymal stem cells and they specifically convey growth and branching morphogenesis of epithelial progenitors. Conclusions: Two distinct functionally specialized fibroblast lineages exist in the normal human breast, of which the lobular fibroblasts have properties in common with mesenchymal stem cells and support epithelial growth and morphogenesis. We propose that lobular fibroblasts constitute a specialized microenvironment for human breast luminal epithelial progenitors, i.e. the putative precursors of breast cancer.