In the normal prostate, most basal and some luminal cells are castration-resistant (CR). The identity of these CR cells and their relation to CR prostate cancer are unresolved. We compared single-cell expression profiles of prostate cells sorted from hormonally nave (HN) and castrated mice. We found both basal and luminal-localized cells, particularly the latter, were molecularly heterogeneous. CR luminal cells and a subset of HN luminal cells exhibited a similar intermediate expression pattern, including high-level expression of multiple prostate stem/progenitor marker genes and androgen receptor gene. We validated LY6D as a marker linking CR luminal cells to luminal progenitors. LY6D+ prostate cells, including LY6D+ luminal cells, were enriched for organoid-forming potential regardless of the presence or absence of androgen. Krt8-based lineage-tracing revealed that LY6D+ CR luminal cells produced LY6D- normal luminal cells upon regeneration, but LY6D+ luminal cancer cells under PTEN-deficiency. Furthermore, prostate cancers originating from CR luminal cells (LY6D+) exhibited a more advanced phenotype than those from HN luminal cells (LY6D+ or LY6D-). Lastly, LY6D amplification/upregulation appear associated with advanced prostate cancer in patient samples. Together, our studies demonstrate LY6D as a novel progenitor marker predictive of lethal CR disease.