The effect of breast cancer heterogeneity on prognosis of patients is still unclear, especially the role of immune cells in prognosis of breast cancer. In this study, single cell transcriptome sequencing data of breast cancer were used to analyze the relationship between breast cancer heterogeneity and prognosis. In this study, 14 cell clusters were identified in two single-cell datasets (GSE75688 and G118389). Proportion analysis of immune cells showed that NK cells were significantly aggregated in triple negative breast cancer, and the proportion of macrophages was significantly increased in primary breast cancer, while B cells, T cells, and neutrophils may be involved in the metastasis of breast cancer. The results of ligand receptor interaction network revealed that macrophages and DC cells were the most frequently interacting cells with other cells in breast cancer. The results of WGCNA analysis suggested that the MEblue module is most relevant to the overall survival time of triple negative breast cancer. Twenty-four prognostic genes in the blue module were identified by univariate Cox regression analysis and KM survival analysis. Multivariate regression analysis combined with risk analysis was used to analyze 24 prognostic genes to construct a prognostic model. The verification result of our prognostic model showed that there were significant differences in the expression of PCDH12, SLIT3, ACVRL1, and DLL4 genes between the high-risk group and the low-risk group, which can be used as prognostic biomarkers.