Pancreatic ductal adenocarcinoma (PDAC) is the most common type of pancreatic cancer featured with high intra-tumoral heterogeneity and poor prognosis. To comprehensively delineate the PDAC intra-tumoral heterogeneity and the underlying mechanism for PDAC progression, we employed single-cell RNA-seq (scRNA-seq) to acquire the transcriptomic atlas of 57,530 individual pancreatic cells from primary PDAC tumors and control pancreases, and identified diverse malignant and stromal cell types, including two ductal subtypes with abnormal and malignant gene expression profiles respectively, in PDAC. We found that the heterogenous malignant subtype was composed of several subpopulations with differential proliferative and migratory potentials. Cell trajectory analysis revealed that components of multiple tumor-related pathways and transcription factors (TFs) were differentially expressed along PDAC progression. Furthermore, we found a subset of ductal cells with unique proliferative features were associated with an inactivation state in tumor-infiltrating T cells, providing novel markers for the prediction of antitumor immune response. Together, our findings provide a valuable resource for deciphering the intra-tumoral heterogeneity in PDAC and uncover a connection between tumor intrinsic transcriptional state and T cell activation, suggesting potential biomarkers for anticancer treatment such as targeted therapy and immunotherapy.