There is a need for better classification and understanding of tumor-infiltrating lymphocytes (TILs). Here, we applied advanced functional genomics to interrogate 9,000 human tumors and multiple single-cell sequencing sets using benchmarked T cell states, comprehensive T cell differentiation trajectories, human and mouse vaccine responses, and other human TILs. Compared with other T cell states, enrichment of T memory/resident memory programs was observed across solid tumors. Trajectory analysis of single-cell melanoma CD8+ TILs also identified a high fraction of memory/resident memory-scoring TILs in anti-PD-1 responders, which expanded post therapy. In contrast, TILs scoring highly for early T cell activation, but not exhaustion, associated with non-response. Late/persistent, but not early activation signatures, prognosticate melanoma survival, and co-express with dendritic cell and IFN-γ response programs. These data identify an activation-like state associated to poor response and suggest successful memory conversion, above resuscitation of exhaustion, is an under-appreciated aspect of successful anti-tumoral immunity.