Whether there is an association between SWI/SNF genomic alterations in tumors and response to immune checkpoint inhibitors (ICIs) remains unclear because prior studies have focused on either an individual gene or a predefined set of genes. Herein, using mutational and clinical data from 832 ICI-treated patients who underwent whole-exome sequencing, including sequencing of all 31 genes of the SWI/SNF complex, we found that SWI/SNF complex alterations were associated with significantly improved overall survival (OS) in melanoma, clear cell renal cell carcinoma, and gastrointestinal cancer, as well as improved progression-free survival (PFS) in non-small cell lung cancer. Including tumor mutational burden as a variable, the multivariate Cox regression analysis showed SWI/SNF genomic alterations had prognostic value in melanoma (HR 0.63 [95% CI 0.47–0.85], p = 0.003), clear cell renal cell carcinoma (HR 0.62 [95% CI 0.46–0.85], p = 0.003) and gastrointestinal cancer (HR 0.42 [95% CI 0.18–1.01], p = 0.053). Further, we used the random forest method for variable screening, identifying 14 genes as a SWI/SNF signature for potential clinical application. Significant correlations were observed between SWI/SNF signature alterations and improved OS and PFS in all cohorts. This suggests that SWI/SNF gene alterations are associated with better clinical outcomes in ICI-treated patients and may serve as a predictive marker for ICI therapy in multiple cancers.