The binding of T cell receptors (TCRs) to peptide-MHC I (pMHC) complexes is critical in triggering adaptive immunity response to potential health threats. Developing highly accurate machine learning (ML) models to predict TCR-pMHC binding could significantly accelerate immunotherapy advancements. However, existing ML models for TCR-pMHC binding prediction often underperform with unseen epitopes, severely limiting their applicability. We introduce TRAP, which leverages contrastive learning to enhance model performance by aligning structural and sequence features of pMHC with TCR sequences. TRAP outperforms previous state-of-the-art models in both random and unseen epitope scenarios, achieving an AUPR of 0.84 (a 22% improvement over the second-best model) and AUC of 0.92 in the random scenario, and AUC of 0.75 (almost 11% higher than the second-best model) in the unseen epitope scenario. Furthermore, TRAP demonstrates a noteworthy capability to diagnose potential issues of cross-reactivity between TCRs and similar epitopes. This highly robust performance makes it a suitable tool for large-scale predictions in real-world setting. A specific case study confirmed that TRAP can discover hit TCRs with binding free energies comparable to the referenced experimental results. These findings highlight TRAP’s potential for practical applications and its role as a powerful tool in developing TCR-based immunotherapies.