PCAFormer
Vision Transformer efficiency project using PCA-based token compression inside attention layers.
PCAFormer explores how to reduce transformer attention cost without losing core visual understanding.
Core ideas:
- Implemented PCA-guided token compression within transformer blocks to shrink attention compute.
- Benchmarked compressed-token variants against baseline ViT settings to analyze accuracy and runtime tradeoffs.
- Used ablation-style experiments to understand where compression is most effective in the architecture.
This project focuses on practical model efficiency for large-scale vision workloads.