AdaMSS: Adaptive Multi-Subspace Approach for Parameter-Efficient Fine-Tuning
NeurIPS 2025 · Second author
Jingjing Zheng, Wanglong Lu, Yiming Dong, Chaojie Ji, Yankai Cao, Zhouchen Lin
Brief description:
AdaMSS is an adaptive multi-subspace approach for parameter-efficient fine-tuning. It models incremental updates with multiple subspaces, improving the expressiveness-efficiency trade-off when adapting large pretrained models while keeping the base weights frozen.
Highlights:
- Multi-subspace-based incremental update for parameter-efficient adaptation.
- Designed to capture richer update structures than single low-rank adaptation under a compact parameter budget.
- Accepted at NeurIPS 2025 and integrated into the Hugging Face PEFT package.
[paper] [doi] [github] [peft integration]
Recommended citation:
@inproceedings{zheng2025adamss,
title={AdaMSS: Adaptive Multi-Subspace Approach for Parameter-Efficient Fine-Tuning},
author={Zheng, Jingjing and Lu, Wanglong and Dong, Yiming and Ji, Chaojie and Cao, Yankai and Lin, Zhouchen},
booktitle={Advances in Neural Information Processing Systems},
volume={38},
year={2025},
doi={10.52202/085713-0664},
}