AdaMSS: Adaptive Multi-Subspace Approach for Parameter-Efficient Fine-Tuning
Published in NeurIPS 2025, 2025
Jingjing Zheng, Wanglong Lu, Yiming Dong, Chaojie Ji, Yankai Cao, Zhouchen Lin
Brief description:
AdaMSS is an adaptive multi-subspace approach for parameter-efficient fine-tuning. It models incremental updates with multiple subspaces, improving the expressiveness-efficiency trade-off when adapting large pretrained models while keeping the base weights frozen.
Highlights:
- Multi-subspace-based incremental update for parameter-efficient adaptation.
- Designed to capture richer update structures than single low-rank adaptation under a compact parameter budget.
- Accepted at NeurIPS 2025 and integrated into the Hugging Face PEFT package.
[paper] [github] [peft integration]
Recommended citation:
@inproceedings{zheng2025adamss,
title={AdaMSS: Adaptive Multi-Subspace Approach for Parameter-Efficient Fine-Tuning},
author={Zheng, Jingjing and Lu, Wanglong and Dong, Yiming and Ji, Chaojie and Cao, Yankai and Lin, Zhouchen},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
}