ACME: Adaptive Customization of Large Models via Distributed Systems

In this work, we propose ACME, an adaptive model customization framework designed to deploy large Transformer-based models efficiently across heterogeneous devices in distributed systems. ACME addresses critical issues such as performance imbalances, energy inefficiency, and privacy concerns when deploying pre-trained models like ViT and BERT at the edge.

The system uses a bidirectional single-loop architecture that progressively customizes models in two phases: (1) backbone customization through Pareto-optimal architecture generation on cloud and edge servers, and (2) header refinement through neural architecture search (NAS) and personalized aggregation based on local data distributions.

This work is a collaboration among researchers at the College of Intelligence and Computing, Tianjin University.

Recommended citation: Ziming Dai, Chao Qiu, Fei Gao, Yunfeng Zhao, Xiaofei Wang, "ACME: Adaptive Customization of Large Models via Distributed Systems" In 2025 IEEE 45th International Conference on Distributed Computing Systems (ICDCS). IEEE.
Download Paper