HiLoRA：面向个性化联邦学习的分层低秩适配
HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learning
把 LoRA 适配器按 root/cluster/leaf 三层级放置，并用子空间相似度做客户端聚类，这套「分层低秩适配 + 子空间正交」思路可直接迁移到视觉模型的多域/多客户端个性化微调与跨域适配场景。
原文依据 we propose HiLoRA, a hierarchical LoRA framework that places adapters at three levels: root, cluster, and leaf
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉 Transformer（ViT）因其强大的迁移能力而被广泛应用于视觉任务。在联邦学习中，完整微调会带来沉重的通信负担，而低秩适配（LoRA）为适配 ViT 提供了一种高效且通信友好的方式。然而，现有基于 LoRA 的联邦微调方法忽略了真实场景中潜在的客户端结构，限制了共享表征学习，并阻碍了对未见客户端的有效适配。为解决这一问题，我们提出 HiLoRA，一个分层 LoRA 框架，将适配器置于根、簇、叶三个层级，分别用于捕获全局知识、子群知识与客户端特定知识。通过跨层级正交性与级联优化，HiLoRA 分离各更新子空间，并使每一层级与其残差个性化目标对齐。特别地，我们设计了 LoRA 子空间自适应聚类机制，通过子空间相似度分析推断潜在的客户端分组，从而促进结构对齐客户端之间的知识共享。在理论上，我们建立了逐层泛化分析，为 HiLoRA 的设计提供支撑。在 CIFAR-100 与 DomainNet 上基于 ViT 主干的实验表明，该方法在个性化与泛化两方面均取得了一致的提升。
英文原文摘要 1302 字符
Vision Transformers (ViTs) have been widely adopted in vision tasks due to their strong transferability. In Federated Learning (FL), where full fine-tuning is communication-heavy, Low-Rank Adaptation (LoRA) provides an efficient and communication-friendly way to adapt ViTs. However, existing LoRA-based federated tuning methods overlook latent client structures in real-world settings, limiting shared representation learning and hindering effective adaptation to unseen clients. To address this, we propose HiLoRA, a hierarchical LoRA framework that places adapters at three levels: root, cluster, and leaf, each designed to capture global, subgroup, and client-specific knowledge, respectively. Through cross-tier orthogonality and cascaded optimization, HiLoRA separates update subspaces and aligns each tier with its residual personalized objective. In particular, we develop a LoRA-Subspace Adaptive Clustering mechanism that infers latent client groups via subspace similarity analysis, thereby facilitating knowledge sharing across structurally aligned clients. Theoretically, we establish a tier-wise generalization analysis that supports HiLoRA's design. Experiments on ViT backbones with CIFAR-100 and DomainNet demonstrate consistent improvements in both personalization and generalization.