ChimeraLoRA:多头LoRA引导的合成数据集
ChimeraLoRA: Multi-Head LoRA-Guided Synthetic Datasets
提出将LoRA分解为类共享A和图像特定B,并用狄利克雷系数混合生成兼具多样性与细节的合成数据,可直接用于少样本/长尾视觉任务的数据增强。
原文依据 we separate the adapter into a class-shared LoRA A for class priors and per-image LoRAs \mathcal B for image-specific characteristics.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在通用识别任务之外,专业领域和细粒度场景常面临数据稀缺问题,尤其是尾部类别。为了在这种稀缺条件下获得偏差更小、更可靠的模型,从业者利用扩散模型来补充真实数据中代表性不足的区域。具体而言,近期研究在少样本真实集上使用LoRA微调预训练扩散模型以合成额外图像。在单张图像上训练的逐图像LoRA能捕捉细粒度细节但多样性有限,而在所有样本上训练的逐类别LoRA能生成多样图像因其编码了类别先验但容易忽略细节。为了结合两者优势,我们将适配器分离为用于类别先验的类共享LoRA A和用于图像特定特征的逐图像LoRA \mathcal B。为了在共享LoRA A中暴露连贯的类别语义,我们提出通过训练时保留类别边界框来进行语义增强。在生成时,我们使用从狄利克雷分布中抽取的系数将A与\mathcal B的混合进行组合。在多个数据集上,我们合成的图像既多样又细节丰富,同时与少样本真实分布紧密对齐,在下游分类准确率上取得稳健提升。
英文原文摘要 1288 字符
Beyond general recognition tasks, specialized domains and fine-grained settings often encounter data scarcity, especially for tail classes. To obtain less biased and more reliable models under such scarcity, practitioners leverage diffusion models to supplement underrepresented regions of real data. Specifically, recent studies fine-tune pretrained diffusion models with LoRA on few-shot real sets to synthesize additional images. While an image-wise LoRA trained on a single image captures fine-grained details yet offers limited diversity, a class-wise LoRA trained over all shots produces diverse images as it encodes class priors yet tends to overlook fine details. To combine both benefits, we separate the adapter into a class-shared LoRA A for class priors and per-image LoRAs \mathcal B for image-specific characteristics. To expose coherent class semantics in the shared LoRA A, we propose a semantic boosting by preserving class bounding boxes during training. For generation, we compose A with a mixture of \mathcal B using coefficients drawn from a Dirichlet distribution. Across diverse datasets, our synthesized images are both diverse and detail-rich while closely aligning with the few-shot real distribution, yielding robust gains in downstream classification accuracy.