近日，实验室博士生张博关于大模型对话生成的研究成果被期刊Transactions of the Association for Computational Linguistics（TACL）录用。TACL期刊是计算语言学和自然语言处理领域的国际顶级期刊，在学术界及业界都享有盛誉，在CCF推荐列表中认定为B类期刊，影响因子4.2。

题目：Efficient Tuning of Large Language Models for Knowledge-Grounded Dialogue Generation

Abstract：Large language models (LLMs) demonstrate remarkable text comprehension and generation capabilities but often lack the ability to utilize up-to-date or domain-specific knowledge not included in their training data. To address this gap, we introduce KEDiT, an efficient method for fine-tuning LLMs for knowledge-grounded dialogue generation. KEDiT operates in two main phases: first, it employs an information bottleneck to compress retrieved knowledge into learnable parameters, retaining essential information while minimizing computational overhead. Second, a lightweight knowledge-aware adapter integrates these compressed knowledge vectors into the LLM during fine-tuning, updating less than 2% of the model parameters. The experimental results on the Wizard of Wikipedia and a newly constructed PubMed-Dialog dataset demonstrate that KEDiT excels in generating contextually relevant and informative responses, outperforming competitive baselines in automatic, LLM-based, and human evaluations. This approach effectively combines the strengths of pretrained LLMs with the adaptability needed for incorporating dynamic knowledge, presenting a scalable solution for fields such as medicine.

中文摘要：大语言模型（LLMs）在文本理解与生成方面表现出了卓越的能力，但往往无法有效利用其训练数据之外的最新知识或特定领域的专业知识。针对这一问题，本文提出了KEDiT，一种用于大模型对话生成的知识引导高效微调方法。KEDiT包含两个主要阶段：首先，使用信息瓶颈方法将检索到的知识压缩为可学习的参数，以保留关键信息同时降低计算开销；其次，通过轻量级的知识感知适配器，在微调过程中将这些压缩后的知识向量整合到LLM中，这一过程仅更新不到2%的模型参数。在Wizard of Wikipedia数据集及新构建的PubMed-Dialog数据集上的实验表明，KEDiT在自动评估、基于LLM的评估及人工评估中，均能生成上下文相关且信息丰富的回复，性能显著优于其他竞争性基线方法。该方法有效结合了预训练LLM的强大能力和整合动态知识的灵活性，为医学等领域提供了一种可扩展的解决方案。