• español
    • English
  • Login
  • English 
    • español
    • English
  • Publication Types
    • bookbook partconference objectdoctoral thesisjournal articlemagazinemaster thesispatenttechnical documentationtechnical report
View Item 
  •   IMDEA Networks Home
  • View Item
  •   IMDEA Networks Home
  • View Item
JavaScript is disabled for your browser. Some features of this site may not work without it.

A few-shot learning method based on knowledge graph in large language models

Share
Files
original version (885.0Kb)
Identifiers
URI: https://hdl.handle.net/20.500.12761/1886
ISSN: 2364-415X
DOI: 10.1007/s41060-024-00699-3
Metadata
Show full item record
Author(s)
Wang, FeiLong; Shi, Donghui; Aguilar, Jose; Cui, Xinyi
Date
2024-12-15
Abstract
The emergence of large language models has significantly transformed natural language processing and text generation. Fine-tuning these models for specific domains enables them to generate answers tailored to the unique requirements of those fields, such as in legal or medical domains. However, these models often perform poorly in few-shot scenarios. Herein, the challenges of data scarcity in fine-tuning large language models in low-sample scenarios were addressed by proposing three different KDGI (Knowledge-Driven Dialog Generation Instances) generation strategies, including entity-based KDGI generation, relation-based KDGI generation, and semantic-based multi-level KDGI generation. These strategies aimed to enhance few-shot datasets to address the issue of low fine-tuning metrics caused by insufficient data. Specifically, knowledge graphs were utilized to define the distinct KDGI generation strategies for enhancing few-shot data. Subsequently, these KDGI data were employed to fine-tune the large language model using the P-tuning v2 approach. Through multiple experiments, the effectiveness of the three KDGI generation strategies was validated using BLEU and ROUGE metrics, and the fine-tuning benefits of few-shot learning on large language models were confirmed. To further evaluate the effectiveness of KDGI, additional experiments were conducted, including LoRA-based fine-tuning in the medical domain and comparative studies leveraging Mask Language Model augmentation, back-translation, and noise injection methods. Consequently, the paper proposes a reference method for leveraging knowledge graphs in prompt data engineering, which shows potential in facilitating few-shot learning for fine-tuning large language models.
Share
Files
original version (885.0Kb)
Identifiers
URI: https://hdl.handle.net/20.500.12761/1886
ISSN: 2364-415X
DOI: 10.1007/s41060-024-00699-3
Metadata
Show full item record

Browse

All of IMDEA NetworksBy Issue DateAuthorsTitlesKeywordsTypes of content

My Account

Login

Statistics

View Usage Statistics

Dissemination

emailContact person Directory wifi Eduroam rss_feed News
IMDEA initiative About IMDEA Networks Organizational structure Annual reports Transparency
Follow us in:
Community of Madrid

EUROPEAN UNION

European Social Fund

EUROPEAN UNION

European Regional Development Fund

EUROPEAN UNION

European Structural and Investment Fund

© 2021 IMDEA Networks. | Accesibility declaration | Privacy Policy | Disclaimer | Cookie policy - We value your privacy: this site uses no cookies!