Korean Medicine Embedding Dataset (한의학 임베딩 데이터셋)

A contrastive-learning dataset for training and evaluating domain-adapted text embedding models for Korean Medicine (KM, 한의학). Each example is a query–positive–negatives triplet built from KM terminology resources and a curated KM ontology, capturing semantic relations such as disease–symptom, disease–prescription, prescription–indication, prescription–herb, and herb–indication.

This dataset accompanies the study "A Domain-Adapted Embedding Model for Semantic Retrieval of Korean Medicine Knowledge", where it was used to domain-adapt BGE-M3 with LoRA and ontology-guided negative sampling. To our knowledge, it is the first publicly released embedding dataset specifically targeting Korean traditional medicine.

Why this dataset

General-purpose and biomedical embedding models do not capture the conceptual and linguistic structure of KM. KM interprets disease states in functional and relational terms and relies on symbolic concepts (e.g., 음양 yin–yang, 한열 cold–heat). It shares historical roots with Traditional Chinese Medicine but developed an independent theoretical system (e.g., Donguibogam, Sasang constitutional medicine). KM texts also mix Hangeul (한글), Hanja (한자), and Hangeul–Hanja parallel notation, where homophonous Sino-Korean terms can differ in meaning depending on the underlying characters. This dataset was constructed to inject that domain knowledge and these orthographic characteristics into embedding training.

Dataset structure

Data fields

Each line is a JSON object:

Example (Hangeul–Hanja format):

{
  "query": "사군산(使君散)",
  "pos": ["사군자(使君子), 빈랑자(檳榔子), 웅황(雄黃)"],
  "neg": ["목통(木通), 청피(靑皮), 내복자(萊菔子), 회향(茴香), ...", "..."]
}

Text representation formats

Every underlying semantic instance is provided in three text representations, applied identically to the query, pos, and neg fields:

The three representation-specific datasets share identical instance IDs, so a given ID denotes the same query–positive relationship across all three.

Splits

The dataset contains 113,299 underlying semantic instances. To prevent different representations of the same instance from being distributed across partitions, synchronized splitting was performed at the instance level: the 113,299 instances were randomly divided 8:1:1 into train / validation / test, and the same partition assignment was applied to all three representation-specific datasets.

This yields 90,639 train / 11,330 validation / 11,330 test instances per representation. The three training subsets were combined into a multi-representation training set of 271,917 instances.

The validation set was used for hyperparameter selection; the test set was held out for final evaluation.

Per-representation, unsplit versions of the full data are also provided under transformed/:

Naming note: both = Hangeul–Hanja, korean = Hangeul, chinese = Hanja.

Dataset composition

Positive pairs span 16 relation types drawn from two source types:

Construction pipeline

1. Source curation. Query–positive pairs were derived from KM terminology resources (term–definition, term–synonym) and a verified KM ontology covering diseases, pattern identifications, prescriptions, and herbs.
2. Negative sampling (graph-based exclusion → similarity + random). All elements of query–positive pairs were placed on a graph with bidirectional edges. Concept nodes (diseases, prescriptions, herbs) connect to other nodes; textual nodes (definitions, efficacies, indications) have no additional connections. Nodes within 2 hops of a query were excluded from the negative candidate pool because of potential semantic relatedness. From the remaining candidates, 10 negatives were selected per query: 3 similarity-based hard negatives (cosine similarity 0.4–0.6 under BGE-M3) and 7 random negatives.
3. Multi-representation generation. Each instance was rendered in Hangeul, Hanja, and Hangeul–Hanja, with the transformation applied consistently to query, positive, and negative texts.
4. Synchronized splitting. Instances were split 8:1:1 with partition assignments shared across all three representations; the training subsets were then unified.

Reference results

The dataset was used to domain-adapt BGE-M3 (568M params) with LoRA (r=16, α=32, dropout=0.1) and MultipleNegativesRankingLoss. Selected configuration: batch size 32, learning rate 5×10⁻⁴, 20 epochs, max sequence length 256, BF16.

Two evaluation settings were used on the same held-out test partition:

• Candidate-pool — each query ranked against 11 candidates (1 positive + its 10 negatives).
• Full-corpus — each query searched against all unique candidate texts: 74,828 (Hangeul–Hanja), 74,134 (Hanja), 73,100 (Hangeul). Corpus sizes differ because distinct source texts can convert to identical representations. Self-matching queries were excluded.

Hangeul–Hanja representation:

Full-corpus performance across representations (fine-tuned model): Hangeul–Hanja MRR@10 0.1483 > Hanja 0.1376 > Hangeul 0.1184.

Performance varied substantially by relation type — Herb–Alias was highest (MRR@10 0.4209, median rank 3), followed by Term–Synonym (0.2755) and Term–Definition (0.1852).

SimGap (mean cosine similarity of positive pairs minus that of negative pairs, computed over the 10 predefined negatives) rose from 0.1245 to 0.5134 for the Hangeul–Hanja representation after domain adaptation.

Ablation analyses reported in the paper found that the effects of similarity-based hard-negative sampling and graph-based filtering were small and not consistent across the three evaluation representations, and that the benefit of including the combined Hangeul–Hanja representation in training varied by evaluation format.

Intended uses

• Training / fine-tuning Korean Medicine–specific embedding models for semantic retrieval.
• Building RAG systems, clinical decision support, and knowledge-retrieval pipelines over KM literature, guidelines, and ontology-based knowledge bases.
• Studying the effect of Hangeul / Hanja / Hangeul–Hanja representations on retrieval in Sino-Korean technical domains.

Limitations

• Train, validation, and test partitions were derived from the same terminology and ontology resources via random splitting. Although splitting was synchronized at the instance level to prevent representation-level leakage, the same entity may still appear across partitions; generalization to fully unseen entities is not guaranteed.
• The data reflects terminology and ontology relations, not real clinical notes or prescription records, which use non-standardized, abbreviated, and varied expressions.
• Negatives were mined under BGE-M3 similarity, so their difficulty distribution is tied to that base model.
• The candidate-pool setting (1 positive + 10 negatives) saturates Recall@10; the full-corpus setting is the more informative evaluation.

Citation

@article{kim2026km_embedding,
  title   = {A Domain-Adapted Embedding Model for Semantic Retrieval of Korean Medicine Knowledge},
  author  = {Kim, Young-Eun and Keum, Kimun and Kim, Sang-Kyun and Kim, Anna and Son, Mi Ju and Seo, SuMin and Jang, Yunji and Oh, Yongtaek and Yi, Mun Yong},
  year    = {2026},
  note    = {Dataset: https://huggingface.co/datasets/cnupo23/korean-medicine-embedding-dataset}
}

License

Released under the Apache License 2.0. The underlying terminology and ontology resources are subject to their original terms; please verify source-specific conditions before use.