This is a repo for A Graph per Persona: Reasoning about Subjective Natural Language Descriptions, accepted at ACL-Findings 2024.

Our code is adapted from AttnIO implementation from this github: https://github.com/rajbsk/kg-cruse

Data

We use sampled_user_responses_20.json data from this github: https://github.com/eujhwang/personalized-llms.

Example Command

python main.py --add_imp=5 --batch_size=8 --conn_type=t5-base --early_stopping=20 --epoch=30 --learning_rate=5e-05 --loss_type=full --num_head=3 --num_op=16 --num_ques=30 --path_len=3 --path_topk=5 --pretrained_model=BAAI/bge-base-en-v1.5 --seed=42 --survey=$1 --threshold=0.1 --weight_decay=0 --do_train --do_eval

Citation

@inproceedings{hwang-etal-2024-graph,
    title = "A Graph per Persona: Reasoning about Subjective Natural Language Descriptions",
    author = "Hwang, EunJeong  and
      Shwartz, Vered  and
      Gutfreund, Dan  and
      Thost, Veronika",
    editor = "Ku, Lun-Wei  and
      Martins, Andre  and
      Srikumar, Vivek",
    booktitle = "Findings of the Association for Computational Linguistics ACL 2024",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand and virtual meeting",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.findings-acl.115",
    pages = "1928--1942",
    abstract = "Reasoning about subjective natural language descriptions, such as opinions and preferences, is a challenging topic that largely remains unsolved to date. In particular, state-of-the-art large language models (LLMs) perform disappointingly in this task, show strong biases, and do not meet the interpretability requirements often needed in these kinds of applications. We propose a novel approach for reasoning about subjective knowledge that integrates potential and implicit meanings and explicitly models the relational nature of the information. We apply supervised graph learning, offer explanations for the model{'}s reasoning, and show that our model performs well across all 15 topics of OpinionQA, outperforming several prominent LLMs. Our detailed analysis further shows its unique advantages and the complementary nature it offers in comparison to LLMs.",
}