Concept Commonality

Concept Commonality is identifying a set of concepts which share some properties of interest. The following are the steps we follow to identify the concept commonalities.

1. Prompt a LLM to generate facets and properties of the concepts. The LLM prompted for facets and properties is Meta-Llama-3-8B-Instruct.
2. Get the embeddings the properties generated by the LLM. Empirically we found that associating facets with properties for getting embedings works well. Currently, the format used for getting the facet: property. We use the embeddings from the LLM LLM2Vec-Mistral-7B-Instruct-v2-mntp from LLM2Vec model.
3. Finally, we cluster the facet:property embeddings using affinity propogation clustering algorithm.

To execute the above steps use the following scripts.

1. Clone the github repo using the following command.

git clone https://github.com/amitgajbhiye/llm_direct_preference_optimisation.git

2. Create the conda environment from file the environment_prompting.yml using the following command.

conda env create -f environment_prompting.yml

3. To prompt the Llama3 model use the following command.

python3 src/concept_facet_property_prompting.py --config configs/facet_prop_generation/3_llama3_1inc_science_repeat_1.json

4. Generate the embeddings using the following script by specifying the facet_colon_property_file in the script.

python src/embeds_llm2vec.py

5. Cluster the facet: property embeddings use the following script.

python3 src/wikidata_facet_property_clustering.py --config_file configs/clustering/10_mistral7b_inst_mntp_facet_colon_property_embeds_5inc_science.json

The final output file will be saved in the output directory specified by the output_dir field in the configuration file used to execute the wikidata_facet_property_clustering.py Python script. The file's name is determined by the max_jaccard_gold_and_predicted_clusters_file field in the same configuration file.