Machine Learning Model Attribution

This repository contains code for the paper Model Attribution in Machine Learning and instructions for reproducing the environmental setup.

Starting the API server

The API server is a Flask application that runs on port 5001. To start the server, using docker-compose, run the following command:

docker-compose -f api/docker-compose.yml up

TODO: add startup without docker

The server will be available at http://localhost:5001.

Prompting a model trough the API

Run the following script to derive responses from a custom set of prompts:

python compute_responses/compute_responses.py

Run the following script to derive responses from the pile prompts:

python compute_responses/compute_pile_responses.py

Both scripts will prompt the model and save the responses in a json file.

Run the classifier on the responses

To run the experiments, use the following command:

python perform_attribution.py

The script will output the model attribution results.

Fine-grained evaluation

python k_auc_prec_recall.py

The report will contain Table 1 and Table 2 from the paper. TODO: Add reference to the paper

Acknowledgement

This work was supported by European Union’s Horizon 2020 research and innovation programme under grant number 951911 – AI4Media.