test_from_database.py	- Evaluates the performance of text embedding models on test datasets, both in-domain and out-of-domain, using fuzzy k-nearest neighbors classification - It measures accuracy, precision, recall, F1 score, MSE, and MAE across different values of K, optimizing model parameters and visualizing results to identify the best configuration.
infer.py	- `algorithm/infer.py` serves as the inference module, utilizing a pre-trained text embedding model to process input text, generate embeddings, and perform k-nearest neighbors search against a serialized index - It classifies the text based on the proximity of its embeddings to known labeled data, determining if the source is human, AI, or mixed.
gen_database.py	- Generates and manages embeddings for textual data, facilitating the creation and updating of searchable databases - It supports distributed processing for scalability and includes functionality to handle both in-domain and out-of-domain datasets, ensuring robustness across different data types - The script also provides tools for serialization and deserialization of indexed data.
requirements.txt	- Specifies the dependencies required for the algorithm component of the project, ensuring compatibility and functionality across various machine learning and data processing libraries - It includes essential libraries for data manipulation, progress tracking, neural network modeling, natural language processing, and visualization, facilitating a robust environment for algorithm development and experimentation.
train_classifier.py	- The `train_classifier.py` file is a critical component of the project's machine learning pipeline, specifically designed for training classification models - This script integrates various machine learning and deep learning libraries and frameworks to set up, train, and evaluate classifiers using textual data - Its primary function is to orchestrate the training process, which includes data loading, model initialization, setting up the training loop, and logging the training progress. Key functionalities of this script include: 1 - Data Preparation: It loads and preprocesses text data, ensuring it is in the correct format for training using utilities like load_dataset and tokenization processes. 2 - Model Setup: It initializes models for text classification, leveraging pre-trained models and custom classifier layers, which are crucial for handling the specifics of the text data. 3 - Training Loop: The script manages the training process, including setting random seeds for reproducibility, configuring data loaders for batch processing of data, and defining the optimization strategy. 4 - Logging and Metrics: Utilizes SummaryWriter for TensorBoard to log training metrics and progress, which is vital for monitoring the training process and outcomes. 5 - Evaluation: Implements functions to compute and calculate metrics, aiding in the assessment of the classifier's performance during and after training. In the broader scope of the project, train_classifier.py serves as the executable script that directly impacts the model's performance by handling the training operations efficiently - It interacts with various modules of the project such as src for model definitions, utils for utility functions, and lightning for leveraging PyTorch Lightning capabilities, indicating its integral role in the machine learning workflow of the codebase.

src

text_embedding.py	- TextEmbeddingModel, defined in algorithm/src/text_embedding.py, serves as a core component for generating normalized text embeddings using pre-trained transformer models - It supports various pooling strategies and can handle different transformer architectures, adapting its behavior based on the model's specific characteristics to optimize text representation within the broader system.
index.py	- Indexer, a core component within the algorithm module, manages the indexing, searching, and serialization of vector data using FAISS - It supports operations on GPU, handles large-scale vector searches efficiently, and maintains a mapping between internal and external identifiers, facilitating quick retrieval and robust data management across the system's architecture.
simclr.py	- Defines and implements a SimCLR-based classifier for text embeddings, integrating a classification head for sentence-level tasks - It utilizes cosine similarity for contrastive learning, supports distributed computing with fabric, and handles various classification scenarios including cross-entropy loss calculations for different label types, enhancing model training and evaluation flexibility within the project's architecture.

script

test_from_database.sh	- Executes testing of deepfake detection models using a specified dataset and model checkpoint - It configures the test environment for different modes including deepfake, TuringBench, and M4 challenges, both monolingual and multilingual, adjusting parameters like device number and batch size to optimize performance.
train.sh	- Train.sh initiates the training of a classifier for detecting deepfakes using a pre-trained RoBERTa model from Princeton NLP - It configures the model to process data from a specified path, setting parameters like device number, batch size, learning rate, and epochs, focusing on optimizing performance across different datasets within the broader architecture.
infer.sh	- Executes a deep learning model for detecting deepfake content by leveraging a pre-trained model and a specified database - The script processes a given text input to assess and output the likelihood of the content being a deepfake, using the top 5 predictions - This functionality is crucial for maintaining the integrity and trustworthiness of digital media within the system.
test.sh	- Executes a Python script for testing a deepfake detection model using a K-nearest neighbors algorithm - It configures the script to process data from a specified path, utilizing a pre-trained model, and saves the resulting database for further analysis - The script is adaptable for various datasets but is currently set for the deepfake domain.
gen_database.sh	- Generates a training database for detecting deepfake content by executing a Python script configured for high-throughput processing on multiple devices - It specifically prepares a dataset using a pre-trained model, targeting the deepfake detection domain, and stores the results in a structured database format for further use in model training and evaluation.

utils

load_dataset.py	Manages the loading and preprocessing of datasets for machine learning models, supporting various dataset configurations including "falconset," "llmdetectaive," and "hart." It categorizes text data into human, human+AI, or AI-generated content, enriching each entry with labels and indices for further processing and analysis in model training and evaluation phases.
utils.py	- Provides utility functions for evaluating and reporting machine learning model performance across various metrics such as accuracy, precision, recall, and F1 score - It includes specialized functions for handling multi-class scenarios, binary outcomes, and specific use cases involving embeddings and unique identifiers, enhancing the interpretability of model results in diverse contexts.