Paper-Replications

This repository contains a collection of code implementations and experiments replicating results from a wide range of influential machine learning and deep learning research papers. Each subfolder corresponds to a specific paper, model, or technique, with code, notes, and sometimes pretrained weights or results.

Deployed models

Visit smolhub to view the deployed models.

Structure

• Attention Mechanisms/: Implementations and experiments with various attention mechanisms.
• BERT/: Replication and exploration of the BERT model.
• CGANs/: Conditional Generative Adversarial Networks.
• CLAP/: Contrastive Language-Audio Pretraining.
• CLiP/: CLIP and related vision-language models.
• CycleGANs/: Cycle-consistent GANs for image translation.
• DCGANs/: Deep Convolutional GANs.
• DDP/: Distributed Data Parallel training experiments.
• DeepSeekV3/: DeepSeek model replications and experiments.
• Differential Transformer/: Differential Transformer architectures.
• DPO/: Direct Preference Optimization and related RLHF methods.
• Encoder-Decoder/: Encoder-decoder architectures for sequence modeling.
• Fine Tuning using PEFT/: Parameter-Efficient Fine-Tuning methods.
• Gemma/, Gemma3/: Replications of Gemma models.
• GPT/: Generative Pretrained Transformer models.
• GRU/: Gated Recurrent Unit models.
• Kimi-K2/: Kimi-K2 model replications and training scripts.
• Llama/, Llama4/: Llama model replications and experiments.
• Llava/: Large Language and Vision Assistant models.
• LoRA/: Low-Rank Adaptation for efficient fine-tuning.
• LSTM/: Long Short-Term Memory models.
• Mixtral/: Mixture-of-Experts Transformer models.
• Moonshine/: Moonshine model experiments.
• ORPO/: Online RLHF Preference Optimization.
• PaliGemma/: PaliGemma model replications.
• Pix2Pix/: Image-to-image translation with Pix2Pix.
• RNNs/: Recurrent Neural Networks.
• Seq2Seq/: Sequence-to-sequence models.
• SigLip/: Sigmoid Loss for Language-Image Pretraining.
• SimplePO/: Simple Preference Optimization.
• Transformer/: Transformer model replications and variants.
• TTS/: Text-to-Speech models.
• VAE/: Variational Autoencoders.
• ViT/: Vision Transformer models.
• WGANs/: Wasserstein GANs.
• Whisper/: Whisper speech recognition model replications.

Usage

Each folder is self-contained and includes code, scripts, and sometimes notebooks for replicating the results of the corresponding paper. Please refer to the README or notes within each subfolder for specific instructions.

Contributing

Feel free to open issues or pull requests if you have suggestions, improvements, or additional replications to add!

Sponsors

Thank you everyone for their support and love for this project!

• abstrait
• Itz-Antaripa

🔗 Links

HuggingFace Account

Authors

• @YuvrajSingh

Citation

If you find this repository useful in your research, please cite it:

@misc{singh_paper_replications_2025,
  author       = {Yuvraj Singh},
  title        = {Paper-Replications: Replication from Scratch Repository using PyTorch},
  year         = {2025},
  publisher    = {GitHub},
  journal      = {GitHub repository},
  howpublished = {\url{https://github.com/YuvrajSingh-mist/Paper-Replications}},
  commit       = {1d7a1b37a82e441cde884f591c9c41fa4e47ddbb}
}