Shadowgraph
TrendingBuzzTop stargazers
Chrome extension
Sign in

pranavrao-qure/MiniTasks

Repo for misc scripts

Jupyter Notebook

0

8 commits

updated Dec 9, 2023

See the code

README

MiniTasks

Repo for misc scripts

RadBERT/ 1. Pretraining Task: a. Script run_mlm.sh : Dataset at /home/users/pranav.rao/MiniTasks/Radbert/data/pretrain_data/full_train_data.txt (@TODO: report cleaning script ) Output model : /models/user_checkpoints/pranav.rao/RadBert/PreTrainModelDir/RadBert/20231007/ b. roberta_pretraining_mlm.py

2. Finetuning Task:
    a. radbert_finetuning.py
        Tags extracted from python script (), 2 Million reports. Dataset in format /home/users/pranav.rao/Downloads/report_tags_25k_train.csv
        To train on a subset of labels give the subset in labels_file argument (/home/users/pranav.rao/MiniTasks/Radbert/labels_top50_withloc.txt)
        Sample run command: python3 radbert_finetuning_lightning.py 0 labels_top50_withloc.txt 0.00003 /models/user_checkpoints/pranav.rao/RadBert/PreTrainModelDir/RadBert/20231007/(optional)
    b. radbert_finetuning_lightning.py
        Same script with Lightning module implemented and training loop replaced by pl.Trainer

Similar repos

sradc/pretraining-BERT

Pre-train BERT from scratch, with HuggingFace. Accompanies the blog post:…

45

patricklai14/cs7643_project

Code for CS 7643 Project

0

ncbi-nlp/bluebert

BlueBERT, pre-trained on PubMed abstracts and clinical notes (MIMIC-III).

600

MuhammetCanGumussu/Bert-Implementation-From-Scratch

2

JunhoKim94/MELT

MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science…

8

alexyalunin/RuBioRoBERTa

3

AI4Bharat/IndicBERT

Pretraining, fine-tuning and evaluation scripts for IndicBERT-v2 and IndicXTREME

124

PEASEC/CySecBERT

12

pranavrao-qure/MiniTasks

Repo for misc scripts

Jupyter Notebook

0

8 commits

updated Dec 9, 2023

See the code

README

MiniTasks

Repo for misc scripts

RadBERT/ 1. Pretraining Task: a. Script run_mlm.sh : Dataset at /home/users/pranav.rao/MiniTasks/Radbert/data/pretrain_data/full_train_data.txt (@TODO: report cleaning script ) Output model : /models/user_checkpoints/pranav.rao/RadBert/PreTrainModelDir/RadBert/20231007/ b. roberta_pretraining_mlm.py

2. Finetuning Task:
    a. radbert_finetuning.py
        Tags extracted from python script (), 2 Million reports. Dataset in format /home/users/pranav.rao/Downloads/report_tags_25k_train.csv
        To train on a subset of labels give the subset in labels_file argument (/home/users/pranav.rao/MiniTasks/Radbert/labels_top50_withloc.txt)
        Sample run command: python3 radbert_finetuning_lightning.py 0 labels_top50_withloc.txt 0.00003 /models/user_checkpoints/pranav.rao/RadBert/PreTrainModelDir/RadBert/20231007/(optional)
    b. radbert_finetuning_lightning.py
        Same script with Lightning module implemented and training loop replaced by pl.Trainer

Similar repos

sradc/pretraining-BERT

Pre-train BERT from scratch, with HuggingFace. Accompanies the blog post:…

45

patricklai14/cs7643_project

Code for CS 7643 Project

0

ncbi-nlp/bluebert

BlueBERT, pre-trained on PubMed abstracts and clinical notes (MIMIC-III).

600

MuhammetCanGumussu/Bert-Implementation-From-Scratch

2

JunhoKim94/MELT

MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science…

8

alexyalunin/RuBioRoBERTa

3

AI4Bharat/IndicBERT

Pretraining, fine-tuning and evaluation scripts for IndicBERT-v2 and IndicXTREME

124

PEASEC/CySecBERT

12