collective-cadence

install WSL, CUDA, PyTorch, dependencies etc

OCR

https://pdf2png.com/ to create folders of png's

https://tesseract-ocr.github.io/ (actually pytesseract) to generate raw txt files.

python pngs/ocr.py

using this llm

python clean-ocr.py -m ../text-gen-install/text-generation-webui/models/Xwin-LM-70B-V0.1-2.3bpw-h6-exl2

then manually search for "______" in generated text files and deal with it...

• other stuff to look for: ©, OCRed

Finetuning

cd axolotl
python -m axolotl.cli.preprocess ../datasets/lora.yml
accelerate launch -m axolotl.cli.train ../datasets/lora.yml
accelerate launch -m axolotl.cli.inference ../datasets/lora.yml \
    --lora_model_dir="./lora-out"

inference

python inference.py ./datasets/qlora.yml     --lora_model_dir="axolotl/qlora-out"

or just use oobabooga

• intro guide

apple M1 inference using llama.cpp:

./main -m /Volumes/Extreme\ SSD/_llama/models/Xwin-LM-13B-V0.2/ggml-model-Q4_K_M --lora /Volumes/Extreme\ SSD/_llama/loras/his-ana-the-512/ggml-adapter-model.bin  -n 8 --prompt "What is dance?"

Paperspace

https://console.paperspace.com/account/settings/ssh-keys

pip install faiss-cpu sentence-transformers groq
scp -i SHA256:my_SSH_key ../datasets/02-HIS-ANA-THE.txt paperspace@74.82.28.60:~
scp -i SHA256:my_SSH_key faiss_index.index paperspace@74.82.28.60:~
scp -i SHA256:my_SSH_key app-groq.py paperspace@74.82.28.60:~
/.
python3 app-groq.py

RAVE

see official tutorial

train:

resume / RTX2070 w/ 8GB VRAM

rave train --name rebe_fem_noguit --db_path "_rave_mp3/rebetika_fem_noguit/pp" --out_path "_rave_mp3/rebetika_fem_noguit/out" --config v2_small --config causal --channels 1 --gpu 0 --augment compress --ckpt "_rave_mp3/rebetika_fem_noguit/out/rebe_fem_noguit_6f53a3ff62/version_14" --batch 4

see this guide

train command:

rave train --config v2 --config wasserstein --override PHASE_1_DURATION=1000000 --db_path ./ --out_path ./out/ --name rebetika_fem_noguit --gpu 0 --n_signal 262144 --val_every 5000 --channels 1 --config causal --batch 8

resume:

rave train --config v2 --config wasserstein --override PHASE_1_DURATION=1000000 --db_path ./ --out_path ./out/ --name rebetika_fem_noguit --gpu 0 --n_signal 262144 --val_every 5000 --channels 1 --config causal --batch 8 --ckpt ./out/rebetika_fem_noguit_5b61af7ec4/version_0/checkpoints/epoch-epoch=61231.ckpt

export command:

cd /out/PATH_TO_CONFIG.GIN
rave export --run ./ --channels 1 --sr 44100 --name test --streaming