talkie-web-13b-base is a 13B language model trained on 260B tokens of FineWeb. It is identical architecturally to talkie-1930-13b-base and exists to make possible controlled comparisons between vintage and modern LMs.
Read more about talkie in our report.
Reference code to run talkie is available on GitHub.
talkie-web-13b-base is a 13B language model trained on 260B tokens of FineWeb. It is identical architecturally to talkie-1930-13b-base and exists to make possible controlled comparisons between vintage and modern LMs.
Read more about talkie in our report.
Reference code to run talkie is available on GitHub.
