Attention Is All You Need
arXiv:1706.03762 · 26 repos reference this paper in their README
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.
vmehtacode/speech-emotion-recognition
2
·
·
Deep learning system for emotion recognition from speech, achieving 50.5% accuracy on 8-class…
demigodmode/pi-web-agent
79
·
·
Reliable web tools for Pi: search for sources, fetch over HTTP, and use headless browsing only when…
Graphify-Labs/graphify
116,721
·
·
·
Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph.…
NVIDIA/NeMo-Agent-Toolkit-Examples
32
·
·
Community examples utilizing NVIDIA NeMo Agent Toolkit.
brexhq/prompt-engineering
9,584
·
Tips and tricks for working with Large Language Models like OpenAI's GPT-4.
karpathy/minGPT
24,880
·
·
A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
mozilla-ai/document-to-podcast
183
·
·
Blueprint by Mozilla.ai for generating podcasts from documents using local AI
kaituoxu/Speech-Transformer
811
·
·
A PyTorch implementation of Speech Transformer, an End-to-End ASR with Transformer network on…
wisnunugroho21/nugie-jax-nemotron-3-nano
1
·
·
A simple, minimalistic, and explainable code implementation of of Nemotron 3 Nano in JAX
jmaczan/tiny-vllm
1,105
·
·
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM
OpenMachine-ai/transformer-tricks
226
·
·
A collection of tricks and tools to speed up transformer models
cahya-wirawan/indonesian-language-models
163
·
·
Indonesian Language Models and its Usage
lukas-blecher/LaTeX-OCR
16,558
·
·
pix2tex: Using a ViT to convert images of equations into LaTeX code.
