Tensorflow 2.0 playground

This is the repo for working with nerual networks with Tensorflow 2.0 beta. Tensorflow 2.0 is out now~

CPU only version installation:

pip install tensorflow

GPU version please consult GPU Guide

Contents

Stand alone examples

1. dataset == MNIST
   • subclass model with eager execution mode in custom loop
   • above + @tf.function decorator
   • above with tf.keras.Sequential model
   • above with tf.keras .fit api
   • metric learning with additive angular margin loss
     Reference: ArcFace: Additive Angular Margin Loss for Deep Face Recognition
2. dataset == cifar10
   • simple cnn model
   • above with mixup
     Reference: mixup: Beyond empirical risk minimization
   • above with ict
     Reference: Interpolation Consistency Training for Semi-Supervised Learning
3. dataset == titanic
   • unsupervised categorical feature embedding

Tutorial Notebooks - I call this 'Learn Tensorflow 2.0 the Hard Way'

1. Tensors Variables Operations and AutoDiff
2. AutoGraph
3. Custom Model and Layer
4. Optimizers
5. Loss Function

Losses

Optimizers

Convolutional Neural Networks

Sequence Models

Graph Neural Networks

Factorization Machines

See the benchmark result on how to use the benchmark runner to train these models on criteo dataset.

Bag of Tricks

• Learning rate finder - Train the model over small number of iterations with increasing learning rates. Plot the loss values against the learning rates to identify sweet spot. Reference
• Monte Carlo Dropout - Kind of TTA procedure, that averages the predictions from model with dropout activated in inference mode. Reference