README.md

State-of-the-art result for all Machine Learning Problems

LAST UPDATE: 13th November 2017

NEWS: I am looking for a Collaborator esp who does research in NLP, Computer Vision and Reinforcement learning. If you are not a researcher, but you are willing, contact me. Email me: [email protected]

This repository provides state-of-the-art (SoTA) results for all machine learning problems. We do our best to keep this repository up to date. If you do find a problem's SoTA result is out of date or missing, please raise this as an issue (with this information: research paper name, dataset, metric, source code and year). We will fix it immediately.

You can also submit this Google Form if you are new to Github.

This is an attempt to make one stop for all types of machine learning problems state of the art result. I can not do this alone. I need help from everyone. Please submit the Google form/raise an issue if you find SOTA result for a dataset. Please share this on Twitter, Facebook, and other social media.

This summary is categorized into:

• Supervised Learning
  • Speech
  • Computer Vision
  • NLP
• Semi-supervised Learning
  • Computer Vision
• Unsupervised Learning
  • Speech
  • Computer Vision
  • NLP
• Transfer Learning
• Reinforcement Learning

Supervised Learning

NLP

1. Language Modelling

Research Paper	Datasets	Metric	Source Code	Year
BREAKING THE SOFTMAX BOTTLENECK: A HIGH-RANK RNN LANGUAGE MODEL	PTB WikiText-2	Perplexity: 47.69 Perplexity: 40.68	Pytorch	2017
DYNAMIC EVALUATION OF NEURAL SEQUENCE MODELS	PTB WikiText-2	Perplexity: 51.1 Perplexity: 44.3	Pytorch	2017
Averaged Stochastic Gradient Descent with Weight Dropped LSTM or QRNN	PTB WikiText-2	Perplexity: 52.8 Perplexity: 52.0	Pytorch	2017
FRATERNAL DROPOUT	PTB WikiText-2	Perplexity: 56.8 Perplexity: 64.1	Pytorch	2017
Factorization tricks for LSTM networks	One Billion Word Benchmark	Perplexity: 23.36	Tensorflow	2017

2. Machine Translation

Research Paper	Datasets	Metric	Source Code	Year
WEIGHTED TRANSFORMER NETWORK FOR MACHINE TRANSLATION	WMT 2014 English-to-French WMT 2014 English-to-German	BLEU: 41.4 BLEU: 28.9	NOT FOUND	2017
Attention Is All You Need	WMT 2014 English-to-French WMT 2014 English-to-German	BLEU: 41.0 BLEU: 28.4	PyTorch Tensorflow	2017
NON-AUTOREGRESSIVE NEURAL MACHINE TRANSLATION	WMT16 Ro→En	BLEU: 31.44	NOT FOUND	2017
Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets	NIST02 NIST03 NIST04 NIST05	38.74 36.01 37.54 33.76	NMTPY	2017

3. Text Classification

Research Paper	Datasets	Metric	Source Code	Year
Learning Structured Text Representations	Yelp	Accuracy: 68.6	NOT FOUND	2017
Attentive Convolution	Yelp	Accuracy: 67.36	NOT FOUND	2017

4. Natural Language Inference

Leader board:

Stanford Natural Language Inference (SNLI)

MultiNLI

Research Paper	Datasets	Metric	Source Code	Year
NATURAL LANGUAGE INFERENCE OVER INTERACTION SPACE	Stanford Natural Language Inference (SNLI)	Accuracy: 88.9	Tensorflow	2017

5. Question Answering

Leader Board

SQuAD

Research Paper	Datasets	Metric	Source Code	Year
Interactive AoA Reader+ (ensemble)	The Stanford Question Answering Dataset	Exact Match: 79.083 F1: 86.450	NOT YET AVAILABLE	2017

6. Named entity recognition

Research Paper	Datasets	Metric	Source Code	Year
Named Entity Recognition in Twitter using Images and Text	Ritter	F-measure: 0.59	NOT YET AVAILABLE	2017

7. Abstractive Summarization

Research Paper	Datasets	Metric	Source Code	Year
Cutting-off redundant repeating generations for neural abstractive summarization	DUC-2004 Gigaword	DUC-2004 ROUGE-1: 32.28 ROUGE-2: 10.54 ROUGE-L: 27.80 Gigaword ROUGE-1: 36.30 ROUGE-2: 17.31 ROUGE-L: 33.88	NOT YET AVAILABLE	2017
Convolutional Sequence to Sequence	DUC-2004 Gigaword	DUC-2004 ROUGE-1: 33.44 ROUGE-2: 10.84 ROUGE-L: 26.90 Gigaword ROUGE-1: 35.88 ROUGE-2: 27.48 ROUGE-L: 33.29	PyTorch	2017

Computer Vision

1. Classification

Research Paper	Datasets	Metric	Source Code	Year
Dynamic Routing Between Capsules	MNIST	Test Error: 0.25±0.005	PyTorch Tensorflow Keras Chainer	2017
High-Performance Neural Networks for Visual Object Classification	NORB	Test Error: 2.53 ± 0.40	NOT FOUND	2011
ShakeDrop regularization	CIFAR-10 CIFAR-100	Test Error: 2.31% Test Error: 12.19%	NOT FOUND	2017
Aggregated Residual Transformations for Deep Neural Networks	CIFAR-10	Test Error: 3.58%	PyTorch	2017
Dynamic Routing Between Capsules	MultiMNIST	Test Error: 5%	PyTorch Tensorflow Keras Chainer	2017
Squeeze-and-Excitation Networks	ImageNet-1k	Top-1 Error 18.68	CAFFE	2017
Aggregated Residual Transformations for Deep Neural Networks	ImageNet-1k	Top-1 Error: 20.4%	PyTorch	2016

2. Instance Segmentation

Research Paper	Datasets	Metric	Source Code	Year
Mask R-CNN	COCO	Average Precision: 37.1%	TensorFlow (This is not official. Performance is worse than the reported AP)	2017

Speech

1. ASR

Research Paper	Datasets	Metric	Source Code	Year
The Microsoft 2017 Conversational Speech Recognition System	Switchboard Hub5'00	WER: 5.1	NOT FOUND	2017

Semi-supervised Learning

Computer Vision

     

Research Paper	Datasets	Metric	Source Code	Year
DISTRIBUTIONAL SMOOTHINGWITH VIRTUAL ADVERSARIAL TRAINING	SVHN NORB	Test error: 24.63 Test error: 9.88	Theano	2016
Virtual Adversarial Training: a Regularization Method for Supervised and Semi-supervised Learning	MNIST	Test error: 1.27	NOT FOUND	2017
Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro	Market-1501 CUHK-03 DukeMTMC-reID CUB-200-2011	Rank-1: 83.97 mAP: 66.07 Rank-1: 84.6 mAP: 87.4 Rank-1: 67.68 mAP: 47.13           Test Accuracy: 84.4	Matconvnet	2017

Unsupervised Learning

Computer Vision

1. Generative Model

Research Paper	Datasets	Metric	Source Code	Year
PROGRESSIVE GROWING OF GANS FOR IMPROVED QUALITY, STABILITY, AND VARIATION	Unsupervised CIFAR 10	Inception score: 8.80	Theano	2017

NLP

Machine Translation

Research Paper	Datasets	Metric	Source Code	Year
UNSUPERVISED MACHINE TRANSLATION USING MONOLINGUAL CORPORA ONLY	WMT16 (en-fr fr-en de-en en-de) Multi30k-Task1(en-fr fr-en de-en en-de)	BLEU:(32.76 32.07 26.26 22.74) BLEU:(15.05 14.31 13.33 9.64)	NOT FOUND	2017

Transfer Learning

Research Paper	Datasets	Metric	Source Code	Year
One Model To Learn Them All	WMT EN → DE WMT EN → FR (BLEU) ImageNet (top-5 accuracy)	BLEU: 21.2 BLEU:30.5 86%	Tensorflow	2017

Reinforcement Learning

Email: [email protected]