Keras-MXNet 2.1.6

Release Notes

Highlights

1. Add Apache MXNet backend support in Keras 2.1.6.
2. Supports Convolutional Neural Network (CNN) and experimental Recurrent Neural Network (RNN) training and inference.
3. Supports high performance, distributed Multi-GPU training of CNN and RNN networks.
4. Supports exporting native MXNet Model from Keras-MXNet trained model. Enabling faster research with Keras interface and high performance, large scale inference in production with the native MXNet engine. You can use all language bindings of MXNet (Scala/Python/Julia/R/Perl) for inference on the exported model.
5. Add Keras benchmarking utility for performing CNN and RNN benchmarks with standard networks and datasets. Supports benchmarking on CPU, one GPU and multi-GPU distributed training.
6. Add keras.utils.to_channels_first() for easy conversion of channels_last data to channels_first.
7. PyPi package for keras-mxnet. pip install keras-mxnet.

Performance

1. Keras-MXNet shows significant improvement in performance with channels_first image_data_format. Performance drops with channels_last data format. Tutorial
2. Keras-MXNet CNN benchmarks shows upto 3X performance improvement on GPUs. See Benchmark Results document for more details.
3. Keras-MXNet RNN support is experimental. Performance on CPU is known to be upto 2X slower (Related issue). However, performance on GPU shows upto 2X performance improvement.

Unsupported Functionalities

1. RNN with Keras-MXNet is experimental and do not support variable length inputs and unroll=False. You need to pad input sequences to make it static length and provide input_length and set unroll=True. See using RNN with Keras-MXNet tutorial for more details.
2. depthwise_conv2d and separable_conv2d operators are not supported.
3. 18 Keras operators are not supported with MXNet backend. Update operators, symbolic gradient, local_conv1d, local_conv2d, higher order functions, other operator liks cumsum, cumprod, stack, ctc and more. See Operators missing with MXNet backend Github Issue for more details.
4. Sparse Tensors are not supported with MXNet backend in this release.
5. Unsupported keras/examples list.
6. Cross backend models are not supported. Training with TensorFlow backend and loading the Keras model with MXNet backend is not supported.

Known Issues

1. MXNet backend performance significantly drops with channels_last image_data_format. It is highly recommended to user channels_first image_data_format. See performance guide for more details.
2. MXNet backend do not support boolean. For example, in_topk operator with MXNet backend uses 0/1 instead of boolean. Issue
3. depthwise_conv2d supports depth_multiplier=1 only. Issue
4. LSTM layer fails if dropout is used. Issue
5. Models with Custom Loss are not serializable. Issue

Contributions

Thanks to all the contributors for their contributions in this release:
@sandeep-krishnamurthy, @jiajiechen, @karan6181, @roywei, @kalyc

Thanks to Keras1.2-MXNet contributors @howard0su, @yajiedesign, @piiswrong, @kevinthesun.

Other Resources and Getting Started

1. Keras-MXNet installation guide - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/installation.md
2. Using multi-gpu distributed training - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/multi_gpu_training.md
3. Performance tuning guide for Keras with MXNet backend - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/performance_guide.md
4. Export MXNet Model from Keras-MXNet - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/save_mxnet_model.md
5. Using RNN in Keras-MXNet. Limitations and workarounds - https://github.com/awslabs/keras-apache-mxnet/blob/master/docs/mxnet_backend/using_rnn_with_mxnet_backend.md
6. Keras-MXNet benchmark results - https://github.com/awslabs/keras-apache-mxnet/tree/master/benchmark