First release

Supported models:

• Openvino (both CPU and FPGA): SSD, Yolo-v3, Faster RCNN, any classification net
• TensorRT (GPU): SSD

Supported protocol:

• HTTP
• GRPC

Bug:

• Maximum of 1 FPGA is allowed in the system; even you have two FPGA and want to run different models on each FPGA, it's not possible by now.
• When running Openvino in hetero mode, if the inference graph has more than two sub-graphs running on FPGA, it will significantly degrade the performance and sometimes can crash the server.
• Some SSD architecture doesn't work on TensorRT.