Python Infer Client is a python inference client library. It provides one interface to interact with many types of inference client as onnxruntime, tritonclient...
With using the tritonclient client, only supported with GRPC
$ pip install infer-client[tritonclient]
With using the onnxruntime client, both CPU and GPU are supported
$ pip install infer-client[onnxruntime]
or
$ pip install infer-client[onnxruntime-gpu]
import numpy as np
from infer_client.adapters.onnx import OnnxInferenceAdapter
from infer_client.inference import Inference
adapter = OnnxInferenceAdapter(model_name="resources/test_classify", version="1", limit_mem_gpu=-1)
infer_client_obj = Inference(adapter)
res = infer_client_obj.inference({"input": np.random.rand(1, 3, 224, 224)}, ["output"])
Please see CHANGELOG for more information on what has changed recently.
Please see CONTRIBUTING for details.
Please review our security policy on how to report security vulnerabilities.
The MIT License (MIT). Please see License File for more information.