Throughput Benchmark Python Sample
This sample demonstrates how to estimate performance of a model using Asynchronous Inference Request API in throughput mode. Unlike demos this sample doesn't have other configurable command line arguments. Feel free to modify sample's source code to try out different options.
The reported results may deviate from what benchmark_app reports. One example is model input precision for computer vision tasks. benchmark_app sets uint8, while the sample uses default model precision which is usually float32.
For more detailed information on how this sample works, check the dedicated article
Requirements
| Options | Values |
|---|---|
| Validated Models | yolo-v3-tf |
| face-detection-0200 | |
| Model Format | OpenVINO™ toolkit Intermediate Representation |
| (*.xml + *.bin), ONNX (*.onnx) | |
| Supported devices | All |
| Other language realization | C++ |
The following Python API is used in the application:
| Feature | API | Description |
|---|---|---|
| OpenVINO API Version | [openvino.__version__] | Get Openvino API version. |
| Basic Infer Flow | [openvino.runtime.Core], | Common API to do inference: compile a model, |
| [openvino.runtime.Core.compile_model] | configure input tensors. | |
| [openvino.runtime.InferRequest.get_tensor] | ||
| Asynchronous Infer | [openvino.runtime.AsyncInferQueue], | Do asynchronous inference. |
| [openvino.runtime.AsyncInferQueue.start_async], | ||
| [openvino.runtime.AsyncInferQueue.wait_all], | ||
| [openvino.runtime.InferRequest.results] | ||
| Model Operations | [openvino.runtime.CompiledModel.inputs] | Get inputs of a model. |
| Tensor Operations | [openvino.runtime.Tensor.get_shape], | Get a tensor shape and its data. |
| [openvino.runtime.Tensor.data] |