This repository contains a number of different examples
that show how to useTF-TRT.
TF-TRT is a part of TensorFlow
that optimizes TensorFlow graphs usingTensorRT.
We have used these examples to verify the accuracy and
performance of TF-TRT. For more information seeVerified Models.
This module provides necessary bindings and introducesTRTEngineOp operator that wraps a subgraph in TensorRT.
This module is under active development.
Installing TF-TRT
Currently Tensorflow nightly builds include TF-TRT by default,
which means you don't need to install TF-TRT separately.
You can pull the latest TF containers from docker hub or
install the latest TF pip package to get access to the latest TF-TRT.
In order to make use of TF-TRT, you will need a local installation
of TensorRT from theNVIDIA Developer website.
Installation instructions for compatibility with TensorFlow are provided on theTensorFlow GPU support guide.
Documentation
TF-TRT documentaiongives an overview of the supported functionalities, provides tutorials
and verified models, explains best practices with troubleshooting guides.
Tests
TF-TRT includes both Python tests and C++ unit tests.
Most of Python tests are located in the test directory
and they can be executed uring bazel test or directly
with the Python command. Most of the C++ unit tests are
used to test the conversion functions that convert each TF op to
a number of TensorRT layers.
Compilation
In order to compile the module, you need to have a local TensorRT installation
(libnvinfer.so and respective include files). During the configuration step,
TensorRT should be enabled and installation path should be set. If installed
through package managers (deb,rpm), configure script should find the necessary
components from the system automatically. If installed from tar packages, user
has to set path to location where the library is installed during configuration.