Arulkumar03/Wheat_HEAD_Detection_Counting_ComputerVision_Model

1# Deployment2 3Models written in Python need to go through an export process to become a deployable artifact.4A few basic concepts about this process:5 6__"Export method"__ is how a Python model is fully serialized to a deployable format.7We support the following export methods:8 9* `tracing`: see [pytorch documentation](https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html) to learn about it10* `scripting`: see [pytorch documentation](https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html) to learn about it11* `caffe2_tracing`: replace parts of the model by caffe2 operators, then use tracing.12 13__"Format"__ is how a serialized model is described in a file, e.g.14TorchScript, Caffe2 protobuf, ONNX format.15__"Runtime"__ is an engine that loads a serialized model and executes it,16e.g., PyTorch, Caffe2, TensorFlow, onnxruntime, TensorRT, etc.17A runtime is often tied to a specific format18(e.g. PyTorch needs TorchScript format, Caffe2 needs protobuf format).19We currently support the following combination and each has some limitations:20 21```eval_rst22+----------------------------+-------------+-------------+-----------------------------+23|       Export Method        |   tracing   |  scripting  |       caffe2_tracing        |24+============================+=============+=============+=============================+25| **Formats**                | TorchScript | TorchScript | Caffe2, TorchScript, ONNX   |26+----------------------------+-------------+-------------+-----------------------------+27| **Runtime**                | PyTorch     | PyTorch     | Caffe2, PyTorch             |28+----------------------------+-------------+-------------+-----------------------------+29| C++/Python inference       | ✅          | ✅          | ✅                          |30+----------------------------+-------------+-------------+-----------------------------+31| Dynamic resolution         | ✅          | ✅          | ✅                          |32+----------------------------+-------------+-------------+-----------------------------+33| Batch size requirement     | Constant    | Dynamic     | Batch inference unsupported |34+----------------------------+-------------+-------------+-----------------------------+35| Extra runtime deps         | torchvision | torchvision | Caffe2 ops (usually already |36|                            |             |             |                             |37|                            |             |             | included in PyTorch)        |38+----------------------------+-------------+-------------+-----------------------------+39| Faster/Mask/Keypoint R-CNN | ✅          | ✅          | ✅                          |40+----------------------------+-------------+-------------+-----------------------------+41| RetinaNet                  | ✅          | ✅          | ✅                          |42+----------------------------+-------------+-------------+-----------------------------+43| PointRend R-CNN            | ✅          | ❌          | ❌                          |44+----------------------------+-------------+-------------+-----------------------------+45| Cascade R-CNN              | ✅          | ❌          | ❌                          |46+----------------------------+-------------+-------------+-----------------------------+47 48```49 50`caffe2_tracing` is going to be deprecated.51We don't plan to work on additional support for other formats/runtime, but contributions are welcome.52 53 54## Deployment with Tracing or Scripting55 56Models can be exported to TorchScript format, by either57[tracing or scripting](https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html).58The output model file can be loaded without detectron2 dependency in either Python or C++.59The exported model often requires torchvision (or its C++ library) dependency for some custom ops.60 61This feature requires PyTorch ≥ 1.8.62 63### Coverage64Most official models under the meta architectures `GeneralizedRCNN` and `RetinaNet`65are supported in both tracing and scripting mode.66Cascade R-CNN and PointRend are currently supported in tracing.67Users' custom extensions are supported if they are also scriptable or traceable.68 69For models exported with tracing, dynamic input resolution is allowed, but batch size70(number of input images) must be fixed.71Scripting can support dynamic batch size.72 73### Usage74 75The main export APIs for tracing and scripting are [TracingAdapter](../modules/export.html#detectron2.export.TracingAdapter)76and [scripting_with_instances](../modules/export.html#detectron2.export.scripting_with_instances).77Their usage is currently demonstrated in [test_export_torchscript.py](../../tests/test_export_torchscript.py)78(see `TestScripting` and `TestTracing`)79as well as the [deployment example](../../tools/deploy).80Please check that these examples can run, and then modify for your use cases.81The usage now requires some user effort and necessary knowledge for each model to workaround the limitation of scripting and tracing.82In the future we plan to wrap these under simpler APIs to lower the bar to use them.83 84## Deployment with Caffe2-tracing85We provide [Caffe2Tracer](../modules/export.html#detectron2.export.Caffe2Tracer)86that performs the export logic.87It replaces parts of the model with Caffe2 operators,88and then export the model into Caffe2, TorchScript or ONNX format.89 90The converted model is able to run in either Python or C++ without detectron2/torchvision dependency, on CPU or GPUs.91It has a runtime optimized for CPU & mobile inference, but not optimized for GPU inference.92 93This feature requires ONNX ≥ 1.6.94 95### Coverage96 97Most official models under these 3 common meta architectures: `GeneralizedRCNN`, `RetinaNet`, `PanopticFPN`98are supported. Cascade R-CNN is not supported. Batch inference is not supported.99 100Users' custom extensions under these architectures (added through registration) are supported101as long as they do not contain control flow or operators not available in Caffe2 (e.g. deformable convolution).102For example, custom backbones and heads are often supported out of the box.103 104### Usage105 106The APIs are listed at [the API documentation](../modules/export).107We provide [export_model.py](../../tools/deploy/) as an example that uses108these APIs to convert a standard model. For custom models/datasets, you can add them to this script.109 110### Use the model in C++/Python111 112The model can be loaded in C++ and deployed with113either Caffe2 or Pytorch runtime.. [C++ examples](../../tools/deploy/) for Mask R-CNN114are given as a reference. Note that:115 116* Models exported with `caffe2_tracing` method take a special input format117  described in [documentation](../modules/export.html#detectron2.export.Caffe2Tracer).118  This was taken care of in the C++ example.119 120* The converted models do not contain post-processing operations that121  transform raw layer outputs into formatted predictions.122  For example, the C++ examples only produce raw outputs (28x28 masks) from the final123  layers that are not post-processed, because in actual deployment, an application often needs124  its custom lightweight post-processing, so this step is left for users.125 126To help use the Caffe2-format model in python,127we provide a python wrapper around the converted model, in the128[Caffe2Model.\_\_call\_\_](../modules/export.html#detectron2.export.Caffe2Model.__call__) method.129This method has an interface that's identical to the [pytorch versions of models](./models.md),130and it internally applies pre/post-processing code to match the formats.131This wrapper can serve as a reference for how to use Caffe2's python API,132or for how to implement pre/post-processing in actual deployment.133 134## Conversion to TensorFlow135[tensorpack Faster R-CNN](https://github.com/tensorpack/tensorpack/tree/master/examples/FasterRCNN/convert_d2)136provides scripts to convert a few standard detectron2 R-CNN models to TensorFlow's pb format.137It works by translating configs and weights, therefore only support a few models.138