Arulkumar03/Wheat_HEAD_Detection_Counting_ComputerVision_Model
0
1# Apply Net2 3`apply_net` is a tool to print or visualize DensePose results on a set of images.4It has two modes: `dump` to save DensePose model results to a pickle file5and `show` to visualize them on images.6 7The `image.jpg` file that is used as an example in this doc can be found [here](http://images.cocodataset.org/train2017/000000117508.jpg)8 9## Dump Mode10 11The general command form is:12```bash13python apply_net.py dump [-h] [-v] [--output <dump_file>] <config> <model> <input>14```15 16There are three mandatory arguments:17 - `<config>`, configuration file for a given model;18 - `<model>`, model file with trained parameters19 - `<input>`, input image file name, pattern or folder20 21One can additionally provide `--output` argument to define the output file name,22which defaults to `output.pkl`.23 24 25Examples:26 271. Dump results of the [R_50_FPN_s1x](https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl) DensePose model for images in a folder `images` to file `dump.pkl`:28```bash29python apply_net.py dump configs/densepose_rcnn_R_50_FPN_s1x.yaml \30https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl \31images --output dump.pkl -v32```33 342. Dump results of the [R_50_FPN_s1x](https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl) DensePose model for images with file name matching a pattern `image*.jpg` to file `results.pkl`:35```bash36python apply_net.py dump configs/densepose_rcnn_R_50_FPN_s1x.yaml \37https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl \38"image*.jpg" --output results.pkl -v39```40 41If you want to load the pickle file generated by the above command:42```43# make sure DensePose is in your PYTHONPATH, or use the following line to add it:44sys.path.append("/your_detectron2_path/detectron2_repo/projects/DensePose/")45 46f = open('/your_result_path/results.pkl', 'rb')47data = pickle.load(f)48```49 50The file `results.pkl` contains the list of results per image, for each image the result is a dictionary.51 52**If you use a [IUV model](DENSEPOSE_IUV.md#-model-zoo-and-baselines)**, the dumped data will have the following format:53 54```55data: [{'file_name': '/your_path/image1.jpg',56 'scores': tensor([0.9884]),57 'pred_boxes_XYXY': tensor([[ 69.6114, 0.0000, 706.9797, 706.0000]]),58 'pred_densepose': [DensePoseChartResultWithConfidences(labels=tensor(...), uv=tensor(...), sigma_1=None,59 sigma_2=None, kappa_u=None, kappa_v=None, fine_segm_confidence=None, coarse_segm_confidence=None),60 DensePoseChartResultWithConfidences, ...]61 }62 {'file_name': '/your_path/image2.jpg',63 'scores': tensor([0.9999, 0.5373, 0.3991]),64 'pred_boxes_XYXY': tensor([[ 59.5734, 7.7535, 579.9311, 932.3619],65 [612.9418, 686.1254, 612.9999, 704.6053],66 [164.5081, 407.4034, 598.3944, 920.4266]]),67 'pred_densepose': [DensePoseChartResultWithConfidences(labels=tensor(...), uv=tensor(...), sigma_1=None,68 sigma_2=None, kappa_u=None, kappa_v=None, fine_segm_confidence=None, coarse_segm_confidence=None),69 DensePoseChartResultWithConfidences, ...]70 }]71```72 73`DensePoseChartResultWithConfidences` contains the following fields:74- `labels` - a tensor of size `[H, W]` of type `torch.long` which contains fine segmentation labels (previously called `I`)75- `uv` - a tensor of size `[2, H, W]` of type `torch.float` which contains `U` and `V` coordinates76- various optional confidence-related fields (`sigma_1`, `sigma_2`, `kappa_u`, `kappa_v`, `fine_segm_confidence`, `coarse_segm_confidence`)77 78 79**If you use a [CSE model](DENSEPOSE_CSE.md#-model-zoo-and-baselines)**, the dumped data will have the following format:80```81data: [{'file_name': '/your_path/image1.jpg',82 'scores': tensor([0.9984, 0.9961]),83 'pred_boxes_XYXY': tensor([[480.0093, 461.0796, 698.3614, 696.1011],84 [78.1589, 168.6614, 307.1287, 653.8522]]),85 'pred_densepose': DensePoseEmbeddingPredictorOutput(embedding=tensor(...), coarse_segm=tensor(...))}86 {'file_name': '/your_path/image2.jpg',87 'scores': tensor([0.9189, 0.9491]),88 'pred_boxes_XYXY': tensor([[734.9685, 534.2003, 287.3923, 254.8859],89 [434.2853, 765.1219, 132.1029, 867.9283]]),90 'pred_densepose': DensePoseEmbeddingPredictorOutput(embedding=tensor(...), coarse_segm=tensor(...))}]91```92 93`DensePoseEmbeddingPredictorOutput` contains the following fields:94- `embedding` - a tensor of size `[N, D, sz, sz]` of type `torch.float`, which contains embeddings of size `D` of the `N` detections in the image95- `coarse_segm` - a tensor of size `[N, 2, sz, sz]` of type `torch.float` which contains segmentation scores of the `N` detections in the image; e.g. a mask can be obtained by `coarse_segm.argmax(dim=1)`96 97`sz` is a fixed size for the tensors; you can resize them to the size of the bounding box, if needed98 99We can use the following code, to parse the outputs of the first100detected instance on the first image (IUV model).101```102img_id, instance_id = 0, 0 # Look at the first image and the first detected instance103bbox_xyxy = data[img_id]['pred_boxes_XYXY'][instance_id]104result = data[img_id]['pred_densepose'][instance_id]105uv = result.uv106```107The array `bbox_xyxy` contains (x0, y0, x1, y1) of the bounding box.108 109 110## Visualization Mode111 112The general command form is:113```bash114python apply_net.py show [-h] [-v] [--min_score <score>] [--nms_thresh <threshold>] [--output <image_file>] <config> <model> <input> <visualizations>115```116 117There are four mandatory arguments:118 - `<config>`, configuration file for a given model;119 - `<model>`, model file with trained parameters120 - `<input>`, input image file name, pattern or folder121 - `<visualizations>`, visualizations specifier; currently available visualizations are:122 * `bbox` - bounding boxes of detected persons;123 * `dp_segm` - segmentation masks for detected persons;124 * `dp_u` - each body part is colored according to the estimated values of the125 U coordinate in part parameterization;126 * `dp_v` - each body part is colored according to the estimated values of the127 V coordinate in part parameterization;128 * `dp_contour` - plots contours with color-coded U and V coordinates;129 * `dp_iuv_texture` - transfers the texture from a given texture image file to detected instances, in IUV mode;130 * `dp_vertex` - plots the rainbow visualization of the closest vertices prediction for a given mesh, in CSE mode;131 * `dp_cse_texture` - transfers the texture from a given list of texture image files (one from each human or animal mesh) to detected instances, in CSE mode132 133 134One can additionally provide the following optional arguments:135 - `--min_score` to only show detections with sufficient scores that are not lower than provided value136 - `--nms_thresh` to additionally apply non-maximum suppression to detections at a given threshold137 - `--output` to define visualization file name template, which defaults to `output.png`.138 To distinguish output file names for different images, the tool appends 1-based entry index,139 e.g. output.0001.png, output.0002.png, etc...140- `--texture_atlas` to define the texture atlas image for IUV texture transfer141- `--texture_atlases_map` to define the texture atlas images map (a dictionary `{mesh name: texture atlas image}`) for CSE texture transfer142 143 144The following examples show how to output results of a DensePose model145with ResNet-50 FPN backbone using different visualizations for image `image.jpg`:146 1471. Show bounding box and segmentation:148```bash149python apply_net.py show configs/densepose_rcnn_R_50_FPN_s1x.yaml \150https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl \151image.jpg bbox,dp_segm -v152```153154 1552. Show bounding box and estimated U coordinates for body parts:156```bash157python apply_net.py show configs/densepose_rcnn_R_50_FPN_s1x.yaml \158https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl \159image.jpg bbox,dp_u -v160```161162 1633. Show bounding box and estimated V coordinates for body parts:164```bash165python apply_net.py show configs/densepose_rcnn_R_50_FPN_s1x.yaml \166https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl \167image.jpg bbox,dp_v -v168```169170 1714. Show bounding box and estimated U and V coordinates via contour plots:172```bash173python apply_net.py show configs/densepose_rcnn_R_50_FPN_s1x.yaml \174https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl \175image.jpg dp_contour,bbox -v176```177178 1795. Show bounding box and texture transfer:180```bash181python apply_net.py show configs/densepose_rcnn_R_50_FPN_s1x.yaml \182https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl \183image.jpg dp_iuv_texture,bbox --texture_atlas texture_from_SURREAL.jpg -v184```185186 1876. Show bounding box and CSE rainbow visualization:188```bash189python apply_net.py show configs/cse/densepose_rcnn_R_50_FPN_s1x.yaml \190https://dl.fbaipublicfiles.com/densepose/cse/densepose_rcnn_R_50_FPN_s1x/251155172/model_final_c4ea5f.pkl \191image.jpg dp_vertex,bbox -v192```193194 1957. Show bounding box and CSE texture transfer:196```bash197python apply_net.py show configs/cse/densepose_rcnn_R_50_FPN_s1x.yaml \198https://dl.fbaipublicfiles.com/densepose/cse/densepose_rcnn_R_50_FPN_s1x/251155172/model_final_c4ea5f.pkl \199image.jpg dp_cse_texture,bbox --texture_atlases_map '{"smpl_27554": "smpl_uvSnapshot_colors.jpg"}' -v200```201202 203The texture files can be found in the `doc/images` folder204