Arulkumar03/Fox_Sheep_Detector_Computer_Vision_model
0
1# Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation2 3Bowen Cheng, Maxwell D. Collins, Yukun Zhu, Ting Liu, Thomas S. Huang, Hartwig Adam, Liang-Chieh Chen4 5[[`arXiv`](https://arxiv.org/abs/1911.10194)] [[`BibTeX`](#CitingPanopticDeepLab)] [[`Reference implementation`](https://github.com/bowenc0221/panoptic-deeplab)]6 7<div align="center">8 <img src="https://github.com/bowenc0221/panoptic-deeplab/blob/master/docs/panoptic_deeplab.png"/>9</div><br/>10 11## Installation12Install Detectron2 following [the instructions](https://detectron2.readthedocs.io/tutorials/install.html).13To use cityscapes, prepare data follow the [tutorial](https://detectron2.readthedocs.io/tutorials/builtin_datasets.html#expected-dataset-structure-for-cityscapes).14 15## Training16 17To train a model with 8 GPUs run:18```bash19cd /path/to/detectron2/projects/Panoptic-DeepLab20python train_net.py --config-file configs/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024_dsconv.yaml --num-gpus 821```22 23## Evaluation24 25Model evaluation can be done similarly:26```bash27cd /path/to/detectron2/projects/Panoptic-DeepLab28python train_net.py --config-file configs/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024_dsconv.yaml --eval-only MODEL.WEIGHTS /path/to/model_checkpoint29```30 31## Benchmark network speed32 33If you want to benchmark the network speed without post-processing, you can run the evaluation script with `MODEL.PANOPTIC_DEEPLAB.BENCHMARK_NETWORK_SPEED True`:34```bash35cd /path/to/detectron2/projects/Panoptic-DeepLab36python train_net.py --config-file configs/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024_dsconv.yaml --eval-only MODEL.WEIGHTS /path/to/model_checkpoint MODEL.PANOPTIC_DEEPLAB.BENCHMARK_NETWORK_SPEED True37```38 39## Cityscapes Panoptic Segmentation40Cityscapes models are trained with ImageNet pretraining.41 42<table><tbody>43<!-- START TABLE -->44<!-- TABLE HEADER -->45<th valign="bottom">Method</th>46<th valign="bottom">Backbone</th>47<th valign="bottom">Output<br/>resolution</th>48<th valign="bottom">PQ</th>49<th valign="bottom">SQ</th>50<th valign="bottom">RQ</th>51<th valign="bottom">mIoU</th>52<th valign="bottom">AP</th>53<th valign="bottom">Memory (M)</th>54<th valign="bottom">model id</th>55<th valign="bottom">download</th>56<!-- TABLE BODY -->57 <tr><td align="left">Panoptic-DeepLab</td>58<td align="center">R50-DC5</td>59<td align="center">1024×2048</td>60<td align="center"> 58.6 </td>61<td align="center"> 80.9 </td>62<td align="center"> 71.2 </td>63<td align="center"> 75.9 </td>64<td align="center"> 29.8 </td>65<td align="center"> 8668 </td>66<td align="center"> - </td>67<td align="center">model | metrics</td>68</tr>69 <tr><td align="left"><a href="configs/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024.yaml">Panoptic-DeepLab</a></td>70<td align="center">R52-DC5</td>71<td align="center">1024×2048</td>72<td align="center"> 60.3 </td>73<td align="center"> 81.5 </td>74<td align="center"> 72.9 </td>75<td align="center"> 78.2 </td>76<td align="center"> 33.2 </td>77<td align="center"> 9682 </td>78<td align="center"> 30841561 </td>79<td align="center"><a href="https://dl.fbaipublicfiles.com/detectron2/PanopticDeepLab/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32/model_final_bd324a.pkl80">model</a> | <a href="https://dl.fbaipublicfiles.com/detectron2/PanopticDeepLab/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32/metrics.json81">metrics</a></td>82</tr>83 <tr><td align="left"><a href="configs/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024_dsconv.yaml">Panoptic-DeepLab (DSConv)</a></td>84<td align="center">R52-DC5</td>85<td align="center">1024×2048</td>86<td align="center"> 60.3 </td>87<td align="center"> 81.0 </td>88<td align="center"> 73.2 </td>89<td align="center"> 78.7 </td>90<td align="center"> 32.1 </td>91<td align="center"> 10466 </td>92<td align="center"> 33148034 </td>93<td align="center"><a href="https://dl.fbaipublicfiles.com/detectron2/PanopticDeepLab/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024_dsconv/model_final_23d03a.pkl94">model</a> | <a href="https://dl.fbaipublicfiles.com/detectron2/PanopticDeepLab/Cityscapes-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_90k_bs32_crop_512_1024_dsconv/metrics.json95">metrics</a></td>96</tr>97</tbody></table>98 99Note:100- [R52](https://dl.fbaipublicfiles.com/detectron2/DeepLab/R-52.pkl): a ResNet-50 with its first 7x7 convolution replaced by 3 3x3 convolutions. This modification has been used in most semantic segmentation papers. We pre-train this backbone on ImageNet using the default recipe of [pytorch examples](https://github.com/pytorch/examples/tree/master/imagenet).101- DC5 means using dilated convolution in `res5`.102- We use a smaller training crop size (512x1024) than the original paper (1025x2049), we find using larger crop size (1024x2048) could further improve PQ by 1.5% but also degrades AP by 3%.103- The implementation with regular Conv2d in ASPP and head is much heavier head than the original paper.104- This implementation does not include optimized post-processing code needed for deployment. Post-processing the network105 outputs now takes similar amount of time to the network itself. Please refer to speed in the106 original paper for comparison.107- DSConv refers to using DepthwiseSeparableConv2d in ASPP and decoder. The implementation with DSConv is identical to the original paper.108 109## COCO Panoptic Segmentation110COCO models are trained with ImageNet pretraining on 16 V100s.111 112<table><tbody>113<!-- START TABLE -->114<!-- TABLE HEADER -->115<th valign="bottom">Method</th>116<th valign="bottom">Backbone</th>117<th valign="bottom">Output<br/>resolution</th>118<th valign="bottom">PQ</th>119<th valign="bottom">SQ</th>120<th valign="bottom">RQ</th>121<th valign="bottom">Box AP</th>122<th valign="bottom">Mask AP</th>123<th valign="bottom">Memory (M)</th>124<th valign="bottom">model id</th>125<th valign="bottom">download</th>126<!-- TABLE BODY -->127 <tr><td align="left"><a href="configs/COCO-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_200k_bs64_crop_640_640_coco_dsconv.yaml">Panoptic-DeepLab (DSConv)</a></td>128<td align="center">R52-DC5</td>129<td align="center">640×640</td>130<td align="center"> 35.5 </td>131<td align="center"> 77.3 </td>132<td align="center"> 44.7 </td>133<td align="center"> 18.6 </td>134<td align="center"> 19.7 </td>135<td align="center"> </td>136<td align="center"> 246448865 </td>137<td align="center"><a href="https://dl.fbaipublicfiles.com/detectron2/PanopticDeepLab/COCO-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_200k_bs64_crop_640_640_coco_dsconv/model_final_5e6da2.pkl138">model</a> | <a href="https://dl.fbaipublicfiles.com/detectron2/PanopticDeepLab/COCO-PanopticSegmentation/panoptic_deeplab_R_52_os16_mg124_poly_200k_bs64_crop_640_640_coco_dsconv/metrics.json139">metrics</a></td>140</tr>141</tbody></table>142 143Note:144- [R52](https://dl.fbaipublicfiles.com/detectron2/DeepLab/R-52.pkl): a ResNet-50 with its first 7x7 convolution replaced by 3 3x3 convolutions. This modification has been used in most semantic segmentation papers. We pre-train this backbone on ImageNet using the default recipe of [pytorch examples](https://github.com/pytorch/examples/tree/master/imagenet).145- DC5 means using dilated convolution in `res5`.146- This reproduced number matches the original paper (35.5 vs. 35.1 PQ).147- This implementation does not include optimized post-processing code needed for deployment. Post-processing the network148 outputs now takes more time than the network itself. Please refer to speed in the original paper for comparison.149- DSConv refers to using DepthwiseSeparableConv2d in ASPP and decoder.150 151## <a name="CitingPanopticDeepLab"></a>Citing Panoptic-DeepLab152 153If you use Panoptic-DeepLab, please use the following BibTeX entry.154 155* CVPR 2020 paper:156 157```158@inproceedings{cheng2020panoptic,159 title={Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation},160 author={Cheng, Bowen and Collins, Maxwell D and Zhu, Yukun and Liu, Ting and Huang, Thomas S and Adam, Hartwig and Chen, Liang-Chieh},161 booktitle={CVPR},162 year={2020}163}164```165 166* ICCV 2019 COCO-Mapillary workshp challenge report:167 168```169@inproceedings{cheng2019panoptic,170 title={Panoptic-DeepLab},171 author={Cheng, Bowen and Collins, Maxwell D and Zhu, Yukun and Liu, Ting and Huang, Thomas S and Adam, Hartwig and Chen, Liang-Chieh},172 booktitle={ICCV COCO + Mapillary Joint Recognition Challenge Workshop},173 year={2019}174}175```176