PanopticFPN算法的简介(论文介绍)

Abstract  

     The recently introduced panoptic segmentation task has  renewed our community’s interest in unifying the tasks of  instance segmentation (for thing classes) and semantic segmentation  (for stuff classes). However, current state-ofthe-art  methods for this joint task use separate and dissimilar  networks for instance and semantic segmentation,  without performing any shared computation. In this work,  we aim to unify these methods at the architectural level,  designing a single network for both tasks. Our approach  is to endow Mask R-CNN, a popular instance segmentation  method, with a semantic segmentation branch using  a shared Feature Pyramid Network (FPN) backbone. Surprisingly,  this simple baseline not only remains effective for  instance segmentation, but also yields a lightweight, topperforming  method for semantic segmentation. In this work,  we perform a detailed study of this minimally extended version  of Mask R-CNN with FPN, which we refer to as Panoptic  FPN, and show it is a robust and accurate baseline for  both tasks. Given its effectiveness and conceptual simplicity,  we hope our method can serve as a strong baseline and  aid future research in panoptic segmentation.

     最近引入的泛光分割任务重新唤起了我们团队对统一实例分割(对象类)和语义分割(对象类)任务的兴趣。然而，目前用于这个联合任务的最先进的方法使用分离和不同的网络，例如语义分割，而不执行任何共享计算。在这项工作中，我们的目标是在体系结构级别统一这些方法，为这两个任务设计一个单一的网络。我们的方法是赋予流行的实例分割方法掩模R-CNN一个使用共享特征金字塔网络(FPN)主干的语义分割分支。令人惊讶的是，这个简单的基线不仅在实例分割方面仍然有效，而且还产生了一个轻量级的、性能***的语义分割方法。在这项工作中，我们使用FPN对这个最小扩展版本的Mask R-CNN进行了详细的研究，我们将其称为Panoptic  FPN，并表明它对于这两个任务来说都是一个健壮和准确的基线。鉴于其有效性和概念的简单性，我们希望我们的方法可以作为一个强大的基线，并帮助未来的研究全光分割。

Conclusion  

     We introduce a conceptually simple yet effective baseline  for panoptic segmentation. The method starts with  Mask R-CNN with FPN and adds to it a lightweight semantic  segmentation branch for dense-pixel prediction. We  hope it can serve as a strong foundation for future research.

     我们介绍了一个概念简单但有效的全光分割基线。该方法以带FPN的Mask R-CNN为起点，在此基础上增加了一个轻量级的语义分割分支，用于密集像素预测。希望为今后的研究打下坚实的基础。

论文

Alexander Kirillov Ross GirshickKaimingHe Piotr Dollár  2019

Panoptic Feature Pyramid Networks

https://arxiv.org/pdf/1901.02446.pdf

0、实验结果

1、使用一个单一的ResNet-101-FPN网络，在COCO(顶部)和Cityscapes(底部)上生成Panoptic FPN结果

Panoptic FPN results on COCO (top) and Cityscapes (bottom) using a single ResNet-101-FPN network.

2、Cityscapes Semantic FPN

3、Multi-Task Training

4、Panoptic Segmentation

Panoptic R50-FPN vs. R50-FPN×2

使用单个FPN网络同时解决这两个任务，与两个独立FPN网络分别产生实例分割和语义分割相当的精度，但计算量只有一半。

1、不同架构比较