Faster-RCNN

Contribution:

提出了RPN（Region Proposal Network）和 anchor box, 可以使用神经网络来提取proposal，优点如下：


 1）几乎是cost-free的，因为RPN和特征提取的CNN共享参数；
2）RPN可以同时预测bounding box 和 objective score；
3）提出了anchor box，可以用多种宽高比和尺度来预测proposal;
4) 可以使得整个目标检测的网络进行end-to-end training。

Architecture:

在Fast-RCNN的基础上加入了RPN(region proposal network)



Faster RCNN 主要包括4个部分： Conv layers, RPN, RoI pooling, Classifier。其中RoI pooling，classifier属于Fast RCNN, 而Conv layers 是RPN和Fast RCNN共享的部分。

1. Shared conv layers: 使用了13个卷积层和4个pooling层来提取图片的feature map, 这部分是Fast RCNN 和RPN共享的；

2. Region Proposal Network: RPN 用于生成region proposals。该层通过softmax判断 achors属于positive 或者negative, 再利用bounding box regression 修正anchors以获得精确的proposals；

3. ROI pooling：根据regional proposal和feature map来提取每个region 对应的feature, 送入后续的全连接层判定目标类别；

4. Classification and Regression：和Fast RCNN类似，这部分主要是完成两个工作：
（1）一个是经过FC layer + softmax进行分类，主要是对object proposal进行分类的，一共是k+1类，包括背景类； （2）另一个是经过FC layer + bbox regressor 输出的，这个就是为k个类各输出4个值。

上图是python版本中的vgg16模型中faster_rcnn_test.pt 的网络结构，该网络对于一幅任意的PxQ的图像，首先缩放至MxN，然后将图像送入网络，而Convlayers中包含了13 conv + 13 relu + 4 pooling; RPN首先经过3x3卷积，在分解生成positive anchors 和对应的bounding box regression偏移量，然后在计算出proposals; 在roi pooling层利用proposals从feature maps中提取proposal feature送入后续的全连接和softmax网络做classification。



RPN 实际包含了两条线:
上面一条通过softmax分类anchors获得positive和negative分类；
下面一条用于计算anchors的bounding box regression偏移量，以获得精确的proposal。最后的proposal层负责综合positive anchors和对应的bounding box regression 偏移量以获取proposals， 同时剔除太小和超出边界的proposals。

ROI pooling: 为何需要roi pooling?

对于传统的CNN, 当网络训练好后输入的图像尺寸必须是固定值，同时网络输出也是固定大小的vector 或 matrix。如果输入图像大小不定，就会出问题。两种解决办法：
1. 从图像中crop一部分传入网络；
2. 将图像warp成需要的大小后传入网络中。

但是无论哪种办法都不好。

ROI pooling 层：
输入shape: (N, W/16, H/16, channels), 为什么除16，因为使用VGG16的话，会经历4个 2*2 的max pooling。

输出shape: (num_rois, expected_H, expected_W, channels), 论文中提出如果使用VGG16的话，expected_H=expected_W=7

RoI pooling 把所有proposals的feature规整到相同大小，比如7x7。

Summary:

Using the produced feature maps from the convolutional layer, we infer regions proposal using a Region Proposal Network rather than relying on an external system. Once we have those proposals, the remaining procedure is the same as Fast-RCNN(forward to ROI layer, classify using SVM and predict the bounding box).

 The trick part is how to train the whole model as we have multiple tasks that need to be addressed:

 1) The region proposal network should decide for each region if it contains an object or not

 2) It needs to produce the bounding box coordinates

 3) The entire model should classify the objects to categories

 4) Again, predict the bounding box offsets










Comments

Post a Comment

Popular posts from this blog

Reading Very Deep VAE

VDVAE:  a hierarchical VAE, generate samples quickly and outperform the PixelCNN in log-likelihood on all the natural image benchmarks. In theory, VAEs can actually represent autoregressive models. VAEs can learn first generate global features at low resolution, then fill in local details in parallel at higher resolutions. Many types of generative models have flourished in recent years, including likelihood-based generative models, which include autoregressive models, VAEs, and invertible flows. Their objective, the negative log-likelihood, is equivalent to the KL divergence between the data distribution and the model distribution.  1) provide theoretical justification for why greater depth could improve VAE performance 2) introduce an architecture capable of scaling past 70 layers 3) verify that depth, independent of model capacity, improves log-likelihood, and allows VAEs to outperform the PixelCNN on all benchmarks 4) uses fewer parameters, generates sam...
Read more

Python Notes

zip: returns a list of tuples, where the i-th tuple contains the i-th element from each of the argument sequences or iterables. a = [1, 2, 3] b = ['a', 'b', 'c'] for i, j in zip(a, b):      print(i,j) =>1 a =>2 b =>3 c Python __repr__() function returns the object representation. It could be any valid python expression such as tuple, dictionary, string etc.  id(object): return the "identity" of an object. This is an integer which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value. hasattr() The hasattr() method returns true if an object has the given named attribute and false if it does not. hasattr(object, name) PIL. Image.resize(size, resample = 0) returns a resized copy of this image Parameters:  -size: the requested size in pixels, as a 2-tuple -resample: an optional resampling filter. This can be one of PIL.I...
Read more

Java Learning

Java 注解： Java 语言中的类、方法、变量、参数和包都可以被标注。 Java标注可以通过反射获取标注内容。 内置的注解： Java定义了一套注解，共有7个，3个在java.lang中，剩下的4个在java.lang.annotation中。 @override--检查该方法是否是重写方法。如果发现其父类，或者引用的接口并没有该方法时，会报编译警告。 @Deprecated--标记过时方法。如果使用该方法，会报编译警告。 @SupperessWarning--指示编译器去忽略注解中声明的警告。 @Retention--标识这个注解应该怎么保存，是只在代码中，还是编入class文件中，或者是在运行时间可以通过反射访问。 @Documented--标记这些注解是否包含在用户文档中。 @Target--标记这个注解应该是哪种Java成员。 @Inherited--标记这个注解是继承于哪个注解类。
Read more