CRF-RNN
Live demo: http://crfasrnn.torr.vision
Updates: * Keras/Tensorflow version is now available.
* We now support the latest Caffe future version.
This package contains code for the "CRF-RNN" semantic image segmentation method, published in the ICCV 2015 paper Conditional Random Fields as Recurrent Neural Networks. This paper was initially described in an arXiv tech report. The online demonstration based on this code won the Best Demo Prize at ICCV 2015. Our software is built on top of the Caffe deep learning library. The current version was developed by:
Sadeep Jayasumana, Shuai Zheng, Bernardino Romera Paredes, Anurag Arnab, and Zhizhong Su.
Supervisor: Philip Torr
Our work allows computers to recognize objects in images, what is distinctive about our work is that we also recover the 2D outline of objects. Currently we have trained this model to recognize 20 classes. This software allows you to test our algorithm on your own images have a try and see if you can fool it, if you get some good examples you can send them to us.
Why are we doing this? This work is part of a project to build augmented reality glasses for the partially sighted. Please read about it here: smart-specs.
For demo and more information about CRF-RNN please visit the project website: http://crfasrnn.torr.vision.
If you use this code/model for your research, please cite the following papers:
@inproceedings{crfasrnn_ICCV2015, author = {Shuai Zheng and Sadeep Jayasumana and Bernardino Romera-Paredes and Vibhav Vineet and Zhizhong Su and Dalong Du and Chang Huang and Philip H. S. Torr}, title = {Conditional Random Fields as Recurrent Neural Networks}, booktitle = {International Conference on Computer Vision (ICCV)}, year = {2015} }
@inproceedings{higherordercrf_ECCV2016, author = {Anurag Arnab and Sadeep Jayasumana and Shuai Zheng and Philip H. S. Torr}, title = {Higher Order Conditional Random Fields in Deep Neural Networks}, booktitle = {European Conference on Computer Vision (ECCV)}, year = {2016} } md5-a3e05a23d802cb9d45ac9bef010d4e68 # This is part of FCN, coarse is a blob coming from FCN layer { type: 'Crop' name: 'crop' bottom: 'bigscore' bottom: 'data' top: 'coarse' } # This layer is used to split the output of FCN into two. This is required by CRF-RNN. layer { type: 'Split' name: 'splitting' bottom: 'coarse' top: 'unary' top: 'Q0' } layer { name: "inference1" # Keep the name "inference1" to load the trained parameters from our caffemodel. type: "MultiStageMeanfield" # Type of this layer bottom: "unary" # Unary input from FCN bottom: "Q0" # A copy of the unary input from FCN bottom: "data" # Input image top: "pred" # Output of CRF-RNN param { lr_mult: 10000 # learning rate for W_G } param { lr_mult: 10000 # learning rate for W_B } param { lr_mult: 1000 # learning rate for compatiblity transform matrix } multi_stage_meanfield_param { num_iterations: 10 # Number of iterations for CRF-RNN compatibility_mode: POTTS # Initialize the compatilibity transform matrix with a matrix whose diagonal is -1. threshold: 2 theta_alpha: 160 theta_beta: 3 theta_gamma: 3 spatial_filter_weight: 3 bilateral_filter_weight: 5 } } md5-80567bc1b65993308725191502f62d3a git clone --recursive https://github.com/torrvision/crfasrnn.git md5-415def6e0b9d788c994db698da213dba sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler sudo apt-get install --no-install-recommends libboost-all-dev md5-f8bf5b9a2f09e4b934e5a1aa7a3627e6 sudo apt-get install freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev md5-f4a7e0fc31fc91d53c15ff5cdfa73c6d blacklist amd76x_edac blacklist vga16fb blacklist nouveau blacklist rivafb blacklist nvidiafb blacklist rivatv md5-6cdb2eb45a3f3913a17565ba7d18a713 sudo apt-get remove --purge nvidia* md5-c7a7a59a1b3cb99b81d72cfa4018de2d sudo service lightdm stop chmod +x cuda*.run sudo ./cuda*.run md5-a2c080b8256490936b5a11cab8a25a35 sudo apt-get install libatlas-base-dev md5-ac5aaacf0c76dbb83298ff532cb4fabd make md5-fd6ecce3939a54e45be63074e78f0a84 make matcaffe md5-d8f195fe6edf5ad2f4125ed8bf9482cd make pycaffe md5-72c3ce44ca6aa06997e665e471449c68 sh download_trained_model.sh md5-a373d63083a8817f84eea3396cf26cc4 python crfasrnn_demo.py md5-aaff26a537158fb6fe134d91c0aad303 sh download_trained_model.sh
Atlernatively, you can also get the model by directly clicking the link in matlab-scripts/README.md
.
Load your MATLAB application and run crfrnn_demo.m.
To use your own images, just replace "input.jpg" in the crfrnn_demo.m file.
You can also find a part of our model in MatConvNet.
If you would like to try out the CRF-RNN model we trained, you should keep the layer name as it is ("inference1"), so that the code will correctly load the parameters from the caffemodel. Otherwise, it will reinitialize parameters.
You should find out that the end-to-end trained CRF-RNN model does better than the alternatives. If you set the CRF-RNN layer name to "inference2", you should observe lower performance since the parameters for both CNN and CRF are not jointly optimized.
If you would like to train CRF-RNN on other datasets, please follow the piecewise training described in our paper. In short, you should first train a strong pixel-wise CNN model. After this, you could plug our CRF-RNN layer into it by adding the MultiStageMeanfieldLayer to the prototxt file. You should then be able to train the CNN and CRF-RNN parts jointly end-to-end.
Notice that the current deploy.prototxt file we have provided is tailored for PASCAL VOC Challenge. This dataset contains 21 class labels including background. You should change the num_output in the corresponding layer if you would like to finetune our model for other datasets. Also, the deconvolution layer in current code does not allow initializing the parameters through prototxt. If you change the num_output there, you should manually re-initialize the parameters in the caffemodel file.
See examples/segmentationcrfasrnn for more information.
This could happen if you change layer names in the model definition prototxt, causing the weights not to load correctly. This could also happen if you change the number of outputs in deconvolution layer in the prototxt but not initialize the deconvolution layer properly.
This error usually occurs when you do not place the spatial.par and bilateral.par files in the script path.
We would like to thank martinkersner and MasazI for providing Python training scripts for CRF-RNN.
It is possible to integrate the CRF-RNN code into upstream Caffe. However, due to the change of the crop layer, the caffemodel we provided might require extra training to provide the same accuracy. mtourne Kindly provided a version that merged the code with upstream caffe.
hyenal kindly provided a purely GPU version of CRF-RNN. This would lead to considerably faster training and testing.
Let us know if we have missed any other works from third parties.
For more information about CRF-RNN please vist the project website http://crfasrnn.torr.vision. Contact: crfasrnn@gmail.com
下一篇:YOLO2
还没有评论,说两句吧!
热门资源
Keras-ResNeXt
Keras ResNeXt Implementation of ResNeXt models...
seetafaceJNI
项目介绍 基于中科院seetaface2进行封装的JAVA...
spark-corenlp
This package wraps Stanford CoreNLP annotators ...
capsnet-with-caps...
CapsNet with capsule-wise convolution Project ...
inferno-boilerplate
This is a very basic boilerplate example for pe...
智能在线
400-630-6780
聆听.建议反馈
E-mail: support@tusaishared.com