OpenPose TensorFlow
'Openpose' for human pose estimation have been implemented using Tensorflow. It also provides several variants that have made some changes to the network structure for real-time processing on the CPU or low-power embedded devices.
You can even run this on your macbook with descent FPS!
Original Repo(Caffe) : https://github.com/CMU-Perceptual-Computing-Lab/openpose
| CMU's Original Model
on Macbook Pro 15" | Mobilenet Variant on Macbook Pro 15" | Mobilenet Varianton Jetson TX2 | |:---------|:--------------------|:----------------| | | | | | ~0.6 FPS | ~4.2 FPS @ 368x368 | ~10 FPS @ 368x368 | | 2.8GHz Quad-core i7 | 2.8GHz Quad-core i7 | Jetson TX2 Embedded Board |
Implemented features are listed here : features
2018.5.21 Post-processing part is implemented in c++. It is required compiling the part. See: https://github.com/ildoonet/tf-pose-estimation/tree/master/src/pafprocess 2018.2.7 Arguments in run.py script changed. Support dynamic input size.
You need dependencies below.
python3
tensorflow 1.4.1+
opencv3, protobuf, python3-tk
slim
slidingwindow
https://github.com/adamrehn/slidingwindow
I copied from the above git repo to modify few things.
Clone the repo and install 3rd-party libraries.
$ git clone https://www.github.com/ildoonet/tf-openpose$ cd tf-openpose$ pip3 install -r requirements.txt
Build c++ library for post processing. See : https://github.com/ildoonet/tf-pose-estimation/tree/master/tf_pose/pafprocess
$ cd tf_pose/pafprocess $ swig -python -c++ pafprocess.i && python3 setup.py build_ext --inplace
Alternatively, you can install this repo as a shared package using pip.
$ git clone https://www.github.com/ildoonet/tf-openpose$ cd tf-openpose$ python setup.py install
python -c 'import tf_pose; tf_pose.infer(image="./images/p1.jpg")'
I have tried multiple variations of models to find optmized network architecture. Some of them are below and checkpoint files are provided for research purpose.
cmu
the model based VGG pretrained network which described in the original paper.
I converted Weights in Caffe format to use in tensorflow.
dsconv
Same architecture as the cmu version except for the depthwise separable convolution of mobilenet.
I trained it using 'transfer learning', but it provides not-enough speed and accuracy.
mobilenet
368x368 : checkpoint weight download
Based on the mobilenet paper, 12 convolutional layers are used as feature-extraction layers.
To improve on small person, minor modification on the architecture have been made.
Three models were learned according to network size parameters.
mobilenet
mobilenet_fast
mobilenet_accurate
I published models which is not the best ones, but you can test them before you trained a model from the scratch.
Before running demo, you should download graph files. You can deploy this graph on your mobile or other platforms.
cmu (trained in 656x368)
mobilenet_thin (trained in 432x368)
CMU's model graphs are too large for git, so I uploaded them on an external cloud. You should download them if you want to use cmu's original model. Download scripts are provided in the model folder.
$ cd models/graph/cmu $ bash download.sh
| Dataset | Model | Inference Time
Macbook Pro i5 3.1G | Inference Time
Jetson TX2 | |---------|--------------------|----------------:|----------------:| | Coco | cmu | 10.0s @ 368x368 | OOM @ 368x368
5.5s @ 320x240| | Coco | dsconv | 1.10s @ 368x368 | | Coco | mobilenet_accurate | 0.40s @ 368x368 | 0.18s @ 368x368 | | Coco | mobilenet | 0.24s @ 368x368 | 0.10s @ 368x368 | | Coco | mobilenet_fast | 0.16s @ 368x368 | 0.07s @ 368x368 |
You can test the inference feature with a single image.
$ python run.py --model=mobilenet_thin --resize=432x368 --image=./images/p1.jpg
The image flag MUST be relative to the src folder with no "~", i.e:
--image ../../Desktop
Then you will see the screen as below with pafmap, heatmap, result and etc.
$ python run_webcam.py --model=mobilenet_thin --resize=432x368 --camera=0
Then you will see the realtime webcam screen with estimated poses as below. This Realtime Result was recored on macbook pro 13" with 3.1Ghz Dual-Core CPU.
This pose estimator provides simple python classes that you can use in your applications.
See run.py or run_webcam.py as references.
e = TfPoseEstimator(get_graph_path(args.model), target_size=(w, h))humans = e.inference(image)image = TfPoseEstimator.draw_humans(image, humans, imgcopy=False)
See : etcs/ros.md
See : etcs/training.md
[1] https://github.com/CMU-Perceptual-Computing-Lab/openpose
[2] Training Codes : https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation
[3] Custom Caffe by Openpose : https://github.com/CMU-Perceptual-Computing-Lab/caffe_train
[4] Keras Openpose : https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation
[5] Keras Openpose2 : https://github.com/kevinlin311tw/keras-openpose-reproduce
[1] Arxiv Paper : https://arxiv.org/abs/1701.00295
[2] https://github.com/DenisTome/Lifting-from-the-Deep-release
[1] Original Paper : https://arxiv.org/abs/1704.04861
[2] Pretrained model : https://github.com/tensorflow/models/blob/master/slim/nets/mobilenet_v1.md
[1] Tensorpack : https://github.com/ppwwyyxx/tensorpack
[1] Freeze graph : https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py
[2] Optimize graph : https://codelabs.developers.google.com/codelabs/tensorflow-for-poets-2
还没有评论,说两句吧!
热门资源
Keras-ResNeXt
Keras ResNeXt Implementation of ResNeXt models...
seetafaceJNI
项目介绍 基于中科院seetaface2进行封装的JAVA...
spark-corenlp
This package wraps Stanford CoreNLP annotators ...
capsnet-with-caps...
CapsNet with capsule-wise convolution Project ...
inferno-boilerplate
This is a very basic boilerplate example for pe...
智能在线
400-630-6780
聆听.建议反馈
E-mail: support@tusaishared.com