Code repository for the paper:
IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation
Juze Zhang, Ye Shi, Yuexin Ma, Lan Xu, Jingyi Yu, Jingya Wang
[Paper
]
[Project Page
]
In AAAI 2023
# 1. Create a conda virtual environment.
conda create -n IKOL python=3.8 pytorch=1.11 cudatoolkit=11.3 torchvision -c pytorch -y
conda activate IKOL
# 2. Install mmcv-full
pip install openmim
mim install mmcv-full
# 3. Install PyTorch3D
conda install -c fvcore -c iopath -c conda-forge fvcore iopath -y
conda install -c bottler nvidiacub -y
conda install pytorch3d -c pytorch3d -y
# 4. Pull our code
git clone https://github.com/Juzezhang/IKOL.git
cd IKOL
# 5. Install
pip install -e .
- Download the SMPL model
basicModel_neutral_lbs_10_207_0_v1.0.0.pkl
from here atcommon/utils/smplpytorch/smplpytorch/native/models
. - Download our pretrained model [ Google Drive] to re-implement our results in Tab. 1 of the main paper.
- Thanks to the great job done by Li et al, we use the dataset formats provided in HybrIK
python ./scripts/validate.py --batch 64 --gpus 0 --world-size 1 --launcher pytorch --rank 0 --dist-url tcp://${HOST}:${PORT} --cfg ${CONFIG} --checkpoint ${CKPT} --flip-test --dataset 3DPW/3DHP --loader_type mmloader/hybrik_loader
e.g. evaluate on 3DPW
CUDA_VISIBLE_DEVICES=0 python ./scripts/validate.py --batch 64 --gpus 0 --world-size 1 --launcher pytorch --rank 0 --dist-url tcp://127.0.0.1:23456 --cfg ./configs/IKOL_resnet34_mmloader.yaml --checkpoint ./checkpoints/pretrained_3dpw_finetune_hybrikloader.pth --flip-test --dataset 3DPW --loader_type hybrikloader
Our approach incorporates two distinct data loader formats, sourced from the HybrIK repository and mmhuman3d. For the fine-tuning process, we have utilized the HybrIK loader. In addition, we train our own model from the stretch with mmloader. Here two implement example. Below are examples of both implementations.
Training with hybrik loader(only finetune):
python ./scripts/train_hybrikloader.py -nThreads 0 --launcher pytorch --rank 0 --dist-url tcp://127.0.0.1:23456 --exp-id train_res34 --cfg ./configs/IKOL_resnet34_hybrikloader_finetune.yaml --seed 123123
Training with mmloader:
python ./scripts/train_mmloader.py -nThreads 0 --launcher pytorch --rank 0 --dist-url tcp://127.0.0.1:23456 --exp-id train_res34 --cfg ./configs/IKOL_resnet34_mmloader.yaml --seed 123123
The implementation was heavily built upon HybrIK and mmhuman3d. We thank the authors for their generosity to release code.
If our code helps your research, please consider citing the following paper:
@inproceedings{
zhang2023ikol,
title={IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation},
author={Juze Zhang and Ye Shi and Yuexin Ma and Lan Xu and Jingyi Yu and Jingya Wang},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
year={2023},
}