Skip to content
/ IKOL Public

[AAAI 2023] Official PyTorch implementation of the paper "IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation"

License

Notifications You must be signed in to change notification settings

Juzezhang/IKOL

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

IKOL

Code repository for the paper:

IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation

Juze Zhang, Ye Shi, Yuexin Ma, Lan Xu, Jingyi Yu, Jingya Wang

[Paper] [Project Page]

In AAAI 2023

Installation instructions

# 1. Create a conda virtual environment.
conda create -n IKOL python=3.8 pytorch=1.11 cudatoolkit=11.3 torchvision -c pytorch -y
conda activate IKOL

# 2. Install mmcv-full
pip install openmim
mim install mmcv-full

# 3. Install PyTorch3D
conda install -c fvcore -c iopath -c conda-forge fvcore iopath -y
conda install -c bottler nvidiacub -y
conda install pytorch3d -c pytorch3d -y

# 4. Pull our code
git clone https://github.com/Juzezhang/IKOL.git
cd IKOL

# 5. Install
pip install -e .

Pretrained models and Dataset

  • Download the SMPL model basicModel_neutral_lbs_10_207_0_v1.0.0.pkl from here at common/utils/smplpytorch/smplpytorch/native/models.
  • Download our pretrained model [ Google Drive] to re-implement our results in Tab. 1 of the main paper.
  • Thanks to the great job done by Li et al, we use the dataset formats provided in HybrIK

Evaluation

python ./scripts/validate.py --batch 64 --gpus 0 --world-size 1 --launcher pytorch --rank 0 --dist-url tcp://${HOST}:${PORT} --cfg ${CONFIG} --checkpoint ${CKPT} --flip-test --dataset 3DPW/3DHP --loader_type mmloader/hybrik_loader

e.g. evaluate on 3DPW

CUDA_VISIBLE_DEVICES=0 python ./scripts/validate.py --batch 64 --gpus 0 --world-size 1 --launcher pytorch --rank 0 --dist-url tcp://127.0.0.1:23456 --cfg ./configs/IKOL_resnet34_mmloader.yaml --checkpoint ./checkpoints/pretrained_3dpw_finetune_hybrikloader.pth --flip-test --dataset 3DPW --loader_type hybrikloader

Training

Our approach incorporates two distinct data loader formats, sourced from the HybrIK repository and mmhuman3d. For the fine-tuning process, we have utilized the HybrIK loader. In addition, we train our own model from the stretch with mmloader. Here two implement example. Below are examples of both implementations.

Training with hybrik loader(only finetune):

python ./scripts/train_hybrikloader.py -nThreads 0 --launcher pytorch --rank 0 --dist-url tcp://127.0.0.1:23456 --exp-id train_res34 --cfg ./configs/IKOL_resnet34_hybrikloader_finetune.yaml --seed 123123

Training with mmloader:

python ./scripts/train_mmloader.py -nThreads 0 --launcher pytorch --rank 0 --dist-url tcp://127.0.0.1:23456 --exp-id train_res34 --cfg ./configs/IKOL_resnet34_mmloader.yaml --seed 123123

Acknowledgement

The implementation was heavily built upon HybrIK and mmhuman3d. We thank the authors for their generosity to release code.

Citing

If our code helps your research, please consider citing the following paper:

  @inproceedings{
        zhang2023ikol,
        title={IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation},
        author={Juze Zhang and Ye Shi and Yuexin Ma and Lan Xu and Jingyi Yu and Jingya Wang},
        booktitle={Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
        year={2023},
  }

About

[AAAI 2023] Official PyTorch implementation of the paper "IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiation"

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages