We propose a method for converting a single RGB-D input image into a 3D photo, i.e., a multi-layer representation for novel view synthesis that contains hallucinated color and depth structures in regions occluded in the original view. We use a Layered Depth Image with explicit pixel connectivity as underlying representation, and present a learning-based inpainting model that iteratively synthesizes new local color-and-depth content into the occluded region in a spatial context-aware manner. The resulting 3D photos can be efficiently rendered with motion parallax using standard graphics engines. We validate the effectiveness of our method on a wide range of challenging everyday scenes and show fewer artifacts when compared with the state-of-the-arts.
3D Photography using Context-aware Layered Depth Inpainting
Meng-Li Shih,
Shih-Yang Su,
Johannes Kopf, and
Jia-Bin Huang
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
and the Python dependencies listed in requirements.txt
conda create -n 3DP python=3.7 anaconda
conda activate 3DP
pip install -r requirements.txt
conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit==10.1.243 -c pytorch
chmod +x download.sh
./download.sh
Please follow the instructions in this section.
This should allow to execute our results.
For more detailed instructions, please refer to DOCUMENTATION.md
.
.jpg
files (e.g., test.jpg) into the image
folder.
image/moon.jpg
python main.py --config argument.yml
depth/moon.npy
, depth/moon.png
depth/moon.png
manually.
depth/moon.png
as input for 3D Photo.
depth_format: '.png'
require_midas: False
save_ply
)
mesh/moon.ply
video/moon_zoom-in.mp4
video/moon_swing.mp4
video/moon_circle.mp4
video/moon_dolly-zoom-in.mp4
DOCUMENTATION.md
and modified argument.yml
.This work is licensed under MIT License. See LICENSE for details.
If you find our code/models useful, please consider citing our paper:
@inproceedings{Shih3DP20,
author = {Shih, Meng-Li and Su, Shih-Yang and Kopf, Johannes and Huang, Jia-Bin},
title = {3D Photography using Context-aware Layered Depth Inpainting},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2020}
}
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。