Chenyang Zhu


My name is Chenyang Zhu. I am currently an Associate Professor at School of Computer Science, National University of Defense Technology (NUDT). I am a faculty member of iGrape Lab @ NUDT, which conducts research in the areas of computer graphics and computer vision. The current directions of interest include data-driven shape analysis and modeling, 3D vision and robot perception & navigation, etc.

I was a Ph.D. student in Gruvi Lab, school of Computing Science at Simon Fraser University, under the supervision of Prof. Hao(Richard) Zhang. I earned my Bachelor and Master degree in computer science from National University of Defense Technology (NUDT) in Jun. 2011 and Dec. 2013 respectively.




Shape analysis


3D Vision


Robotic applications

  • Graduate School Funding, National University of Defense Technology. 国防科技大学校科研项目. 2019-2022
  • National Natural Science Foundation of China. 国家自然科学基金青年项目. 2020-2023.
  • Young Elite Scientists Sponsorship Program by CAST. 中国科协青年人才托举工程. 2020-2023
  • Hunan Provincial Science and Technology Department Funding. 湖湘青年英才. 2021-2024


Computational Visual Media

6DOF Pose Estimation of a 3D Rigid Object based on Edge-enhanced Point Pair Features

Chenyi Liu, Fei Chen, Lu Deng, Renjiao Yi, Lintao Zheng, Chenyang Zhu and Kai Xu

The point pair feature (PPF) is widely used for 6D pose estimation. In this paper, we propose an efficient 6D pose estimation method based on the PPF framework.We introduce a well-targeted down-sampling strategy that focuses more on edge area for efficient feature extraction of complex geometry. A pose hypothesis validation approach is proposed to resolve the symmetric ambiguity by calculating edge matching degree. We perform evaluations on two challenging datasets and one real-world collected dataset, demonstrating the superiority of our method on pose estimation of geometrically complex, occluded, symmetrical objects. We further validate our method by applying it to simulated punctures.

CVPR 2022

DisARM: Displacement Aware Relation Module for 3D Detection

Yao Duan, Chenyang Zhu, Yuqing Lan, Renjiao Yi, Xinwang Liu and Kai Xu

The core idea of DisARM is that contextual information is critical to tell the difference between different objects when the instance geometry is incomplete or featureless. We find that relations between proposals provide a good representation to describe the context. Rather than working with all relations, we find that training with relations only between the most representative ones, or anchors, can significantly boost the detection performance.

Computational Visual Media

ARM3D: Attention-based relation module for indoor 3D object detection

Yuqing Lan, Yao Duan, Chenyi Liu, Chenyang Zhu, Yueshan Xiong, Hui Huang and Kai Xu

Relation contexts have been proved to be useful for many challenging vision tasks. In the field of 3D object detection, previous methods have been taking the advantage of context encoding, graph embedding, or explicit relation reasoning to extract relation contexts. However, there exist inevitably redundant relation contexts due to noisy or low-quality proposals. In fact, invalid relation contexts usually indicate underlying scene misunderstanding and ambiguity, which may, on the contrary, reduce the performance in complex scenes...

Science China (Information Sciences)

Learning Practically Feasible Policies for Online 3D Bin Packing

Hang Zhao*, Chenyang Zhu*, Xin Xu, Hui Huang and Kai Xu

This is a follow-up of our AAAI 2021 work on online 3D BPP. In this work, we aim to learn more PRACTICALLY FEASIBLE policies with REAL ROBOT TESTING! To that end, we propose three critical designs: (1) an online analysis of packing stability based on a novel stacking tree which is highly accurate and computationally efficient and hence especially suited for RL training, (2) a decoupled packing policy learning for different dimensions of placement for high-res spatial discretization and hence high packing precision, and (3) a reward function dictating the robot to place items in a far-to-near order and therefore simplifying motion planning of the robotic arm.

SIGGRAPH 2021, ACM Transactions on Graphics

ROSEFusion: Random Optimization for Online Dense Reconstruction under Fast Camera Motion

Jiazhao Zhang, Chenyang Zhu, Lintao Zheng and Kai Xu

Online reconstruction based on RGB-D sequences has thus far been restrained to relatively slow camera motions. Under very fast camera motion, the reconstruction can easily crumble even for the stateof- the-art methods. Fast motion brings two challenges to depth fusion: 1) the high nonlinearity of camera pose optimization due to large inter-frame rotations and 2) the lack of reliably trackable features due to motion blur.We propose to tackle the difficulties of fast-motion camera tracking in the absence of inertial measurements using random optimization, in particular, the Particle Filter Optimization (PFO). To surmount the computation-intensive particle sampling and update in standard PFO, we propose to accelerate the randomized search via updating a particle swarm template (PST).

MMM 2021

Fine-Grained Video Deblurring with Event Camera

Limeng Zhang, Hongguang Zhang, Chenyang Zhu, Shasha Guo, Jihua Chen, Lei Wang

Despite CNN-based deblurring models have shown their superiority on solving motion blurs, how to restore photorealistic images from severe motion blurs remains an ill-posed problem due to the loss of temporal information and textures. In this paper, we propose a deep fine-grained video deblurring pipeline consisting of a deblurring module and a recurrent module to address severe motion blurs. Concatenating the blurry image with event representations at a fine-grained temporal period, our proposed model achieves state-of-the-art performance on both popular GoPro and real blurry datasets captured by DAVIS, and is capable of generating high frame-rate video by applying a tiny shift to event representations in the recurrent module.

AAAI 2021

Online 3D Bin Packing with Constrained Deep Reinforcement Learning

Hang Zhao, Qijin She, Chenyang Zhu, Yin Yang and Kai Xu

We solve a challenging yet practically useful variant of 3D Bin Packing Problem (3D-BPP). In our problem, the agent has limited information about the items to be packed into the bin, and an item must be packed immediately after its arrival without buffering or readjusting. The item's placement also subjects to the constraints of collision avoidance and physical stability.

CVPR 2020

Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation

Jiazhao Zhang*, Chenyang Zhu*, Lintao Zheng and Kai Xu

Online semantic scene segmentation with high speed (12 FPS) and SOTA accuracy (avg. IoU=0.72 measured w.r.t. per-frame ground-truth image labels). We have also submitted our results to the ScanNet benchmark, demonstrating an avg. IoU of 0.63 on the leaderboard. Note, however, the number was obtained by spatially transferring the point-wise labels of our online recontructed point clouds to the pre-reconstructed point clouds of the benchmark scenes...

CVPR 2020, Oral

AdaCoSeg: Adaptive Shape Co-Segmentation with Group Consistency Loss

Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Li Yi, Leonidas J. Guibas and Hao Zhang

We introduce AdaCoSeg, a deep neural network architecture for adaptive co-segmentation of a set of 3D shapes represented as point clouds. Differently from the familiar single-instance segmentation problem, co-segmentation is intrinsically contextual: how a shape is segmented can vary depending on the set it is in. Hence, our network features an adaptive learning module to produce a consistent shape segmentation which adapts to a set.

Pacific Graphics 2019, Computer Graphics Forum

Active Scene Understanding via Online Semantic Reconstruction

Lintao Zheng, Chenyang Zhu, Jiazhao Zhang, Hang Zhao, Hui Huang, Matthias Niessner and Kai Xu

We propose a novel approach to robot-operated active understanding of unknown indoor scenes, based on online RGBD reconstruction with semantic segmentation. In our method, the exploratory robot scanning is both driven by and targeting at the recognition and segmentation of semantic objects from the scene. Our algorithm is built on top of the volumetric depth fusion framework (e.g., KinectFusion) and performs real-time voxel-based semantic labeling over the online reconstructed volume. The robot is guided by an online estimated discrete viewing score field (VSF) parameterized over the 3D space of ...

CVPR 2019

PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation

Fenggen Yu, Kun Liu, Yan Zhang, Chenyang Zhu and Kai Xu

Deep learning approaches to 3D shape segmentation are typically formulated as a multi-class labeling problem. Existing models are trained for a fixed set of labels, which greatly limits their flexibility and adaptivity. We opt for topdown recursive decomposition and develop the first deep learning model for hierarchical segmentation of 3D shapes, based on recursive neural networks. Starting from a full shape represented as a point cloud, our model performs recursive binary decomposition, where the decomposition network at all nodes in the hierarchy share weights. At each node, a node classifier is trained to determine the type (adjacency or symmetry) and stopping criteria of its decomposition ...

SIGGRAPH ASIA 2018, ACM Transactions on Graphics

SCORES: Shape Composition with Recursive Substructure Priors

Chenyang Zhu, Kai Xu, Siddhartha Chaudhuri, Renjiao Yi and Hao Zhang

We introduce SCORES, a recursive neural network for shape composition. Our network takes as input sets of parts from two or more source 3D shapes and a rough initial placement of the parts. It outputs an optimized part structure for the composed shape, leading to high-quality geometry construction. A unique feature of our composition network is that it is not merely learning how to connect parts. Our goal is to produce a coherent and plausible 3D shape, despite large incompatibilities among the input parts. The network may significantly alter the geometry and structure of the input parts ...

ECCV 2018

Faces as Lighting Probes via Unsupervised Deep Highlight Extraction

Renjiao Yi, Chenyang Zhu, Ping Tan and Stephen Lin

We present a method for estimating detailed scene illumination using human faces in a single image. In contrast to previous works that estimate lighting in terms of low-order basis functions or distant point lights, our technique estimates illumination at a higher precision in the form of a non-parametric environment map...

SIGGRAPH 2017, ACM Transactions on Graphics

Deformation-Driven Shape Correspondence via Shape Recognition

Chenyang Zhu, Renjiao Yi, Wallace Lira, Ibraheem Alhashim, Kai Xuand Hao Zhang

Many approaches to shape comparison and recognition start by establishing a shape correspondence. We “turn the table” and show that quality shape correspondences can be obtained by performing many shape recognition tasks. What is more, the method we develop computes a fine-grained, topology-varying part correspondence between two 3D shapes where the core evaluation mechanism only recognizes shapes globally. This is made possible by casting the part correspondence problem in a deformation-driven framework and relying on a data-driven “deformation energy” which rates visual similarity between deformed shapes and models from a shape repository. Our basic premise is that if a correspondence between two chairs (or airplanes, bicycles, etc.) is correct, then a reasonable deformation between the two chairs anchored on ...

SIGGRAPH 2015, ACM Transactions on Graphics

Interaction Context (ICON): Towards a Geometric Functionality Descriptor

Ruizhen Hu, Chenyang Zhu, Oliver van Kaick, Ligang Liu, Ariel Shamir and Hao Zhang

We introduce a contextual descriptor which aims to provide a geometric description of the functionality of a 3D object in the context of a given scene. Differently from previous works, we do not regard functionality as an abstract label or represent it implicitly through an agent. Our descriptor, called interaction context or ICON for short, explicitly represents the geometry of object-to-object interactions...

SIGGRAPH 2014, ACM Transactions on Graphics

Organizing Heterogeneous Scene Collections through Contextual Focal Points

Kai Xu, Rui Ma, Hao Zhang, Chenyang Zhu, Ariel Shamir, Daniel Cohen-Or and Hui Huang

We introduce focal points for characterizing, comparing, and organizing collections of complex and heterogeneous data and apply the concepts and algorithms developed to collections of 3D indoor scenes. We represent each scene by a graph of its constituent objects and define focal points as representative substructures in a scene collection. To organize a heterogeneous scene collection, we cluster the scenes...



School Of Computing Science
National University of Defense Technology
109 Deya Rd.
Kaifu District
Changsha, Hunan. 410073