SHORT BIO
I am a ZJU100 Young Professor at Zhejiang University. My group is recruiting Graduate Students, Post-doctoral Researchers, and Research Assistants for Fall 2023. My research focuses on Interactive Graphics and Computer Vision, especially the algorithms that allow computers to understand and generate graphics and image data. I worked in Huawei Canada for three years where I founded a graphics team in Vancouver before joining Zhejiang University. Prior to that, I was a research assistant professor at UMIACS of the University of Maryland at College Park. I spent two years as a postdoctoral researcher at SFU, Canada, supervised by Prof. Hao (Richard) Zhang (张皓) and Prof. Ping Tan (谭平) from 2015 to 2017. I also spent a couple of months in visiting the graphics team led by Dr. Hans-Peter Seidel in Saarbrucken of Germany around early 2019. I was the first PhD graduate of the Multimedia Lab at SIAT of Chinese Academy of Sciences which was founded by Prof. Xiaoou Tang (汤晓鸥). I grew up, academically, under the supervision of Dr. Jianzhuang Liu (刘健庄), Prof. Hongbo Fu (傅红波), Dr. Changhu Wang (王长虎), and Dr. Lei Zhang (张磊). |
Links to some other close professors
Xin Tong (童老), Shifeng Chen (陈世峰), Baoquan Chen (陈宝权) , Kai Xu (徐凯), Yue Gao (高越), Renjie chen (陈仁杰) , Rynson. Lau, Alla Sheffer, Marie Paule Cani, Alec Jacobson Matthias Zwicker, Edgar Simo-Serra. Links to some academic buddies
Limin Wang (王利民), Xiaojiang Peng (彭小江), Weilin Huang, Zhuowei Cai (蔡卓伟), Qiang Hao (郝强), Heng Yang (杨恒), Tianfan Xue (薛天帆), Zhaoliang Lun, Haibin Huang, Linjie Yang (杨林杰), Huixuan Tang (汤慧璇), Zheng Xu (许正), Xiaoshuai Sun (孙晓帅), Ruofei Du (杜若飞), ZiShun Liu (刘子舜), Kuiyuan Yang (杨奎元), James Wong (王哲), ALI MAHDAVI-AMIRI, Qian Yu (于茜) |
My group at Zhejiang University and Zhejiang Lab is hiring post-docs, graduate students, interns, and research scientists who want to do amazing work in cutting-edge vision & graphics-based AR/VR applications. Please email me directly.
News since 2015
- Feb. 2021: Our another paper "Heterogeneous Hypergraph Variational Autoencoder for Link Prediction" was accepted by TPAMI.
- Nov. 2020: Big News! Our paper "Hypergraph Learning: Methods and Practices" was accepted by TPAMI.
- Oct. 2020: I will serve the senior program committee member of IJCAI, 2021.
- May. 2020: I will serve the program committee member of Pacific Graphics, 2020.
- Apr. 2020: Our paper "Sketch-R2CNN: An Attentive Network for Vector Sketch Recognition" was accepted by TVCG.
- Mar. 2020: Our paper "Image Generation from Freehand Scene Sketches" was accepted as oral presentation in CVPR 2020.
- Feb. 2020: I will serve the program committee member of SIGGRAPH ASIA Course, 2020.
- Feb. 2020: Two papers "Image Generation from Freehand Scene Sketches" and "Universal Physical Camouflage Attacks on Object Detectors" was accepted by CVPR 2020 with high review scores.
- Feb. 2020: Two papers "Hypergraph Label Propagation Network (Oral)" and "attention-based multimodal fusion for semantic scene completion" was accepted by AAAI 2020.
- Aug. 2019: Our Paper “Language-based Colorization of Scene Sketches” was accepted by SIGGRAPH ASIA 2019.
- Mar. 2019: Our paper "ADCrowdNet: An Attention-injective Deformable Convolutional Network for Crowd Understanding" was accepted by CVPR 2019.
- Nov. 2018: Our paper "PVRNet: Point-View Relation Neural Network for 3D Shape Recognition" was accepted by AAAI 2019.
- Oct. 2018: I visited the School of Software of Tsinghua University and gave a talk about my recent work on shape understanding.
- Oct. 2018: I gave a talk "Plane Shape Understanding and Its Applications" at Huawei AI Lab of Toronto Branch.
- Aug. 2018: Our paper "Construction and Fabrication of Reversible Shape Transforms" was accepted by SIGGRAPH ASIA 2018.
- Jun. 2018: I gave a talk “From Object to Scene: An Evolution of 2D Shape Understanding" at the State Key Laboratory of Virtual Reality Technology and Systems of Beihang University and the School of Data and Computer Science of Sun Yat-sen University.
- Jun. 2018: Our scene level sketch dataset paper "SketchyScene: Richly-Annotated Scene Sketches " was accepted by ECCV2018. Our team are now working hard to improve it to be a work of cornerstone values in the sketch community.
- Mar. 2018: I will serve the program committee member of Pacific Graphics, 2018.
- Mar. 2018: Completed the first large scale scene level sketch dataset.
- Dec. 2017: I gave a talk "Learning to Group Discrete Graphical Patterns" at SIGGRAPH ASIA 2017.
- Dec. 2017: I arrived at College Park and worked at GVIL lab together with Dr. Varshney.
- Aug. 2017: Our 2D-shape analysis project was accepted by SIGGRAPH ASIA 2017.
- Jun. 2017: Our image color transfer work was accepted by Pacific Graphics 2017.
- Jun. 2017: Our paper "Multi-modal Feature Fusion for Geographic Image Annotation" was accepted by Pattern Recognition.
- Mar. 2017: I served the program committee for CAD/Graphics, Changsha, China, 2017.
- Oct. 2016: Our paper "Model-driven Sketch Reconstruction with Structure-Oriented Retrieval" was accepted by SIGGRAPH ASIA Technical Brief 2016.
- Aug. 2016: Our paper "Action-Driven 3D Indoor Scene Evolution" was accepted by SIGGRAPH ASIA 2016.
- Jul. 2016: Our paper "Full and Partial Shape Similarity through Sparse Descriptor Reconstruction" was accepted by The Visual Computer.
- Jun. 2016: Our paper "Shape similarity assessment based on partial feature aggregation and ranking lists" was accepted by Pattern Recognition Letters.
- Jun. 2016: Our paper "An example-based approach to 3D man-made object reconstruction from line drawings" was accepted by Pattern Recognition.
- Mar. 2016: Our paper "Legible Compact Calligrams" was conditionally accepted by SIGGRAPH 2016.
- Aug. 18, 2015: My proposal for National Science Foundation for Young Scientists was approved.
- Jun. 24, 2015: I arrived at SFU and started my Post-doc life.
- Jan. 17, 2015: I received my PhD degree.
Editorial Board Service
|
Paper Reviewer
|
Recent Program Committees
- SIGGRAPH Asia Course: 2020
- AAAI: 2019
- NeurIPS: 2020
- BMVC: 2020
- Pacific Graphics (PG): 2018-2020
- International Conference on Geometric Modeling and Processing (GMP): 2018-2020
- Computer Graphics International (CGI): 2018-2020
- International Symposium on Visual Computing (ISVC): 2018-2020
- The International Conference on Computer-Aided Design and Graphics: 2017
- International Conference on Virtual Reality and Visualization: 2017
Selected Projects (See my pages on Google Scholar Citations and DBLP)
“Life is too short to work on a project you’re not incredibly excited about.”— Oren Etzioni
Projects on Plane Shape Understanding

Lei Li, Changqing Zou, Youyi Zheng, Qingkun Su, Hongbo Fu, Chiew-Lan Tai. "Sketch-R2CNN: An Attentive Network for Vector Sketch Recognition", TVCG, 2020.
[Paper | Project]
We propose a novel single branch attentive network architecture RNN-Rasterization CNN (Sketch-R2CNN for short) to fully leverage the dynamics in sketches for recognition. Sketch-R2CNN takes as input only a vector sketch with grouped sequences of points, and uses an RNN for stroke attention estimation in the vector space and a CNN for 2D feature extraction in the pixel space respectively...
[Paper | Project]
We propose a novel single branch attentive network architecture RNN-Rasterization CNN (Sketch-R2CNN for short) to fully leverage the dynamics in sketches for recognition. Sketch-R2CNN takes as input only a vector sketch with grouped sequences of points, and uses an RNN for stroke attention estimation in the vector space and a CNN for 2D feature extraction in the pixel space respectively...
Chengying Gao, Qi Liu, Qi Xu, Limin Wang, Jianzhuang Liu, Changqing Zou* (corresponding author).
"SketchyCOCO: Image Generation from Freehand Scene Sketches", CVPR (oral), 2020. [Paper | Project] We introduce the first method for automatic image generation from scene-level freehand sketches. Our model allows for controllable image generation by specifying the synthesis goal via freehand sketches... |
Changqing Zou, Haoran Mo (joint first author), Chengying Gao (corresponding author), Ruofei Du, Hongbo Fu. "Language-based Colorization of Scene Sketches", SIGGRAPH ASIA, 2019. [Project]
Being natural, touch-less, and fun-embracing, language-based inputs have demonstrated effective for various tasks from image generation to literacy education for children. This paper for the first time presents a language-based system for interactive colorization of scene sketches... |
Changqing Zou, Qian Yu (joint first author), Ruofei Du, Haoran Mo, Yi-Zhe Song, Tao Xiang, Chengyin Gao, Baoquan Chen (corresponding author), and Hao Zhang. "SketchyScene: Richly-Annotated Scene Sketches", ECCV, 2018. [Paper | Supplementary] | Project | Poster]
We contribute the first large-scale dataset of scene sketches, SketchyScene, with the goal of advancing research on sketch understanding at both the object and scene level. The dataset is created through a novel and carefully designed crowdsourcing pipeline, enabling users to efficiently generate large quantities of realistic and diverse scene sketches. SketchyScene contains more than 29,000 scene-level sketches, 7,000+ pairs of scene templates and photos, and 11,000+ object sketches...... |
Shuhua Li, Ali Mahdavi-Amiri, Ruizhen Hu, Han Liu, Changqing Zou, Oliver Van Kaick, Xiuping Liu, Hui Huang, Hao Zhang. "Construction and Fabrication of Reversible Shape Transforms", SIGGRAPH ASIA, 2018. [Paper]
We study a new and elegant instance of geometric dissection of 2D shapes: reversible hinged dissection, which corresponds to a dual transform between two shapes where one of them can be dissected in its interior and then inverted inside-out, with hinges on the shape boundary, to reproduce the other shape, and vice versa.We call such a transform reversible inside-out transform or RIOT...... |
Changqing Zou, Haoran Mo (joint first author), Ruofei Du, Xing Wu, Chengying Gao, Hongbo FU.
"LUCSS: Language-based User-customized Colorization of Scene Sketches", arXiv, 2018. [Paper | project] We introduce LUCSS, a language-based system for interactive colorization of scene sketches, based on their semantic understanding. LUCSS is built upon deep neural networks trained via a large-scale repository of scene sketches and cartoon-style color images with text descriptions..... |
Zhaoliang Lun*, Changqing Zou* (equal contribution), Haibin Huang, Evangelos Kalogerakis, Ping Tan, Marie-Paule Cani, Hao Zhang. "Learning to Group Discrete Graphical Patterns". SIGGRAPH ASIA (ACM Transactions on Graphics, Vol. 36, No. 6), 2017. [Paper | Project | Video ]
We introduce a deep learning approach for grouping discrete patterns common in graphical designs. Our approach is based on a convolutional neural network architecture that learns a grouping measure defined over a pair of pattern elements. Motivated by perceptual grouping principles, the key feature of our network is the encoding of element shape, context, symmetries, and structural arrangements. These element properties are all jointly considered and appropriately weighted...... |
Changqing Zou, Junjie Cao, Warunika Ranaweera, Ibraheem Alhashim, Ping Tan, Alla Sheffer, Hao Zhang, "Legible Compact Calligrams. SIGGRAPH (ACM Transactions on Graphics, Vol. 35, No. 4), 2016. [Paper | Slides ]
A calligram is an arrangement of words or letters that creates a visual image, and a compact calligram fits one word into a 2D shape. We introduce a fully automatic method for the generation of legible compact calligrams which provides a balance between conveying the input shape, legibility, and aesthetics. Our method has three key elements: a path generation step which computes a global layout path suitable for embedding the input word; an alignment step to place the letters so as to achieve feature alignment between letter and shape protrusions...... |
Projects on 3D Reconstruction
Lei Li, Zhe Huang, Changqing Zou*(corresponding author), Chiew-Lan Tai, Rynson W. H. Lau, Hao Zhang, Ping Tan, Hongbo Fu. "Model-driven sketch reconstruction with structure-oriented retrieval", SIGGRAPH Asia Technical Briefs 2016: 28. [Paper |Project |Demo]
We propose an interactive system that aims at lifting a 2D sketch into a 3D sketch with the help of existing models in shape collections. The key idea is to exploit part structure for shape retrieval and sketch reconstruction. We adopt sketch-based shape retrieval and develop a novel matching algorithm which considers structure in addition to traditional shape features. From a list of retrieved models, users select one to serve as a 3D proxy, providing abstract 3D information. Then our reconstruction method transforms the sketch into 3D geometry by back-projection...... |
Rui Ma, Honghua Li, Changqing Zou, Zicheng Liao, Xin Tong, Hao Zhang. "Action-Driven 3D Indoor Scene Evolution", SIGGRAPH ASIA (ACM Transactions on Graphics, Vol. 35, No. 6), 2016.
[Paper | Code | Data | Slides] We introduce a framework for action-driven evolution of 3D indoor scenes, where the goal is to simulate how scenes are altered by human actions, and specifically, by object placements necessitated by the actions. To this end, we develop an action model with each type of action combining information about one or more human poses, one or more object categories, and spatial configurations of objects belonging to these categories...... |
Changqing Zou, Shifeng Chen, Hongbo Fu, Jianzhuang Liu. Progressive 3D Reconstruction of Planar-Faced Manifold Objects with DRF-Based Line Drawing Decomposition, IEEE Trans. Vis. Comput. Graph. 21(2): 252-263, 2015. [Paper]
This paper presents an approach for reconstructing polyhedral objects from single-view line drawings. Our approach separates a complex line drawing representing a manifold object into a series of simpler line drawings, based on the degree of reconstruction freedom (DRF). We then progressively reconstruct a complete 3D model from these simpler line drawings...... |
Changqing Zou, Heng Yang, Jianzhuang Liu. "Separation of Line Drawings Based on Split Faces for 3D Object Reconstruction", CVPR, 692-699. 2014. [Paper].
Reconstructing 3D objects from single line drawings is often desirable in computer vision and graphics applications. If the line drawing of a complex 3D object is decomposed into primitives of simple shape, the object can be easily reconstructed. We propose an effective method to conduct the line drawing separation and turn a complex line drawing into parametric 3D models...... |
Changqing Zou, Tianfan Xue, Xiaojiang Peng, Honghua Li, Baochang Zhang, Ping Tan, Jianzhuang Liu. "An example-based approach to 3D man-made object reconstruction from line drawings", Pattern Recognition 60: 543-553, 2016. [Paper | Data]
3D reconstruction from a single 2D line drawing is an important but challenging problem in computer vision. Existed methods usually fail when line drawings contain large degree of noise named sketch errors. In this paper, we present an example-based approach to reconstructing 3D object, either planar or curved, from a single-view line drawing with sketch errors...... |
Projects on Visual Recognition
Lifeng Huang, Chengying Gao, Yuyin Zhou, Cihang Xie, Alan Yuille, Changqing Zou, Ning Liu. "UPC: Learning Universal Physical Camouflage Attacks on Object Detectors", CVPR, 2020.
[Paper | Project page | Dataset] In this paper, we study physical adversarial attacks on object detectors in the wild. Prior arts on this matter mostly craft instance-dependent perturbations only for rigid and planar objects. To this end, we propose to learn an adversarial pattern to effectively attack all instances belonging to the same object category(e.g., person, car), referred to as Universal Physical Camouflage Attack (UPC)... To fairly evaluate the effectiveness of different physical-world attacks on object detectors, we present the first standardized virtual database, AttackScenes, which simulates the real 3D world in a controllable and reproducible environment... |
Chunyu Xie, Ce Li, Baochang Zhang, Chen Chen, Jungong Han, Changqing Zou, Jianzhuang Liu.
"Memory Attention Networks for Skeleton-based Action Recognition", IJCIA, 2018. [Paper] Skeleton-based action recognition task is entangled with complex spatio-temporal variations of skeleton joints, and remains challenging for Recurrent Neural Networks (RNNs). In this work, we propose a temporal-then-spatial recalibration scheme to alleviate such complex variations, resulting in an end-to-end Memory Attention Networks (MANs) which consist of a Temporal Attention Recalibration Module (TARM) and a Spatio-Temporal Convolution Module (STCM)...... |
Ke Li, Changqing Zou (corresponding author), Shuhui Bu, Yun Liang, Jian Zhang, Minglun Gong. "Multi-modal feature fusion for geographic image annotation", Pattern Recognition 73: 1-14, 2018.
[Paper | Dataset] This paper presents a multi-modal feature fusion based framework to improve the geographic image an- notation. To achieve effective representations of geographic images, the method leverages a low-to-high learning flow for both the deep and shallow modality features. It first extracts low-level features for each input image pixel, such as shallow modality features (SIFT, Color, and LBP) and deep modality fea- tures (CNNs). It then constructs mid-level features for each superpixel from low-level features...... |
Xiaojiang Peng, Changqing Zou, Yu Qiao, Qiang Peng. "Action Recognition with Stacked Fisher Vectors", ECCV, 2014. [Paper]
Representation of video is a vital problem in action recognition. This paper proposes Stacked Fisher Vectors (SFV), a new representation with multi-layer nested Fisher vector encoding, for action recognition. In the first layer, we densely sample large subvolumes from input videos, extract local features, and encode them using Fisher vectors (FVs). The second layer compresses the FVs of subvolumes obtained in previous layer, and then encodes them again with Fisher vectors...... |
Heng Yang, Changqing Zou, Ioannis Patras. Face Sketch Landmarks Localization in the Wild. IEEE Signal Process. Lett. 21(11): 1321-1325, 2014. [Paper]
We propose a method for facial landmarks localization in face sketch images. As recent approaches and the corresponding datasets are designed for ordinary face photos, the performance of such models drop significantly when they are applied on face sketch images. We first propose a scheme to synthesize face sketches from face photos based on random-forests edge detection and local face region enhancement. Then we jointly train a Cascaded Pose Regression based method for facial landmarks localization for both face photos and sketches. We build an evaluation dataset, called Face Sketches in the Wild (FSW)...... |
Projects on Shape Retrieval
Lili Wan, Changqing Zou, Hao Zhang. "Full and partial shape similarity through sparse descriptor reconstruction".The Visual Computer 33(12): 1497-1509, 2017. [Paper | Code].
We introduce a novel approach to measuring similarity between two shapes based on sparse reconstruction of shape descriptors. The main feature of our approach is its applicability in situations where either of the two shapes may have moderate to significant portions of its data missing. Let the two shapes be A and B. Without loss of generality, we characterize A by learning a sparse dictionary from its local descriptors. The similarity between A and B is defined by the error incurred when reconstructing B’s descriptor set using the basis signals from A’s dictionary. Benefits of using sparse dictionary learning and reconstruction are twofold. First, sparse dictionary learning reduces data redundancy and facilitates similarity computations...... |
Changqing Zou, Changhu Wang, Yafei Wen, Lei Zhang, Jianzhuang Liu. "Viewpoint-Aware Representation for Sketch-Based 3D Model Retrieval". IEEE Signal Process. Lett. 21(8): 966-970, 2014. [Paper]
We study the problem of sketch-based 3D model retrieval, and propose a solution powered by a new query-to-model distance metric and a powerful feature descriptor based on the bag-of-features framework. The main idea of the proposed query-to-model distance metric is to represent a query sketch using a compact set of sample views (called basic views) of each model, and to rank the models in ascending order of the representation errors...... |
Changqing Zou, Zhe Huang, Rynson W. H. Lau, Jianzhuang Liu, Hongbo Fu. "Sketch-based Shape Retrieval using Pyramid-of-Parts". ArXiv 2015. [Paper]
We present a multi-scale approach to sketch-based shape retrieval. It is based on a novel multi-scale shape descriptor called Pyramid-of-Parts, which encodes the features and spatial relationship of the semantic parts of query sketches. The same descriptor can also be used to represent 2D projected views of 3D shapes, allowing effective matching of query sketches with 3D shapes across multiple scales. Experimental results show that the proposed method outperforms the state-of-the-art method, whether the sketch segmentation information is obtained manually or automatically by considering each stroke as a semantic part...... |
Zhenzhong Kuang, Zongmin Li, Yujie Liu, Changqing Zou."Shape similarity assessment based on partial feature aggregation and ranking lists". Pattern Recognition Letters 83: 368-378, 2016. [Paper]
We focus on the problem of similarity assessment of isometric 3D shapes, which is of great relevance in improving the effectiveness of retrieval tasks. We first present an effective shape representation technique by proposing a partial aggregation model based on the bag-of-words paradigm. This technique can effectively encode our multiscale local features and has a good discriminatory ability. We then develop a parameter-free distance mapping approach to re-evaluate the similarity results based on intrinsic analysis of a well organized reciprocal k-nearest neighborhood graph....... |
Bo Li, Yijuan Lu, Chunyuan Li, Afzal Godil, Tobias Schreck, Masaki Aono, Martin Burtscher, Qiang Chen, Nihad Karim Chowdhury, Bin Fang, Hongbo Fu, Takahiko Furuya, Haisheng Li, Jianzhuang Liu, Henry Johan, Ryuichi Kosaka, Hitoshi Koyanagi, Ryutarou Ohbuchi, Atsushi Tatsuma, Yajuan Wan, Chaoli Zhang, Changqing Zou. "A comparison of 3D shape retrieval methods based on a large-scale benchmark supporting multimodal queries". Computer Vision and Image Understanding 131: 1-27, 2015. [Paper]
Large-scale 3D shape retrieval has become an important research direction in content-based 3D shape retrieval. To promote this research area, two Shape Retrieval Contest (SHREC) tracks on large scale comprehensive and sketch-based 3D model retrieval have been organized by us in ...... |
Projects on Mesh/Image Processing
Xiuping Liu, Pingping Tao, Junjie Cao, He Chen, Changqing Zou, "Mesh saliency detection via double absorbing Markov chain in feature space". The Visual Computer 32(9): 1121-1132, 2016. [Paper]
We propose a mesh saliency detection approach using absorbing Markov chain. Unlike most of the existing methods based on some center-surround operator, our method employs feature variance to obtain insignificant regions and considers both background and foreground cues. Firstly, we partition an input mesh into a set of segments using N-cuts algorithm and then each segment is over segmented into patches based on Zernike coefficients. Afterwards, some background patches are selected by computing feature variance within the segments...... |
Junjie Cao, He Chen, Jie Zhang, Yujiao Li, Xiuping Liu, Changqing Zou,"Normal estimation via shifted neighborhood for point cloud." J. Computational Applied Mathematics 329: 57-67, 2018. [Paper]
For accurately estimating the normal of a point, the structure of its neighborhood has to be analyzed. All the previous methods use some neighborhood centering at the point, which is prone to be sampled from different surface patches when the point is near sharp features. Then more inaccurate normals or higher computation cost may be unavoidable. To conquer this problem, we present a fast and quality normal estimator based on neighborhood shift. Instead of using the neighborhood centered at the point, we wish to locate a neighborhood containing the point but clear of sharp features, which is usually not centering at the point.Two specific neighborhood shift techniques are designed in view of the complex structure of sharp features and the characteristic of raw point clouds...... |
Chengying Gao, Mengyue Tang, Xiangguo Liang, Zhuo Su, Changqing Zou. "PencilArt: A Chromatic Penciling Style Generation Framework". Comput. Graph. Forum 37(6): 395-409, 2018. [Paper]
Non-photorealistic rendering has been an active area of research for decades whereas few of them concentrate on rendering chromatic penciling style. In this paper, we present a framework named as PencilArt for the chromatic penciling style generation from wild photographs. The structural outline and textured map for composing the chromatic pencil drawing are generated, respectively. First, we take advantage of deep neural network to produce the structural outline with proper intensity variation and conciseness...... |
Dong Wang, Changqing Zou, Guiqing Li, Chengying Gao, Zhuo Su, Ping Tan. "ℒ0 Gradient-Preserving Color Transfer". Comput. Graph. Forum 36(7): 93-103, 2017. [Paper]
We presents a new two-step color transfer method which includes color mapping and detail preservation. To map source colors to target colors, which are from an image or palette, the proposed similarity-preserving color mapping algorithm uses the similarities between pixel color and dominant colors as existing algorithms and emphasizes the similarities between source image pixel colors...... |