publications

2024

1. 

   Rethinking Video-Text Understanding: Retrieval from Counterfactually Augmented Data

   Wufei Ma, Kai Li, Zhongshi Jiang, Moustafa Meshry, Qihao Liu, Huiyu Wang, Christian Häne, and Alan Yuille

   In European Conference on Computer Vision , 2024

   Dataset Vision-Lanugage

   Abs Webpage arXiv Data

   We propose a novel task, retrieval from counterfacually augmented data, and a dataset, Feint6K, for video-text understanding.

2. 

   NOVUM: Neural Object Volumes for Robust Object Classification

   Artur Jesslen, Guofeng Zhang , Angtian Wang, Wufei Ma, Alan Yuille, and Adam Kortylewski

   In European Conference on Computer Vision , 2024

   3D Vision

   Abs arXiv

   We introduce NOVUM, a 3D compositional model for 3D-aware image classification.

3. 

   ImageNet3D: Towards General-Purpose Object-Level 3D Understanding

   Wufei Ma, Guanning Zeng, Guofeng Zhang, Qihao Liu , Letian Zhang, Adam Kortylewski , Yaoyao Liu, and Alan Yuille

   arXiv preprint arXiv:2406.09613, 2024

   Dataset 3D Vision

   Abs Webpage arXiv Data Code

   We present ImageNet3D, a large dataset for general-purpose object-level 3D understanding.

4. 

   Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answering

   Xingrui Wang, Wufei Ma , Angtian Wang, Shuo Chen, Adam Kortylewski, and Alan Yuille

   arXiv preprint arXiv:2406.00622, 2024

   3D Vision Vision-Lanugage

   Abs arXiv

   We develop a video question answering dataset and study the dynamics properties of objects.

5. 

   Uncertainty-Aware Deep Video Compression with Ensembles

   Wufei Ma , Jiahao Li , Bin Li, and Yan Lu

   IEEE Transactions on Multimedia, 2024

   Compression

   Abs arXiv Journal

   We propose an uncertainty-aware video compression model that effectively captures the predictive uncertainty with deep ensembles and saves bits by more than 20% when compared to DVC Pro.

6. 

   Generating Images with 3D Annotations Using Diffusion Models

   Wufei Ma^*, Qihao Liu^* , Jiahao Wang^* , Angtian Wang, Xiaoding Yuan , Yi Zhang, Zihao Xiao, Guofeng Zhang , Beijia Lu, Ruxiao Duan, Yongrui Qi, Adam Kortylewski , Yaoyao Liu, and Alan Yuille

   In The Twelfth International Conference on Learning Representations , 2024

   (Spotlight, 5%)

   Dataset 3D Vision

   Abs arXiv Code

   We propose 3D-DST that generates synthetic data with 3D groundtruth by incorporating 3D geomeotry control into diffusion models. With our diverse prompt generation, we effectively improve both in-distribution (ID) and out-of-distribution (OOD) performance for various 2D and 3D vision tasks.

7. 

   3d-aware visual question answering about parts, poses and occlusions

   Xingrui Wang, Wufei Ma , Zhuowan Li, Adam Kortylewski, and Alan Yuille

   Advances in Neural Information Processing Systems, 2024

   Dataset 3D Vision Vision-Lanugage

   Abs arXiv Code

   We introduce Super-CLEVR-3D, a compositional reasoning dataset that contains questions about object parts, their 3D poses, and occlusions. We propose PO3D-VQA, a 3D-aware VQA model that combines probabilistic neural symbolic program execution with 3D generative representations of objects.

8. 

   Neural textured deformable meshes for robust analysis-by-synthesis

   Angtian Wang^*, Wufei Ma^*, Alan Yuille, and Adam Kortylewski

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024

   3D Vision

   Abs arXiv

   We introduce Neural Textured Deformable Meshes (NTDM), which learns a neural mesh model with deformable geometry and enables optimization on both camera parameters and object geometries.

9. 

   Robust Category-Level 3D Pose Estimation from Diffusion-Enhanced Synthetic Data

   Jiahao Yang, Wufei Ma , Angtian Wang, Xiaoding Yuan, Alan Yuille, and Adam Kortylewski

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024

   3D Vision

   Abs arXiv

   We introduce SyntheticP3D, a synthetic dataset for object pose estimation, and CC3D that adapts neural mesh models from synthetic to real data.

2023

1. 

   Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape

   Jiacong Xu , Yi Zhang, Jiawei Peng, Wufei Ma, Artur Jesslen, Pengliang Ji, Qixin Hu , Jiehua Zhang, Qihao Liu , Jiahao Wang, and  others

   In Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023

   Dataset 3D Vision

   Abs arXiv HTML Data

   Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model.

2. 

   OOD-CV-v2: An Extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images

   Bingchen Zhao , Jiahao Wang, Wufei Ma, Artur Jesslen , Siwei Yang, Shaozuo Yu, Oliver Zendel, Christian Theobalt, Alan Yuille, and Adam Kortylewski

   arXiv preprint arXiv:2304.10266, 2023

   Dataset 3D Vision

   Abs arXiv HTML

   We introduce OOD-CV-v2, a benchmark dataset that includes out-of-distribution examples of 10 object categories in terms of pose, shape, texture, context and the weather conditions, and enables benchmarking of models for image classification, object detection, and 3D pose estimation.

3. 

   Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning

   Zhuowan Li , Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan Yuille

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023

   (Highlight)

   Dataset Vision-Lanugage

   Abs arXiv Code

   We introduce Super-CLEVR, where different factors in VQA domain shifts can be isolated in order that their effects can be studied independently. We propose probabilistic NSVQA (P-NSVQA), which extends NSVQA with uncertainty reasoning.

2022

1. 

   Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features

   Wufei Ma , Angtian Wang, Alan Yuille, and Adam Kortylewski

   In European Conference on Computer Vision , 2022

   3D Vision

   Abs arXiv Code

   We introduce a coarse-to-fine optimization strategy that utilizes the neural features to estimate a sparse set of 6D object proposals, which are subsequently refined with gradient-based optimization.

2. 

   OOD-CV: A Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images

   Bingchen Zhao, Shaozuo Yu, Wufei Ma , Mingxin Yu, Shenxiao Mei , Angtian Wang, Ju He, Alan Yuille, and Adam Kortylewski

   In European conference on computer vision , 2022

   (Oral)

   Dataset 3D Vision

   Abs arXiv

   We introduce OOD-CV, a benchmark dataset for diagnosing the robustness of 2D and 3D vision algorithms to individual nuisances in real-world images.

2021

1. 

   Making group decisions from natural language-based preferences

   Farhad Mohsin, Lei Luo, Wufei Ma, Inwon Kang , Zhibing Zhao , Ao Liu, Rohit Vaish, and Lirong Xia

   In Proceedings of the 8th International Workshop on Computational Social Choice (COMSOC) , 2021

   Dataset Preference Learning

   Abs PDF Data Journal

   We propose a framework for making group decisions from natural language-based preferences. Experiments on the real world data confirms the efficacy of our method.

2. 

   Adoption of Image-Driven Machine Learning for Microstructure Characterization and Materials Design: A Perspective

   Arun Baskaran, Elizabeth J Kautz, Aritra Chowdhary, Wufei Ma, Bulent Yener, and Daniel Lewis

   JOM, 2021

   Microstructure

   Abs arXiv Journal

   We first review the application of image-driven machine learning approaches to the field of materials characterization. Then we analyze and discuss the impact of various approaches at each step of the experiments.

2020

1. 

   Image-driven discriminative and generative machine learning algorithms for establishing microstructure–processing relationships

   Wufei Ma, Elizabeth J Kautz, Arun Baskaran, Aritra Chowdhury, Vineet Joshi, Bulent Yener, and Daniel Lewis

   Journal of Applied Physics, 2020

   Microstructure

   Abs arXiv HTML Code Journal

   Characterize 10 different microstructure representations with image texture features and quantitative metrics from image segmentation. For the microstructure generation task, two schemes are considered: 1) generating high-resolution (1024x1024) microstructure images from random noise; and 2) train a style transfer GAN for image generation conditioned on the segmentation label.

2. 

   An image-driven machine learning approach to kinetic modeling of a discontinuous precipitation reaction

   Elizabeth Kautz, Wufei Ma, Saumyadeep Jana, Arun Devaraj, Vineet Joshi, Bülent Yener, and Daniel Lewis

   Materials Characterization, 2020

   Microstructure

   Abs arXiv Code Journal

   Kinetic modeling of a discontinuous precipitation reaction (5 phases) by 1) deep learning with CNN, and 2) performing image segmentation of various microstructure and quantizing the area fractions.