publications

2026

1. 

   LychSim: A Controllable and Interactive Simulation Framework for Vision Research

   Wufei Ma , Chloe Wang , Siyi Chen, Jiawei Peng , Patrick Li, and Alan Yuille

   arXiv preprint, 2026

   Dataset 3D Vision Vision-Lanugage

   Abs Webpage arXiv Code

   While self-supervised pretraining has reduced vision systems’ reliance on synthetic data, simulation remains an indispensable tool for closed-loop optimization and rigorous out-of-distribution (OOD) evaluation. However, modern simulation platforms often present steep technical barriers, requiring extensive expertise in computer graphics and game development. In this work, we present LychSim, a highly controllable and interactive simulation framework built upon Unreal Engine 5 to bridge this gap. LychSim is built around three key designs: (1) a streamlined Python API that abstracts away underlying engine complexities; (2) a procedural data pipeline capable of generating diverse, high-fidelity environments with varying out-of-distribution (OOD) visual challenges, paired with rich 2D and 3D ground truths; and (3) a native integration of the Model Context Protocol (MCP) that transforms the simulator into a dynamic, closed-loop playground for reasoning agentic LLMs.

2. 

   Pursuing Minimal Sufficiency in Spatial Reasoning

   Yejie Guo, Yunzhong Hou, Wufei Ma, Meng Tang , and Ming-Hsuan Yang

   In The Fourteenth International Conference on Learning Representations , 2026

   3D Vision Vision-Lanugage

   Abs arXiv

   We introduce MSSR (Minimal Sufficient Spatial Reasoner), a dual-agent framework that implements this principle. A Perception Agent programmatically queries 3D scenes using a versatile perception toolbox to extract sufficient information, including a novel SOG (Situated Orientation Grounding) module that robustly extracts language-grounded directions. A Reasoning Agent then iteratively refines this information to pursue minimality, pruning redundant details and requesting missing ones in a closed loop until the MSS is curated.

3. 

   4D-Animal: Freely Reconstructing Animatable 3D Animals from Videos

   Shanshan Zhong, Jiawei Peng, Zehan Zheng, Zhongzhan Huang, Wufei Ma, Guofeng Zhang, Qihao Liu, Alan Yuille, and Jieneng Chen

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2026

   3D Vision

   Abs arXiv Code

   We propose 4D-Animal, a novel framework that reconstructs animatable 3D animals from videos without requiring sparse keypoint annotations.

2025

1. 

   Generative Adversarial Reasoner: Enhancing LLM Reasoning with Adversarial Reinforcement Learning

   Qihao Liu, Luoxin Ye, Wufei Ma, Yu-Cheng Chou, and Alan Yuille

   arXiv preprint, 2025

   LLM

   Abs arXiv

   We introduce Generative Adversarial Reasoner, an on-policy joint training framework designed to enhance reasoning by co-evolving an LLM reasoner and an LLM-based discriminator through adversarial reinforcement learning.

2. 

   SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning

   Wufei Ma, Yu-Cheng Chou, Qihao Liu , Xingrui Wang, Celso M de Melo, Jianwen Xie, and Alan Yuille

   In Advances in Neural Information Processing Systems , 2025

   3D Vision Vision-Lanugage

   Abs Webpage arXiv

   We introduce SpatialReasoner, a novel large vision-language model (LVLM) that address 3D spatial reasoning with explicit 3D representations shared between stages – 3D perception, computation, and reasoning. Explicit 3D representations provide a coherent interface that supports advanced 3D spatial reasoning and enable us to study the factual errors made by LVLMs.

3. 

   3DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark

   Wufei Ma , Haoyu Chen, Guofeng Zhang, Jieneng Chen, Celso M de Melo, and Alan Yuille

   In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , Oct 2025

   3D Vision Vision-Lanugage

   Abs Webpage arXiv

   We present 3DSRBench, a comprehensive 3D spatial reasoning benchmark.

4. 

   DINeMo: Learning Neural Mesh Models with no 3D Annotations

   Weijie Guo, Guofeng Zhang, Wufei Ma, and Alan Yuille

   3rd Workshop on Compositional 3D Vision at CVPR, Oct 2025

   3D Vision

   Abs Webpage arXiv

   We present DINeMo, a novel neural mesh model that is trained with no 3D annotations by leveraging pseudo-correspondence obtained from large visual foundation models.

5. 

   SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Models

   Wufei Ma, Luoxin Ye, Celso M de Melo, Jieneng Chen, and Alan Yuille

   In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , Oct 2025

   (Highlight, 3.0%)

   3D Vision Vision-Lanugage

   Abs Webpage arXiv

   We systematically study the impact of 3D-informed data, architecture, and training setups and present SpatialLLM, an LMM with advanced 3D spatial reasoning abilities.

6. 

   Spatial457: A Diagnostic Benchmark for Comprehensive Spatial Reasoning of Large Multimodal Models

   Xingrui Wang, Wufei Ma , Tiezheng Zhang, Celso M de Melo, Jieneng Chen, and Alan Yuille

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , Oct 2025

   (Highlight, 3.0%)

   Dataset 3D Vision Vision-Lanugage

   Abs Webpage arXiv

   We present PulseCheck457, a scalable and unbiased synthetic dataset designed with 4 key capability for spatial reasoning.

7. 

   Compositional 4D Dynamic Scenes Understanding with Physics Priors for Video Question Answering

   Xingrui Wang, Wufei Ma , Angtian Wang , Shuo Chen, Adam Kortylewski, and Alan Yuille

   In The Thirteenth International Conference on Learning Representations , Oct 2025

   Dataset 3D Vision Vision-Lanugage

   Abs arXiv

   We develop a video question answering dataset and study the dynamics properties of objects.

2024

1. 

   ImageNet3D: Towards General-Purpose Object-Level 3D Understanding

   Wufei Ma, Guofeng Zhang, Qihao Liu, Guanning Zeng, Adam Kortylewski , Yaoyao Liu, and Alan Yuille

   In Advances in Neural Information Processing Systems , Oct 2024

   Dataset 3D Vision

   Abs Webpage arXiv Data Code

   We present ImageNet3D, a large dataset for general-purpose object-level 3D understanding.

2. 

   Rethinking Video-Text Understanding: Retrieval from Counterfactually Augmented Data

   Wufei Ma, Kai Li, Zhongshi Jiang, Moustafa Meshry, Qihao Liu, Huiyu Wang, Christian Häne, and Alan Yuille

   In European Conference on Computer Vision , Oct 2024

   (Strong Double Blind)

   Dataset Vision-Lanugage

   Abs Webpage arXiv Data

   We propose a novel task, retrieval from counterfacually augmented data, and a dataset, Feint6K, for video-text understanding.

3. 

   NOVUM: Neural Object Volumes for Robust Object Classification

   Artur Jesslen, Guofeng Zhang , Angtian Wang, Wufei Ma, Alan Yuille, and Adam Kortylewski

   In European Conference on Computer Vision , Oct 2024

   3D Vision

   Abs arXiv

   We introduce NOVUM, a 3D compositional model for 3D-aware image classification.

4. 

   Uncertainty-Aware Deep Video Compression with Ensembles

   Wufei Ma , Jiahao Li , Bin Li, and Yan Lu

   IEEE Transactions on Multimedia, Oct 2024

   Compression

   Abs arXiv Journal

   We propose an uncertainty-aware video compression model that effectively captures the predictive uncertainty with deep ensembles and saves bits by more than 20% when compared to DVC Pro.

5. 

   Generating Images with 3D Annotations Using Diffusion Models

   Wufei Ma^*, Qihao Liu^* , Jiahao Wang^* , Angtian Wang, Xiaoding Yuan , Yi Zhang, Zihao Xiao, Guofeng Zhang , Beijia Lu, Ruxiao Duan, Yongrui Qi, Adam Kortylewski , Yaoyao Liu, and Alan Yuille

   In The Twelfth International Conference on Learning Representations , Oct 2024

   (Spotlight, 5%)

   Dataset 3D Vision

   Abs arXiv Code

   We propose 3D-DST that generates synthetic data with 3D groundtruth by incorporating 3D geomeotry control into diffusion models. With our diverse prompt generation, we effectively improve both in-distribution (ID) and out-of-distribution (OOD) performance for various 2D and 3D vision tasks.

6. 

   Neural textured deformable meshes for robust analysis-by-synthesis

   Angtian Wang^*, Wufei Ma^*, Alan Yuille, and Adam Kortylewski

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , Oct 2024

   3D Vision

   Abs arXiv

   We introduce Neural Textured Deformable Meshes (NTDM), which learns a neural mesh model with deformable geometry and enables optimization on both camera parameters and object geometries.

7. 

   Robust Category-Level 3D Pose Estimation from Diffusion-Enhanced Synthetic Data

   Jiahao Yang, Wufei Ma , Angtian Wang, Xiaoding Yuan, Alan Yuille, and Adam Kortylewski

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , Oct 2024

   3D Vision

   Abs arXiv

   We introduce SyntheticP3D, a synthetic dataset for object pose estimation, and CC3D that adapts neural mesh models from synthetic to real data.

8. 

   OOD-CV-v2: An Extended Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images

   Bingchen Zhao , Jiahao Wang, Wufei Ma, Artur Jesslen , Siwei Yang, Shaozuo Yu, Oliver Zendel, Christian Theobalt, Alan Yuille, and Adam Kortylewski

   IEEE Transactions on Pattern Analysis and Machine Intelligence, Oct 2024

   Dataset 3D Vision

   Abs arXiv HTML

   We introduce OOD-CV-v2, a benchmark dataset that includes out-of-distribution examples of 10 object categories in terms of pose, shape, texture, context and the weather conditions, and enables benchmarking of models for image classification, object detection, and 3D pose estimation.

2023

1. 

   3d-aware visual question answering about parts, poses and occlusions

   Xingrui Wang, Wufei Ma , Zhuowan Li, Adam Kortylewski, and Alan Yuille

   In Advances in Neural Information Processing Systems , Oct 2023

   Dataset 3D Vision Vision-Lanugage

   Abs arXiv Code

   We introduce Super-CLEVR-3D, a compositional reasoning dataset that contains questions about object parts, their 3D poses, and occlusions. We propose PO3D-VQA, a 3D-aware VQA model that combines probabilistic neural symbolic program execution with 3D generative representations of objects.

2. 

   Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape

   Jiacong Xu , Yi Zhang, Jiawei Peng, Wufei Ma, Artur Jesslen, Pengliang Ji, Qixin Hu , Jiehua Zhang, Qihao Liu , Jiahao Wang, and  others

   In Proceedings of the IEEE/CVF International Conference on Computer Vision , Oct 2023

   Dataset 3D Vision

   Abs arXiv HTML Data

   Animal3D consists of 3379 images collected from 40 mammal species, high-quality annotations of 26 keypoints, and importantly the pose and shape parameters of the SMAL model.

3. 

   Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning

   Zhuowan Li , Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan Yuille

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , Oct 2023

   (Highlight)

   Dataset Vision-Lanugage

   Abs arXiv Code

   We introduce Super-CLEVR, where different factors in VQA domain shifts can be isolated in order that their effects can be studied independently. We propose probabilistic NSVQA (P-NSVQA), which extends NSVQA with uncertainty reasoning.

2022

1. 

   Robust Category-Level 6D Pose Estimation with Coarse-to-Fine Rendering of Neural Features

   Wufei Ma , Angtian Wang, Alan Yuille, and Adam Kortylewski

   In European Conference on Computer Vision , Oct 2022

   3D Vision

   Abs arXiv Code

   We introduce a coarse-to-fine optimization strategy that utilizes the neural features to estimate a sparse set of 6D object proposals, which are subsequently refined with gradient-based optimization.

2. 

   OOD-CV: A Benchmark for Robustness to Out-of-Distribution Shifts of Individual Nuisances in Natural Images

   Bingchen Zhao, Shaozuo Yu, Wufei Ma , Mingxin Yu, Shenxiao Mei , Angtian Wang, Ju He, Alan Yuille, and Adam Kortylewski

   In European conference on computer vision , Oct 2022

   (Oral)

   Dataset 3D Vision

   Abs arXiv

   We introduce OOD-CV, a benchmark dataset for diagnosing the robustness of 2D and 3D vision algorithms to individual nuisances in real-world images.

2021

1. 

   Making group decisions from natural language-based preferences

   Farhad Mohsin, Lei Luo, Wufei Ma, Inwon Kang , Zhibing Zhao , Ao Liu, Rohit Vaish, and Lirong Xia

   In Proceedings of the 8th International Workshop on Computational Social Choice (COMSOC) , Oct 2021

   Dataset Preference Learning

   Abs PDF Data Journal

   We propose a framework for making group decisions from natural language-based preferences. Experiments on the real world data confirms the efficacy of our method.

2. 

   Adoption of Image-Driven Machine Learning for Microstructure Characterization and Materials Design: A Perspective

   Arun Baskaran, Elizabeth J Kautz, Aritra Chowdhary, Wufei Ma, Bulent Yener, and Daniel Lewis

   JOM, Oct 2021

   Microstructure

   Abs arXiv Journal

   We first review the application of image-driven machine learning approaches to the field of materials characterization. Then we analyze and discuss the impact of various approaches at each step of the experiments.

2020

1. 

   Image-driven discriminative and generative machine learning algorithms for establishing microstructure–processing relationships

   Wufei Ma, Elizabeth J Kautz, Arun Baskaran, Aritra Chowdhury, Vineet Joshi, Bulent Yener, and Daniel Lewis

   Journal of Applied Physics, Oct 2020

   Microstructure

   Abs arXiv HTML Code Journal

   Characterize 10 different microstructure representations with image texture features and quantitative metrics from image segmentation. For the microstructure generation task, two schemes are considered: 1) generating high-resolution (1024x1024) microstructure images from random noise; and 2) train a style transfer GAN for image generation conditioned on the segmentation label.

2. 

   An image-driven machine learning approach to kinetic modeling of a discontinuous precipitation reaction

   Elizabeth Kautz, Wufei Ma, Saumyadeep Jana, Arun Devaraj, Vineet Joshi, Bülent Yener, and Daniel Lewis

   Materials Characterization, Oct 2020

   Microstructure

   Abs arXiv Code Journal

   Kinetic modeling of a discontinuous precipitation reaction (5 phases) by 1) deep learning with CNN, and 2) performing image segmentation of various microstructure and quantizing the area fractions.