Publications

3x2: 3D Object Part Segmentation by 2D Semantic Correspondences

3x2: 3D Object Part Segmentation by 2D Semantic Correspondences

Anh Thai, Weiyao Wang, Hao Tang, Stefan Stojanov, James M. Rehg, Matt Feiszli
PDF Project
LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning

LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning

Bolin Lai, Xiaoliang Dai, Lawrence Chen, Guan Pang, James M. Rehg, Miao Liu
PDF Project
Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipation

Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipation

Bolin Lai, Fiona Ryan, Wenqi Jia, Miao Liu*, James M. Rehg*
PDF Project
PointInfinity: Resolution-Invariant Point Diffusion Models

PointInfinity: Resolution-Invariant Point Diffusion Models

Zixuan Huang, Justin Johnson, Shoubhik Debnath, James M. Rehg, Chao-Yuan Wu
MAPLM: A Real-World Large-Scale Vision-Language Dataset for Map and Traffic Scene Understanding

MAPLM: A Real-World Large-Scale Vision-Language Dataset for Map and Traffic Scene Understanding

Xu Cao*, Tong Zhou*, Yunsheng Ma*, Wenqian Ye, Can Cui, Kun Tang, Zhipeng Cao, Kaizhao Liang, Ziran Wang, James M. Rehg, Chao Zheng
Project
ZeroShape: Regression-based Zero-shot Shape Reconstruction

ZeroShape: Regression-based Zero-shot Shape Reconstruction

Zixuan Huang*, Stefan Stojanov*, Anh Thai, Varun Jampani, James M. Rehg
PDF Project
The Audio-Visual Conversational Graph: From an Egocentric- Exocentric Perspective

The Audio-Visual Conversational Graph: From an Egocentric- Exocentric Perspective

Wenqi Jia, Miao Liu, Hao Jiang, Ishwarya Ananthabhotla, James M. Rehg, Vamsi Krishna Ithapu, Ruohan Gao
PDF Project
RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Models

RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Models

Ozgur Kara*, Bariscan Kurtkaya*, Hidir Yesiltepe, James M. Rehg, Pinar Yanardag
PDF Project
Modeling Multimodal Social Interactions: New Challenges and Baselines with Densely Aligned Representations

Modeling Multimodal Social Interactions: New Challenges and Baselines with Densely Aligned Representations

Sangmin Lee, Bolin Lai, Fiona Ryan, Bikram Boote, James M. Rehg
PDF Project
LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs

LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs

Yunsheng Ma*, Can Cui*, Xu Cao*, Wenqian Ye, Peiran Liu, Juanwu Lu, Amr Abdelraouf, Rohit Gupta, Kyungtae Han, Aniket Bera, James M. Rehg, Ziran Wang
PDF
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives

Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives

Kristen Grauman et al. (including Bikram Boote, Fiona Ryan, James M. Rehg)
PDF Project
REBAR: Retrieval-Based Reconstruction For Time-series Contrastive Learning

REBAR: Retrieval-Based Reconstruction For Time-series Contrastive Learning

Maxwell A. Xu, Alexander Moreno, Hui Wei, Benjamin M. Marlin, James M. Rehg
PDF
Low-shot Object Learning with Mutual Exclusivity Bias

Low-shot Object Learning with Mutual Exclusivity Bias

Anh Thai, Ahmad Humayun, Stefan Stojanov, Zixuan Huang, Bikram Boote, James M. Rehg
PDF Project
In the Eye of Transformer: Global–Local Correlation for Egocentric Gaze Estimation and Beyond

In the Eye of Transformer: Global–Local Correlation for Egocentric Gaze Estimation and Beyond

Bolin Lai, Miao Liu, Fiona Ryan, James M. Rehg
PDF
Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games

Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games

Bolin Lai, Hongxin Zhang, Miao Liu, Aryan Pariani, Fiona Ryan, Wenqi Jia, Shirley Anugrah Hayati, James M. Rehg, Diyi Yang
PDF Project
ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistency

ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistency

Zixuan Huang, Varun Jampani, Anh Thai, Yuanzhen Li, Stefan Stojanov, James M. Rehg
PDF Project
Egocentric Auditory Attention Localization in Conversations

Egocentric Auditory Attention Localization in Conversations

Fiona Ryan, Hao Jiang, Abhinav Shukla, James M. Rehg, Vamsi Krishna Ithapu
PDF Project
Which way is ‘right’?: Uncovering limitations of Vision-and-Language Navigation models

Which way is ‘right’?: Uncovering limitations of Vision-and-Language Navigation models

Meera Hahn, Amit Raj, James M. Rehg
PDF
Explaining a machine learning decision to physicians via counterfactuals

Explaining a machine learning decision to physicians via counterfactuals

Supriya Nagesh, Nina Mishra, Yonatan Naamad, James M. Rehg, Mehul A Shah, Alexei Wagner
PDF
In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation

In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation

Bolin Lai, Miao Liu, Fiona Ryan, James M. Rehg
PDF Project Poster
Kernel Multimodal Continuous Attention

Kernel Multimodal Continuous Attention

Alexander Moreno, Zhenke Wu, Supriya Nagesh, Walter Dempsey, James M. Rehg
PulseImpute: A Novel Benchmark Task for Pulsative Physiological Signal Imputation

PulseImpute: A Novel Benchmark Task for Pulsative Physiological Signal Imputation

Maxwell A. Xu, Alexander Moreno, Supiya Nagesh, V. Burak Aydemir, David W. Wetter, Santosh Kumar, James M. Rehg
PDF Code Project
Learning Dense Object Descriptors from Multiple Views for Low-shot Category Generalization

Learning Dense Object Descriptors from Multiple Views for Low-shot Category Generalization

Stefan Stojanov, Anh Thai, Zixuan Huang, James M Rehg
PDF Code Video
Discovering Novel Predictors of Minimally Verbal Outcomes in Autism through Computational Modeling

Discovering Novel Predictors of Minimally Verbal Outcomes in Autism through Computational Modeling

Maxwell Xu, James M. Rehg, Agata Rozga, Jena McDaniel, Paul Yoder, Linda R. Watson, Nancy Brady
PDF
Egocentric Activity Recognition and Localization on a 3D Map

Egocentric Activity Recognition and Localization on a 3D Map

Miao Liu, Lingni Ma, Kiran Somasundaram, Yin Li, Kristen Grauman, James M. Rehg, Chao Li
Generative Adversarial Network for Future Hand Segmentation from Egocentric Video

Generative Adversarial Network for Future Hand Segmentation from Egocentric Video

Wenqi Jia, Miao Liu, James M. Rehg
PDF Code Project Poster
Planes vs. Chairs: Category-guided 3D shape learning without any 3D cues

Planes vs. Chairs: Category-guided 3D shape learning without any 3D cues

Zixuan Huang, Stefan Stojanov, Anh Thai, Varun Jampani, James M. Rehg
PDF Code Project Poster
Ego4D: Around the World in 3,000 Hours of Egocentric Video

Ego4D: Around the World in 3,000 Hours of Egocentric Video

Kristen Grauman et al., (including lab members Miao Liu, Fiona Ryan, Wenqi Jia, Audrey Southerland, and James M. Rehg)
PDF Dataset Project
The Surprising Positive Knowledge Transfer in Continual 3D Object Shape Reconstruction

The Surprising Positive Knowledge Transfer in Continual 3D Object Shape Reconstruction

Anh Thai, Stefan Stojanov, Zixuan Huang, James M. Rehg
The Surprising Positive Knowledge Transfer in Continual 3D Object Shape Reconstruction

The Surprising Positive Knowledge Transfer in Continual 3D Object Shape Reconstruction

Anh Thai, Stefan Stojanov, Zixuan Huang, James M. Rehg
PDF Code
mRisk: Continuous Risk Estimation for Smoking Lapse from Noisy Sensor Data with Incomplete and Positive-Only Labels

mRisk: Continuous Risk Estimation for Smoking Lapse from Noisy Sensor Data with Incomplete and Positive-Only Labels

Md Azim Ullah, Soujanya Chatterjee, Christopher P Fagundes, Cho Lam, Inbal Nahum-Shani, James M. Rehg, David W Wetter, Santosh Kumar
No RL, No Simulation: Learning to Navigate without Navigating

No RL, No Simulation: Learning to Navigate without Navigating

Meera Hahn, Devendra Singh Chaplot, Shubham Tulsiani, Mustafa Mukadam, James M. Rehg, Abhinav Gupta
4D Human Body Capture from Egocentric Video via 3D Scene Grounding

4D Human Body Capture from Egocentric Video via 3D Scene Grounding

Miao Liu, Dexin Yang, Yan Zhang, Zhaopeng Cui, James M. Rehg, Siyu Tang
PDF Project
Transformers for prompt-level EMA non-response prediction

Transformers for prompt-level EMA non-response prediction

Supriya Nagesh, Alexander Moreno, Stephanie M. Carpenter, Jamie Yap, Soujanya Chatterjee, Steven Lloyd Lizotte, Neng Wan, Santosh Kumar, Cho Lam, David W. Wetter, Inbal Nahum-Shani, James M. Rehg
PDF
Efficient Learning and Decoding of the Continuous-Time Hidden Markov Model for Disease Progression Modeling

Efficient Learning and Decoding of the Continuous-Time Hidden Markov Model for Disease Progression Modeling

Yu-Ying Liu, Alexander Moreno, Maxwell A. Xu, Shuang Li, Jena C. McDaniel, Nancy C. Brady, Agata Rozga, Fuxin Li, Le Song, James M. Rehg
PDF DOI
Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Bias

Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Bias

Stefan Stojanov, Anh Thai, James M. Rehg
PDF Code Dataset Project
Orthogonal Over-Parameterized Training

Orthogonal Over-Parameterized Training

Weiyang Liu, Rongmei Lin, Zhen Liu, James M. Rehg, Liam Paull, Li Xiong, Le Song, Adrian Weller
Discriminative Appearance Modeling with Multi-track Pooling for Real-time Multi-object Tracking

Discriminative Appearance Modeling with Multi-track Pooling for Real-time Multi-object Tracking

Chanho Kim, Li Fuxin, Mazen Alotaibi, James M. Rehg
Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning

Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning

Keuntaek Lee, Bogdan Vlahov, Jason Gibson, James M. Rehg, Evangelos A Theodorou
3D Reconstruction of Novel Object Shapes from Single Images

3D Reconstruction of Novel Object Shapes from Single Images

Anh Thai, Stefan Stojanov, Vijay Upadhya, James M. Rehg
The mobile assistance for regulating smoking (MARS) micro-randomized trial design protocol

The mobile assistance for regulating smoking (MARS) micro-randomized trial design protocol

Inbal Nahum-Shani, Lindsey N Potter, Cho Y Lam, Jamie Yap, Alexander Moreno, Rebecca Stoffel, Zhenke Wu, Neng Wan, Walter Dempsey, Santosh Kumar, Emre Ertin, Susan A Murphy, James M. Rehg, David W Wetter
In the Eye of the Beholder: Gaze and Actions in First Person Video

In the Eye of the Beholder: Gaze and Actions in First Person Video

Yin Li, Miao Liu, James M. Rehg
PDF
Continuous measurement of attachment behavior: A multimodal view of the strange situation procedure

Continuous measurement of attachment behavior: A multimodal view of the strange situation procedure

Emily B Prince, Arridhana Ciptadi, Yudong Tao, Agata Rozga, Katherine B Martin, James M. Rehg, Daniel S Messinger
Where Are You? Localization from Embodied Dialog

Where Are You? Localization from Embodied Dialog

Meera Hahn, Jacob Krantz, Dhruv Batra, Devi Parikh, James M. Rehg, Stefan Lee, Peter Anderson
PDF Code Dataset Project Video
Attention Distillation for Learning Video Representations

Attention Distillation for Learning Video Representations

Miao Liu, Xin Chen, Yun Zhang, Yin Li, James M. Rehg
PDF Project
Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Egocentric Activity

Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Egocentric Activity

Miao Liu, Siyu Tang, Yin Li, James M Rehg
PDF Project
Tripping through time: Efficient Localization of Activities in Videos

Tripping through time: Efficient Localization of Activities in Videos

Meera Hahn, Asim Kadav, James M. Rehg, Hans Peter Graf
PDF Video
Toys4K 3D Object Dataset

Toys4K 3D Object Dataset

4,000 3D object instances from 105 categories of developmentally plausible objects
Stefan Stojanov, Anh Thai, James M Rehg
Dataset
Georgia Tech Egocentric Activity Datasets

Georgia Tech Egocentric Activity Datasets

Summary Text for GTEA dataset
Yin Li, Alireza Fathi, Zhefan Ye, Miao Liu, James M Rehg
Dataset
Detecting Attended Visual Targets in Video

Detecting Attended Visual Targets in Video

Eunji Chong, Yongxin Wang, Nataniel Ruiz, James M Rehg
PDF
Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning

Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning

Keuntaek Lee, Bogdan Vlahov, Jason Gibson, James M Rehg, Evangelos A Theodorou
PDF
3D Reconstruction of Novel Object Shapes from Single Images

3D Reconstruction of Novel Object Shapes from Single Images

Anh Thai, Stefan Stojanov, Vijay Upadhya, James M. Rehg
PDF Code Project
Locally Weighted Regression Pseudo-Rehearsal for Adaptive Model Predictive Control

Locally Weighted Regression Pseudo-Rehearsal for Adaptive Model Predictive Control

Grady R Williams, Brian Goldfain, Keuntaek Lee, Jason Gibson, James M. Rehg, Evangelos A Theodorou
PDF
Classification of Decompensated Heart Failure from Clinical and Home Ballistocardiography

Classification of Decompensated Heart Failure from Clinical and Home Ballistocardiography

Varol Burak Aydemir, Supriya Nagesh, Md Mobashir Hasan Shandhi, Joanna Fan, Liviu Klein, Mozziyar Etemadi, James Alex Heller, Omer T. Inan, James M. Rehg
PDF DOI
Incremental Object Learning from Contiguous Views

Incremental Object Learning from Contiguous Views

Stefan Stojanov, Samarth Mishra*, Anh Thai*, Nikhil Dhanda, Ahmad Humayun, Chen Yu, Linda B. Smith, James M. Rehg
PDF Project
In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video

In the Eye of Beholder: Joint Learning of Gaze and Actions in First Person Video

Yin Li, Miao Liu, James M. Rehg
PDF
Watching the TV Watchers

Watching the TV Watchers

Yun C. Zhang, James M. Rehg
PDF Video DOI
SyncWISE: Window Induced Shift Estimation for Synchronization of Video and Accelerometry from Wearable Sensors

SyncWISE: Window Induced Shift Estimation for Synchronization of Video and Accelerometry from Wearable Sensors

Yun C. Zhang*, Shibo Zhang*, Miao Liu, Elyse Daly, Samuel Battalio, Santosh Kumar, Bonnie Spring, James M. Rehg, Nabil Alshurafa
PDF Code Project DOI