南洋理工大学洪方舟-学生论坛.pdf

1、Egocentric 3D Capture and Understanding第一人称视角下的三维捕捉与理解Fangzhou Hong（洪方舟）Advisor:Prof.Ziwei Liu（刘子纬）S-Lab,Nanyang Technological University1.Natural Human Experience and CognitionWhy study egocentric perception?Egocentric Perspective Provides:2.Better Context for InteractionEgocentric Perception Enabl

2、es:1.Personal AI AssistantGaze,AttentionNavigation,Spatial AwarenessObject ManipulationHand-Eye CoordinationSocial InteractionEgoLife,CVPR2025EgoLM,CVPR2025Egocentric Personal AI Assistant EgoLife:Towards Egocentric Life Assistant Jingkang Yang,Shuai Liu,Hongming Guo,Yuhao Dong,Xiamengwei Zhang,Sich

3、eng Zhang,Pengyun Wang,Zitang Zhou,Binzhu Xie,Ziyue Wang,Bei Ouyang,Zhengyu Lin,Marco Cominelli,Zhongang Cai,YuanhanZhang,Peiyuan Zhang,Fangzhou Hong,Joerg Widmer,Francesco Gringoli,Lei Yang,Bo Li,Ziwei LiuCVPR 20251.Natural Human Experience and CognitionWhy study egocentric perception?Egocentric Pe

4、rspective Provides:2.Better Context for InteractionEgocentric Perception Enables:1.Personal AI Assistant2.Embodied AI LearningGaze,AttentionNavigation,Spatial AwarenessObject ManipulationHand-Eye CoordinationSocial InteractionDexCap,EgoMimic,PH2DEgoLife,CVPR2025EgoLM,CVPR2025Holistic Capture from Eg

5、ocentric PerceptionsEgocentric VideoSparse Motion SensorHumanObjectSceneEgocentric Motion LearningEgoLM:Multi-Modal Language Model of Egocentric Motions Fangzhou Hong,Vladimir Guzov,Hyo Jin Kim,Yuting Ye,Richard Newcombe,Ziwei Liu,Lingni MaCVPR 2025(Oral)Egocentric Motion Tracking and Understanding“

6、The person is standing straight as she puts the piece of clothing on the hanger.”“The person turns around then walks out of the bedroom.”Egocentric VideosMotion TrackingMotion UnderstandingSparse Motion SensorsKey Challenge 1Lack of Direct Observation of the WearerSparse Motion S