arXiv 2610.30340 · cs.CV · 2026-10-21
Foundation4D: Towards a unified perception and reconstruction model for autonomous vehicles
NVIDIA, :, Sérgio Agostinho, Maximilian Apodaca, Emmanuel Attia, Guillem Braso, George Dong, Ilya Fedorov 等 28 人
NVIDIA
一句话要点
Foundation4D提出一个统一模型，融合多摄像头视频实现几何估计、3D检测和语义占位等任务，在多个基准上取得最优性能，并通过联合训练提升检测与几何精度。
Foundation4D unifies perception and reconstruction, achieving state-of-the-art results in depth, detection, and occupancy with co-training benefits.
要点
- 自动驾驶中多任务感知（几何估计、3D检测、语义占位）存在信息割裂问题，Foundation4D通过共享骨干网络和时空注意力机制实现统一建模。
- 模型采用因子化时空注意力结构，结合视频推理中的位姿图优化，有效融合多摄像头时序观测并提升轨迹与物体跟踪精度。
- 在Waymo、nuScenes和OpenOccupancy上，Foundation4D在深度误差、检测mAP和占位mIoU等指标上均达到最优，验证了其综合性能优势。
- 联合训练检测与几何任务显著提升检测准确率和相机到车身的位移估计精度，表明多任务协同具有明确的性能增益。
English overview
Foundation4D presents a unified feed-forward model for autonomous driving perception and reconstruction, leveraging factorized spatiotemporal attention to jointly estimate geometry, 3D detection, semantic occupancy, and Gaussian surfaces from multi-camera videos. It incorporates pose graph optimization for trajectory refinement and object tracking. The model achieves state-of-the-art performance on depth, detection, and occupancy benchmarks through task co-training, demonstrating mutual benefits between perception tasks.
只根据 arXiv 官方摘要，由开源模型整理，经数字/专名自动核对（必须在摘要中出现），以论文原文为准。
阅读 arXiv 原文 PDF Hugging Face 讨论
主题与学科
同主题：视觉理解与 3D
- Spectral-Spatial Interpretation ▲753
- VisionHOPE: Visual Backbones as Self-Modifying Learning Systems ▲296
- EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience ▲189
- Holi-World: Building Holistic World Intelligence from Human-Ego Experience ▲172
- DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations ▲161
同主题：评测与数据
- StreamEval: Evaluating Interactive Streaming Video Generation Models 2026-10-20 · ▲649
- Has GPT-6-Astra Achieved Embodied Intelligence? 2026-10-15 · ▲421
- BizAgentBench: Unveiling Hidden Challenges in Deploying Long-Horizon Agents for Real-World Business 2026-10-14 · ▲388
- RealCompanion: Benchmarking Human Understanding from Reasoning over Longitudinal Real-World Conversations 2026-10-01 · ▲287
- FDE-Gym: The Last Mile of AI Delivery 2026-10-15 · ▲270
同机构：NVIDIA
- Joints, Camera, Action! Co-Generating 3D Motion and Video for Arbitrary Rigs 2026-10-20
- Long-WAM: Scaling the Context of World-Action Models 2026-10-07
- NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter Scale 2026-10-06
- LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches 2026-10-05
- Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents 2026-09-30
数据说明
元数据来自 arXiv API（已核实）；点赞来自 Hugging Face 每日论文；引用来自 Semantic Scholar。最近刷新 2026-10-21。
要点由本站根据摘要自写，不转载摘要原文，请阅读原文。