LiteReality is a new pipeline that transforms RGB-D indoor environment scans into compact, realistic, and interactive 3D virtual replicas. LiteReality not only reconstructs scenes that are visually similar to reality, but also supports key features essential to graphics pipelines such as object individuality, joint motion, high-quality physically-based rendering materials, and physics-based interactions. It reconstructs the scene by searching for the most visually similar 3D artist-created models from a curated asset database, focusing on performing scene understanding using a structured scene graph and parsing the results into consistent 3D layouts and objects. It then reconstructs the scene by restoring high-quality, spatially-varying materials through a material painting module to enhance realism. Finally, it integrates the reconstructed scene into a simulation engine with basic physics properties to enable interactive behavior. The resulting scene is compact, editable, and fully compatible with standard graphics pipelines, making it ideal for AR/VR, gaming, robotics, and digital twin applications. LiteReality also introduces a learning-free object detection module that achieves state-of-the-art similarity performance on the Scan2CAD benchmark, and a powerful material painting module that can transfer the appearance of any style of image to 3D assets, even with severe alignment errors, occlusions, and poor lighting. We demonstrate the effectiveness of LiteReality on both real-world scans and public datasets.
