My topics of interest lie in efficient 3D representations,
with a particular focus on compression and acceleration of NeRF and Gaussian Splatting.
I'm always open to research collaborations — feel free to reach out anytime!
TL;DR. Real-time, high-quality 3D Gaussian Splatting for light field displays via subpixel-level rasterization that reuses computation across coherent views.
Light field displays require rendering an interlaced image that encodes many view-dependent observations, introducing substantial computational overhead that makes real-time rendering difficult. While 3D Gaussian Splatting (3DGS) is efficient for single-view rendering, directly extending it to light field displays is expensive, and prior accelerations either suffer from GPU inefficiency under incoherent subpixel layouts or rely on heavy multi-plane intermediates. We propose CoherentRaster, a 3DGS-based light field rendering framework that performs subpixel-level rasterization: Cross-view Coherent Attribute Reuse eliminates redundant computation across neighboring viewpoints, and View-coherent Remapping restores warp-level memory efficiency degraded by the interlaced subpixel layout, together enabling real-time, high-quality light field synthesis on consumer-grade hardware.
Leveraging Learned Image Prior for 3D Gaussian Compression
TL;DR. A learned image prior restores compression artifacts, improving the rate-distortion trade-off of 3D Gaussian compression.
Existing 3D Gaussian Splatting (3DGS) compression methods reduce storage while preserving rendering quality, but the lack of learned priors restricts further advances in the rate-distortion trade-off. We introduce a 3DGS compression framework that leverages the representational capacity of learned image priors to recover compression-induced quality degradation. Built upon initially compressed Gaussians, our restoration network models compression artifacts in image space, using coarse rendering residuals as side information. Supervised by the restored images, the compressed Gaussians are refined into a highly compact representation with enhanced rendering. Compatible with existing Gaussian compression methods, our framework achieves superior rate-distortion performance while requiring substantially less storage.
Locality-aware Gaussian Compression for Fast and High-quality Rendering
TL;DR. A locality-aware 3DGS representation that exploits spatial coherence for up to 96.6× smaller storage and 2.4× faster rendering.
We present LocoGS, a locality-aware 3D Gaussian Splatting (3DGS) framework that exploits the spatial coherence of 3D Gaussians for compact modeling of volumetric scenes. We analyze the local coherence of Gaussian attributes and propose a locality-aware representation that encodes locally-coherent attributes using a neural field with minimal storage. On top of this representation, LocoGS adds dense initialization, an adaptive spherical harmonics bandwidth scheme, and attribute-specific encoding to maximize compression. Our approach outperforms existing compact Gaussian representations in rendering quality while achieving 54.6× to 96.6× compressed storage and 2.1× to 2.4× faster rendering than 3DGS.
We propose Binary Radiance Fields (BiRF), a storage-efficient radiance field representation that encodes local features using binary encoding parameters in a format of either +1 or -1. With a 2D-3D hybrid feature grid design for enhanced compactness, BiRF achieves superior reconstruction performance compared to state-of-the-art efficient radiance field models while using significantly less storage — for example, 32.03 dB on Synthetic-NeRF, 34.48 dB on Synthetic-NSVF, and 28.20 dB on Tanks and Temples, each with only 0.5 MB of storage.
Deep 3D Reconstruction of Synchrotron X-ray Computed Tomography for Intact Lungs
Seungjoo Shin*, Min Woo Kim*, Kyong Hwan Jin, Kwang Moo Yi, Yoshiki Kohmura, Tetsuya Ishikawa, Jung Ho Je, Jaesik Park(*Equal contribution)
TL;DR. A deep-image-prior network reconstructs the 3D alveolar structure of intact mouse lungs from synchrotron X-ray CT, without ground-truth data.
Synchrotron X-rays can be used to obtain highly detailed images of parts of the lung. However, micro-motion artifacts induced by such as cardiac motion impede quantitative visualization of the alveoli in the lungs. This paper proposes a method that applies a neural network for synchrotron X-ray Computed Tomography (CT) data to reconstruct the high-quality 3D structure of alveoli in intact mouse lungs at expiration, without needing ground-truth data. Our approach reconstructs the spatial sequence of CT images by using a deep-image prior with interpolated input latent variables, and in this way significantly enhances the images of alveolar structure compared with the prior art. The approach successfully visualizes 3D alveolar units of intact mouse lungs at expiration and enables us to measure the diameter of the alveoli. We believe that our approach helps to accurately visualize other living organs hampered by micro-motion.
TL;DR. The first large-scale radiance field dataset for 2D/3D perception tasks.
Neural Radiance Fields (NeRFs) enable accurate, photorealistic 3D reconstruction in a differentiable manner, conveying the information of hundreds of high-resolution images in one compact format. Using the Plenoxels variant of NeRF, we create PeRFception, the first large-scale implicit representation dataset for perception tasks, incorporating both object-centric and scene-centric scans for classification and segmentation. It achieves a significant memory compression rate (96.4%) from the original dataset while containing both 2D and 3D information in a unified form. We construct classification and segmentation models that directly take this implicit format as input, and propose a novel augmentation technique to avoid overfitting on image backgrounds.
Open-Source Library
NeRF-Factory: An awesome PyTorch NeRF collection
A library providing easily extensible and usable PyTorch-implementation of representative NeRF models.
