Multi-Modal Foundation · ADSKAILab
WaLa RGB4 1B
A 1-billion-parameter diffusion model that generates 3D shapes from four-view RGB images in 2 to 4 seconds.
What WaLa RGB4 1B is
WaLa RGB4 1B is a 3D generative model developed by researchers at ADSKAILab. It addresses the challenge of reconstructing detailed geometry and complex 3D structures directly from multi-view colour photographs. By generating full 3D representations from four standard RGB perspectives, it offers a rapid path from visual observations to volumetric digital models.
The model forms part of the Wavelet Latent Diffusion (WaLa) framework, combining a compact wavelet-based latent space with a large transformer architecture. It was developed for theoretical exploration and academic research in fields such as engineering and materials science, enabling computational scientists to reconstruct shapes and explore multi-view visual conditioning without manual 3D design pipelines.
How it works
WaLa RGB4 1B employs a modified U-ViT (U-shaped Vision Transformer) architecture operating over a compact wavelet-tree representation. 3D training shapes are initially converted into a truncated signed distance function (TSDF) at 256³ resolution, which is then decomposed using a discrete wavelet transform into coarse and fine components. Conditioning images from four viewpoints are each passed individually through a DINOv2 vision encoder to extract feature vectors, which are concatenated together to form the global conditioning sequence for the diffusion model.
Training was conducted on NVIDIA H100 GPUs using a corpus of more than 10 million publicly available 3D shapes. This collection incorporates 19 sub-datasets, including ModelNet, ShapeNet, SMLP, Thingi10K, SMAL, COMA, House3D, ABC, Fusion 360, 3D-FUTURE, BuildingNet, DeformingThings4D, FG3D, Toys4K, ABO, Infinigen, Objaverse, and subsets of ObjaverseXL.
Key facts
- Architecture
- Modified U-ViT with DINOv2 visual encoder and wavelet-tree latent space
- Parameter count
- Approximately 956 million (~1B)
- Input format
- Four-view RGB images
- Output format
- 3D shape (derived from a 256³ TSDF wavelet representation)
- Training data
- Over 10 million 3D shapes from 19 public datasets
- Inference speed
- 2 to 4 seconds (typically 2.5 seconds)
- Hardware used for training
- NVIDIA H100 GPUs
What you can use it for
- Multi-view 3D reconstruction
- Generating complete digital 3D shapes from four calibrated or pre-selected RGB viewpoints of an object.
- Geometric shape exploration
- Synthesising and investigating digital geometries and complex structures for engineering and design research.
- Generative diffusion research
- Benchmarking and studying compact wavelet-tree latent representations against alternative 3D neural representations.
Run it here
We list this model but cannot run it yet: no library and no architectures — nothing can load it.
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Rapid inference capable of producing complete 3D geometries in 2 to 4 seconds.
- Trained on a vast collection of over 10 million diverse 3D shapes spanning 19 sub-datasets.
- Uses compact discrete wavelet transforms to balance high geometric detail with structural plausibility.
- Conditioned on multi-view visual inputs via robust DINOv2 feature embeddings.
Watch out for
- Output geometric quality degrades if input images are ambiguous, unclear, or of low resolution.
- Performance may drop when generating shapes belonging to categories underrepresented in the training data.
- Generated meshes cannot be relied upon for real-world structural soundness and must not be used for load-bearing physical objects.
Inputs and outputs
You provide
Image
PNG, JPEG or TIFF. Sent to the model as-is.
You get back
A generated mesh, offered for download — GLB or OBJ.
Licence
Check the licence before commercial use · other
The other licence carries terms worth reading before you depend on it.
Questions
Can I use WaLa RGB4 1B commercially?
Check the licence before commercial use. The other licence carries terms worth reading before you depend on it.
What inputs does the model require?
The model requires four RGB images of an object taken from pre-selected viewpoints that collectively cover the target geometry.
How fast does WaLa RGB4 1B generate a 3D shape?
Generation typically takes around 2.5 seconds, running within an overall window of 2 to 4 seconds.
How does the model represent 3D geometry internally?
It represents objects as truncated signed distance functions (TSDF) at 256³ resolution, decomposed into latent wavelet trees using a discrete wavelet transform.
How well does the model score on standard reconstruction benchmarks?
On the Google Scanned Objects (GSO) benchmark, it achieves an Intersection over Union (IoU) of 0.8500, a Chamfer Distance of 0.00182, and a Light Field Distance of 1260.64.
Can this model be used to design load-bearing components?
No. The model can generate structurally implausible geometries and is explicitly not intended for designing load-bearing parts where failure could cause injury or damage.
How to cite it
Aditya Sanghi, Aliasghar Khani, Pradyumna Reddy, Arianna Rampini, Derek Cheung, Kamal Rahimi Malekshan, Kanika Madan, Hooman Shayani. Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings. arXiv:2411.08017, 2024.
Written from WaLa RGB4 1B's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.