Multi-Modal Foundation · ADSKAILab
WaLa DM4 1B
Generates 3D shapes from four-view depth maps in four seconds using wavelet latent diffusion.
What WaLa DM4 1B is
WaLa DM4 1B is a 3D generative model developed by researchers at Autodesk AI Lab. It forms part of the Wavelet Latent Diffusion (WaLa) framework, which investigates large-scale 3D synthesis. The model is specifically configured to reconstruct 3D shapes from four-view depth map inputs, targeting applications in scientific research, engineering, and materials science.
Generating detailed 3D structures from sparse sensor or depth data is often computationally intensive or limited in geometric resolution. WaLa DM4 1B addresses this by employing a compact wavelet-based latent space, enabling the rapid generation of 3D geometry with fine details in approximately four seconds.
Trained on more than 10 million publicly available 3D shapes, the model provides an avenue for academic exploration of generative geometry, spatial representation, and depth-driven reconstruction.
How it works
The model architecture is based on a modified U-shaped Vision Transformer (U-ViT) operating alongside a DINO v2 visual encoder. To process inputs, four depth maps taken from fixed viewpoints covering the full object are individually passed through the DINO v2 encoder to extract latent vectors. These multi-view feature sequences are concatenated to form the complete conditioning vector.
For 3D data representation, training meshes are converted into truncated signed distance functions (TSDF) at a resolution of 256³. The TSDF volumes are decomposed via a discrete wavelet transform into a compact wavelet-tree representation. The U-ViT generative model is trained to synthesize these wavelet latent encodings conditioned on the depth representations, running on NVIDIA H100 GPUs.
Key facts
- Model type
- 3D generative model (depth-map-to-3D)
- Architecture
- Modified U-ViT with DINO v2 conditioning encoder
- Parameter count
- Approximately 956 million
- Input format
- Four-view depth maps
- Target representation
- Discrete wavelet transform of 256³ TSDF
- Generation time
- Approximately 4 seconds
- Training data size
- Over 10 million 3D shapes from 19 public sub-datasets
- Hardware used
- NVIDIA H100 GPUs
What you can use it for
- Multi-view depth-to-3D reconstruction
- Reconstruct full 3D geometric shapes from calibrated four-angle depth measurements.
- Theoretical 3D generative modeling
- Investigate wavelet-tree encodings and large-scale latent diffusion architectures in academic and scientific research.
- Rapid geometry prototyping
- Generate candidate 3D surface representations within four seconds for research simulations and visual analysis.
Run it here
We list this model but cannot run it yet: no library and no architectures — nothing can load it.
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Rapidly generates 3D geometries within approximately four seconds.
- Combines a billion-parameter modified U-ViT with compact discrete wavelet transforms to retain fine geometric details.
- Trained on an aggregated collection of over 10 million 3D objects spanning 19 public datasets.
- Achieves high fidelity on the Google Scanned Objects benchmark with an IoU of 0.87884 and a Chamfer Distance of 0.00164.
Watch out for
- Output geometric quality depends directly on the accuracy and resolution of the four input depth maps.
- Performance may degrade on object categories or styling poorly represented across the training datasets.
- May produce ambiguous or implausible shapes that must not be relied upon for real-world structural soundness or load-bearing applications.
Inputs and outputs
You provide
Image
PNG, JPEG or TIFF. Sent to the model as-is.
You get back
A generated mesh, offered for download — GLB or OBJ.
Licence
Check the licence before commercial use · other
The other licence carries terms worth reading before you depend on it.
Questions
Can I use WaLa DM4 1B commercially?
Check the licence before commercial use. The other licence carries terms worth reading before you depend on it.
What inputs are required to run WaLa DM4 1B?
The model requires four depth maps captured from predetermined viewpoints that offer comprehensive visual coverage of the object.
How are the input depth maps processed?
Each depth map is passed individually through a DINO v2 encoder to produce latent vectors, which are then concatenated into a single conditioning vector for the diffusion model.
What internal 3D format does the model synthesize?
It generates a wavelet-tree representation derived from a 256³ resolution truncated signed distance function (TSDF) using a discrete wavelet transform.
Can the model be used to design load-bearing components?
No. The model is intended strictly for scientific research and theoretical exploration; generated shapes are not verified for physical structural soundness.
How to cite it
Aditya Sanghi, Aliasghar Khani, Pradyumna Reddy, Arianna Rampini, Derek Cheung, Kamal Rahimi Malekshan, Kanika Madan, Hooman Shayani. Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings. arXiv:2411.08017, 2024.
Written from WaLa DM4 1B's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.