Multi-Modal Foundation · ADSKAILab
WaLa MVDream RGB4
Text-to-multiview diffusion model generating four consistent RGB views from text prompts for 3D shape synthesis.
What WaLa MVDream RGB4 is
WaLa MVDream RGB4 is a text-to-multiview generative model developed by ADSKAILab. It adapts the MVDream architecture to synthesise four distinct RGB viewpoints of an object from natural language descriptions. The model serves as an intermediate stage in the Wavelet Latent Diffusion (WaLa) framework for text-conditioned 3D shape generation.
In 3D generative workflows, producing consistent multi-angle views directly from descriptive prompts provides essential geometric conditioning. The four-view RGB images produced by this model are designed to feed downstream 3D reconstruction models, particularly WaLa-RGB4-1B, assisting researchers in engineering and materials science with text-driven 3D concept exploration.
How it works
The model builds on the MVDream architecture and is fine-tuned to generate four-view RGB images from text inputs. During training on the WaLa dataset, captions for 3D objects were prepared using four renderings and two distinct prompts per object.
Initial training captions were produced using the multimodal model InternVL 2.0 and subsequently augmented using LLaMA 3.1 to increase linguistic diversity and descriptive detail. The model was trained using NVIDIA H100 GPUs.
Key facts
- Base architecture
- MVDream
- Model type
- Text-to-multiview diffusion
- Output modality
- Four-view RGB images
- Training hardware
- NVIDIA H100 GPUs
- Associated paper
- arXiv:2411.08017
What you can use it for
- Text-to-Multiview Generation
- Synthesising four perspective-aligned RGB images of an object directly from descriptive text prompts.
- Conditioning for 3D Shape Synthesis
- Supplying multi-view RGB visual inputs to downstream reconstruction models such as WaLa-RGB4-1B to produce 3D shapes.
- Engineering Concept Visualisation
- Generating multi-angle visual representations of conceptual physical shapes and structures from written specifications.
Run it here
We list this model but cannot run it yet: no library and no architectures — nothing can load it.
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Generates four aligned perspective RGB views simultaneously from a single text description.
- Designed to integrate directly with WaLa-RGB4-1B for end-to-end text-to-3D shape generation.
- Trained on enriched multi-view captions generated with InternVL 2.0 and expanded via LLaMA 3.1.
Watch out for
- Output quality directly influences the fidelity of downstream 3D shape synthesis.
- Synthesised images may occasionally fail to represent the input prompt accurately or maintain consistency across all four viewpoints.
- Model performance varies according to the complexity and specificity of the input text prompt.
Inputs and outputs
You provide
Prompt
What to generate.
You get back
The returned image.
Licence
Check the licence before commercial use · other
The other licence carries terms worth reading before you depend on it.
Questions
Can I use WaLa MVDream RGB4 commercially?
Check the licence before commercial use. The other licence carries terms worth reading before you depend on it.
What is the primary function of this model within the WaLa pipeline?
It serves as an intermediate text-to-image generator that translates text descriptions into four-view RGB images, which are then used by WaLa-RGB4-1B to construct 3D shapes.
Does this model output 3D geometry directly?
No, it outputs four-view 2D RGB images rather than 3D meshes or volumetric representations.
How were the training captions constructed?
Captions were generated from four renderings of 3D objects using InternVL 2.0 and two separate prompts, followed by augmentation with LLaMA 3.1 to improve descriptive richness.
What base architecture does this model use?
It is a fine-tuned version of the MVDream architecture trained on NVIDIA H100 GPUs.
How to cite it
Aditya Sanghi, Aliasghar Khani, Pradyumna Reddy, Arianna Rampini, Derek Cheung, Kamal Rahimi Malekshan, Kanika Madan, Hooman Shayani. Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings. arXiv:2411.08017, 2024.
Written from WaLa MVDream RGB4's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.