NVIDIA’s latest research, “Diffusion Texture Painting,” introduces a revolutionary technique that leverages 2D generative diffusion models (DMs) for interactive texture painting on 3D meshes. This innovative approach allows artists to use any complex image texture as a brush, enabling the real-time generation of seamless strokes and realistic transitions between different textures. The method significantly enhances the creative capabilities of digital artists by providing greater control and flexibility in texture creation.
1. Stamp-Based Approach:
- Uses an adapted pre-trained diffusion model to inpaint patches in local render space.
- Projects these patches into the texture image, allowing precise control over brush stroke shape and texture orientation.
2. Real-Time Interaction:
- Supports seamless strokes and realistic transitions in real-time, unlike traditional texture synthesis methods.
- Ensures stable brush identity, preventing unwanted variations and preserving the artist’s original intent.
3. 3D Texture Painting:
- Extends capabilities to 3D meshes, generating stamps in the local camera space for seamless integration with existing textures.
- Applicable to UV-mapped surface meshes, making it suitable for a wide range of 3D texturing workflows.
Implemented as an extension in NVIDIA Omniverse, this system leverages optimized inference with a custom TensorRT engine, ensuring efficient and responsive performance. The results from experimental evaluations demonstrate superior performance in texture consistency and diversity compared to existing methods, highlighting the potential of this technique to become a standard tool in digital art and 3D modeling.
With the promise of enhancing creative workflows and expanding the possibilities of digital texture painting, “Diffusion Texture Painting” represents a significant advancement in the field. This innovative method not only empowers artists to create complex, realistic textures in real-time but also paves the way for further integration of generative models into interactive and creative applications.
