FROMAT: Tri-Stream Feature Routing for Material Appearance Transfer in Generative Novel View Synthesis

We present Tri-Stream Feature Routing, a lightweight adaptation framework that enables precise appearance control, including textures, materials, and lighting, within 3D-consistent novel view synthesis (NVS) diffusion models. As generative models increasingly produce frame sequences along complex camera trajectories, the ability to manipulate an object's appearance without compromising its underlying structure becomes critical. Existing frame-based diffusion approaches struggle to decouple appearance from identity, often yielding distorted geometries or failing to reflect visual cues from an appearance exemplar. Our method addresses these limitations through a self-attention-based adaptation mechanism that learns to route appearance features from a reference image and identity features from the input object, fusing them via a lightweight optimization. This guides the diffusion model to generate faithful, 3D-consistent images of a target object with the desired appearance. Notably, our method learns this generalized routing mechanism from only a few positive examples and generalizes to novel objects and materials. We demonstrate high-quality visual outputs with complex appearance modifications when integrated with state-of-the-art NVS models such as Stable Virtual Camera and Era3D.

@article{kompanowski2025fromat,
    title={FROMAT: Multiview Material Appearance Transfer via Few-Shot Self-Attention Adaptation},
    author={Hubert Kompanowski and Varun Jampani and Aaryaman Vasishta and Binh-Son Hua},
    journal={arXiv preprint arXiv:2512.09617},
    year={2025},
}
          

This work was conducted with the financial support of the Research Ireland Centre for Research Training in Digitally Enhanced Reality (d-real) under Grant No. 18/CRT/6224. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.

This project is supported by Research Ireland under the Research Ireland Frontiers for the Future Programme, award number 22/FFP-P/11522.