A team of researchers from the University of Wisconsin-Madison, NVIDIA, the University of Michigan, and Stanford University have developed a new vision-language model (VLM) called Dolphins. It is a conversational driving assistant that can process multimodal inputs to provide informed driving instructions. Dolphins are designed to address the complex driving scenarios faced by autonomous vehicles…

How can the effectiveness of vision transformers be leveraged in diffusion-based generative learning? This paper from NVIDIA introduces a novel model called Diffusion Vision Transformers (DiffiT), which combines a hybrid hierarchical architecture with a U-shaped encoder and decoder. This approach has pushed the state of the art in generative models and offers a solution to…

How can Neural Radiance Fields (NeRFs) be improved to handle scale variations and reduce aliasing artifacts in scene reconstruction? A new research paper from CMU and Meta addresses this issue by proposing PyNeRF (Pyramidal Neural Radiance Fields). It improves neural radiation fields (NeRFs) by training model heads at different spatial grid resolutions, which helps…