The problem of video understanding and generation scenarios has been addressed by researchers of Tencent AI Lab and The University of Sydney by presenting GPT4Video. This unified multi-model framework supports LLMs with the capability of both video understanding and generation. GPT4Video developed an instruction-following-based approach integrated with the stable diffusion generative model, which effectively and…

Human Activity Recognition (HAR) is a field of study that focuses on developing methods and techniques to automatically identify and classify human activities based on data collected from various sensors. HAR aims to enable machines like smartphones, wearable devices, or smart environments to understand and interpret human activities in real-time. Traditionally, wearable sensor-based and camera-based…

Researchers from the University of Southern California, the University of Washington, Bar-Ilan University, and Google Research introduced DreamSync, which addresses the problem of enhancing alignment and aesthetic appeal in diffusion-based text-to-image (T2I) models without the need for human annotation, model architecture modifications, or reinforcement learning. It achieves this by generating candidate images, evaluating them using…

In generative modeling, diffusion models (DMs) have assumed a pivotal role, facilitating recent progress in producing high-quality picture and video synthesis. Scalability and iterativeness are two of DMs’ main advantages; they enable them to do intricate tasks like picture creation from free-form text cues. Unfortunately, the many sample steps required for the iterative inference process…

Natural picture production is now on par with professional photography, thanks to a notable recent improvement in quality. This advancement is attributable to creating technologies like DALL·E3, SDXL, and Imagen. Key elements driving these developments are using the potent Large Language Model (LLM) as a text encoder, scaling up training datasets, increasing model complexity, better…

High-quality 3D content synthesis is a crucial yet challenging problem for many applications, such as autonomous driving, robotic simulation, gaming, filmmaking, and future VR/AR situations. The topic of 3D geometry generation has seen a surge in research interest from the computer vision and graphics community due to the availability of more and more 3D content…

NeRF represents scenes as continuous 3D volumes. Instead of discrete 3D meshes or point clouds, it defines a function that calculates color and density values for any 3D point within the scene. By training the neural network on multiple scene images captured from different viewpoints, NeRF learns to generate consistent and accurate representations that align…

Finer control over the visual characteristics and notions represented in a produced picture is typically required by artistic users of text-to-image diffusion models, which is presently not achievable. It can be challenging to accurately modify continuous qualities, such as an individual’s age or the intensity of the weather, using simple text prompts. This constraint makes…

Providing a virtual environment that matches the actual world, the recent widespread rise of 3D applications, including metaverse, VR/AR, video games, and physical simulators, has improved human lifestyle and increased productive efficiency. These programs are based on triangle meshes, which stand in for the intricate geometry of actual environments. Most current 3D applications rely on…

Researchers from Shanghai AI Laboratory, Fudan University, Northwestern Polytechnical University, and The Hong Kong University of Science and Technology have collaborated to develop a 3D Gaussian representation-based Simultaneous Localization and Mapping (SLAM) system named GS-SLAM. The goal of the plan is to achieve a balance between accuracy and efficiency. GS-SLAM uses a real-time differentiable splatting…