Kategorie: KI Trends

With AI at its tipping point, AI-enabled computer vision is being used to address the world’s most challenging problems in nearly every industry.

At GTC, a global conference for the era of AI and the metaverse running through Thursday, March 23, NVIDIA announced technology updates poised to drive the next wave of vision AI adoption. These include NVIDIA TAO Toolkit 5.0 for creating customized, production-ready AI models; expansions to the NVIDIA DeepStream software development kit for developing vision AI applications and services; and early access to Metropolis Microservices for powerful, cloud-native building blocks that accelerate vision AI.

Exploding Adoption and Ecosystem

More than 1,000 companies are using NVIDIA Metropolis developer tools to solve Internet of Things (IoT), sensor processing and operational challenges with vision AI — and the rate of adoption is quickening. The tools have now been downloaded over 1 million times by those looking to build vision AI applications.

PepsiCo is optimizing its operations with NVIDIA Metropolis to improve throughput, reduce downtime and minimize energy consumption.

The convenience-food and beverages giant is developing AI-powered digital twins of its distribution centers using the NVIDIA Omniverse platform to visualize how different setups in its facilities will impact operational efficiency before implementing them in the real world. PepsiCo is also using advanced machine vision technology, powered by the NVIDIA AI platform and GPUs, to improve efficiency and accuracy in its distribution process.

Siemens, a technology leader in industrial automation and digitalization, is adding next-level perception into its edge-based applications through NVIDIA Metropolis. With millions of sensors across factories, Siemens uses NVIDIA Metropolis — a key application framework for edge AI — to connect entire fleets of robots and IoT devices and bring AI into its industrial environments.

Automaker BMW Group is using computer vision technologies based on lidar and cameras — built by Seoul Robotics and powered by the NVIDIA Jetson edge AI platform — at its manufacturing facility in Munich to automate the movement of cars. This automation has resulted in significant time and cost savings, as well as employee safety improvements.

Making World-Class Vision AI Accessible to Any Developer on Any Device

As AI is made accessible to developers of any skill level, the next phase of AI adoption will arrive.

GTC is showcasing major expansions of Metropolis workflows, which put some of the latest AI capabilities and research into the hands of developers through NVIDIA TAO Toolkit, Metropolis Microservices and the DeepStream SDK, as well as the NVIDIA Isaac Sim synthetic data generation tool and robotics simulation applications.

NVIDIA TAO Toolkit is a low-code AI framework that supercharges vision AI model development for practically any developer, in any service, on any device. TAO 5.0 is filled with new features, including vision transformer pretrained AI models, the ability to deploy models on any platform with standard ONNX export, automatic hyperparameter tuning with AutoML, and AI-assisted data annotation.

STMicroelectronics, a global leader in embedded microcontrollers, integrates TAO into its STM32Cube AI developer workflow. TAO has enabled the company to run sophisticated AI in widespread IoT and edge use cases that STM32 microcontrollers power within their compute and memory budget.

The NVIDIA DeepStream SDK has emerged as a powerful tool for developers looking to create vision AI applications across a wide range of industries. With its latest update, a new graph execution runtime (GXF) allows developers to expand beyond the open-source GStreamer multimedia framework. DeepStream’s addition of GXF is a game-changer for users seeking to build applications that require tight execution control, advanced scheduling and critical thread management. This feature unlocks a host of new applications, including those in industrial quality control, robotics and autonomous machines.

Adding perception to physical spaces often requires applying vision AI to numerous cameras covering multiple regions.

Challenges in computer vision include monitoring the flow of packaged goods across a warehouse or analyzing individual customer flow across a large retail space. Metropolis Microservices make these sophisticated vision AI tasks easy to integrate and deploy into users’ applications.

Leading IT services company Infosys is using NVIDIA Metropolis to supercharge its vision AI application development and deployment. The NVIDIA TAO low-code training framework and pretrained models help Infosys reduce AI training efforts. Metropolis Microservices, along with the DeepStream SDK, optimize the company’s vision processing pipeline throughput and cut overall solution costs. Infosys can also generate troves of synthetic data with the NVIDIA Omniverse Replicator SDK to easily train AI models with new stock keeping units and packaging.

Latest Metropolis Features

Tap into the latest in NVIDIA vision AI technologies:

Read the TAO 5.0 blog. Try TAO Toolkit on NVIDIA LaunchPad.
GXF runtime, now part of NVIDIA DeepStream, unlocks new use cases that require tight scheduling control. Try it on NVIDIA LaunchPad.
Sign up for early access to Metropolis Microservices, a suite of cloud-native microservices and reference applications that accelerate efforts to create API-driven solutions for the edge and the cloud.
Learn more about NVIDIA Metropolis — through corporate blogs, technical blogs and case studies — to see how vision AI is transforming the world.

Register free to attend GTC, and watch these sessions to learn how to accelerate vision AI application development and understand its many use cases.

Watch NVIDIA founder and CEO Jensen Huang’s GTC keynote in replay:

Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology improves creative workflows. We’re also deep diving on new GeForce RTX 40 Series GPU features, technologies and resources, and how they dramatically accelerate content creation.

Powerful AI technologies are revolutionizing 3D content creation — whether by enlivening realistic characters that show emotion or turning simple texts into imagery.

The brightest minds, artists and creators are gathering at NVIDIA GTC, a free, global conference on AI and the metaverse, taking place online through Thursday, March 23.

NVIDIA founder and CEO Jensen Huang’s GTC keynote announced a slew of advancements set to ease creators’ workflows, including using generative AI with the Omniverse Audio2Face app.

NVIDIA Omniverse, a platform for creating and operating metaverse applications, further expands with an updated Unreal Engine Connector, open-beta Unity Connector and new SimReady 3D assets.

New NVIDIA RTX GPUs, powered by the Ada Lovelace architecture, are fueling next-generation laptop and desktop workstations to meet the demands of the AI, design and the industrial metaverse.

The March NVIDIA Studio Driver adds support for the popular RTX Video Super Resolution feature, now available for GeForce RTX 40 and 30 Series GPUs.

And this week In the NVIDIA Studio, the Adobe Substance 3D art and development team explores the process of collaborating to create the animated short End of Summer using Omniverse USD Composer (formerly known as Create). 

Omniverse Overdrive

Specialized generative AI tools can boost creator productivity, even for users who don’t have extensive technical skills. Generative AI brings creative ideas to life, producing high-quality, highly iterative experiences — all in a fraction of the time and cost of traditional asset development.

The Omniverse Audio2Face AI-powered app allows 3D artists to efficiently animate secondary characters,  generating realistic facial animations with just an audio file — replacing what is often a tedious, manual process.

The latest release delivers significant upgrades in quality, usability and performance including a new headless mode and a REST API — enabling game developers and other creators to run the app and process numerous audio files from multiple users in the data center.

A new Omniverse Connector developed by NVIDIA for Unity workflows is available in open beta. Unity scenes can be added directly onto Omniverse Nucleus servers with access to platform features: the DeepSearch tool, thumbnails, bookmarks and more. Unidirectional live-sync workflows enable real-time updates.

With the Unreal Engine Connector’s latest release, Omniverse users can now use Unreal Engine’s USD import utilities to add skeletal mesh blend shape importing, and Python USD bindings to access stages on Omniverse Nucleus. This release also delivers improvements in import, export and live workflows, as well as updated software development kits.

In addition, over 1,000 new SimReady assets are available for creators. SimReady assets are built to real-world scale with accurate mass, physical materials and center of gravity for use within Omniverse PhysX for the most photorealistic visuals and accurate movements.

March Studio Driver Brings Superfly Super Resolution

Over 90% of online videos consumed by NVIDIA RTX GPU owners are 1080p resolution or lower, often resulting in upscaling that further degrades the picture despite the hardware being able to handle more.

The solution: RTX Video Super Resolution. The new feature, available on GeForce RTX 30 and 40 Series GPUs, uses AI to improve the quality of any video streamed through Google Chrome and Microsoft Edge browsers.

Click the image to see the differences between bicubic upscaling (left) and RTX Video Super Resolution.

This improves video sharpness and clarity. Users can watch online content in its native resolution, even on high-resolution displays.

RTX Video Super Resolution is now available in the March Studio Driver, which can be downloaded today.

New NVIDIA RTX GPUs Power Professional Creators

Six new professional-grade NVIDIA RTX GPUs — based on the Ada Lovelace architecture — enable creators to meet the demands of their most complex workloads using laptops and desktops.

The NVIDIA RTX 5000, RTX 4000, RTX 3500, RTX 3000 and RTX 2000 Ada Generation laptop GPUs deliver up to 2x the performance compared with the previous generation. The NVIDIA RTX 4000 Small Form Factor (SFF) Ada Generation desktop GPU features new RT Cores, Tensor Cores and CUDA cores with up to 20GB of graphics memory.

These include the latest NVIDIA Max-Q and RTX technologies and are backed by the NVIDIA Studio platform with RTX optimizations in over 110 creative apps, NVIDIA RTX Enterprise Drivers for the highest levels of stability and performance, and exclusive AI-powered NVIDIA tools: Omniverse, Canvas and Broadcast.

Professionals using these laptop GPUs can run advanced technologies like DLSS 3 to increase frame rates by up to 4x compared to the previous generation, and Omniverse Enterprise for real-time collaboration and simulation.

Next-generation mobile workstations featuring NVIDIA RTX GPUs will be available starting this month.

Creative Boosts at GTC

Experience GTC for more inspiring content, expert-led sessions and a must-see keynote to accelerate your life’s creative work.
Catch these sessions on Omniverse, AI and 3D workflows — live or on demand:
Fireside Chat With OpenAI Founder Ilya Sutskever and Jensen Huang: AI Today and Vision of the Future [S52092]
How Generative AI Is Transforming the Creative Process: Fireside Chat With Adobe’s Scott Belsky and NVIDIA’s Bryan Catanzaro [S52090]
Generative AI Demystified [S52089]
3D by AI: How Generative AI Will Make Building Virtual Worlds Easier [S52163]
Custom World Building With AI Avatars: The Little Martians Sci-Fi Project [S51360]
AI-Powered, Real-Time, Markerless: The New Era of Motion Capture [S51845]
3D and Beyond: How 3D Artists Can Build a Side Hustle in the Metaverse [SE52117]
NVIDIA Omniverse User Group [SE52047]
Accelerate the Virtual Production Pipeline to Produce an Award-Winning Sci-Fi Short Film [S51496]

As part of the Watch ‘n Learn Giveaway with valued partner 80LV, GTC attendees who register for any Omniverse for creators session — or watch on-demand before March 30 — have a chance to win a powerful GeForce RTX 4080 GPU. Simply fill out this form and tag #GTC23 and @NVIDIAOmniverse with the name of the session.

Search the GTC session catalog and check out the “Media and Entertainment” and “Omniverse” topics for additional creator-focused sessions.

A Father-Daughter Journey Back Home

The short animation End of Summer, created by the Substance 3D art and development team at Adobe, may evoke a surprising amount of heart. That was the team’s intent.

“We loved the idea of allowing the artwork to invoke an emotion in the viewer, letting them develop their own version of a story they felt was unfolding before their eyes,” said team member Wes McDermott.

“End of Summer” design boards.

End of Summer, a nod to stop-motion animation studios such as Laika, began as an internal Adobe Substance 3D project aimed at accomplishing two goals.

First, to encourage a relatively new group of artists to work together as a team by leaning into a creative endeavor. And second, to test their pipeline feature set for the potential of the Universal Scene Description (USD) framework.

Early concept work for “End of Summer.”

The group divided the task of creating assets across the most popular 3D apps, including Adobe Substance 3D Modeler, Autodesk 3ds Max, Autodesk Maya, Blender and Maxon’s Cinema 4D. Their GeForce RTX GPUs unlocked AI denoising in the viewport for fast, interactive rendering and GPU-accelerated filters to speed up and simplify material creation.

“We quickly started to see the power of using USD as not only an export format but also a way to build assets,” McDermott said. “USD enables artists on the team to use whatever 3D app they felt most comfortable with.”

The Adobe team relied heavily on the Substance 3D asset library of materials, models and lights to create their studio environment. All textures were applied in Substance 3D Painter, where RTX-accelerated light and ambient occlusion baking optimized assets in mere moments.

Then, they imported all models into Omniverse USD Composer, where the team simultaneously refined and assembled assets.

“This was also during the pandemic, and we were all quarantined in our homes,” McDermott said. “Having a project we could work on together as a team helped us to communicate and be creative.”

Accelerate scene composition, and assemble, simulate and render 3D scenes in real time in Omniverse USD Composer.

Lastly, the artists imported the scene into Unreal Engine as a stage for lighting and rendering.

Final scene edits in Unreal Engine.

McDermott stressed the importance of hardware in his team’s workflows. “The bakers in Substance Painter are GPU accelerated and benefit greatly from NVIDIA RTX GPUs,” he said. “We were also heavily working on Unreal Engine and reliant on real-time rendering.”

For more on this workflow, check out the GTC session, 3D Art Goes Multiplayer: Behind the Scenes of Adobe Substance’s ‘End of Summer’ Project With Omniverse. Registration is free.

Adobe Substance 3D team lead and artist Wes McDermott.

Check out McDermott’s portfolio on Instagram.

Follow NVIDIA Studio on Instagram, Twitter and Facebook. Access tutorials on the Studio YouTube channel and get updates directly in your inbox by subscribing to the Studio newsletter. Learn more about Omniverse on Instagram, Medium, Twitter and YouTube for additional resources and inspiration. Check out the Omniverse forums, and join our Discord server and Twitch channel to chat with the community.

The automotive industry is undergoing a digital revolution, driven by breakthroughs in accelerated computing, AI and the industrial metaverse.

Automakers are digitalizing every phase of the product lifecycle — including concept and styling, design and engineering, software and electronics, smart factories, autonomous driving and retail — using the NVIDIA Omniverse platform and AI.

Based on the Universal Scene Description (USD) framework, Omniverse transforms complex 3D workflows, allowing teams to connect and customize 3D pipelines and simulate large-scale, physically accurate virtual worlds. By taking the automotive product workflow into the virtual world, automakers can bypass traditional bottlenecks to save critical time and reduce cost.

Bringing Ideas to Life

Designing new vehicle models — and refreshing current ones — is a collaborative process that requires review and alignment of even the tiniest details.

By refining concepts in Omniverse, designers can visualize every facet of a car’s interior and exterior in the full context of the broader vehicle. Global teams can iterate quickly with real-time, physically based, photorealistic rendering. For example, they can collaborate to design the cockpit’s critical components, such as digital instrument clusters and infotainment systems, which must strike a balance of communicating information while minimizing distraction.

Omniverse enables designers to flexibly lay out the cabin and cockpit onscreen user experience along with the vehicle’s physical interior to ensure a harmonious look and feel.

With this next-generation design process, automakers can catch flaws early and make real-time improvements, reducing the number of physical prototypes to test and validate.

Virtual Validation

Once the design is complete, developers can use Omniverse to kick the tires on their new concepts.

Perfecting the interior is necessary for customer experience as well as safety.

Developers can take these in-cabin designs for a spin in the virtual world, collaborating and sharing designs for efficient refinement and validation.

Digitalization is also transforming the way automakers approach vehicle engineering. Teams can test different materials and components in a virtual environment to further reduce physical prototyping. For example, engineers can use computational fluid dynamics to refine aerodynamics and perform virtual crash simulations for safer vehicle designs.

Continuous Improvement

The coming generation of vehicles are highly advanced computers on wheels, packed with complex, centralized electronic systems and software for enhanced safety, intelligence and security.

Typically, vehicle functions are controlled by dozens of electronic control units distributed throughout a vehicle. By centralizing computing into core domains, automakers can replace many components and simplify what has been an incredibly complex supply chain.

With a digital representation of this entire architecture, automakers can simulate and test the vehicle software, and then provide over-the-air updates for continuous improvement throughout the car’s lifespan — from remote diagnostics to autonomous-driving capabilities to subscriptions for entertainment and other services.

Digital-First Production

Vehicle production is a colossal undertaking that requires thousands of parts and workers moving in sync. Any supply chain or production issues can lead to costly delays.

With Omniverse, automakers can develop and operate complex, AI-enabled virtual environments for factory and warehouse design. These physically based, precision-timed digital twins are the key to unlocking operational efficiencies with predictive analysis and process automation.

Factory planners can access the digital twin of the factory to review and improve the plant as needed. Every change can be quickly evaluated and validated in the virtual world, then implemented in the real world to ensure maximum efficiency and optimal ergonomics for factory workers.

Additionally, automakers can synchronize plant locations anywhere in the world for scalable design and iteration.

Autonomous Vehicle Proving Grounds

On top of enhancing traditional product development and manufacturing, Omniverse offers a complete toolchain for developing and validating automated and autonomous-driving systems.

NVIDIA DRIVE Sim is a physically based simulation platform, built on NVIDIA Omniverse, designed for fast and efficient autonomous-vehicle testing and validation at scale. It is time-accurate and supports the complete development toolchain, so developers can run simulation at the component level or for the entire system.

With DRIVE Sim, developers can repeatedly simulate routine driving scenarios, as well as rare and hazardous conditions that are too risky to test in the real world. Additionally, real-world driving recordings can be turned into reactive simulation scenarios using the platform’s Neural Reconstruction Engine.

Automakers can also fine-tune their advanced driver-assistance and autonomous-vehicle systems for New Car Assessment Program (NCAP) regulations, which evaluate the safety performance of new cars based on several crash tests and safety features.

The DRIVE Sim NCAP tool provides high-fidelity NCAP test protocols in simulation, so automakers can efficiently perform dedicated development and validation at scale.

The ability to drive in physically based virtual environments can significantly accelerate the autonomous-vehicle development process, overcoming data collection and scenario diversity hurdles that occur in real-world testing.

Omniverse’s generative AI reconstructs previously driven routes into 3D so past experiences can be reenacted or modified.

Try Before You Buy

The end customer benefits from digitalization, too.

Immersive technologies in Omniverse — including 3D visualization, augmented reality (AR) and virtual reality (VR) streamed using NVIDIA CloudXR — deliver consumers a more engaging experience, allowing them to explore features before making a purchase.

Prospective buyers can customize their vehicle in a car configurator — choosing colors, interior materials, trim levels and more — without being limited by the physical inventory of a dealership. They can then check out the car from every angle using 3D visualization. And with AR and VR, they can view and virtually test drive a car from anywhere.

The benefits of digitalization extend beyond the automotive industry. With Omniverse, any enterprise can reimagine their workflows to increase efficiency, productivity and speed, revolutionizing the way they do business. Omniverse is the digital-to-physical operating system to realize industrial digitalization.

Learn more about the latest in AI and the metaverse by watching NVIDIA founder and CEO Jensen Huang’s GTC keynote address:

Transportation industry trailblazers are propelling their next-generation vehicles by building on NVIDIA DRIVE end-to-end solutions, which span the cloud to the car.

The world’s best-selling new energy vehicle (NEV) brand BYD announced at NVIDIA GTC that it’s using the NVIDIA DRIVE Orin centralized compute platform to power an even wider range of vehicles within its mainstream Dynasty and Ocean series of NEVs.

This comes hot on the heels of BYD’s recent announcement that it’s working to bring the NVIDIA GeForce NOW cloud gaming service to its vehicles to further enhance the in-car experience.

DeepRoute.ai, a developer of production-ready autonomous driving solutions, has launched its Driver 3.0 HD Map-Free solution. Built on NVIDIA DRIVE Orin, this product is designed to offer a non-geo-fenced solution for mass-produced advanced driver-assistance system (ADAS) vehicles, and will be available at the end of the year.

By using the computational power of the automotive-grade DRIVE Orin system-on-a-chip, which delivers 254 trillion operations per second (TOPS) of compute performance, DeepRoute’s HD Map-Free solution promises to accelerate deployment of driver-assistance functions to consumer cars and robotaxis.

Plus, Pony.ai announced that its autonomous-driving domain controller (ADC), powered by NVIDIA DRIVE, will be deployed for large-scale commercial use in autonomous-delivery vehicles for Beijing-based companies Meituan and Neolix.

With NVIDIA DRIVE Orin as the AI brain of their driverless vehicles, Meituan and Neolix are well-positioned to fulfill growing consumer demand for safe, scalable autonomous delivery of goods.

Lenovo announced it is a tier-one manufacturer of a new ADC based on the next-generation NVIDIA DRIVE Thor centralized computer. Packed with up to 2,000 TOPS of performance, DRIVE Thor will power Lenovo’s ADC, which is set to become the company’s top-tier vehicle computing product line, with mass production expected in 2025.

Rimac Technology, the engineering arm of Croatian-based Rimac Group, is working on a new central vehicle computer, or R-CVC, that will power ADAS, in-vehicle cockpit systems, the vehicle dynamics logic and the body and comfort software stack.

NVIDIA DRIVE hardware and software will be used in this platform to accelerate Rimac Technology’s development efforts and enable its manufacturer customers to speed time to market, reduce development costs, streamline maintenance, and boost vehicle performance.

Rimac Technology’s central vehicle computer.

New premium intelligent all-electric auto brand smart is now developing next-generation intelligent mobility solutions with NVIDIA. The startup will build its future all-electric portfolio  using the NVIDIA DRIVE Orin platform to create a “smarter” urban mobility experience for its global customers. The start of vehicle production is expected by the end of 2024. 

In addition, smart will collaborate with NVIDIA to build a dedicated data center for the development of highly advanced assisted-driving and AI systems to explore cutting-edge mobility solutions.

Changing the Rules of the Road

The transportation industry is undergoing a revolution, and NVIDIA is leading the charge with its game-changing DRIVE end-to-end platform, which is transforming the way mobility leaders are building advanced driving systems.

NVIDIA’s dedication to safer, smarter and more enjoyable in-vehicle experiences is core to all aspects of its DRIVE platform, from the ability to train AI in the data center to delivering high-performance centralized compute in the car.

The NVIDIA DRIVE AV and DRIVE IX software stacks enable custom applications, and the DRIVE Sim platform powered by Omniverse provides a comprehensive testing and validation platform for autonomous vehicles.

Learn more about the latest technology breakthroughs in automotive and other industries by watching NVIDIA founder and CEO Jensen Huang’s GTC keynote:

Mitsui & Co., Ltd., one of Japan’s largest business conglomerates, is collaborating with NVIDIA on Tokyo-1 — an initiative to supercharge the nation’s pharmaceutical leaders with technology, including high-resolution molecular dynamics simulations and generative AI models for drug discovery.

Announced today at the NVIDIA GTC global AI conference, the Tokyo-1 project features an NVIDIA DGX AI supercomputer that will be accessible to Japan’s pharma companies and startups. The effort is poised to accelerate Japan’s $100 billion pharma industry, the world’s third largest following the U.S. and China.

“Japanese pharma companies are experts in wet lab research, but they have not yet taken advantage of high performance computing and AI on a large scale,” said Yuhi Abe, general manager of the digital healthcare business department at Mitsui. “With Tokyo-1, we are creating an innovation hub that will enable the pharma industry to transform the landscape with state-of-the-art tools for AI-accelerated drug discovery.”

The project will provide customers with access to NVIDIA DGX H100 nodes supporting molecular dynamics simulations, large language model training, quantum chemistry, generative AI models that create novel molecular structures for potential drugs, and more. Tokyo-1 users can also harness large language models for chemistry, protein, DNA and RNA data formats through the NVIDIA BioNeMo drug discovery software and service.

Xeureka, a Mitsui subsidiary focused on AI-powered drug discovery, will be operating Tokyo-1, which is expected to go online later this year. The initiative will also include workshops and technical training on accelerated computing and AI for drug discovery.

Invigorating Drug Discovery Research With AI, HPC

According to Abe, Japan’s pharmaceutical environment has long faced drug lag: delays in both drug development and the approval of treatments that are already available elsewhere. The problem received renewed attention during the race to develop vaccines during the COVID-19 pandemic.

The nation’s pharmaceutical companies see AI adoption as part of the solution — a key tool to strengthen and accelerate the industry’s drug development pipeline. Training and fine-tuning AI models for drug discovery require enormous compute resources, such as the Tokyo-1 supercomputer, which in its first iteration will include 16 NVIDIA DGX H100 systems, each with eight NVIDIA H100 Tensor Core GPUs.

The DGX H100 is based on the powerful NVIDIA Hopper GPU architecture, which features a Transformer Engine designed to accelerate the training of transformer models, including generative AI models for biology and chemistry. Xeureka plans to add more nodes to the system as the project grows.

“Tokyo-1 is designed to address some of the barriers to implementing data-driven, AI-accelerated drug discovery in Japan,” said Hiroki Makiguchi, product engineering manager in the science and technology division at Xeureka. “This initiative will uplevel the Japanese pharmaceutical industry with high performance computing and unlock the potential of generative AI to discover new therapies.”

Customers will be able to access a dedicated server on the supercomputer, receive technical support from Xeureka and NVIDIA, and participate in workshops from the two companies. For larger training runs that require more computational resources, customers can request access to a server with more nodes. Users can also purchase Xeureka’s software solutions for molecular dynamics, docking, quantum chemistry and free-energy perturbation calculations.

By using NVIDIA BioNeMo software on the Tokyo-1 supercomputer, researchers will be able to scale state-of-the-art AI models to millions and billions of parameters for applications including protein structure prediction, small molecule generation and pose prediction estimation.

Tokyo-1 Accelerates Japanese Companies in Pharma and Beyond 

Major Japanese pharma companies including Astellas Pharma, Daiichi-Sankyo and Ono Pharmaceutical are already making plans to advance their drug discovery projects with Tokyo-1.

Tokyo-based Astellas Pharma is pursuing innovative digital solutions across its business — including in sales, manufacturing, and research and development — to maximize outcomes for patients and reduce the costs of healthcare. With Tokyo-1, the company will accelerate its research with molecular simulations and large language models for generative AI through NVIDIA BioNeMo software.

“AI and large-scale simulations can be used for applications including small molecule compounds, antibodies, gene therapy, cell therapy, targeted protein degradation, engineered phage therapy and mRNA medicine,” said Kazuhisa Tsunoyama, head of digital research solutions, advanced informatics and analytics at Astellas. “By enabling us to take full advantage of recent advances in AI and simulation technology, Tokyo-1 will be one of the foundations on which Astellas can achieve its VISION for the future of pharmaceutical research.”

Tokyo-based Daiichi Sankyo will use Tokyo-1 to establish a drug discovery process that fully integrates AI and machine learning.

“By adopting AI and the cutting-edge GPU resources of Tokyo-1, we will be able to perform large-scale computations to accelerate our drug discovery efforts,” said Takayuki Serizawa, senior researcher at Daiichi Sankyo. “These advancements will provide new value to patients by improving drug delivery and potentially enabling personalized medicine.”

Ono Pharmaceutical, based in Osaka, focuses on drug discovery in the fields of oncology, immunology and neurology.

“Training AI models requires significant computational power, and we believe that the massive GPU resources of Tokyo-1 will solve this problem,” said Hiromu Egashira, director of the Drug Discovery DX Office in the drug discovery technology department at Ono. “We envision our use of the DGX supercomputer to be very broad, including high-quality simulations, image analysis, video analysis and language models.”

Beyond the pharmaceutical industry, Mitsui plans to make the Tokyo-1 supercomputer accessible to Japanese medical-device companies and startups — and to connect Tokyo-1 customers to AI solutions developed by global healthcare startups in the NVIDIA Inception program. NVIDIA will also connect Tokyo-1 users with the hundreds of global life science customers in its developer network.

Discover the latest in AI and healthcare at GTC, running online through Thursday, March 23. Registration is free. 

Watch the GTC keynote address by NVIDIA founder and CEO Jensen Huang below:

Digitalization that combines AI and simulation is redefining how industrial products are created and transforming how people interact with the digital world.

To help enterprises tackle complex new workloads, NVIDIA has unveiled the third generation of its NVIDIA OVX computing system.

OVX is designed to power large-scale digital twins built on NVIDIA Omniverse Enterprise, a platform for creating and operating metaverse applications. The latest OVX system provides the breakthrough graphics and AI required to accelerate massive digital twin simulations and other demanding applications by combining NVIDIA BlueField-3 DPUs with NVIDIA L40 GPUs, ConnectX-7 SmartNICs and the NVIDIA Spectrum Ethernet platform.

Some of the world’s largest systems makers will be bringing the latest OVX systems to customers worldwide later this year, providing enterprises with the technology to handle complex manufacturing, design and Omniverse-based workloads. Businesses can take advantage of the real-time, true-to-reality capabilities of OVX to collaborate on the most challenging visualization, virtual workstation and data center processing workflows.

Reimagining Digital Twin Simulation 

Customers using third-generation OVX systems can speed their workflows and optimize simulations through immersive digital twins used to model factories, cities, autonomous vehicles and more before deployment in the real world. This helps maximize operational efficiency and predictive planning capabilities.

For example, DB Netze’s Digitale Schiene Deutschland is leveraging the capabilities of OVX to power large-scale digital twins of dynamic physical systems, including rail networks. Others, like Jaguar Land Rover, are leveraging the graphics and simulation capabilities of OVX systems in conjunction with the NVIDIA DRIVE Sim platform to accelerate the testing and development of next-generation autonomous vehicles.

Next-Generation Platform Features 

The third generation of OVX features a new architecture, with a server design based on a dual-CPU platform with four NVIDIA L40 GPUs. Based on the Ada Lovelace architecture, the L40 GPU delivers revolutionary neural graphics, AI compute and the performance needed for the most demanding Omniverse workloads.

Each OVX server also includes two high-performance ConnectX-7 SmartNICs to enable multi-node scalability and precise time synchronization. The Ethernet adapters enable the multi-node scalability of OVX systems and provide networking capabilities for the low-latency, high-bandwidth communication that globally dispersed teams need.

New with this generation, the BlueField-3 data processing unit offloads, accelerates and isolates CPU-intensive infrastructure tasks. For deploying Omniverse at data center scale, BlueField-3 DPUs provide a secure foundation for running the data center control-plane, enabling higher performance, limitless scaling, zero-trust security and better economics.

Helping users keep up with networking performance, the accelerated NVIDIA Spectrum Ethernet platform provides high bandwidth and network synchronization to enhance real-time simulation capabilities.

Availability 

In addition to original NVIDIA OVX partners Lenovo and Supermicro, third-generation OVX systems will be available later this year through Dell Technologies, GIGABYTE and QCT. NVIDIA is also working on Digital Twin as a Service offerings based on OVX with HPE Greenlake.

To learn more about OVX, watch NVIDIA founder and CEO Jensen Huang’s GTC keynote.

Register free for NVIDIA GTC, a global AI conference, to attend sessions with NVIDIA and industry leaders:

Building a Digital Twin of the German Rail Network to Deliver Next-Generation Railway Systems by Digitale Schiene Deutschland
Optimizing Distribution and Fulfillment Center Operations with Computer Vision and Digital Twins by PepsiCo
Connect With the Experts: How to Build a Digital Twin in Omniverse
Hit the Ground Running With Data Center Digital Twin Automation

Healthcare enterprises globally are working with NVIDIA to drive AI-accelerated solutions that are detecting diseases earlier from medical images, delivering critical insights to care teams and revolutionizing drug discovery workflows.

NVIDIA Clara, a suite of software and services that powers AI healthcare solutions, is enabling this transformation industry-wide. The Clara ecosystem includes BioNeMo for drug discovery, Holoscan for medical devices, Parabricks for genomics and MONAI for medical imaging.

Using NVIDIA Clara, healthcare researchers and companies have recently achieved milestones including generating blueprints for two novel proteins with BioNeMo, conducting a first-of-its-kind surgery with Holoscan, and deploying MONAI-powered solutions in radiology departments.

BioNeMo Enables Generative AI for Drug Discovery

Traditional drug discovery is a time- and resource-intensive process. Many drugs take more than a decade to go to market, with an average drug candidate success rate of just 10%. Generative AI, which makes use of large language models, can help increase the chances of success in less time with fewer costs.

Just as the large language models behind services like ChatGPT can generate text, generative AI models trained on biomolecular data can generate blueprints for new molecules and proteins, a critical step in drug discovery.

NVIDIA BioNeMo is a cloud service for generative AI in biology, offering a variety of AI models for small molecules and proteins. With BioNeMo, pharmaceutical research and industry professionals can use generative AI to accelerate the identification and optimization of new drug candidates.

Startup Evozyne used NVIDIA BioNeMo for AI protein identification to engineer new proteins with enhanced functionality. A joint paper describes the engineered proteins — one to potentially be used for treating disease and another designed for carbon consumption.

Deloitte is using AI models ESM and OpenFold in BioNeMo for its AI drug discovery platform for 3D protein structure prediction, model rank classification and druggable region prediction.

NVIDIA Inception member Innophore uses BioNeMo with its product Cavitomix, a tool that allows users to analyze protein cavities from any input structure. PyTorch-based AI model OpenFold is accelerated up to 6x in BioNeMo, resulting in lightning-fast 3D protein structure prediction of linear amino acids.

Holoscan Powers Real-Time AI in Medical Devices

Millions of medical devices are used every day across hospitals to enable robot-assisted surgery, radiation therapy, CT scans and more. NVIDIA Holoscan — a scalable, software-defined AI computing platform for processing real-time data at the edge — accelerates these devices to deliver the low-latency inference required for AI in a clinical setting.

In a landmark step, doctors at Belgium-based surgical training center ORSI Academy brought NVIDIA Holoscan into the operating room to support real-world, robot-assisted surgery for the first time.

At Onze-Lieve-Vrouw Hospital, urologists trained at ORSI successfully removed the patient’s kidney using Intuitive’s da Vinci robotic-assisted surgical system, with the help of an augmented reality overlay of the patient’s anatomy from a CT scan, rendered in real time and AI-augmented with Holoscan. The video feed overlay allowed the surgeon to clearly view the patient’s vascular and tissue structures that may have been obstructed from view by the surgical instruments used during the procedure.

ORSI Academy surgeons interact with NVIDIA Holoscan in the operating room. Image courtesy of ORSI Academy.

Parabricks Accelerates Genomics for Precision Medicine

Accelerating genomic sequencing, the process of determining the genetic makeup of a specific organism or cell type, is critical to unlocking the full potential of precision medicine.

NVIDIA Parabricks is a suite of AI-accelerated genomic analysis applications that enhances the speed and accuracy of the entire sequencing process, from gathering genetic data to analyzing and reporting it. A whole genome can be analyzed in 16 minutes vs. about 24 hours on CPU, meaning that around 32,000 genomes can be analyzed in a year on a single server.

Accessible from either the genomics instrument itself or through cloud services, Parabricks allows for flexible, scalable and efficient genomics analysis that can lead to more accurate diagnoses and tailored treatments.

Form Bio has recently integrated NVIDIA Parabricks into its computational life sciences platform, resulting in a 52% reduction in overall costs and an over 80x speedup, enabling life sciences professionals to accelerate whole genome sequence analysis.

PacBio began shipping its Revio system, a long-read sequencer designed to deliver accurate, complete genomes at high throughput. With on-board NVIDIA GPUs, Revio has 20x more computing power than prior PacBio systems. The compute is used to handle the increased scale and to utilize advanced AI models for basecalling and methylation analysis. For spatial biology workflows, Nanostring is using NVIDIA technology in its CosMx instrument to power 5-20x faster cell segmentation.

MONAI Helps to Build and Deploy Medical AI

Accurate, detailed processing of medical images is crucial for precise diagnosis. MONAI, a medical imaging AI framework accelerated by NVIDIA, simplifies the creation of healthcare AI applications that can label and analyze medical images.

MONAI recently surpassed 1 million downloads, solidifying its position as an industry-standard tool for healthcare AI developers. MONAI MAPs streamline the deployment of AI models created with the framework as applications that integrate within healthcare workflows and medical software ecosystems.

Biomedical research data platform Flywheel is incorporating MONAI in its offerings. In collaboration with the University of Wisconsin Radiology Department, Flywheel has used MONAI to develop a model-based image classifier that predicts and labels the body regions present in medical images. The AI application speeds up data preparation from up to eight months to just one day.

MLOps platform Weights & Biases is bringing MONAI to Cincinnati Children’s Hospital, providing AI researchers there with a full suite of tools to train and tune computer vision algorithms for AI-assisted object detection to aid diagnosis.

AI Available Anytime, Anywhere 

With the vast applications and impact of AI in healthcare, strategic implementation of the technology is essential. NVIDIA Clara is reaching developers wherever they are, however it’s needed, through global systems integrators, original design manufacturers, cloud platforms and more. 

Bringing AI to a global network: Global systems integrator Deloitte is helping solution providers around the world bring NVIDIA Clara to the healthcare ecosystem. With access to Clara, Deloitte’s professionals are leveraging MONAI for medical imaging, NVIDIA FLARE for federated learning and BioNeMo for drug discovery to develop innovative solutions for customers across the industry.
AI solutions expertise: Service delivery partner Quantiphi consults with clients on AI solutions using its expertise in NVIDIA healthcare software, including Clara Discovery, MONAI, BioMegatron and BioNeMo.
Managing data in the cloud: MONAI has been integrated with all major cloud hyperscalers, allowing for optimized processing and data sharing in a single environment. NVIDIA Parabricks is available in every public cloud and on genomics-specific cloud platforms, including the Terra cloud platform, which is co-developed by The Broad Institute of MIT and Harvard, Microsoft and Verily and has more than 25,000 users.
Software-defined devices: System builder Advantech is adopting NVIDIA IGX, an industrial-grade edge AI platform, for low-latency, real-time healthcare applications in its all-in-one, medical-grade computers.

Discover the latest in AI and healthcare at GTC, running online through Thursday, March 23. Registration is free. 

Watch the GTC keynote address by NVIDIA founder and CEO Jensen Huang below:

Powerful AI technologies are making a massive impact in 3D content creation and game development. Whether creating realistic characters that show emotion or turning simple texts into imagery, AI tools are becoming fundamental to developer workflows — and this is just the start.

At NVIDIA GTC and the Game Developers Conference (GDC), learn how the NVIDIA Omniverse platform for creating and operating metaverse applications is expanding with new Connectors and generative AI services for game developers.

Part of the excitement around generative AI is because of its ability to capture the creator’s intent. The technology learns the underlying patterns and structures of data, and uses that to generate new content, such as images, audio, code, text, 3D models and more.

Announced today, the NVIDIA AI Foundations cloud services enable users to build, refine and operate custom large language models (LLMs) and generative AI trained with their proprietary data for their domain-specific tasks.

And through NVIDIA Omniverse, developers can get their first taste of using generative AI technology to enhance game creation and accelerate development pipelines with the Omniverse Audio2Face app.

Accelerating 3D Content With Generative AI

Specialized generative AI tools can boost creator productivity, even for users who don’t have extensive technical skills. Anyone can use generative AI to bring their creative ideas to life, producing high-quality, highly iterative experiences — all in a fraction of the time and cost of traditional game development.

For example, NVIDIA Omniverse Avatar Cloud Engine (ACE) offers the fastest, most versatile solution for bringing interactive avatars to life at scale. Game developers could leverage ACE to seamlessly integrate NVIDIA AI into their applications, including NVIDIA Riva for creating expressive character voices using speech and translation AI, or Omniverse Audio2Face and Live Portrait for AI-powered 2D and 3D character animation.

Today, game developers are already taking advantage of Audio2Face, where artists are more efficiently animating secondary characters without a tedious manual process. The app’s latest release brings major quality, usability and performance updates, including headless mode and a REST API — enabling developers to run the app and process numerous audio files from multiple users in the data center.

Mandarin Chinese language support can now be previewed in Audio2Face, along with improved lip-sync quality, more robust multi-language support and a new pretrained female model. The world’s first fully real-time, ray-traced subsurface scattering shader is also demonstrated in the demo with Diana, a new digital human model.

GSC Game World, one of Europe’s leading game developers, is adopting Omniverse Audio2Face in its upcoming game, S.T.A.L.K.E.R. 2 Head of Chernobyl. Join the NVIDIA and GCS session at GDC to learn how developers are implementing generative AI technology in Omniverse.

A scene from “S.T.A.L.K.E.R. 2 Head of Chernobyl.”

Fallen Leaf, an indie game developer, is also using Omniverse Audio2Face for character facial animation in Fort Solis, a third-person sci-fi thriller game that takes place on Mars.

New generative AI services such as NVIDIA Picasso, announced at GTC, preview the future of building and deploying assets for game production pipelines. Omniverse is opening portals to enrich workflows with generative AI tools powered by NVIDIA and its partners, and the momentum around unifying the game asset pipeline is growing.

Unifying Game Asset Pipelines With Universal Scene Description

Based on the Universal Scene Description (USD) framework, NVIDIA Omniverse is the connecting fabric that helps creators and developers build interoperability between their favorite tools — like Autodesk Maya, Autodesk 3ds Max and Adobe Substance 3D Painter — or make their own custom applications.

And with USD — an open, extensible framework and ecosystem for composing, simulating and collaborating within 3D worlds — developers can achieve non-destructive, collaborative workflows when creating scenes, as well as simplify asset aggregation so content creation teams can iterate faster.

Image courtesy of Tencent Games.

Tencent Games is adopting USD workflows based on Omniverse to better streamline content creation pipelines. To create vast worlds in every level of a game, the artists at Tencent use design tools such as Autodesk Maya, SideFX Houdini and Unreal Engine to produce up to millions of trees, buildings and other properties to enrich their scenes. The technical artists often look to optimize their content creation pipelines to speed up this process, so they developed a proprietary Unreal Engine workflow powered by OmniObjects.

With USD, Tencent Games’ teams saw the opportunity to easily streamline and seamlessly connect their workflows. Building on Omniverse as the platform for developing USD workflows, the artists at Tencent no longer need to install plug-ins for each software they use. Using just one USD plug-in enables interoperability across all their favorite software tools. Learn more about Tencent Games by joining this session at GDC.

New and updated Omniverse Connectors for game engines are also now available.

The open-beta Omniverse Connector for Unity workflows helps users of Omniverse and Unity collaborate on projects. Developed by NVIDIA, the Connector delivers USD support alongside Unity workflows, enabling Unity users to take advantage of interoperable workflows. It offers Omniverse Nucleus connection and browsing, USD geometry export, lights, cameras, Material Definition Language and preview for USD materials. Early features also include physics export, USD import and unidirectional live sync.

And with the Unreal Engine Connector’s latest release, Omniverse users can now use Unreal Engine’s USD import utilities to add skeletal mesh blend shape importing, and Python USD bindings to access stages on Omniverse Nucleus. The latest release also delivers improvements in import, export and live workflows, as well as updated software development kits.

Learn more about these latest technologies by joining NVIDIA at GDC.

And catch up on all the groundbreaking announcements in generative AI and the metaverse by watching the NVIDIA GTC keynote.

Follow NVIDIA Omniverse on Instagram, Medium, Twitter and YouTube for additional resources and inspiration. Check out the Omniverse forums, and join our Discord server and Twitch channel to chat with the community.

The results are in, and they point to a new era in energy-efficient computing.

In tests of real workloads, the NVIDIA Grace CPU Superchip scored 2x performance gains over x86 processors at the same power envelope across major data center CPU applications. That opens up a whole new set of opportunities.

It means data centers can handle twice as much peak traffic. They can slash their power bills by as much as half. They can pack more punch into the confined spaces at the edge of their networks — or any combination of the above.

Energy Efficiency, a Data Center Priority

Data center managers need these options to thrive in today’s energy-efficient era.

Moore’s law is effectively dead. Physics no longer lets engineers pack more transistors in the same space at the same power.

That’s why new x86 CPUs typically offer gains over prior generations of less than 30%. It’s also why a growing number of data centers are power capped.

With the added threat of global warming, data centers don’t have the luxury of expanding their power, but they still need to respond to the growing demands for computing.

Wanted: Same Power, More Performance

Compute demand is growing 10% a year in the U.S., and will double in the eight years from 2022-2030, according to a McKinsey study.

“Pressure to make data centers sustainable is therefore high, and some regulators and governments are imposing sustainability standards on newly built data centers,” it said.

With the end of Moore’s law, the data center’s progress in computing efficiency has stalled, according to a survey that McKinsey cited (see chart below).

In today’s environment, the 2x gains NVIDIA Grace offers are the eye-popping equivalent of a multi-generational leap. It meets the requirements of today’s data center executives.

Zac Smith — the head of edge infrastructure at Equinix, a global service provider that manages more than 240 data centers — articulated these needs in an article about energy-efficient computing.

“The performance you get for the carbon impact you have is what we need to drive toward,” he said.

“We have 10,000 customers counting on us for help with this journey. They demand more data and more intelligence, often with AI, and they want it in a sustainable way,” he added.

A Trio of CPU Innovations

The Grace CPU delivers that efficient performance thanks to three innovations.

It uses an ultra-fast fabric to connect 72 Arm Neoverse V2 cores in a single die that sports 3.2 terabytes per second in fabric bisection bandwidth, a standard measure of throughput. Then it connects two of those dies in a superchip  package with the NVIDIA NVLink-C2C interconnect, delivering 900 GB/s of bandwidth.

Finally, it’s the first data center CPU to use server-class LPDDR5X memory. That provides up to 50% more memory bandwidth at similar cost but one-eighth the power of typical server memory. And its compact size enables 2x the density of typical card-based memory designs.

Compared to current x86 CPUs, NVIDIA Grace is a simpler design that offers more bandwidth and uses less power.

The First Results Are In

NVIDIA engineers are running real data center workloads on Grace today.

They found that compared to the leading x86 CPUs in data centers using the same power footprint, Grace is:

2.3x faster for microservices,
2x faster in memory intensive data processing
and 1.9 x faster in computational fluid dynamics, used in many technical computing apps.

Data centers usually have to wait two or more CPU generations to get these benefits, summarized in the chart below.

Net gains (in light green) are the product of server-to-server advances (in dark green) and additional Grace servers that fit in the same x86 power envelope (middle bar) thanks to the energy efficiency of Grace.

Even before these results on working CPUs, users responded to the innovations in Grace.

The Los Alamos National Laboratory announced in May it will use Grace in Venado, a 10 exaflop AI supercomputer that will advance the lab’s work in areas such as materials science and renewable energy. Meanwhile, data centers in Europe and Asia are evaluating Grace for their workloads.

NVIDIA Grace is sampling now with production in the second half of the year. ASUS, Atos, GIGABYTE, Hewlett Packard Enterprise, QCT, Supermicro, Wistron and ZT Systems are building servers that use it.

Go Deep on Sustainable Computing

To dive into the details, read this whitepaper on the Grace architecture.

Learn more about sustainable computing from this session at NVIDIA GTC (March 20-23, free with registration): Three Strategies to Maximize Your Organization’s Sustainability and Success in an End-to-End AI World.

Read a whitepaper about the NVIDIA BlueField DPU to find out how to build energy-efficient networks.

And watch NVIDIA founder and CEO Jensen Huang’s GTC keynote to get the big picture.

Microsoft, Tencent and Baidu are adopting NVIDIA CV-CUDA for computer vision AI.

NVIDIA CEO Jensen Huang highlighted work in content understanding, visual search and deep learning Tuesday as he announced the beta release for NVIDIA’s CV-CUDA — an open-source, GPU-accelerated library for computer vision at cloud scale.

“Eighty percent of internet traffic is video, user-generated video content is driving significant growth and consuming massive amounts of power,” said Huang in his keynote at NVIDIA’s GTC technology conference. “We should accelerate all video processing and reclaim the power.”

CV-CUDA promises to help companies across the world build and scale end-to-end, AI-based computer vision and image processing pipelines on GPUs.

Optimizing Internet-Scale Visual Computing With AI

The majority of internet traffic is video and image data, driving incredible scale in applications such as content creation, visual search and recommendation, and mapping.

These applications use a specialized, recurring set of computer vision and image-processing algorithms to process image and video data before and after they’re processed by neural networks.

Microsoft Bing’s Visual Search Engine uses AI Computer Vision
to search for images (dog food, for example) within images on the Internet.

While neural networks are normally GPU accelerated, the computer vision and image processing algorithms that support them are often CPU bottlenecks in today’s AI applications.

CV-CUDA helps process 4x as many streams on a single GPU by transitioning the pre- and post-processing steps from CPU to GPU. In effect, it processes the same workloads at a quarter of the cloud-computing cost.

The CV-CUDA library provides developers more than 30 high-performance computer vision algorithms with native Python APIs and zero-copy integration with the PyTorch, TensorFlow2, ONNX and TensorRT machine learning frameworks.

The result is higher throughput, reduced computing cost and a smaller carbon footprint for cloud AI businesses.

Global Adoption for Computer Vision AI

Adoption by industry leaders around the globe highlights the benefits and versatility of CV-CUDA for a growing number of large-scale visual applications. Companies with massive image processing workloads can save tens to hundreds of millions of dollars.

Microsoft is working to integrate CV-CUDA into Bing Visual Search, which lets users search the web using an image instead of text to find similar images, products and web pages.

In 2019, Microsoft shared at GTC how they’re using NVIDIA technologies to help bring speech recognition, intelligent answers, text to speech technology and object detection together seamlessly and in real time.

Tencent has deployed CV-CUDA to accelerate its ad creation and content understanding pipelines, which process more than 300,000 videos per day.

The Shenzhen-based multimedia conglomerate has achieved a 20% reduction in energy and cost for image processing over their previous GPU-optimized pipelines.

And Beijing-based search giant Baidu is integrating CV-CUDA into FastDeploy, one of the open-source deployment toolkits of the PaddlePaddle Deep Learning Framework, which enables seamless computer vision acceleration to developers in the open-source community.

From Content Creation to Automotive Use Cases

Applications for CV-CUDA are growing. More than 500 companies have reached out with over 100 use cases in just the first few months of the alpha release.

In content creation and e-commerce, images use pre- and post-processing operators to help recommender engines recognize, locate and curate content.

In mapping, video ingested from mapping survey vehicles requires preprocessing and post-processing operators to train neural networks in the cloud to identify infrastructure and road features.

In infrastructure applications for self-driving simulation and validation software, CV-CUDA enables GPU acceleration for algorithms that are already occurring in the vehicle, such as color conversion, distortion correction, convolution and bilateral filtering.

Looking to the future, generative AI is transforming the world of video content creation and curation, allowing creators to reach a global audience.

New York-based startup Runway has integrated CV-CUDA, alleviating a critical bottleneck in preprocessing high-resolution videos in their video object segmentation model.

Implementing CV-CUDA led to a 3.6x speedup, enabling Runway to optimize real-time, click-to-content responses across its suite of creation tools.

“For creators, every second it takes to bring an idea to life counts,” said Cristóbal Valenzuela, co-founder and CEO of Runway. “The difference CV-CUDA makes is incredibly meaningful for the millions of creators using our tools.”

To access CV-CUDA, visit the CV-CUDA GitHub.

Or learn more by checking out the GTC sessions featuring CV-CUDA. Registration is free.

Overcoming Pre- and Post-Processing Bottlenecks in AI-Based Imaging and Computer Vision Pipelines [S51182],
Building AI-Based HD Maps for Autonomous Vehicles [SE50001],
Connect With the Experts: GPU-Accelerated Data Processing with NVIDIA Libraries [CWES52014],
Advancing AI Applications with Custom GPU-Powered Plugins for NVIDIA DeepStream [S51612].