Kategorie: KI Trends

The latest trends and technologies in the automotive industry are in the spotlight at the Beijing International Automotive Exhibition, aka Auto China, which opens to the public on Saturday, April 27.

An array of NVIDIA auto partners is embracing this year’s theme, “New Era, New Cars,” by making announcements and showcasing their latest offerings powered by NVIDIA DRIVE, the platform for AI-defined vehicles.

NVIDIA Auto Partners Announce New Vehicles and Technologies

Image courtesy of JIYUE.

Electric vehicle (EV) makers Chery (booth E107) and JIYUE (booth W206), a joint venture between Baidu and Geely (booth W204), announced they have adopted the next-generation NVIDIA DRIVE Thor centralized car computer.

DRIVE Thor will integrate the new NVIDIA Blackwell GPU architecture, designed for transformer, large language model and generative AI workloads.

In addition, a number of automakers are building next-gen vehicles on NVIDIA DRIVE Orin, including:

smart, a joint venture between Mercedes-Benz and Geely, previewed its largest and most spacious model to date, an electric SUV called #5. It will be built on its Pilot Assist 3.0 intelligent driving-assistance platform, powered by NVIDIA DRIVE Orin, which supports point-to-point automatic urban navigation. smart #5 will be available for purchase in the second half of this year. smart will be at booth E408.

NIO, a pioneer in the premium smart EV market, unveiled its updated ET7 sedan, featuring upgraded cabin intelligence and smart-driving capabilities. NIO also showcased its 2024 ET5 and ES7. All 2024 models are equipped with four NVIDIA DRIVE Orin systems-on-a-chip (SoCs). Intelligent-driving capabilities in urban areas will fully launch soon. NIO will be at booth E207.

Image courtesy of GWM.

GWM revealed the WEY Blue Mountain (Lanshan) Intelligent Driving Edition, its luxury, high-end SUV. This upgraded vehicle is built on GWM’s Coffee Pilot Ultra intelligent-driving system, powered by NVIDIA DRIVE Orin, and can support features such as urban navigate-on-autopilot (NOA) and cross-floor memory parking. GWM will be at booth E303.

XPENG, a designer and manufacturer of intelligent EVs, announced that it is streamlining the design workflow of its flagship XPENG X9 using the NVIDIA Omniverse platform. In March, XPENG announced it will adopt NVIDIA DRIVE Thor for its next-generation EV fleets. XPENG will be at booth W402.

Innovation on Display

On the exhibition floor, NVIDIA partners are showcasing their NVIDIA DRIVE-powered vehicles:

Image courtesy of BYD.

BYD, DENZA and YANGWANG are featuring their latest vehicles built on NVIDIA DRIVE Orin. The largest EV maker in the world, BYD is building both its Ocean and Dynasty series on NVIDIA DRIVE Orin. In addition, BYDE, a subsidiary of BYD, will tap into the NVIDIA Isaac and NVIDIA Omniverse platforms to develop tools and applications for virtual factory planning and retail configurators. BYD will be at booth W106, DENZA at W408 and YANGWANG at W105.

DeepRoute.ai is showcasing its new intelligent driving-platform, DeepRoute IO, and highlighting its end-to-end model. Powered by NVIDIA DRIVE Orin, the first mass-produced car built on DeepRoute IO will focus on assisted driving and parking. DeepRoute.ai will be at booth W4-W07.

Hyper, a luxury brand owned by GAC AION, is displaying its latest Hyper GT and Hyper HT models, powered by NVIDIA DRIVE Orin. These vehicles feature advanced level 2+ driving capabilities in high-speed environments. Hyper recently announced it selected DRIVE Thor for its next-generation EVs with level 4 driving capabilities. Hyper will be at booth W310.

IM Motors is exhibiting the recently launched L6 Super Intelligent Vehicle. The entire lineup of the IM L6 is equipped with NVIDIA DRIVE Orin to power intelligent driving abilities, including urban NOA features. IM Motors will be at booth W205.

Li Auto is showcasing its recently released L6 model, as well as L7, L8, L9 and MEGA. Models equipped with Li Auto’s AD Max system are powered by dual NVIDIA DRIVE Orin SoCs, which help bring ever-upgrading intelligent functionality to Li Auto’s NOA feature. Li Auto will be at booth E405.

Image courtesy of Lotus.

Lotus is featuring a full range of vehicles, including the Emeya electric hyper-GT powered by NVIDIA DRIVE Orin. Lotus will be at booth E403.

Mercedes-Benz is exhibiting its Concept CLA Class, the first car to be developed on the all-new Mercedes-Benz Modular Architecture. The Concept CLA Class fully runs on MB.OS, which handles infotainment, automated driving, comfort and charging. Mercedes-Benz will be at booth E404.

Momenta is rolling out a new NVIDIA DRIVE Orin solution to accelerate commercialization of urban NOA capabilities at scale.

Image courtesy of Polestar.

Polestar is featuring the Polestar 3, the Swedish car manufacturer’s battery electric mid-size luxury crossover SUV powered by DRIVE Orin. Polestar will be at booth E205.

SAIC R Motors is showcasing the Rising Auto R7 and F7 powered by NVIDIA DRIVE Orin at booth W406.

WeRide is exhibiting Chery’s Exeed Sterra ET SUV and ES sedan, both powered by NVIDIA DRIVE Orin. The vehicles demonstrate progress made by Bosch and WeRide on level 2 to level 3 autonomous-driving technology. WeRide will be at booth E1-W04.

Xiaomi is displaying its NVIDIA DRIVE Orin-powered SU7 and “Human x Car x Home” smart ecosystem, designed to seamlessly connect people, cars and homes, at booth W203.

ZEEKR unveiled its SEA-M architecture and is showcasing the ZEEKR 007 powered by NVIDIA DRIVE Orin at booth E101.

Auto China runs through Saturday, May 4, at the China International Exhibition Center in Beijing.

Learn more about the industry-leading designs and technologies NVIDIA is developing with its automotive partners.

Featured image courtesy of JIYUE.

Support for Battle.net on GeForce NOW expands this GFN Thursday, as titles from the iconic StarCraft and Diablo series come to the cloud.

StarCraft Remastered, StarCraft II, Diablo II: Resurrected and Diablo III are part of 16 new games joining the GeForce NOW library of more than 1,900 titles.

Plus, a new update rolling out for members this week brings AV1 streaming to Mac M3 computers. This feature will improve game-streaming quality for members on M3, M3 Pro and M3 Max devices.

Plenty of Space in Hell

Dive into the original Blizzard games that set the stage for real-time strategy and action role-playing games (RPGs). StarCraft Remastered, StarCraft II, Diablo II: Resurrected and Diablo III bring galactic warfare, epic quests and legendary battles to the cloud.

Oh my Zerg.

In StarCraft Remastered, command one of three races — Terran, Zerg or Protoss — as they desperately struggle for survival. Build bases, gather resources and engage in intense battles using unique units and strategies.

Time to Plyon to the cloud.

Continue the saga with StarCraft II, with enhanced graphics and extended storytelling. Save the galaxy from emergent threats in full-length Terran, Zerg and Protoss campaigns. Take charge of all multiplayer units solo in Versus Mode, team up with a friend for Co-Op Missions or explore community-created game modes in the Arcade.

The fires of hell heat up the cloud once again.

In Diablo III, become a hero to battle the forces of darkness, uncover ancient secrets and face powerful foes in the action RPG set in the world of Sanctuary. With various character classes, intense combat and a rich loot system, members can experience a gripping single-player experience and cooperative multiplayer adventures.

Remastered goodness.

Pursue the mysterious Dark Wanderer and battle the denizens of hell in the remastered action RPG Diablo II: Resurrected. The title’s classic Diablo gameplay — enhanced with stunning 3D visuals for all the environments, characters and monsters — enable a nostalgic, high-quality return to hell.

Stream all the action at up to 4K resolution or up to 240 frames per second with an Ultimate membership. These top games join the Battle.net games first added to GeForce NOW, including Diablo IV, Overwatch 2, Call of Duty HQ and Hearthstone.

Remember the Cloud

Unrelenting odds are no problem for the cloud.

The second downloadable content (DLC) for Ark Games’ Remnant II is available for members to stream. Experience a brand-new storyline, area, weapons, bosses and more in The Forgotten Kingdom.

Piece together the forgotten history of the lost tribe of Yaesha in an attempt to quell the vengeful wrath of Lydusa, an ancient stone spirit. Navigate the lingering traces of torment, treachery and death that haunt the land’s once-proud ziggurats. Traverse new dungeons, acquire powerful gear — including a new Archetype, “The Invoker” — meet unexpected allies and face new threats to return a semblance of peace to the forgotten kingdom.

GeForce NOW members will be able to stream the DLC without waiting around for downloads. Uncover the secrets of the lost tribe with an Ultimate membership for eight-hour gaming sessions and support for ultrawide resolutions.

New Adventures

Grow from a humble hamlet to a hub for the kingdom in “Manor Lords.”

Guide a medieval village as it grows into a bustling city in Manor Lords, streaming this week on GeForce NOW. Manage resources and production chains in this historically accurate city builder while expanding the land through large-scale tactical battles.

Check out the full list of new games this week:

Dead Island 2 (New release on Steam, April 22)
Bellwright (New release on Steam, April 23)
Phantom Fury (New release on Steam, April 23)
Oddsparks: An Automation Adventure (New release on Steam, April 24)
Age of Water (New release on Steam, April 25)
Manor Lords (New release on Steam and Xbox, April 26, available on PC Game Pass)
9-Bit Armies: A Bit Too Far (Steam)
Diablo II: Resurrected (Battle.net)
Diablo III (Battle.net)
Dragon’s Dogma 2 Character Creator & Storage (Steam)
Islands of Insight (Steam)
Metaball (Steam)
StarCraft Remastered (Battle.net)
StarCraft II (Battle.net)
Stargate: Timekeepers (Steam)
Tortuga – A Pirate’s Tale (Steam)

What are you planning to play this weekend? Let us know on X or in the comments below.

The last thing in your hand is your weapon against the final boss, what item is it?

— NVIDIA GeForce NOW (@NVIDIAGFN) April 24, 2024

Editor’s note: This post is part of Into the Omniverse, a series focused on how artists, developers and enterprises can transform their workflows using the latest advances in OpenUSD and NVIDIA Omniverse.

Universal Scene Description, aka OpenUSD, is elevating the manufacturing game. Siemens, a leader in industrial technology, has embraced OpenUSD as a cornerstone of its digital transformation journey, using it to help bridge the gap between physical and virtual worlds.

Siemens is adding support for OpenUSD in its Siemens Xcelerator platform applications, starting with Teamcenter X software.

The integration empowers manufacturers to create photorealistic, robust digital twins that mirror real-world counterparts with unprecedented fidelity and efficiency. This allows for optimized resource utilization, minimized waste and enhanced product quality through comprehensive simulation and analysis — all of which align with sustainability and quality objectives.

For a company such as Siemens — one whose software touches all parts of the manufacturing cycle — digitalization can mean helping customers save time and costs, streamline workflows and reduce risk of manufacturing defects.

Ian Fisher, a member of Siemens Digital Industries Software team, is no stranger to the impact of embracing digital transformation — especially one powered by OpenUSD and generative AI.

“We are an industrial company where data is king,” he said. “OpenUSD comes in from the media side of the world, and we are looking to bring its openness and flexibility into the industrial world.”

Enterprises of all sizes depend on Siemens’ Teamcenter software, part of the Siemens Xcelerator platform, to develop and deliver products at scale. By connecting NVIDIA Omniverse — a platform of APIs and services based on OpenUSD — with Teamcenter X, Siemens’ cloud-based product lifecycle management software, engineering teams can make their physics-based digital twins more photorealistic and immersive, improving accuracy and minimizing waste and errors within workflows.

Siemens’ adoption of OpenUSD means that companies like HD Hyundai, a leader in sustainable ship manufacturing, can consolidate and visualize complex engineering projects directly within Teamcenter X. Find out more in the demo:

OpenUSD is touching other parts of Siemens as well. Siemens produces inverters, drive controllers and motors for more than 30,000 customers worldwide. Its lead electronics plant, GWE, in Erlangen, Germany, has been developing use cases from AI-enabled computer vision for defect detection to training pick-and-place robots.

One of their main challenges has been acquiring data to train the AI models that fuel these use cases. By building custom synthetic data generation pipelines using Omniverse Replicator, powered by OpenUSD, the engineers were able to generate large sets of diverse training data by varying many parameters including color, texture, background, lighting and more — allowing them to not only bootstrap but also quickly iterate on their AI models.

Committed to a future of widespread OpenUSD integration, Siemens was one of eight new general members that joined the Alliance for OpenUSD (AOUSD) last month, an organization dedicated to interoperability of 3D content through standardization.

Watch Fisher and other special guests discuss the impact of OpenUSD on industrial digitization workflows in this livestream replay:

Get Plugged Into the World of OpenUSD

Siemens and OpenUSD took center stage this week at Hannover Messe, the world’s leading industrial trade fair. Siemens CEO Roland Busch and Rev Lebaredian, vice president of Omniverse and simulation technology at NVIDIA, shared their vision on the potential of OpenUSD for customers in all industries.

For more on how Siemens is using OpenUSD to build and test complex AI-based automation systems completely virtually, watch the replay of the GTC session, “Virtual Commissioning of AI Vision Systems With OpenUSD.” All other sessions from GTC’s OpenUSD Day are available for viewing on demand.

Watch @BuschRo and @RevLebaredian discuss how #digitaltwins, powered by #AI and #OpenUSD, can drive productivity across all industries at #HM24.

— NVIDIA Design & Visualization (@NVIDIADesign) April 22, 2024

Get started with NVIDIA Omniverse by downloading the standard license free, access OpenUSD resources and learn how Omniverse Enterprise can connect teams. Follow Omniverse on Instagram, Medium and X. For more, join the Omniverse community on the forums, Discord server, Twitch and YouTube channels. 

Featured image courtesy of Siemens, HD Hyundai.

To address the shift to electric vehicles, increased semiconductor demand, manufacturing onshoring, and ambitions for greater sustainability, manufacturers are investing in new factory developments and re-engineering their existing facilities.

These projects often run over budget and schedule, due to complex and manual planning processes, legacy technology infrastructure, and disconnected tools, data and teams.

To address these challenges, manufacturers are embracing digitalization and virtual factories, powered by technologies like digital twins, the Universal Scene Description (OpenUSD) ecosystem and generative AI, that enable new possibilities from planning to operations.

What Is a Virtual Factory?

A virtual factory is a physically accurate representation of a real factory. These digital twins of factories allow manufacturers to model, simulate, analyze and optimize their production processes, resources and operations without the need for a physical prototype or pilot plant.

Benefits of Virtual Factories

Virtual factories unlock many benefits and possibilities for manufacturers, including:

Streamlined Communication: Instead of teams relying on in-person meetings and static planning documents for project alignment, virtual factories streamline communication and ensure that critical design and operations decisions are informed by the most current data.
Contextualized Planning: During facility design, construction and commissioning, virtual factories allow project stakeholders to visualize designs in the context of the entire facility and production process. Planning and operations teams can compare and verify built structures with the virtual designs in real time and decrease costs by identifying errors and incorporating feedback early in the review process.
Optimized Facility Designs: Connecting virtual factories to simulations of processes and discrete events enables teams to optimize facility designs for production and material flow, ergonomic work design, safety and overall utilization.
Intelligent and Optimized Operations: Operations teams can integrate their virtual factories with valuable production data from Internet of Things technology at the edge, and tap AI to drive further optimizations.

Virtual Factories: A Testing Ground for AI and Robotics

Robotics developers are increasingly using virtual factories to train and test AI and autonomous systems that run in physical factories. For example, virtual factories can enable developers and manufacturing teams to simulate digital workers and autonomous mobile robots (AMRs), vision AI agents and sensors to create a centralized map of worker activity throughout a facility. By fusing data from simulated camera streams with multi-camera tracking, developers can generate occupancy maps that inform optimal AMR routes.

Developers can also use these physically accurate virtual factories to train and test AI agents capable of managing their robot fleets, to ensure AI-enabled robots can adapt to real-world unpredictability and to identify streamlined configurations for human-robot collaboration.

What Are the Foundations of a Virtual Factory

Building large-scale, physically accurate virtual factories that unlock these transformational possibilities requires bringing together many tools, data formats and technologies to harmonize the representation of real-world aspects in the digital world.

Originally invented by Pixar Animation Studios, OpenUSD encompasses a collection of tools and capabilities that enable the data interoperability developers and manufacturers require to achieve their digitalization goals.

OpenUSD’s core superpower is flexible data modeling. 3D input can be accepted from source applications and combined with a variety of data, including from computer-aided design software, live sensors, documentation and maintenance records, through a unified data pipeline. OpenUSD enables developers to share these data types across different simulation tools and AI models, providing insights for all stakeholders. Data can be synced from the factory floor to the digital twin, surfacing real-time insights for factory managers and teams.

By developing virtual factory solutions on OpenUSD, developers can enhance collaboration for factory teams, allowing them to review plans, discuss optimization opportunities and make decisions in real time.

To support and accelerate the development of the OpenUSD ecosystem, Pixar, Adobe, Apple, Autodesk and NVIDIA formed the Alliance for OpenUSD, which is building open standards for USD in core specification, materials, geometry and more.

Industrial Use Cases for Virtual Factories

To unlock the potential of virtual factories, industry leaders including Autodesk, Continental, Pegatron, Rockwell Automation, Siemens and Wistron are developing virtual-factory solutions on OpenUSD and NVIDIA Omniverse, a platform of application programming interfaces (APIs) and software development kits that enable developers to build applications for complex 3D and industrial digitalization workflows based on OpenUSD.

FlexSim, an Autodesk company, uses OpenUSD to enable factory teams to analyze, visualize and optimize real-world processes with its simulation modeling for complex systems and operations. The discrete-event simulation software provides an intuitive drag-and-drop interface to create 3D simulation models, account for real-world variability, run “what-if” scenarios and perform in-depth analyses.

Developers at Continental, a leading German automotive technology company, developed ContiVerse, a factory planning and manufacturing operations application on OpenUSD and NVIDIA Omniverse. The application helps Continental optimize factory layouts and plan production processes collaboratively, leading to an expected 13% reduction in time to market. 

Partnering with software company SoftServe, Continental also developed Industrial Co-Pilot, which combines AI-driven insights with immersive visualization to deliver real-time guidance and predictive analytics to engineers. This is expected to reduce maintenance effort and downtime by 10%.

Pegatron, one of the world’s largest manufacturers of smartphones and consumer electronics, is developing virtual-factory solutions on OpenUSD to accelerate the development of new factories — as well as to minimize change orders, optimize operations and maximize production-line throughput in existing facilities.

Rockwell Automation is integrating NVIDIA Omniverse Cloud APIs and OpenUSD with its Emulate3D digital twin software to bring manufacturing teams data interoperability, live collaboration and physically based visualization for designing, building and operating industrial-scale digital twins of production systems.

Siemens, a leading technology company for automation, digitalization and sustainability and a member of the Alliance for OpenUSD, is adopting Omniverse Cloud APIs within its Siemens Xcelerator Platform, starting with Teamcenter X, the industry-leading cloud-based product lifecycle management software. This will help teams design, build and test next-generation products, manufacturing processes and factories virtually, before they’re built in the physical world.

Wistron, a leading global technology service provider and electronics manufacturer, is digitalizing new and existing factories with OpenUSD. By developing virtual-factory solutions on NVIDIA Omniverse, Wistron enables its factory teams to collaborate remotely to refine layout configurations, optimize surface mount technology and in-circuit testing lines, and transform product-on-dock testing. 

With these solutions, Wistron has achieved a 51% boost in worker efficiency and 50% reduction in production process times. Layout optimization and real-time monitoring have decreased defect rates by 40%. And construction time on Wistron’s new NVIDIA DGX factory was cut in half, from about five months to just two and a half months.

Learn more at the Virtual Factory Use Case page, where a reference architecture provides an overview of components and capabilities developers should consider when developing virtual-factory solutions.

Get started with NVIDIA Omniverse by downloading the standard license free, access OpenUSD resources, and learn how Omniverse Enterprise can connect your team. Stay up to date on Instagram, Medium and X. For more, join the Omniverse community on the forums, Discord server, Twitch and YouTube channels. 

Can machine learning help predict extreme weather events and climate change? Christopher Bretherton, senior director of climate modeling at the Allen Institute for Artificial Intelligence, or AI2, explores the technology’s potential to enhance climate modeling with AI Podcast host Noah Kravitz in an episode recorded live at the NVIDIA GTC global AI conference. Bretherton explains how machine learning helps overcome the limitations of traditional climate models and underscores the role of localized predictions in empowering communities to prepare for climate-related risks. Through ongoing research and collaboration, Bretherton and his team aim to improve climate modeling and enable society to better mitigate and adapt to the impacts of climate change.

Stay tuned for more episodes recorded live from GTC, and watch the replay of Bretherton’s GTC session on using machine learning for climate modeling.

Time Stamps

2:03: What is climate modeling and how can it prepare us for climate change?

5:28: How can machine learning help enhance climate modeling?

7:21: What were the limitations of traditional climate models?

10:24: How does a climate model work?

12:11: What information can you get from a climate model?

13:26: What are the current climate models telling us about the future?

15:56: How does machine learning help enable localized climate modeling?

18:39: What, if anything, can individuals or small communities do to prepare for what climate change has in store for us?

25:59: How do you measure the accuracy or performance of an emulator that’s doing something like climate modeling out into the future?

You Might Also Like…

ITIF’s Daniel Castro on Energy-Efficient AI and Climate Change – Ep. 215

AI-driven change is in the air, as are concerns about the technology’s environmental impact. In this episode of NVIDIA’s AI Podcast, Daniel Castro, vice president of the Information Technology and Innovation Foundation and director of its Center for Data Innovation, speaks with host Noah Kravitz about the motivation behind his AI energy use report, which addresses misconceptions about the technology’s energy consumption.

DigitalPath’s Ethan Higgins on Using AI to Fight Wildfires – Ep. 211

DigitalPath is igniting change in the golden state — using computer vision, generative adversarial networks and a network of thousands of cameras to detect signs of fire in real-time. In the latest episode of NVIDIA’s AI Podcast, host Noah Kravtiz spoke with DigitalPath system architect Ethan Higgins about the company’s role in the ALERTCalifornia initiative, a collaboration between California’s wildfire fighting agency CAL FIRE and the University of California, San Diego.

Anima Anandkumar on Using Generative AI to Tackle Global Challenges – Ep. 203

Generative AI-based models can not only learn and understand natural languages — they can learn the very language of nature itself, presenting new possibilities for scientific research. On the latest episode of NVIDIA’s AI Podcast, host Noah Kravitz spoke with Anandkumar on generative AI’s potential to make splashes in the scientific community.

How Alex Fielding and Privateer Space Are Taking on Space Debris – Ep. 196

In this episode of the NVIDIA AI Podcast, host Noah Kravitz dives into an illuminating conversation with Alex Fielding, co-founder and CEO of Privateer Space. Privateer Space, Fielding’s latest venture, aims to address one of the most daunting challenges facing our world today: space debris.

Subscribe to the AI Podcast

Get the AI Podcast through iTunes, Google Podcasts, Google Play, Amazon Music, Castbox, DoggCatcher, Overcast, PlayerFM, Pocket Casts, Podbay, PodBean, PodCruncher, PodKicker, Soundcloud, Spotify, Stitcher and TuneIn.

Make the AI Podcast better: Have a few minutes to spare? Fill out this listener survey.

Editor’s note: This post is part of the AI Decoded series, which demystifies AI by making the technology more accessible, and which showcases new hardware, software, tools and accelerations for RTX PC users.

AI continues to raise the bar for PC gaming.

DLSS 3.5 with Ray Reconstruction creates higher quality ray-traced images for intensive ray-traced games and apps. This advanced AI-powered neural renderer is a groundbreaking feature that elevates ray-traced image quality for all GeForce RTX GPUs, outclassing traditional hand-tuned denoisers by using an AI network trained by an NVIDIA supercomputer. The result improves lighting effects like reflections, global illumination, and shadows to create a more immersive, realistic gaming experience.

A Ray of Light

Ray tracing is a rendering technique that can realistically simulate the lighting of a scene and its objects by rendering physically accurate reflections, refractions, shadows and indirect lighting. Ray tracing generates computer graphics images by tracing the path of light from the view camera — which determines the view into the scene — through the 2D viewing plane, out into the 3D scene, and back to the light sources. For instance, if rays strike a mirror, reflections are generated.

A visualization of how ray tracing works.

It’s the digital equivalent to real-world objects illuminated by beams of light and the path of the light being followed from the eye of the viewer to the objects that light interacts with. That’s ray tracing.

Simulating light in this manner — shooting rays for every pixel on the screen — is computationally intensive, even for offline renderers that calculate scenes over the course of several minutes or hours. Instead, ray samples fire a handful of rays at various points across the scene for a representative sample of the scene’s lighting, reflectivity and shadowing.

However, there are limitations. The output is a noisy, speckled image with gaps, good enough to ascertain how the scene should look when ray traced. To fill in the missing pixels that weren’t ray traced, hand-tuned denoisers use two different methods, temporally accumulating pixels across multiple frames, and spatially interpolating them to blend neighboring pixels together. Through this process, the noisy raw output is converted into a ray-traced image.

This adds complexity and cost to the development process, and reduces the frame rate in highly ray-traced games where multiple denoisers operate simultaneously for different lighting effects.

DLSS 3.5 Ray Reconstruction introduces an NVIDIA supercomputer-trained, AI-powered neural network that generates higher-quality pixels in between the sampled rays. It recognizes different ray-traced effects to make smarter decisions about using temporal and spatial data, and retains high frequency information for superior-quality upscaling. And it recognizes lighting patterns from its training data, such as that of global illumination or ambient occlusion, and recreates it in-game.

Portal with RTX is a great example of Ray Reconstruction in action. With DLSS OFF, the denoiser struggles to reconstruct the dynamic shadowing alongside the moving fan.

With DLSS 3.5 and Ray Reconstruction enabled, the denoiser is trained on AI and recognizes certain patterns associated with shadows and keeps the image stable, accumulating accurate pixels while blending neighboring pixels to generate high-quality reflections.

Deep Learning, Deep Gaming

Ray Reconstruction is just one of the AI graphics breakthroughs that multiply performance in DLSS. Super Resolution, the cornerstone of DLSS, samples multiple lower resolution images and uses motion data and feedback from prior frames to reconstruct native-quality images. The result is high image quality without sacrificing game performance.

DLSS 3 introduced Frame Generation, which boosts performance by using AI to analyze data from surrounding frames to predict what the next generated frame should look like. These generated frames are then inserted in between rendered frames. Combining the DLSS-generated frames with DLSS Super Resolution enables DLSS 3 to reconstruct seven-eighths of the displayed pixels with AI, boosting frame rates by up to 4x compared to without DLSS.

Because DLSS Frame Generation is post-processed (applied after the main render) on the GPU, it can boost frame rates even when the game is bottlenecked by the CPU.

Generative AI is transforming gaming, videoconferencing and interactive experiences of all kinds. Make sense of what’s new and what’s next by subscribing to the AI Decoded newsletter.

To help customers make more efficient use of their AI computing resources, NVIDIA today announced it has entered into a definitive agreement to acquire Run:ai, a Kubernetes-based workload management and orchestration software provider.

Customer AI deployments are becoming increasingly complex, with workloads distributed across cloud, edge and on-premises data center infrastructure.

Managing and orchestrating generative AI, recommender systems, search engines and other workloads requires sophisticated scheduling to optimize performance at the system level and on the underlying infrastructure.

Run:ai enables enterprise customers to manage and optimize their compute infrastructure, whether on premises, in the cloud or in hybrid environments.

The company has built an open platform on Kubernetes, the orchestration layer for modern AI and cloud infrastructure. It supports all popular Kubernetes variants and integrates with third-party AI tools and frameworks.

Run:ai customers include some of the world’s largest enterprises across multiple industries, which use the Run:ai platform to manage data-center-scale GPU clusters.

“Run:ai has been a close collaborator with NVIDIA since 2020 and we share a passion for helping our customers make the most of their infrastructure,” said Omri Geller, Run:ai cofounder and CEO. “We’re thrilled to join NVIDIA and look forward to continuing our journey together.”

The Run:ai platform provides AI developers and their teams:

A centralized interface to manage shared compute infrastructure, enabling easier and faster access for complex AI workloads.
Functionality to add users, curate them under teams, provide access to cluster resources, control over quotas, priorities and pools, and monitor and report on resource use.
The ability to pool GPUs and share computing power — from fractions of GPUs to multiple GPUs or multiple nodes of GPUs running on different clusters — for separate tasks.
Efficient GPU cluster resource utilization, enabling customers to gain more from their compute investments.

NVIDIA will continue to offer Run:ai’s products under the same business model for the immediate future. And NVIDIA will continue to invest in the Run:ai product roadmap as part of NVIDIA DGX Cloud, an AI platform co-engineered with leading clouds for enterprise developers, offering an integrated, full-stack service optimized for generative AI.

NVIDIA DGX and DGX Cloud customers will gain access to Run:ai’s capabilities for their AI workloads, particularly for large language model deployments. Run:ai’s solutions are already integrated with NVIDIA DGX, NVIDIA DGX SuperPOD, NVIDIA Base Command, NGC containers, and NVIDIA AI Enterprise software, among other products.

NVIDIA’s accelerated computing platform and Run:ai’s platform will continue to support a broad ecosystem of third-party solutions, giving customers choice and flexibility.

Together with Run:ai, NVIDIA will enable customers to have a single fabric that accesses GPU solutions anywhere. Customers can expect to benefit from better GPU utilization, improved management of GPU infrastructure and greater flexibility from the open architecture.

NVIDIA announced today its acceleration of Microsoft’s new Phi-3 Mini open language model with NVIDIA TensorRT-LLM, an open-source library for optimizing large language model inference when running on NVIDIA GPUs from PC to cloud.

Phi-3 Mini packs the capability of 10x larger models and is licensed for both research and broad commercial usage, advancing Phi-2 from its research-only roots. Workstations with NVIDIA RTX GPUs or PCs with GeForce RTX GPUs have the performance to run the model locally using Windows DirectML or TensorRT-LLM.

The model has 3.8 billion parameters and was trained on 3.3 trillion tokens in only seven days on 512 NVIDIA H100 Tensor Core GPUs.

Phi-3 Mini has two variants, with one supporting 4k tokens and the other supporting 128K tokens, which is the first model in its class for very long contexts. This allows developers to use 128,000 tokens — the atomic parts of language that the model processes — when asking the model a question, which results in more relevant responses from the model.

Developers can try Phi-3 Mini with the 128K context window at ai.nvidia.com, where it is packaged as an NVIDIA NIM, a microservice with a standard application programming interface that can be deployed anywhere.

Creating Efficiency for the Edge

Developers working on autonomous robotics and embedded devices can learn to create and deploy generative AI through community-driven tutorials, like on Jetson AI Lab, and deploy Phi-3 on NVIDIA Jetson.

With only 3.8 billion parameters, the Phi-3 Mini model is compact enough to run efficiently on edge devices. Parameters are like knobs, in memory, that have been precisely tuned during the model training process so that the model can respond with high accuracy to input prompts.

Phi-3 can assist in cost- and resource-constrained use cases, especially for simpler tasks. The model can outperform some larger models on key language benchmarks while delivering results within latency requirements.

TensorRT-LLM will support Phi-3 Mini’s long context window and uses many optimizations and kernels such as LongRoPE, FP8 and inflight batching, which improve inference throughput and latency. The TensorRT-LLM implementations will soon be available in the examples folder on GitHub. There, developers can convert to the TensorRT-LLM checkpoint format, which is optimized for inference and can be easily deployed with NVIDIA Triton Inference Server.

Developing Open Systems

NVIDIA is an active contributor to the open-source ecosystem and has released over 500 projects under open-source licenses.

Contributing to many external projects such as JAX, Kubernetes, OpenUSD, PyTorch and the Linux kernel, NVIDIA supports a wide variety of open-source foundations and standards bodies as well.

Today’s news expands on long-standing NVIDIA collaborations with Microsoft, which have paved the way for innovations including accelerating DirectML, Azure cloud, generative AI research, and healthcare and life sciences.

Learn more about our recent collaboration.

Whether they’re monitoring miniscule insects or delivering insights from satellites in space, NVIDIA-accelerated startups are making every day Earth Day.

Sustainable Futures, an initiative within the NVIDIA Inception program for cutting-edge startups, is supporting 750+ companies globally focused on agriculture, carbon capture, clean energy, climate and weather, environmental analysis, green computing, sustainable infrastructure and waste management.

This Earth Day, discover how five of these sustainability-focused startups are advancing their work with accelerated computing and the NVIDIA Earth-2 platform for climate tech.

Earth-2 features a suite of AI models that help simulate, visualize and deliver actionable insights about weather and climate.

Insect Farming Catches the AI Bug

Image courtesy of Bug Mars

Amid a changing climate, a key component of environmental resilience is food security: the ability to produce and provide enough food to meet the nutrition needs of all people. Edible insects, such as crickets and black soldier flies, are one solution that could reduce humans’ reliance on resource-intensive livestock farming for protein.

Bug Mars, a startup based in Ontario, Canada, supports insect protein production with AI tools that monitor variables including temperature, pests and number of insects — and predict issues and recommend actions based on that data. It can help insect farmers increase yield by 30%.

The company uses NVIDIA Jetson Orin Nano modules to accelerate its work, and recently announced it’s using synthetic data and digital twin technology to further advance its AI solutions for insect agriculture.

Seeing the Forest for the Trees

Based in Truckee, Calif., Vibrant Planet is modeling trillions of trees and other flammable vegetation such as shrublands and grasslands to help land managers, counties and fire districts across North America build wildfire and climate resilience.

NVIDIA hardware and software has helped Vibrant Planet develop transformer models for forest and ecosystem management and AI-enhanced operational planning.

Visualization courtesy of Vibrant Planet

The startup collects and analyzes data from lidar sensors, satellites and aircraft to train AI models that can map vegetation with high precision, estimate canopy height and detect characteristics of forest and vegetation areas such as carbon, water, biodiversity and built infrastructure. Customers can use this data to understand fire and drought hazards, and, with these insights, conduct scenario planning to forecast the effects of potential forest thinning, prescribed fire or other actions.

Delivering Tomorrow’s Forecast

Tomorrow.io, based in Boston, is a leading resilience platform that helps organizations adapt to increasing weather and climate volatility. Powered by next-generation space technology, advanced AI models and proprietary modeling capabilities, the startup enables businesses and governments to proactively mitigate risk, ensure operational resilience and drive critical decision-making.

Image courtesy of Tomorrow.io

The startup is developing weather forecasting AI and is launching its own satellites to collect environmental data to further train its models. It’s also conducting experiments using Earth-2 AI forecast models to determine the optimal configurations of satellites to improve weather-forecasting conditions.

One of Tomorrow.io’s projects is an initiative in Kenya with the Bill and Melinda Gates Foundation that provides daily alerts to 6 million farmers with insights around when to water their crops, when to spray pesticides, when to harvest or when to change crops altogether due to changes in the local climate. The team hopes to scale up their user base to 100 million farmers in Africa by 2030.

Winds of Change

Palo Alto, Calif.-based WindBorne Systems is developing weather sensing balloons equipped with WeatherMesh, a state-of-the-art AI model for real-time global weather forecasts.

Image courtesy of WindBorne Systems

WeatherMesh predicts factors including surface temperature, pressure, winds, precipitation and radiation. The model has set world records for accuracy and is lightweight enough to run on a gaming laptop, unlike traditional models that run on supercomputers.

WindBorne uses NVIDIA GPUs to develop its AI and is an early-access user of Earth-2. The company’s weather balloon development is funded in part by the National Oceanic and Atmospheric Administration’s Weather Program Office.

Taking the Temperature of Global Cities

FortyGuard, a startup founded in Abu Dhabi with headquarters in Miami, is developing a system to measure urban heat with AI models that present insights for public health officials, city planners, landscape architects and environmental engineers.

FortyGuard presented in the Expo Hall Theater at NVIDIA GTC.

The company — an early-access user of the Earth-2 platform — aims for its temperature AI models to provide a more granular view into urban heat dynamics, providing data that can help industries and governments shape cooler and more livable cities.

FortyGuard’s technology, offered via application programming interfaces, could integrate with existing enterprise platforms to enable use cases including temperature-based route navigation, predictive enhanced EV performance and property insights.

To learn more about the Sustainable Futures program, watch the “AI Nations and Sustainable Futures Day” session from NVIDIA GTC. 

NVIDIA is a member of the U.S. Department of State’s Coalition for Climate Entrepreneurship, which aims to address the United Nations’ Sustainable Development Goals using emerging technologies. Learn more in the GTC session, “Global Strategies: Startups, Venture Capital, and Climate Change Solutions.”

Video at top courtesy of Vibrant Planet.

NVIDIA today announced optimizations across all its platforms to accelerate Meta Llama 3, the latest generation of the large language model (LLM).

The open model combined with NVIDIA accelerated computing equips developers, researchers and businesses to innovate responsibly across a wide variety of applications.

Trained on NVIDIA AI

Meta engineers trained Llama 3 on a computer cluster packing 24,576 NVIDIA H100 Tensor Core GPUs, linked with an NVIDIA Quantum-2 InfiniBand network. With support from NVIDIA, Meta tuned its network, software and model architectures for its flagship LLM.

To further advance the state of the art in generative AI, Meta recently described plans to scale its infrastructure to 350,000 H100 GPUs.

Putting Llama 3 to Work

Versions of Llama 3, accelerated on NVIDIA GPUs, are available today for use in the cloud, data center, edge and PC.

From a browser, developers can try Llama 3 at ai.nvidia.com. It’s packaged as an NVIDIA NIM microservice with a standard application programming interface that can be deployed anywhere.

Businesses can fine-tune Llama 3 with their data using NVIDIA NeMo, an open-source framework for LLMs that’s part of the secure, supported NVIDIA AI Enterprise platform. Custom models can be optimized for inference with NVIDIA TensorRT-LLM and deployed with NVIDIA Triton Inference Server.

Taking Llama 3 to Devices and PCs

Llama 3 also runs on NVIDIA Jetson Orin for robotics and edge computing devices, creating interactive agents like those in the Jetson AI Lab.

What’s more, NVIDIA RTX and GeForce RTX GPUs for workstations and PCs speed inference on Llama 3. These systems give developers a target of more than 100 million NVIDIA-accelerated systems worldwide.

Get Optimal Performance with Llama 3

Best practices in deploying an LLM for a chatbot involves a balance of low latency, good reading speed and optimal GPU use to reduce costs.

Such a service needs to deliver tokens — the rough equivalent of words to an LLM — at about twice a user’s reading speed which is about 10 tokens/second.

Applying these metrics, a single NVIDIA H200 Tensor Core GPU generated about 3,000 tokens/second — enough to serve about 300 simultaneous users — in an initial test using the version of Llama 3 with 70 billion parameters.

That means a single NVIDIA HGX server with eight H200 GPUs could deliver 24,000 tokens/second, further optimizing costs by supporting more than 2,400 users at the same time.

For edge devices, the version of Llama 3 with eight billion parameters generated up to 40 tokens/second on Jetson AGX Orin and 15 tokens/second on Jetson Orin Nano.

Advancing Community Models

An active open-source contributor, NVIDIA is committed to optimizing community software that helps users address their toughest challenges. Open-source models also promote AI transparency and let users broadly share work on AI safety and resilience.

Learn more about how NVIDIA’s AI inference platform, including how NIM, TensorRT-LLM and Triton use state-of-the-art techniques such as low-rank adaptation to accelerate the latest LLMs.