Autor: MK

What can we learn about human intelligence by studying how machines “think?” Can we better understand ourselves if we better understand the artificial intelligence systems that are becoming a more significant part of our everyday lives?

These questions may be deeply philosophical, but for Phillip Isola, finding the answers is as much about computation as it is about cogitation.

Isola, the newly tenured associate professor in the Department of Electrical Engineering and Computer Science (EECS), studies the fundamental mechanisms involved in human-like intelligence from a computational perspective.

While understanding intelligence is the overarching goal, his work focuses mainly on computer vision and machine learning. Isola is particularly interested in exploring how intelligence emerges in AI models, how these models learn to represent the world around them, and what their “brains” share with the brains of their human creators.

“I see all the different kinds of intelligence as having a lot of commonalities, and I’d like to understand those commonalities. What is it that all animals, humans, and AIs have in common?” says Isola, who is also a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL).

To Isola, a better scientific understanding of the intelligence that AI agents possess will help the world integrate them safely and effectively into society, maximizing their potential to benefit humanity.

Asking questions

Isola began pondering scientific questions at a young age.

While growing up in San Francisco, he and his father frequently went hiking along the northern California coastline or camping around Point Reyes and in the hills of Marin County.

He was fascinated by geological processes and often wondered what made the natural world work. In school, Isola was driven by an insatiable curiosity, and while he gravitated toward technical subjects like math and science, there was no limit to what he wanted to learn.

Not entirely sure what to study as an undergraduate at Yale University, Isola dabbled until he came upon cognitive sciences.

“My earlier interest had been with nature — how the world works. But then I realized that the brain was even more interesting, and more complex than even the formation of the planets. Now, I wanted to know what makes us tick,” he says.

As a first-year student, he started working in the lab of his cognitive sciences professor and soon-to-be mentor, Brian Scholl, a member of the Yale Department of Psychology. He remained in that lab throughout his time as an undergraduate.

After spending a gap year working with some childhood friends at an indie video game company, Isola was ready to dive back into the complex world of the human brain. He enrolled in the graduate program in brain and cognitive sciences at MIT.

“Grad school was where I felt like I finally found my place. I had a lot of great experiences at Yale and in other phases of my life, but when I got to MIT, I realized this was the work I really loved and these are the people who think similarly to me,” he says.

Isola credits his PhD advisor, Ted Adelson, the John and Dorothy Wilson Professor of Vision Science, as a major influence on his future path. He was inspired by Adelson’s focus on understanding fundamental principles, rather than only chasing new engineering benchmarks, which are formalized tests used to measure the performance of a system.

A computational perspective

At MIT, Isola’s research drifted toward computer science and artificial intelligence.

“I still loved all those questions from cognitive sciences, but I felt I could make more progress on some of those questions if I came at it from a purely computational perspective,” he says.

His thesis was focused on perceptual grouping, which involves the mechanisms people and machines use to organize discrete parts of an image as a single, coherent object.

If machines can learn perceptual groupings on their own, that could enable AI systems to recognize objects without human intervention. This type of self-supervised learning has applications in areas such autonomous vehicles, medical imaging, robotics, and automatic language translation.

After graduating from MIT, Isola completed a postdoc at the University of California at Berkeley so he could broaden his perspectives by working in a lab solely focused on computer science.

“That experience helped my work become a lot more impactful because I learned to balance understanding fundamental, abstract principles of intelligence with the pursuit of some more concrete benchmarks,” Isola recalls.

At Berkeley, he developed image-to-image translation frameworks, an early form of generative AI model that could turn a sketch into a photographic image, for instance, or turn a black-and-white photo into a color one.

He entered the academic job market and accepted a faculty position at MIT, but Isola deferred for a year to work at a then-small startup called OpenAI.

“It was a nonprofit, and I liked the idealistic mission at that time. They were really good at reinforcement learning, and I thought that seemed like an important topic to learn more about,” he says.

He enjoyed working in a lab with so much scientific freedom, but after a year Isola was ready to return to MIT and start his own research group.

Studying human-like intelligence

Running a research lab instantly appealed to him.

“I really love the early stage of an idea. I feel like I am a sort of startup incubator where I am constantly able to do new things and learn new things,” he says.

Building on his interest in cognitive sciences and desire to understand the human brain, his group studies the fundamental computations involved in the human-like intelligence that emerges in machines.

One primary focus is representation learning, or the ability of humans and machines to represent and perceive the sensory world around them.

In recent work, he and his collaborators observed that the many varied types of machine-learning models, from LLMs to computer vision models to audio models, seem to represent the world in similar ways.

These models are designed to do vastly different tasks, but there are many similarities in their architectures. And as they get bigger and are trained on more data, their internal structures become more alike.

This led Isola and his team to introduce the Platonic Representation Hypothesis (drawing its name from the Greek philosopher Plato) which says that the representations all these models learn are converging toward a shared, underlying representation of reality.

“Language, images, sound — all of these are different shadows on the wall from which you can infer that there is some kind of underlying physical process — some kind of causal reality — out there. If you train models on all these different types of data, they should converge on that world model in the end,” Isola says.

A related area his team studies is self-supervised learning. This involves the ways in which AI models learn to group related pixels in an image or words in a sentence without having labeled examples to learn from.

Because data are expensive and labels are limited, using only labeled data to train models could hold back the capabilities of AI systems. With self-supervised learning, the goal is to develop models that can come up with an accurate internal representation of the world on their own.

“If you can come up with a good representation of the world, that should make subsequent problem solving easier,” he explains.

The focus of Isola’s research is more about finding something new and surprising than about building complex systems that can outdo the latest machine-learning benchmarks.

While this approach has yielded much success in uncovering innovative techniques and architectures, it means the work sometimes lacks a concrete end goal, which can lead to challenges.

For instance, keeping a team aligned and the funding flowing can be difficult when the lab is focused on searching for unexpected results, he says.

“In a sense, we are always working in the dark. It is high-risk and high-reward work. Every once in while, we find some kernel of truth that is new and surprising,” he says.

In addition to pursuing knowledge, Isola is passionate about imparting knowledge to the next generation of scientists and engineers. Among his favorite courses to teach is 6.7960 (Deep Learning), which he and several other MIT faculty members launched four years ago.

The class has seen exponential growth, from 30 students in its initial offering to more than 700 this fall.

And while the popularity of AI means there is no shortage of interested students, the speed at which the field moves can make it difficult to separate the hype from truly significant advances.

“I tell the students they have to take everything we say in the class with a grain of salt. Maybe in a few years we’ll tell them something different. We are really on the edge of knowledge with this course,” he says.

But Isola also emphasizes to students that, for all the hype surrounding the latest AI models, intelligent machines are far simpler than most people suspect.

“Human ingenuity, creativity, and emotions — many people believe these can never be modeled. That might turn out to be true, but I think intelligence is fairly simple once we understand it,” he says.

Even though his current work focuses on deep-learning models, Isola is still fascinated by the complexity of the human brain and continues to collaborate with researchers who study cognitive sciences.

All the while, he has remained captivated by the beauty of the natural world that inspired his first interest in science.

Although he has less time for hobbies these days, Isola enjoys hiking and backpacking in the mountains or on Cape Cod, skiing and kayaking, or finding scenic places to spend time when he travels for scientific conferences.

And while he looks forward to exploring new questions in his lab at MIT, Isola can’t help but contemplate how the role of intelligent machines might change the course of his work.

He believes that artificial general intelligence (AGI), or the point where machines can learn and apply their knowledge as well as humans can, is not that far off.

“I don’t think AIs will just do everything for us and we’ll go and enjoy life at the beach. I think there is going to be this coexistence between smart machines and humans who still have a lot of agency and control. Now, I’m thinking about the interesting questions and applications once that happens. How can I help the world in this post-AGI future? I don’t have any answers yet, but it’s on my mind,” he says.

Adoption of new tools and technologies occurs when users largely perceive them as reliable, accessible, and an improvement over the available methods and workflows for the cost. Five PhD students from the inaugural class of the MIT-IBM Watson AI Lab Summer Program are utilizing state-of-the-art resources, alleviating AI pain points, and creating new features and capabilities to promote AI usefulness and deployment — from learning when to trust a model that predicts another’s accuracy to more effectively reasoning over knowledge bases. Together, the efforts from the students and their mentors form a through-line, where practical and technically rigorous research leads to more dependable and valuable models across domains.

Building probes, routers, new attention mechanisms, synthetic datasets, and program-synthesis pipelines, the students’ work spans safety, inference efficiency, multimodal data, and knowledge-grounded reasoning. Their techniques emphasize scaling and integration, with impact always in sight.

Learning to trust, and when

MIT math graduate student Andrey Bryutkin’s research prioritizes the trustworthiness of models. He seeks out internal structures within problems, such as equations governing a system and conservation laws, to understand how to leverage them to produce more dependable and robust solutions. Armed with this and working with the lab, Bryutkin developed a method to peer into the nature of large learning models (LLMs) behaviors. Together with the lab’s Veronika Thost of IBM Research and Marzyeh Ghassemi — associate professor and the Germeshausen Career Development Professor in the MIT Department of Electrical Engineering and Computer Science (EECS) and a member of the Institute of Medical Engineering Sciences and the Laboratory for Information and Decision Systems — Bryutkin explored the “uncertainty of uncertainty” of LLMs. 

Classically, tiny feed-forward neural networks two-to-three layers deep, called probes, are trained alongside LLMs and employed to flag untrustworthy answers from the larger model to developers; however, these classifiers can also produce false negatives and only provide point estimates, which don’t offer much information about when the LLM is failing. Investigating safe/unsafe prompts and question-answer tasks, the MIT-IBM team used prompt-label pairs, as well as the hidden states like activation vectors and last tokens from an LLM, to measure gradient scores, sensitivity to prompts, and out-of-distribution data to determine how reliable the probe was and learn areas of data that are difficult to predict. Their method also helps identify potential labeling noise. This is a critical function, as the trustworthiness of AI systems depends entirely on the quality and accuracy of the labeled data they are built upon. More accurate and consistent probes are especially important for domains with critical data in applications like IBM’s Granite Guardian family of models.

Another way to ensure trustworthy responses to queries from an LLM is to augment them with external, trusted knowledge bases to eliminate hallucinations. For structured data, such as social media connections, financial transactions, or corporate databases, knowledge graphs (KG) are natural fits; however, communications between the LLM and KGs often use fixed, multi-agent pipelines that are computationally inefficient and expensive. Addressing this, physics graduate student Jinyeop Song, along with lab researchers Yada Zhu of IBM Research and EECS Associate Professor Julian Shun created a single-agent, multi-turn, reinforcement learning framework that streamlines this process. Here, the group designed an API server hosting Freebase and Wikidata KGs, which consist of general web-based knowledge data, and a LLM agent that issues targeted retrieval actions to fetch pertinent information from the server. Then, through continuous back-and-forth, the agent appends the gathered data from the KGs to the context and responds to the query. Crucially, the system uses reinforcement learning to train itself to deliver answers that strike a balance between accuracy and completeness. The framework pairs an API server with a single reinforcement learning agent to orchestrate data-grounded reasoning with improved accuracy, transparency, efficiency, and transferability.

Spending computation wisely

The timeliness and completeness of a model’s response carry similar weight to the importance of its accuracy. This is especially true for handling long input texts and those where elements, like the subject of a story, evolve over time, so EECS graduate student Songlin Yang is re-engineering what models can handle at each step of inference. Focusing on transformer limitations, like those in LLMs, the lab’s Rameswar Panda of IBM Research and Yoon Kim, the NBX Professor and associate professor in EECS, joined Yang to develop next-generation language model architectures beyond transformers.

Transformers face two key limitations: high computational complexity in long-sequence modeling due to the softmax attention mechanism, and limited expressivity resulting from the weak inductive bias of RoPE (rotary positional encoding). This means that as the input length doubles, the computational cost quadruples. RoPE allows transformers to understand the sequence order of tokens (i.e., words); however, it does not do a good job capturing internal state changes over time, like variable values, and is limited to the sequence lengths seen during training.

To address this, the MIT-IBM team explored theoretically grounded yet hardware-efficient algorithms. As an alternative to softmax attention, they adopted linear attention, reducing the quadratic complexity that limits the feasible sequence length. They also investigated hybrid architectures that combine softmax and linear attention to strike a better balance between computational efficiency and performance.

Increasing expressivity, they replaced RoPE with a dynamic reflective positional encoding based on the Householder transform. This approach enables richer positional interactions for deeper understanding of sequential information, while maintaining fast and efficient computation. The MIT-IBM team’s advancement reduces the need for transformers to break problems into many steps, instead enabling them to handle more complex subproblems with fewer inference tokens.

Visions anew

Visual data contain multitudes that the human brain can quickly parse, internalize, and then imitate. Using vision-language models (VLMs), two graduate students are exploring ways to do this through code.

Over the past two summers and under the advisement of Aude Oliva, MIT director of the MIT-IBM Watson AI Lab and a senior research scientist in the Computer Science and Artificial Intelligence Laboratory; and IBM Research’s Rogerio Feris, Dan Gutfreund, and Leonid Karlinsky (now at Xero), Jovana Kondic of EECS has explored visual document understanding, specifically charts. These contain elements, such as data points, legends, and axes labels, that require optical character recognition and numerical reasoning, which models still struggle with. In order to facilitate the performance on tasks such as these, Kondic’s group set out to create a large, open-source, synthetic chart dataset from code that could be used for training and benchmarking. 

With their prototype, ChartGen, the researchers created a pipeline that passes seed chart images through a VLM, which is prompted to read the chart and generate a Python script that was likely used to create the chart in the first place. The LLM component of the framework then iteratively augments the code from many charts to ultimately produce over 200,000 unique pairs of charts and their codes, spanning nearly 30 chart types, as well as supporting data and annotation like descriptions and question-answer pairs about the charts. The team is further expanding their dataset, helping to enable critical multimodal understanding to data visualizations for enterprise applications like financial and scientific reports, blogs, and more.

Instead of charts, EECS graduate student Leonardo Hernandez Cano has his eyes on digital design, specifically visual texture generation for CAD applications and the goal of discovering efficient ways to enable to capabilities in VLMs. Teaming up with the lab groups led by Armando Solar-Lezama, EECS professor and Distinguished Professor of Computing in the MIT Schwarzman College of Computing, and IBM Research’s Nathan Fulton, Hernandez Cano created a program synthesis system that learns to refine code on its own. The system starts with a texture description given by a user in the form of an image. It then generates an initial Python program, which produces visual textures, and iteratively refines the code with the goal of finding a program that produces a texture that matches the target description, learning to search for new programs from the data that the system itself produces. Through these refinements, the novel program can create visualizations with the desired luminosity, color, iridescence, etc., mimicking real materials.

When viewed together, these projects, and the people behind them, are making a cohesive push toward more robust and practical artificial intelligence. By tackling the core challenges of reliability, efficiency, and multimodal reasoning, the work paves the way for AI systems that are not only more powerful, but also more dependable and cost-effective, for real-world enterprise and scientific applications.

Coding with large language models (LLMs) holds huge promise, but it also exposes some long-standing flaws in software: code that’s messy, hard to change safely, and often opaque about what’s really happening under the hood. Researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) are charting a more “modular” path ahead. 

Their new approach breaks systems into “concepts,” separate pieces of a system, each designed to do one job well, and “synchronizations,” explicit rules that describe exactly how those pieces fit together. The result is software that’s more modular, transparent, and easier to understand. A small domain-specific language (DSL) makes it possible to express synchronizations simply, in a form that LLMs can reliably generate. In a real-world case study, the team showed how this method can bring together features that would otherwise be scattered across multiple services.

The team, including Daniel Jackson, an MIT professor of electrical engineering and computer science (EECS) and CSAIL associate director, and Eagon Meng, an EECS PhD student, CSAIL affiliate, and designer of the new synchronization DSL, explore this approach in their paper “What You See Is What It Does: A Structural Pattern for Legible Software,” which they presented at the Splash Conference in Singapore in October. The challenge, they explain, is that in most modern systems, a single feature is never fully self-contained. Adding a “share” button to a social platform like Instagram, for example, doesn’t live in just one service. Its functionality is split across code that handles posting, notification, authenticating users, and more. All these pieces, despite being scattered across the code, must be carefully aligned, and any change risks unintended side effects elsewhere.

Jackson calls this “feature fragmentation,” a central obstacle to software reliability. “The way we build software today, the functionality is not localized. You want to understand how ‘sharing’ works, but you have to hunt for it in three or four different places, and when you find it, the connections are buried in low-level code,” says Jackson.

Concepts and synchronizations are meant to tackle this problem. A concept bundles up a single, coherent piece of functionality, like sharing, liking, or following, along with its state and the actions it can take. Synchronizations, on the other hand, describe at a higher level how those concepts interact. Rather than writing messy low-level integration code, developers can use a small domain-specific language to spell out these connections directly. In this DSL, the rules are simple and clear: one concept’s action can trigger another, so that a change in one piece of state can be kept in sync with another.

“Think of concepts as modules that are completely clean and independent. Synchronizations then act like contracts — they say exactly how concepts are supposed to interact. That’s powerful because it makes the system both easier for humans to understand and easier for tools like LLMs to generate correctly,” says Jackson. “Why can’t we read code like a book? We believe that software should be legible and written in terms of our understanding: our hope is that concepts map to familiar phenomena, and synchronizations represent our intuition about what happens when they come together,” says Meng. 

The benefits extend beyond clarity. Because synchronizations are explicit and declarative, they can be analyzed, verified, and of course generated by an LLM. This opens the door to safer, more automated software development, where AI assistants can propose new features without introducing hidden side effects.

In their case study, the researchers assigned features like liking, commenting, and sharing each to a single concept — like a microservices architecture, but more modular. Without this pattern, these features were spread across many services, making them hard to locate and test. Using the concepts-and-synchronizations approach, each feature became centralized and legible, while the synchronizations spelled out exactly how the concepts interacted.

The study also showed how synchronizations can factor out common concerns like error handling, response formatting, or persistent storage. Instead of embedding these details in every service, synchronization can handle them once, ensuring consistency across the system. 

More advanced directions are also possible. Synchronizations could coordinate distributed systems, keeping replicas on different servers in step, or allow shared databases to interact cleanly. Weakening synchronization semantics could enable eventual consistency while still preserving clarity at the architectural level.

Jackson sees potential for a broader cultural shift in software development. One idea is the creation of “concept catalogs,” shared libraries of well-tested, domain-specific concepts. Application development could then become less about stitching code together from scratch and more about selecting the right concepts and writing the synchronizations between them. “Concepts could become a new kind of high-level programming language, with synchronizations as the programs written in that language.”

“It’s a way of making the connections in software visible,” says Jackson. “Today, we hide those connections in code. But if you can see them explicitly, you can reason about the software at a much higher level. You still have to deal with the inherent complexity of features interacting. But now it’s out in the open, not scattered and obscured.”

“Building software for human use on abstractions from underlying computing machines has burdened the world with software that is all too often costly, frustrating, even dangerous, to understand and use,” says University of Virginia Associate Professor Kevin Sullivan, who wasn’t involved in the research. “The impacts (such as in health care) have been devastating. Meng and Jackson flip the script and insist on building interactive software on abstractions from human understanding, which they call ‘concepts.’ They combine expressive mathematical logic and natural language to specify such purposeful abstractions, providing a basis for verifying their meanings, composing them into systems, and refining them into programs fit for human use. It’s a new and important direction in the theory and practice of software design that bears watching.”

„It’s been clear for many years that we need better ways to describe and specify what we want software to do,” adds Thomas Ball, Lancaster University honorary professor and University of Washington affiliate faculty, who also wasn’t involved in the research. “LLMs’ ability to generate code has only added fuel to the specification fire. Meng and Jackson’s work on concept design provides a promising way to describe what we want from software in a modular manner. Their concepts and specifications are well-suited to be paired with LLMs to achieve the designer’s intent.”

Looking ahead, the researchers hope their work can influence how both industry and academia think about software architecture in the age of AI. “If software is to become more trustworthy, we need ways of writing it that make its intentions transparent,” says Jackson. “Concepts and synchronizations are one step toward that goal.”

This work was partially funded by the Machine Learning Applications (MLA) Initiative of CSAIL Alliances. At the time of funding, the initiative board was British Telecom, Cisco, and Ernst and Young. 

A robot searching for workers trapped in a partially collapsed mine shaft must rapidly generate a map of the scene and identify its location within that scene as it navigates the treacherous terrain.

Researchers have recently started building powerful machine-learning models to perform this complex task using only images from the robot’s onboard cameras, but even the best models can only process a few images at a time. In a real-world disaster where every second counts, a search-and-rescue robot would need to quickly traverse large areas and process thousands of images to complete its mission.

To overcome this problem, MIT researchers drew on ideas from both recent artificial intelligence vision models and classical computer vision to develop a new system that can process an arbitrary number of images. Their system accurately generates 3D maps of complicated scenes like a crowded office corridor in a matter of seconds. 

The AI-driven system incrementally creates and aligns smaller submaps of the scene, which it stitches together to reconstruct a full 3D map while estimating the robot’s position in real-time.

Unlike many other approaches, their technique does not require calibrated cameras or an expert to tune a complex system implementation. The simpler nature of their approach, coupled with the speed and quality of the 3D reconstructions, would make it easier to scale up for real-world applications.

Beyond helping search-and-rescue robots navigate, this method could be used to make extended reality applications for wearable devices like VR headsets or enable industrial robots to quickly find and move goods inside a warehouse.

“For robots to accomplish increasingly complex tasks, they need much more complex map representations of the world around them. But at the same time, we don’t want to make it harder to implement these maps in practice. We’ve shown that it is possible to generate an accurate 3D reconstruction in a matter of seconds with a tool that works out of the box,” says Dominic Maggio, an MIT graduate student and lead author of a paper on this method.

Maggio is joined on the paper by postdoc Hyungtae Lim and senior author Luca Carlone, associate professor in MIT’s Department of Aeronautics and Astronautics (AeroAstro), principal investigator in the Laboratory for Information and Decision Systems (LIDS), and director of the MIT SPARK Laboratory. The research will be presented at the Conference on Neural Information Processing Systems.

Mapping out a solution

For years, researchers have been grappling with an essential element of robotic navigation called simultaneous localization and mapping (SLAM). In SLAM, a robot recreates a map of its environment while orienting itself within the space.

Traditional optimization methods for this task tend to fail in challenging scenes, or they require the robot’s onboard cameras to be calibrated beforehand. To avoid these pitfalls, researchers train machine-learning models to learn this task from data.

While they are simpler to implement, even the best models can only process about 60 camera images at a time, making them infeasible for applications where a robot needs to move quickly through a varied environment while processing thousands of images.

To solve this problem, the MIT researchers designed a system that generates smaller submaps of the scene instead of the entire map. Their method “glues” these submaps together into one overall 3D reconstruction. The model is still only processing a few images at a time, but the system can recreate larger scenes much faster by stitching smaller submaps together.

“This seemed like a very simple solution, but when I first tried it, I was surprised that it didn’t work that well,” Maggio says.

Searching for an explanation, he dug into computer vision research papers from the 1980s and 1990s. Through this analysis, Maggio realized that errors in the way the machine-learning models process images made aligning submaps a more complex problem.

Traditional methods align submaps by applying rotations and translations until they line up. But these new models can introduce some ambiguity into the submaps, which makes them harder to align. For instance, a 3D submap of a one side of a room might have walls that are slightly bent or stretched. Simply rotating and translating these deformed submaps to align them doesn’t work.

“We need to make sure all the submaps are deformed in a consistent way so we can align them well with each other,” Carlone explains.

A more flexible approach

Borrowing ideas from classical computer vision, the researchers developed a more flexible, mathematical technique that can represent all the deformations in these submaps. By applying mathematical transformations to each submap, this more flexible method can align them in a way that addresses the ambiguity.

Based on input images, the system outputs a 3D reconstruction of the scene and estimates of the camera locations, which the robot would use to localize itself in the space.

“Once Dominic had the intuition to bridge these two worlds — learning-based approaches and traditional optimization methods — the implementation was fairly straightforward,” Carlone says. “Coming up with something this effective and simple has potential for a lot of applications.

Their system performed faster with less reconstruction error than other methods, without requiring special cameras or additional tools to process data. The researchers generated close-to-real-time 3D reconstructions of complex scenes like the inside of the MIT Chapel using only short videos captured on a cell phone.

The average error in these 3D reconstructions was less than 5 centimeters.

In the future, the researchers want to make their method more reliable for especially complicated scenes and work toward implementing it on real robots in challenging settings.

“Knowing about traditional geometry pays off. If you understand deeply what is going on in the model, you can get much better results and make things much more scalable,” Carlone says.

This work is supported, in part, by the U.S. National Science Foundation, U.S. Office of Naval Research, and the National Research Foundation of Korea. Carlone, currently on sabbatical as an Amazon Scholar, completed this work before he joined Amazon.