Engineers teach spacecraft to ‘dream’ their way to the space station

Docking with the ISS may seem simple. However, actually doing so shows how difficult orbital mechanics can be. It’s like traveling down a highway at 28,000 km/hr (17,000 mph) and parallel parking in an open garage on a multibillion-dollar laboratory traveling at the same speed. If you try to accelerate forward, you actually drift up, and there’s no air friction to naturally slow you down. Oh, and if you hit the lab, everyone aboard both your craft and the station dies, and the resulting debris field could wipe out dozens of satellites and even harm people on the ground. No pressure, obviously.

For decades, aerospace engineers have docked successfully using hard-coded physics equations and human pilots to make corrections. But now, a new paper posted to the arXiv preprint server from researchers at Stanford is taking a shot at building an AI to perform a series of “mental simulations” that could fundamentally change how future spacecraft interact with each other.

Their solution is called the Out-of-this-World-Model (OWM), but before we get to what that is, it’s best to recap how we typically navigate in low Earth orbit (LEO). Traditionally, navigation computers use a type of algorithm called a guidance, navigation and control (GNC) algorithm. They also take advantage of another mathematical tool called an extended Kalman filter, which helps them take in data from GPS receivers and star trackers and output thruster burn duty cycles.

Unfortunately, Kalman filters are terrible at processing high-speed video. So engineers turn to traditional computer vision techniques. However, those techniques face a huge hurdle in space—they’re very fragile. If sunlight glints off a solar array or part of the target is suddenly covered in shadow, those computer vision algorithms can be thrown off.

Sounds like a perfect place for AI, right? Researchers thought so too, so they began applying reinforcement learning (RL) algorithms—the kind famously behind computers winning at chess or Dota. However, they’re only good at dealing with scenarios with very specific sets of rules. Change the rules—such as moving the ISS docking port they’re supposed to attach to from one side of the station to the other—and the algorithms don’t know what to do.

Enter the “world model.” The best way to describe this type of algorithm is through an analogy with a baseball (or cricket) player. When an outfielder moves to catch a fly ball, they don’t immediately begin solving partial differential equations in their head—they have a mental simulation that predicts where the ball will land. That mental simulation is based on past visual cues but doesn’t involve a lot of direct calculations.

In machine learning, a “world model” is a type of AI architecture that learns the fundamental physics of the environment it’s interacting with directly from experience rather than from equations programmed into it or updates based on a specific reinforcement algorithm. The Stanford OWM model uses this idea to “dream” dozens of potential futures and controls the system outputs to “step” toward that dream future.

Crucially, instead of just guessing, it measures an outcome’s probability. That measure of confidence is key to ensuring an AI can react to unexpected situations or adjust an output when some of its predicted outcomes don’t occur.

Teaching such a model remained the biggest challenge, though, with a world model requiring hundreds of thousands of simulated flights. Doing so on a traditional computer CPU would have taken weeks. So the researchers came up with a library they dubbed AstroJAX, which is designed to run on graphics processing units (GPUs), similar to those used to train early AI models but originally intended for use in video gaming computers.

The algorithm that resulted from this effort performed admirably—the OWM model required only 500,000 iterations to master docking maneuvers, whereas a comparable RL system required 25,000,000 permutations. The OWM model also performed better when asked to dock at a never-before-seen docking port on the ISS and was better able to deal with unexpected situations—such as when the researchers intentionally placed a docked capsule at the docking port the AI was supposed to use.

Admittedly, there’s still a long way to go in this software’s development. Across all the docking ports on the ISS, the model docked successfully about 53% of the time—though that should be compared with the RL algorithm’s 29% success rate. However, close-up operations appeared difficult for the OWM, likely because collisions were heavily penalized, which the authors say can be adjusted in future iterations.

It will likely be a while before we see a fully automated AI take control of a docking sequence—and probably much longer before one takes control of anything with humans on either end. But with the increasing number of satellites that either need servicing or deorbiting, and potentially plenty more people in space in the coming decades, it’s only a matter of time before we let the robot take the wheel. Or thruster control, in this case.

Who’s behind this story?

Lisa Lock

BA art history, MA material culture. Former museum editor, paramedic, and transplant coordinator. Editing for Science X since 2021.

Full profile →

Andrew Zinin

Master’s in physics with research experience. Long-time science news enthusiast. Plays key role in Science X’s editorial success.

Full profile →