VTT develops tool based on AR and AI to support space repairs

03 Jan 2020

European Space Agency has been testing the astronaut assistance tool as part of its training and operations system.

The European Space Agency has been testing a tool developed by Finnish research agency VTT as part of its training and operations system. The tool, using augmented reality, has been developed to support astronauts in training and operations during a space flight.

In space, says VTT, astronauts need to perform maintenance and repair tasks. To support them, they have completed thorough training, and have access to detailed instructions on computer screen and, if necessary, to experts on ground.

A permanent contact with ground could improve the crew performance, but communications is limited on exploration missions to Moon and Mars. However, the agency warns, delving any deeper into operations in training could make the training period unreasonably long.

VTT has now developed a tool that contributes to ESA's goal to give unambiguous guidance to astronauts in complex maintenance and repair tasks both during the space mission and when training for it before the flight.

The use of augmented reality makes the tool unique. The work instructions are provided as text, graphics or speech to the astronaut's Microsoft HoloLens-AR head mounted display. For example, an arrow [simulation] may directly indicate a lever in the device being serviced and show in which direction it is to be turned. The astronaut may also access the device's service history on his or her AR display as well as its status report that is being observed both in space and on the ground.

"The tool we have developed reduces the risk of human error and significantly speeds up the work performance. It is also extremely well suited for both supervised and individual training of astronauts. The tool uses augmented reality in a new way that is of great help in demanding maintenance and installation tasks of critical importance with a view to security or financial costs in other sectors as well, such as mines, paper mills and nuclear power plants," commented Principal Scientist Kaj Helin from VTT.

AR on Mars?

The cooperation between ESA and VTT has been close for well over a decade. In the latest project, which began in September, the AR tool will be integrated into the training and operations system currently under development.

"There is a strong interest in the AR field from ESA and the way VTT works with challenges of interest to ESA. So VTT is on a path that we ESA see as very fruitful - both for spacecraft testing and human space flight applications," said technical officer Mikael Wolff from ESA's European Space Research and Technology Centre ESTEC.

The importance of the AR tool will increase as the distances become longer on space missions. At the moment, a space mission lasting slightly over one year is preceded by training that takes about two years. When flying to Mars, the outward journey alone will take more than six months, and when the mission arrives on the planet, the crew has to manage novel environments and new technologies.

It is clear that, using traditional training methods, the duration of training would need to be multiplied. With the AR tool, the astronauts could practice the maintenance and repair of equipment to be used on Mars during the long outward journey. VTT has already created a future scenario where astronauts can practice the maintenance of, for example, the Mars rover in a setting resembling the real environment.

The initial work was carried out in a joint VTT and ESA project with the contract number 4000125238/18/NL/AF/as, and the work will continue for the next 18 months in the project 4000127710/19/NL/GLC that was launched in September. Another partner in the project is the Irish SKYTEK, which is developing mobiPV, the training and operations system for ESA astronauts.

Video

The following VTT Research video shows the use of the new AR package in a simulated repair scenario:

• SPIE will be hosting the third annual AR, VR, MR Conference coincident with Photonics West 2020 at the Moscone Center, San Francisco, between February 2nd-4th, 2020. The event will feature must-see presentations and demonstrations from the biggest names in consumer electronics and up-and-coming XR companies.