Augmented reality onboard the ISS?
According to a brief news release from Human Factors and Ergonomics Society, researchers from NASA and Lockheed Martin have successfully tested a software for a robotic extension device that has been shown to dramatically improve astronauts’ ability to perform remote tasks on the International Space Station (ISS).' Their usage of augmented reality has been developed to be used with the Special Purpose Dexterous Manipulator (SPDM), which is scheduled for 2008 and works in conjunction with the current Space Station Remote Manipulator System (SSRMS). And they concluded that their system should be used for space shuttle missions and on the ISS.
You can see above a diagram showing the components of the Special Purpose Dexterous Manipulator (SPDM) designed in Canada for NASA, and scheduled to be used in 2008. "The SPDM is a two-armed robotic device that will be used as an extension to the end effector of the Space Station Remote Manipulator System (SSRMS), currently in use on the International Space Station (ISS). SPDM tasks include external orbital replacement unit (ORU) maintenance and temporary storage management of ORU's and payloads" (Credit: NASA and Lockeed Martin).
Crew training for the SPDM includes the use a hardware simulator called the Dexterous Manipulator Trainer (DMT) -- see above -- located in the Multi-use Remote Manipulator Development Center (MRMDF) at the NASA Johnson Space Center (JSC). (Credit: NASA & Lockeed Martin)
So how the researchers tested this robotic device? "Twelve highly skilled robotics operators were tested on four installation tasks under conditions of dynamic sunlight and very dark nights with and without the overlay. In all cases, accuracy and efficiency improved significantly when using the new overlay system, and all 12 operators found the overlay information extremely helpful in performing the ORU alignment operation. Time to complete the task was also reduced."
The researchers, James Maida (NASA JSC), Charles Bowen and John Pace (Lockheed Martin), presented their findings at the Human Factors and Ergonomics Society's 2007 Annual Meeting on October 3, 2007 in Baltimore, Maryland. Here is a link to their paper, "Improving Robotic Operator Performance Using Augmented Reality" (PDF format, 5 pages, 768 KB), from which the above illustrations have been extracted.
Here is the abstract. "Robotic operations performed in earth orbit face unique challenges. In the case of the Special Purpose Dexterous Manipulator (SPDM), a two-armed robotic device, operations can last multiple orbital days, with dynamic sunlit days and very dark nights. Direct and indirect visibility can be difficult. This project developed and tested the use of augmented reality technology with a ground based trainer to reduce the negative effects of these conditions. Results of the project showed that twelve skilled operators can significantly reduce positioning errors and time to task completion when using augmented reality techniques."
So how did it work? "The results of this experiment indicate that overlays improve performance in maneuvering an ORU in preparation for inserting it into a receptacle. Despite the small number of subjects available to participate, three of the four performance metrics showed statistically significant improvements in pre-positioning accuracy using overlays. Trial completion time results are slightly less clear cut, but variability in performance was reduced for all metrics during trials using overlays."
Finally, here are the conclusions of this technical report. "Because the graphics are relatively simple and the computational requirements are low the system could be implemented on existing hardware such as the avionics vision unit (AVU) used on the shuttle and the space station. A previous technology development project demonstrated this feasibility. The potential benefits provided by augmented reality techniques in robotic control should be considered in the modification of existing systems or the designing of future systems for operations in space."
Sources: Human Factors and Ergonomics Society news release, October 11, 2007; and various websites
You'll find related stories by following the links below.