Due to infrastructure, training, and physical ability requirements for at sea oceanographic research, most people haven’t had much access to scientific study of the deep ocean.
The public, citizen scientists, and shore-based researchers can now observe and manage robotic sampling procedures with ease thanks to a brand-new, ground-breaking framework for oceanographic research.
The Shared Autonomy for Remote Collaboration (SHARC) framework “enables remote participants to conduct shipboard operations and control robotic manipulators” such as on remotely operated vehicles (ROVs) “using only a basic internet connection and consumer-grade hardware, regardless of their prior piloting experience,” according to a paper in Science Robotics, “Enhancing scientific exploration of the deep sea through shared autonomy in remote manipulation.”
A research team from the Woods Hole Oceanographic Institution (WHOI), the Massachusetts Institute of Technology (MIT), and the Toyota Technological Institute at Chicago (TTIC) have developed the framework.
Multiple remote operators can work together in real-time using the SHARC framework, issuing goal-directed orders with simple speech and hand gestures while donning virtual reality goggles and interacting with an intuitive three-dimensional workspace representation.
Through SHARC, “we can open up the operational aspects of deep sea exploration to citizen scientists, whether they be kids in a classroom or people who can’t be present on a ship because of logistical or physical requirements,” said co-author Richard Camilli, a principal investigator for the project and scientist in WHOI’s Applied Ocean Physics and Engineering Department.
“Citizen scientists can interact with the ROV’s robotic manipulator arm in a virtual world, somewhat analogous to the science fiction ‘holodeck’ holographic system used on Federation starships on Star Trek.”
By using the SHARC framework for scientific exploration in the deep sea which is a very challenging and unstructured environment it highlights that this technology can be transferable to many different operational contexts, as well, potentially including subsea scientific infrastructure maintenance, deep space operations, nuclear decommissioning, and even unexploded ordnance remediation.
Richard Camilli
The human operator and the robot are given responsibilities based on their complementing capabilities in the human-robot interaction, which is frequently referred to as shared autonomy, made possible by SHARC.
For instance, while human operators are in charge of high-level scene analysis, goal selection, and task-level planning, the robot can handle kinematics, motion planning, obstacle avoidance, and other low-level activities. In addition, SHARC allows for parallel, rather than sequential operation.
“We just give the robot its goal, and it finds a solution,” said Camilli. “People and the robot can collaborate together, where we’re not waiting for one thing to happen in order to do the next thing. While the robotic arm is executing a task, we can be focusing on the next goal.”
In September 2021, during the height of the Covid pandemic, scientists successfully tested SHARC. During an oceanographic expedition in the San Pedro Basin of the Eastern Pacific Ocean, SHARC team members operated WHOI’s Nereid Under Ice (NUI) hybrid ROV from thousands of kilometers away using SHARC’s virtual reality and desktop interfaces.
The team members physically located in Chicago, Boston, and Woods Hole collaboratively collected a physical push core sample and recorded in-situ X-ray fluorescence measurements of seafloor microbial mats and sediments at water depths exceeding 1000 meters.
“This paper really highlights shared autonomy’s potential to help democratize access to the deep sea,” said lead author Amy Phung, who is a student in the MIT-WHOI Joint Program in Oceanography/Applied Ocean Science and Engineering, graduate degree program. Phung was one of the scientists operating the NUI vehicle during the 2021 test of SHARC.
“With SHARC, our shore-side team was able to collect seafloor samples from over 4000 kilometers away without specialized hardware or extensive prior training. In the future, I believe that further advancements in robotics and autonomy research can someday enable shore-side scientists, students, and enthusiasts to actively participate in and contribute to deep-ocean exploration operations as they occur, which in turn can help to foster ocean literacy among the general public.”
“Whether it is on land, air, or in the ocean, most robots that operate today do so in one of two distinctly different ways: full autonomy or full remote control by highly trained pilots, with the latter being standard for settings like underwater manipulation that involve complex interactions between robots and their environment. This paper describes a new framework that enables robots to operate in between these two extremes in a way that takes advantage of the complementary capabilities of robots and humans,” said co-author Matthew Walter, associate professor at TTIC.
Walter also currently is a WHOI guest investigator; previously, he was a student in the MIT-WHOI Joint Program. “SHARC allows people with little-to-no training to perform sophisticated tasks with deep-sea robots, with pilot oversight, from the comfort of their homes and offices through a combination of speech and virtual reality and, in turn, promises to redefine how we use robots for marine science and engineering.”
In addition, SHARC is not dependent on a specific kind of ROV, manipulator arm, or other factors. “We can apply the same SHARC technology with totally different robotic arms and vehicles in completely different contexts,” said Camilli. The SHARC framework “is flexible and hardware-agnostic.”
“By using the SHARC framework for scientific exploration in the deep sea which is a very challenging and unstructured environment it highlights that this technology can be transferable to many different operational contexts, as well, potentially including subsea scientific infrastructure maintenance, deep space operations, nuclear decommissioning, and even unexploded ordnance remediation,” Camilli added.
The National Science Foundation (NSF) National Robotics Initiative, NASA’s Planetary Science and Technology provided funding for this research from Analog Research (PSTAR) program, NSF Graduate Research Fellowship, and the Link Foundation.
The NOAA Ocean Exploration Cooperative Institute provided in-kind support with ship and robotic vehicle operations during 2021 Pacific Ocean demonstrations in the San Pedro Basin.
