Currents are slowly pushing autonomous ships into the mainstream.

AV Ahoy!

By Nathan Eddy, Contributing Writer

Autonomous vehicle technology is being adapted for use in everything from container ships to tankers. Engineers are also equipping other types of commercial vessels, including ferries, tugboats and warships, with state-of-the-art navigation systems that can be safely used in harbors and on lakes, rivers and oceans.

Autonomous shipping is the future of the maritime industry, with smart ships changing the way that cargo vessels are designed, built and operated. Advanced technology will also revolutionize how ships communicate with ports, with each other and with human operators on land.

These crewless and remotely controlled ships must be able to monitor their own health, establish and communicate what is around them, and make decisions based on that information.

To realize the benefits autonomous ships can bring the global economy, an extraordinary number of players in the ecosystem, from the International Maritime Organization (IMO) and technology providers to government agencies and port authorities, will need to develop common standards for automation, communication, navigation and cybersecurity.

On the upside, most of the technologies needed to make autonomous ships a reality already exist, including different types of high-definition cameras, lidar, radar, satellites and thermal imaging systems.

The major question now is how to fuse multiple sensor inputs in a way that provides the best results, in the most cost-efficient way, considering the unique challenges of the maritime environment, such as corrosion, salty air, solar flares and unpredictable weather conditions.

“The main players in the shipping industry see automation as something extremely important to the future,” says Pedro Pacheco, an analyst at Gartner Inc. “The question now is when? Nobody is really in a position to answer this.”

Another big hurdle is the fact there is a certain level of aversion to risk from shipping companies and their customers.

“It will be hard to convince the insurance companies, the owners of the cargo and the shipping companies themselves that autonomous technology is worth it,” explains Pacheco. “This conundrum must first be broken. In the shipping world, it’s not so much a matter of enthusiasm for a new technology; it’s more about whether it makes a sound business case.”

Thomas Pedersen, director of Smart Vessel at Wärtsilä Voyage, which already supplies digital navigation systems to thousands of commercial ships, believes a step-by-step approach will be required.

“We don’t have the expectation to do a research project over the next four years and then out comes a fully autonomous ship,” says Pedersen. “We are looking toward automating smaller parts of a ship’s operation, either to increase efficiency or safety, and to first automate ships that have a simpler operating profile. For example, equipping a ship to sail automatically between port A and port B.”

When it comes to modernizing ship technology, Pedersen believes connectivity is the critical enabling platform, because the further you get from shore, the more expensive and the more complicated it becomes.

“When you reach a certain distance, the only thing you have left is satellites, which are expensive and relatively slow, especially if you want redundancy, monitoring and control segregation,” Pedersen points out. 

The deployment of low-orbit satellites tasked with providing better connectivity may eventually help alleviate that issue. 

“It’s going to be a mix between different technologies, so you have a prime and a fallback if the prime stops working,” explains Pedersen. “It also depends on the operational profile.”

For example, Wärtsilä is working on an unmanned project that involves a container ship traveling between two piers. According to Pedersen, it will be a simple operation in a fixed area, with one player controlling the infrastructure. “It’s an obvious first step,” he says. 

Making the First Waves

As autonomous shipping evolves, the first commercial applications will be confined to inland waterways and short-distance ferries, as well as restricted areas of ports where leisure craft are not allowed.

Wärtsilä recently announced plans to develop an autonomous, zero emission barge for the Port of Rotterdam through a research project dubbed MAGPIE (sMArt Green Ports as Integrated Efficient multimodal hubs). The effort was borne out of a collaboration between the port authorities of Rotterdam, DeltaPort in Germany, HAROPA in France and Sines in Portugal.

The installation will include the Wärtsilä SmartMove Suite, which pairs sensor technology with navigation systems to facilitate automated ship movement.

“Rotterdam is really helping to drive innovation, which they have been doing for many years,” says Pedersen. “MAGPIE is a small part of a bigger project.”

Tugboat operators also have a keen interest in using unmanned technology for near-shore applications, because autonomy offers the potential to improve safety and efficiency.

Autonomous operation enables unmanned tugboats to maneuver alongside traditional manned vessels without the risk of serious injury. That’s because one of the most perilous elements of a towage operation is when a tug has to engage with a much larger cargo ship.

The tugboat crew has to physically catch the ship’s weighted heaving line from the deck, which can be particularly dangerous in bad weather conditions or heavy seas. Once the heaving line is captured, a crew member also has to attach the tug’s towing line to it so it can be winched aboard the connecting ship and secured for towage.

One of the first attempts to integrate autonomous systems in real-world commercial tugboat operations was launched earlier this year. The 100-foot-long Rachael Allen, which was built at Nichols Brothers Boat Builders, in Freeland, WA, will operate off the coast of California towing ships for Foss Maritime.

It features a SM300 autonomous-command-and-control system supplied by Sea Machines Robotics Inc. Capabilities include transit autonomy, as well as remote access to the tugboat’s onboard machinery, a feature that allows personnel to manage and support operations from anywhere on board the vessel or from shore. The tugboat is also equipped with navigation obstacle detection and avoidance capabilities.

The project marks Sea Machines’ first installation of an autonomous system on a vessel over 5,000-hp.

“While our space has seen a number of government-innovation backed prototype initiatives for autonomous tugs in Europe, Japan and Singapore, it’s notable that Foss is launching the first commercially funded project in U.S. waters,” says Michael Johnson, CEO of Sea Machines. “Our technology has been in use aboard commercial vessels around the world for nearly three years and is backed up by thousands of hours of testing and validation.

“This is yet another real-world example of how autonomy is reshaping the way work is being done by vessel crews on the water,” notes Johnson.

Capitalizing on Sea Machines’ remote command and control capabilities, Foss will use its existing Fleet Monitoring Center personnel to survey the tugboat’s systems and operating domain via streaming video and sensor data.

“By having an extra set of eyes and hands with access to vessel command and control, the Sea Machines system allows [us] to maximize productive time, safety and crew welfare,” explains Will Roberts, president and CEO of Foss Maritime. “[We are] leveraging cutting-edge technology to take on the routine work and allow crew to focus on higher-level tasks and improve safety, while also increasing productivity and efficiency during harbor operations.”

Back to the Future

The first attempt by an autonomous ship to cross the Atlantic Ocean has revealed just how hard implementing such technology can be. In mid-June, the Mayflower Autonomous Ship (MAS400) set off to retrace the historic 1620 voyage of the Puritans from Plymouth, England, to Cape Cod, MA.

But, a few days later, the 49-foot vessel operated by IBM and Promare, a nonprofit ocean research organization, encountered a "minor technical issue" and was forced to abort the attempt and return back to port. They plan to try again later this year.

The state-of-the-art trimaran is equipped with 30 sensors, 15 edge devices and six cameras that will be monitored by a land-based crew. GPS, lidar, radar and other devices will also help the new-age Mayflower safely navigate around buoys, cargo ships and other ocean hazards.

The next-gen Mayflower is powered by a hybrid propulsion system that uses a solar-powered electric motor, in addition to batteries and a diesel generator for emergency backup power.

With no human captain or onboard crew, the research vessel uses automation, artificial intelligence and edge computing technology developed by IBM engineers. The 5-ton ship will assess its status, environment and mission, and make decisions about what to do next while at sea.

One goal of the MAS400 project is to help scientists gather data they need to advance understanding of key global issues affecting ocean health, including acidification, microplastics and marine mammal conservation.

A European project called RECOTUG will enable a tugboat to be fully operated from a remote operation center on shore in the port of Copenhagen. Kongsberg Maritime will provide the remote control systems and autonomous technology. The American Bureau of Shipping and Svitzer A/S, a leading tugboat operator, are also involved in the project.

According to Anders Thybo, program lead for operational innovation at Svitzer, the primary technical challenge to widespread adoption of unmanned technology is the development of algorithms for object detection and decision making, as well as the interface between human operated ships vs. autonomous ships.

“It would be key to control how an autonomous vessel should behave if it encounters a manned vessel that does not obey the international regulations for preventing collisions at sea,” says Thybo. “From a towage perspective, our mission is to explore and exploit how advanced autonomous technology and innovation can improve safety and [create] more operational efficiency.”

However, despite recent advancements, humans will likely play an important role in the maritime industry for years to come.

“Vessels will still need to have small maintenance and repair jobs done while in operation, such as simple things like changing engine filters or painting surfaces,” explains Thybo. “As such, we foresee a human in the loop of our operations for the foreseeable future.”

Common Standards

The IMO aims to integrate autonomous technology into the regulatory framework, balancing the benefits derived against important factors such as safety and security concerns, environmental impact and potential costs to the maritime industry. Among other things, it must ensure that standards for Maritime Autonomous Surface Ships (MASS) keep pace with rapidly evolving technological developments.

In May, the organization’s Maritime Safety Committee completed a regulatory scoping exercise to analyze relevant ship safety treaties to assess how MASS could be regulated. The IMO identified various degrees of automation ranging from crewed ship with automated processes and decision support (Degree One); remotely controlled ship with seafarers on board (Degree Two); remotely controlled ship without seafarers on board (Degree Three); and fully autonomous ship (Degree Four).

IMO officials also highlighted several high-priority issues that need to be addressed at a policy level to determine future work. These involve the development of MASS terminology and definitions, including an internationally agreed definition of MASS and clarifying the meaning of the term “master,” “crew” and “responsible person,” particularly in degrees Three (remotely controlled ship) and Four (fully autonomous ship).

Other key issues include addressing the functional and operational requirements of a remote-control station or center, and the possible designation of a remote operator as a “seafarer.” The organization is currently seeking input from shipping companies, equipment providers and other stakeholders as it develops a regulatory framework.

With the pace of technology changing rapidly, the IMO plans to address the benefits derived from these technologies against safety and security concerns.

Aligning Currents of Development

Kongsberg’s recent acquisition of Rolls-Royce Commercial Marine enabled the company to consolidate its own developments in autonomous docking with Rolls-Royce’s autonomous transit technology, resulting in a product that could cover an entire voyage from port to port.

“Our focus used to be on vessel capabilities and how to navigate safely with sensors, but if you want to remove the crew, you still need to have an environment with a remote operation center with the right data and connectivity links with redundancy and security,” says An-Magritt Tinlund Ryste, director of next-generation shipping at Kongsberg.

Then there’s the question of what technologies to use and where. As Ryste points out, lidar can be good for a harbor, but not cost optimal for deep sea operations. “You have to think of the cost benefit of adding more equipment,” she points out. “We’re not doing autonomy for the sake of it; we’re doing it to solve safety, emissions or logistical challenges.”

Part of the focus at Kongsberg is adding optical sensors for navigation that can pan, tilt and zoom to track objects and apply machine learning to predict the movement of an obstacle. The devices also need to recognize an obstacle’s shape, how it will move and in which direction.

“Then, you have to think about how you’re able to collect data from multiple sources and make sense of it, and utilize those data sets, as well as have trust that what you have detected is actually accurate,” explains Ryste. “Remote piloting has been more heavily investigated in the last year. It has changed our way of thinking and has set the scene for a change of pace that hasn’t been there before.”

According to Ryste, another challenge the maritime industry faces when developing autonomous shipping technology is finding a balance between complex algorithms fed into artificial intelligence and machine learning systems and the hard-won knowledge that comes from years of captaining and seamanship. For instance, while rules of sea take into account a meeting of one or two vessels, in a more chaotic environment the correct course of action to take comes down to good seamanship.

“Finding the right operational patterns is essential when writing code for these algorithms, because it’s about capturing that operational knowledge into the technology,” says Ryste. “It’s just expertise, knowledge and experience.”

That requires hiring software developers and AI specialists who are can follow, monitor and ask the right questions to the right group of veteran sea captains over time.

“Finding that combination is really rare,” warns Ryste. “We look for those kinds of people who are great at listening and observe what the captain is doing.”

While it’s easy to design reactive technology that responds to a certain event, software developers who can design proactive technology are the ones who understand the complexity of what companies like Kongsberg are trying to do.

“How do you plan to avoid something that hasn’t occurred yet?” argues Ryste. “That’s where seamanship comes into play.”

Gartner’s Pacheco also points out the myriad edge cases—from heavy fog to heavily armed pirates—that require evasive maneuvers or complicated interactions with land-based operation centers.

“How does an autonomous cargo ship deal with that situation in terms of having the right awareness to make the correct response?” asks Pacheco. “It’s about having that discernment.”

He points to other critical situations where a technical problem on board arises, noting that maintenance is a big part of merchant crew responsibilities today.

“If something happens and the ship gets stranded, and nobody is on board to deal with that situation, someone will need to go out and solve that problem,” says Pacheco. “A situation like that could easily derail the cost savings.”

Time and Tide

While some autonomous ships are currently in operation, it will be several more years before the technology is widely adopted. Some experts cautiously predict it will be at least five years before fully autonomous cargo vessels will be plying global trade routes without human crews. 

The maritime industry is gradually moving toward using smaller onboard crews. This will be followed by a period of testing more automated functionality until shipping companies receive external approval to operate vessels with no crew on board.

Kongsberg already boasts an autonomous passenger ferry that operates between two Norwegian cities. The short crossing is fully automated, but monitored by an onboard captain. Many passengers don’t even realize the ship is navigating by itself.

There’s broad consensus throughout the maritime industry that the operation of autonomous ships will need to be at least as safe as existing vessels if they are to secure regulatory approval, as well as the support of ship owners, operators and seafarers.

However, when new, emerging technology is involved—particularly in a relatively conservative industry like shipping—wider and deeper understanding of potential risks, as well as known and unknown hazards, is needed, which will take time.

“The type of automation we will see in ships will come progressively, so the technology [will need to be] tested and proven,” says Pacheco. “You start automating some functionalities, then you move onto the next level, until you reach full automation.”

Wärtsilä’s Pedersen views the situation similarly, but predicts there will be a tipping point where shipping companies will go from a fear of the technology not working to a fear of missing out. Once this feeling starts to take effect, he believes it will have a significant impact on the global maritime industry.

“Efforts to automate shipping are already happening in small areas where like-minded people get together from different companies and make it happen,” explains Pedersen. “Would it be nice to have a more streamlined process? Absolutely. Is it something I expect to happen anytime soon? No.”

U.S. Navy Anchors Autonomous Ship R&D Efforts

Naval engineers see a bright future for unmanned surface ships. In fact, the U.S. Navy has several autonomous vessels currently operating in sea trials.

Recently, two supply ships successfully travelled more than 4,500 miles, via the Gulf Coast and the Panama Canal, to the West Coast. It’s part of the Navy’s secretive Ghost Fleet Overlord program that is studying the benefits of unmanned vessel technology. Except while transiting the Panama Canal, the ships operated almost entirely in autonomous mode as engineers collected data.

Earlier this year, the U.S. Navy Pacific Fleet conducted an exercise dubbed Unmanned Integrated Battle Problem 21 (IBP21). A variety of unmanned aerial, surface and underwater vehicles were put into real-world, “blue-water” environments, working in sync with manned platforms in combat drills.

“Advancements in technology have created the opportunity to provide our military with an operational advantage by developing improved command and control capabilities,” says Dorothy Engelhardt, director of unmanned systems for the deputy assistant secretary of the Navy for ships. “This enables our military to be more agile, lethal and decisive.”

“Large-scale exercises such as IBP21 are critical for the Navy and Marine Corps to make the transition to a hybrid manned-unmanned force in the future,” adds Rear Admiral Lorin Selby, chief of naval research. “These demonstrations ensure what works in theory will work in the fleet—in an environment that is messier, dirtier and wetter than a lab. They also allow us to get valuable feedback from the Sailors and Marines themselves.”

According to Selby, the purpose of IBP21 was to explore a variety of questions about how unmanned systems can be incorporated into fleet operations. For example, how can unmanned and manned systems work together effectively in diverse warfighting scenarios, and how can unmanned systems be seamlessly integrated into existing platforms?

“Unmanned systems are resilient, enable better beyond-line-of-sight targeting, and improve battlespace awareness and command and control,” explains Selby. “They also provide significant advantages in ISR (intelligence, surveillance, reconnaissance) and targeting and firing capabilities, without creating additional risks to the mission or warfighters. The result—more effective offensive and defensive postures.”