Demand for unpaved applications is driving unmanned vehicle development.

Forging a Path to Off-Road Autonomy

By Nathan Eddy, Contributing Writer

Autonomous cars and trucks are still in development and years away from real-world deployment. But, many types of off-road vehicles are already emerging for daily use in agriculture, construction, mining, port logistics and other applications.

Recent technological advances promise lower barriers to investment, higher cost savings, and improved safety and reliability. However, rugged and harsh operating environments are typically devoid of the typical guidance infrastructure critical to the seamless operation of blacktop-bound autonomous vehicles.

In addition, strenuous demands for reliability and the need to perform coordinated, complex tasks raise several barriers to widespread adoption.

But, those challenges haven’t stopped engineers at some companies, such as Volvo Construction Equipment, from forging ahead. It recently established a new business venture called Volvo Autonomous Solutions.

“While great strides have been taken in the switch to hybrid and all-electric drive, creating autonomous transport is proving harder to perfect,” says Perjohan Rosdahl, head of off-road at Volvo Autonomous Solutions. “Automation has struggled in the mainstream automotive world because [engineers] are trying to get autonomous vehicles to work everywhere and safely coexist with all the variables of life—cars, trucks, bikes, people, dogs, cats, you name it.

“Solving all of these issues at the same time is proving to be an enormously complex challenge, even for the world’s biggest automotive and technology companies,” explains Rosdahl. “Our approach is to start small in a tightly confined environment and build on our successes over time. A perfect place to start is quarries, which have clearly defined load-and-dump locations over generally short circuits.”

One of the first commercially available AV products developed by Volvo engineers, called TARA, is already being deployed at several mining sites in Europe. The TARA system consists of a fleet of TA15 unmanned haulers, plus infrastructure such as charging stations and a control tower. A series of autonomous vehicles can be connected together to form a “train” of machines for maximum loading, hauling and unloading efficiency.

The TA15 machines, which feature a 15-ton hauling capacity, are assembled at Volvo’s factory in Braas, Sweden. They operate on a battery-electric drivetrain that is based on shared technology from the Volvo Group. The machines use GPS, lidar, radar and other sensors that enable them to interact closely with traditional manned loading equipment, such as excavators and wheel loaders.

The charging process is fully autonomous, using a 150-kilowatt high-power charger that connects via a pantograph on the ground to a pick-up on the bottom side of the TA15. Charge times vary from one to three minutes, depending on the amount of energy needed.

“There’s no need for drivers to control these machines, which makes it attractive for several reasons,” says Luca Delgrossi, head of technology at Volvo Autonomous Solutions. “There’s the cost savings, of course, but also increased safety.”

However, Delgrossi warns that some use cases are not yet possible. He says everything becomes more complicated when things are hard to control, like the presence of people on the site that are not trained or do not know how to behave around AVs.

“All these factors increase complexity,” notes Delgrossi. “The more the environment is controlled, the easier the safety case is to build. If a customer comes to us with a use case, we determine how it maps into our safety case diagram. From there, we decide whether or not we can immediately take the challenge or if more study is required.”

According to Delgrossi, there are ways to compensate for the loss of road-based infrastructure, such as building it at the site where the vehicles are operating. However, ideal candidates for off-road AV applications require extremely high productivity—close to 100 percent, in some cases.

“You really need to be flawless and work 24/7 the entire time,” claims Delgrossi. “This, of course, adds to the complexity. For example, when a vehicle completes a mission, it must assess what the battery level is, go charge automatically and then return for the next mission without any human intervention.”

One technology that can help bring order to this complexity is using cloud-based services to orchestrate the fleet. Volvo also offers the option to package the same services into an edge computer installed on site, which can be helpful for remote locations or special security needs enforced by the customer.

Partnerships with chipmakers like NVIDIA, which supplies perception systems along with machine learning and artificial intelligence, are also helping the company expand and scale up in the mining and construction industry.

“We’re starting to learn how to walk,” says Delgrossi. “We want to address more complicated scenarios, but we are also trying to be careful and not do too many things at the same time. We know our machines are suitable for agriculture, too. It’s a just a matter of priorities.”

Hazards, Labor and Cost Benefits

Komatsu Ltd. is another leading construction equipment manufacturer that is bullish on autonomous mining operations. Since 2008, its FrontRunner Autonomous Haulage System has moved more than 3 billion metric tons of material. Several hundred unmanned Komatsu vehicles are used to haul coal, copper, iron ore and oil sands in remote regions of the world ranging from Australia to Chile.

“This is practical technology that’s been used for quite a while,” says Pedro Pacheco, an analyst at Gartner Inc., a research and advisory firm that specializes in cutting-edge technology. “It’s not theoretical or experimental—it’s being put into practice.”

As the talent pool for heavy machinery operators dwindles, the need for autonomous vehicles in less-than-desirable locations or for the handling of hazardous materials will increase.

“When you think in terms of safety and efficiency, if you don’t pay a person to drive the truck you have a machine that works efficiently with less risk of destruction or fatigue operating in places where health and safety are of extreme importance,” explains Pacheco.

For instance, in Germany, the country’s federal agency for handling radioactive waste is currently developing a project in partnership with IAV GmbH, an automotive engineering company. A fleet of autonomous vehicles will transport nuclear waste to underground disposal facilities remotely through Wi-Fi 6 technology.

For the project, IAV is setting up a test vehicle including a workstation for the control system. Engineers are equipping a scaled vehicle with sensors, control hardware and software, and communication technology, and installing a control station for remote control. IAV is also implementing a control body for communication and recording of measurement data.

“Setting up a test vehicle and integrating it into a corresponding communications infrastructure is now day-to-day business for us as automotive developers of interconnected systems,” says Mirko Taubenreuther, head of the Automated Driving Functions department at IAV. “The appeal of the project stems from the unfamiliar terrain for us and the opportunity to contribute toward the safe operation of underground applications.

“After initial trials on the surface, we will take the system to around 700 meters underground and test it there in different environments,” explains Taubenreuther. “If the project succeeds, we would have proven that Wi-Fi 6 remote-controlled vehicles can be used safely under the required conditions.”

During tests, IAV engineers will validate the propagation of radio waves in different rock environments, such as clay, rock or salt, the latency, shadowing and signal stall of the WLAN standard Wi-Fi 6 underground. In the process, they will also test whether the system functions without interference at higher travel speeds. Specific overloads will also be simulated.

In addition to heavy-load applications in agriculture, mining and construction, many companies are developing automated systems for last-mile delivery, operating on sidewalks. Similar AV technology has already been deployed on large campuses of hospitals, research centers and universities. According to Pacheco, it’s much easier to deploy AV technology is those types of closed environments vs. running a vehicle on public roads.

“The most important factors are that the tech can expand in terms of different use cases, and you have more customers adopting them,” says Pacheco. “It’s about proof of concept to potential customers and being able to develop the applications beyond specific use cases.”

The Ultimate Self-Driving Machine

Continental AG is a 150-year-old company that develops radar, lidar and Wi-Fi-enabled camera systems, as well as hardware components, for off-highway autonomous vehicles in the agriculture, construction and mining sectors.

“Full autonomy is the ultimate goal, where you eliminate the operator and the vehicle is driving fully autonomously—a typical example of this is in material handling, like automated guided vehicles in factories or warehouses,” says Mario Branco, who heads business development for off-highway applications across all Continental business areas. “Our focus is on OEMs, but we also offer retrofit solutions that a mining company could buy directly from us.”

For Branco, what makes the off-road autonomy market interesting is that the business case from an end user perspective is clearer than it is in other industries.

“Those OEMs selling automated guided vehicles or forklifts to organizations can sell a return on investment of less than two years, and I’ve seen even better numbers,” notes Branco. “If the business case can be calculated and sold, the increased productivity and efficiency is attractive.”

In addition, certain objectives, such as sustainability, can be achieved with off-road vehicles. For example, agricultural producers are increasingly being asked to limit their use of fertilizers, herbicides and insecticides. Autonomous off-road vehicles can be used to remove weeds between rows of crops or to apply chemicals more judiciously.

“You get to a point where the technology you need is available and affordable, like sensors, artificial intelligence and software,” Branco points out. “Thanks to those synergies, the off-highway industry is benefiting, and the target to increase this operational efficiency is achievable.”

However, what doesn’t work is to take AV technology developed for use on public roads or highways and apply it to the off-highway world, where there is a much different environment. In other words, you can’t take the road environment to the construction site or the field.

“One of the points with off-highway is that we are talking about mobile machines that are doing a process; simply driving is not enough,” says Branco. “You need to make sure all those processes are automated. Otherwise, you’re not reaping the full benefits, and the complexity of the use cases has to be considered.”

Sensors and Sensibilities

Aceinna Inc., a leading supplier of current sensors, flow sensors and inertial measurement unit (IMU) sensing, has been applying its cutting-edge technology to off-road autonomous applications in the agricultural and construction markets.

In agriculture, IMUs can be used in precision driving for row-crop spacing, with additional corrections for centimeter-gauge accuracy, as well as in determining angular rates for planting seeds while a tractor makes a turn.

In construction, IMUs attached to the chassis, boom and arm of an excavator enable operators to compute the location of the bucket. If all the angles and the geometry of the excavator are known, through trigonometry one can piece together an accurate position of the bucket doing the digging.

“IMU technology, unlike cameras or lidar, is based on gravity, so rain, dust and storms don’t affect it,” says John Newton, vice president of Aceinna. “If you put one of these real-time kinematic (RTK) global navigation satellite system (GNSS) systems on the vehicle, then you know where you are, and you know exactly where you’re digging. That’s the next step that will allow more autonomous use, but the cost of those needs to come down significantly.”

“Autonomous vehicle systems require position accuracy better than 10 centimeters to achieve [critical] performance targets and safety goals,” claims Yang Zhao, chairman and CEO of Aceinna. “Unfortunately, GNSS alone falls short, enabling only meter-level position accuracy even under ideal operating conditions. A navigation system that includes an RTK positioning engine and GNSS corrections delivered from a ground-based network of secure base stations will reliably improve position accuracy to centimeter-level accuracy.”

To address the issue, Aceinna recently launched OpenARC, a precise positioning hardware and software platform that offers easy system integration of GNSS corrections with high-performance inertial navigation system and RTK hardware. OpenARC is powered by Point One Navigation, a leading supplier of precise positioning technology. The OpenARC positioning platform combines Point One’s Polaris GNSS correction service with Aceinna’s OpenRTK330 hardware and software system.

For robustness, Aceinna’s devices are sealed in casings with an IP69 rating for use in highly ruggedized environments. That’s because many qualification requirements are more stringent for off-highway applications where vibration, testing reliability, accelerated life tests, temperature cycles, chemical resistance and pressure washing tests have to be addressed.

“There also is not as much of a consensus on safety as there is in automotive,” explains Newton. “There’s not one dominant standard that everyone’s deciding to comply with. It would be easier if everyone knew what to comply with, but we’re not at that stage. It’s not clear how that’s all going to work out.”

However, one recent development that promises to improve off-road autonomous machine performance is a strategic alliance between Trimble Inc. and VayaVision, an Israeli company that is a division of LeddarTech Co. The companies plan to bring advanced perception hardware and software to the agriculture, construction and mining markets.

The integration of VayaVision’s raw data sensor fusion and perception software platform with Trimble’s existing machine control technology will enable intelligent automation in complex work environments.

The alliance is focused on advancing data fusion for multi-sensor systems such as lidar, cameras and GNSS. These sensors are used to produce reliable environmental mapping, which is critical for autonomous systems. The result is expected to enable improvements in the performance of autonomous machines operating in challenging work environments or when supporting safety-critical applications.

Trimble and VayaVision engineers will collaborate to advance autonomous development and adoption with a customizable portfolio that can potentially benefit agriculture, construction and mining professionals in several ways.

“[Goals include] augmenting or replacing human control to create a safer work environment through path optimization, obstacle avoidance and situational awareness,” says Finlay Wood, business area director for Trimble's Autonomous Solutions Division. “[We also hope to improve] sustainability by optimizing vehicle functionality and utilization with automation to reduce idle time, fuel consumption and environmental impact.

“We see the demand for operator assistance, autonomous machines and workflow optimization accelerating dramatically,” explains Wood. “Alliances such as this are a critical part of how we will bring added value to our customers in all levels of autonomy.”

Off the Blacktop, Onto the Battlefield

Practical applications for off-road autonomy go far beyond farm fields and rock quarries. In the near future, military convoys could be made safer for soldiers by using AV technology to reduce the number of people needed to operate trucks, tanks and other vehicles.

Safety is a key driver for defense departments when considering unmanned vehicle technology, such as replacing a solider in a front-line location—or behind enemy lines—with autonomous vehicles. It allows soldiers to see things and go places that they traditionally can’t.

“It’s tremendously compelling from a human impact perspective,” says John Lizzi, executive leader for robotics and autonomous systems at GE Research. “The next frontier is moving autonomy into these more complex environments, up that ladder from the factory to the road and into environments like the forest, and it’s extremely exciting.”

Lizzi and his colleagues are currently working on a project with the U.S. Army through its Scalable Adaptive Resilient Autonomy Program. The goal is to develop a new kind of “risk-aware” autonomous technology that enables military ground vehicles to move safely in complex off-road conditions.

“If you step back and look at the AV industry today, they’re building systems that operate in somewhat structured and constrained environments, with lines on the road, signs and tremendous amounts of map data,” explains Lizzi. “When you think about the domain the army and our defense partners operate in, there’s a lot less structure; you don’t necessarily have map data to rely on.”

The challenge, from Lizzi’s perspective, is to build autonomy in complex, unstructured environments. The current phase of the R&D program is focused on developing and demonstrating a system that can navigate through a forest.

The process requires designing a system that can look at the terrain, perceive it, figure out which parts are less risky and navigate through it. And, if it’s not able to go through it, the navigation system will fall back to a different behavior, and figure out how to reduce its risk.

“The reason this uncertainly aware capability is so important is because you’re trying to make a real-time determination of whether it’s safe or not,” Lizzi points out. “You need to have some concept of how your algorithms are performing and do some introspection on how aggressively you should pursue your path."

GE Research engineers have been testing the AV technology at the Army Research Laboratory’s R&D facility in Middle River, MD. The next step is transitioning to their proving grounds, where everyone will see how the system performs in the types of environments they’re meant to work in.

“Right now, the hardware is much more capable than the software that runs on it,” says Shiraj Sen, a senior scientist at GE Research. “[We have] done a lot of work on rugged systems and sensing is built-in, as well.

“Where the systems have been lacking is in the autonomy itself,” adds Sen. “We are trying to flip that and make the autonomy as capable as the vehicle it’s running on, or more capable.”

To learn more about the latest military AV applications, see Army R&D Advances Autonomous Off-Road Navigation in this issue.