BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Waymo And Cruise Vie For Supremacy In Murky California Self-Driving Data

Following
This article is more than 4 years old.

Waymo once again had the best annual statistics for how infrequently humans need to take control of the self-driving vehicles it tests in California but faces more intense competition from General Motors-backed Cruise, which reported numbers slightly behind the Alphabet Inc. unit’s in 2019–and apparently much better in the second half. The stats are by no means definitive, but the progress is encouraging.

Waymo, which runs a small-scale driverless ride service in metro Phoenix, racked up 1.45 million autonomous miles on California roads in 2019, the most of any company, based on Department of Motor Vehicle data. It said “disengagements”—situations when a human had to take the wheel—occurred only once every 13,219 miles, an improvement from once every 11,017 miles in 2018. Cruise said its San Francisco-based test fleet drove 831,040 autonomous miles last year, averaging just one disengagement per 12,221 miles. 

In the second half alone, however, Cruise’s disengagements fell to just once every 20,110 miles, a distance that’s equivalent to driving from San Francisco to New York nearly seven times. Chinese tech firm Baidu, a major autonomous player in that market, said its four-vehicle Silicon Valley test fleet logged 108,300 miles in California last year and just six disengagements–or one every 18,050 miles.

To be sure, the California numbers don’t correlate between different companies or varying approaches to testing and market plans. Much like Churchill’s quip about democracy being the “worst form of government except for all the other forms,” disengagement stats are of limited use but are the only public information available in the absence of standardized federal tests to evaluate self-driving technology. And on that basis, improvements by the biggest, best-funded programs indicate the technology is moving closer to commercial viability.

“It’s like a Yelp rating for a restaurant: The information can be useful but not definitive,” said Mike Ramsey, a senior research director for Gartner. “It's obvious that Cruise has put a ton of energy into getting better, and they are testing in a more difficult environment.” 

Waymo mixes its road tests between Silicon Valley and Bay Area suburbs and downtown San Francisco, while Cruise is fixated on mastering that crowded, congested city. Last July, Cruise CEO Dan Ammann conceded it wouldn’t meet an initial goal of launching a commercial ride service by the end of 2019 but said the company would more than double test miles in San Francisco, which it did. In January, Cruise debuted the Origin, an electric passenger shuttle with no steering wheel or pedals that will serve as its robotaxi vehicle. Separately, the company also received a permit this month to operate a driverless ride service in California, though it can’t yet charge customers for those rides.

“The cynical part of me says if it’s super important to GM to be compared against Waymo, they would have a really strong incentive to make sure that their disengagement reports improve a lot,” Ramsey said.  

A total of 60 companies with autonomous vehicle permits issued by California’s Department of Motor Vehicles filed reports for 2019 and drove a combined 2.88 million test miles of state roads. Waymo and Cruise alone accounted for 79% of those miles. 

“The disengagement metric does not provide relevant insights into the capabilities of the Waymo Driver or distinguish its performance from others in the self-driving space,” the Mountain View, California-based company said after the reports were released. “While most of the development, learning and validation of the Waymo Driver comes through billions of miles driven within our simulation environments, our real-world driving experience is primarily outside of California, in markets like Detroit and Phoenix.”

Tesla, which is working to launch a commercial ride network for its vehicles once it achieves what Elon Musk calls “full self-driving” capability, reported very brief testing in California. “In April, we operated one vehicle in autonomous mode to record one demo run on a 12.2-mile route around Tesla’s Palo Alto headquarters,” the company said in a letter to the DMV. “The route covered surface streets and highways. We did not experience any autonomous mode disengagements during this run and, as a result, do not have any disengagements to report for Reporting Year 2019.”

Apple, which has been testing autonomous vehicles in California for a few years but hasn’t clarified business plans for the technology, logged 7,544 test miles last year and 64 disengagements–once every 118 miles. 

San Francisco-based startup Zoox, aiming to compete with Cruise and Waymo, said it logged 67,000 miles of autonomous driving last year, mainly in the city, with a disengagement once every 1,600 miles.

Disengagement data “has historically been viewed as a `report card for self-driving companies. But as we have repeatedly said, using these statistics as a measuring stick is misguided,” Chris Urmson, CEO of self-driving tech company Aurora and the former head of Google’s self-driving car project, said in a blog post. “The self-driving teams I’ve led over the past 17 years have championed various attitudes toward on-road testing. While huge numbers of on-road miles may initially seem impressive, experience has taught me which approaches merely look like progress and which can actually move the needle toward real progress.”

In Aurora’s case, the company says it’s focused more on virtual testing, using advanced computer simulation to create more complex scenarios than the real-world driving may offer. Currently, it’s doing 735,000 virtual tests per day, 100 times more than the rate a year ago.

Follow me on LinkedInCheck out my websiteSend me a secure tip