The Final Frontier — Self-Driving Cars on the Streets of New York (and They've Already Been Here for 6 Weeks!)

With Ninth Avenue in Hell’s Kitchen this week named as one of the most dangerous streets in the United States, how are you feeling about autonomous vehicles (AV) being added into the mix?

A self-driving vehicle from Mobileye’s autonomous test fleet navigates New York City’s Times Square. Photo: Mobileye.

New York State’s Department of Motor Vehicles has given Mobileye the go-ahead for testing on the streets of the Big Apple. The company, owned by Intel, has previously tested its vehicles on busy streets around the world, from Israel (its home base) to China, Munich to Tokyo.

The company recognizes that New York is one of the world’s most challenging driving environments, which is why it is testing here. “Driving in complex urban areas such as New York City is a crucial step in vetting the capabilities of an autonomous system and moving the industry closer to commercial readiness,” said Professor Amnon Shashua, CEO of Mobileye.

A Mobileye autonomous vehicle riding on the streets of New York. Photo: Mobileye.

Mobileye’s camera-only subsystem AV is now successfully driving through New York City, while back in Tel Aviv, they are fine-tuning their radar/lidar system. When the cars go into production, they will use both systems.

Mobileye’s special permit in NYC allows the “safety drivers” in its cars to keep their hands off the steering wheel.

Jaywalking is common in many cities, but in New York City it is particularly rampant

The company has been driving hands-free in New York for six weeks, utilizing two cars, and plans to increase the number of vehicles on the city’s roads to seven in the coming months.

Mobileye is the only company currently holding an AV testing permit in the city. After extensive day and nighttime driving, they’ve identified seven things that make New York’s streets a major challenge — listed below:

Pedestrians. Jaywalking is common in many cities, but in New York City it is particularly rampant, and is coupled with a high number of pedestrians. An AV must make assumptions about the behavior of those pedestrians and factor those assumptions into its driving policy. Humans do this instinctively; machines must be programmed for it.

Driving behavior. When streets are clogged, drivers become impatient and aggressive. New York City drivers – especially cabbies and other professionals – are much more assertive than drivers in other cities.

View of the systems onboard Mobileye autonomous vehicle riding on the streets of New York. Photo: Mobileye.

Traffic density and road user diversity. Although car ownership in New York City is low compared with other large US cities, the number and variety of road users is especially dense. New York City has more than its share of cabs and limousines, buses, trucks, food carts, horse-drawn carriages, emergency vehicles, bicycles, scooters, skateboards – you name it.

Double-parking. Who’s parked and who’s not? The question is easier for a human to answer than a machine, and New York City’s population density contributes to a high number of delivery vehicles that must stop to unload. As a result, double-parking is ubiquitous. AVs struggle with this, although Mobileye’s AVs take clues from other road users to decide when to maneuver around.

A car is a car — who are you going to yell at if no one is driving them? Christine Berthet

Construction. New York City is one big construction zone, and Mobileye knows this thanks to all the data saved in its always-updating AV maps. While competitors rely on either their own test cars to build maps or spend millions of dollars driving special mapping vehicles, Mobileye receives data about blocked or closed lanes from cars already on the road (data it can, and does, license back to municipal services, too).

Tunnels and bridges. The island of Manhattan is connected to the surrounding areas via 15 tunnels and 21 bridges, many of which contain narrow lanes framed with bollards or cones – the Achilles heel of many an AV. In the face of all that traffic “furniture” and even multi-level roads, Mobileye’s crowd-source mapping technology and its well-trained sensing system understands all of this and handles it.

This city really never sleeps (the lights!). Though Paris gets the “city of lights” moniker, Manhattan is electrified at night. The visual noise and light pollution is daunting to an AV’s sensing system. Mobileye AVs handle it easily with only a bit of algorithm tuning.

A self-driving vehicle from Mobileye’s autonomous test fleet passes Radio City. Photo: Mobileye.

Shashua, the boss of Mobileye, was asked at a press conference if he thought New York was the toughest challenge. It looks like we lag behind India in technical difficulty: “Driving in India is something that I don’t see us doing in the next decade,” he said. When asked about any advice he had for New Yorkers dealing with self-driving cars, he added: “Nobody’s going to heed to my advice in New York — definitely not the drivers. We need to manage with what we have. We’re not going to change the DNA of New Yorkers.”

Christine Berthet, the founder of CHEKPEDS — started 16 years ago to reclaim Ninth Avenue from the hellish Lincoln Tunnel traffic — summed up the feelings of locals: “A car is a car: there are too many of them in our limited space. And who are you going to yell at if no one is driving them?”

Mobileye has released an unedited video showing their cars in action on the streets (and tunnels) of the city over the past six weeks. Take a look below…