Google’s New Street View Cameras Will Help Algorithms Index The Real World – WIRED
Steve Silverman helped build cameras for two NASA rovers that went to Mars. In the less exotic landscape of a Google parking lot, he looks up fondly at his latest creation, bolted onto the roof of a Hyundai hatchback. The gawky assemblage almost doubles the carâ€™s height: four white legs holding up a vertical black stalk sporting eight cameras. â€œWe thought about covering it up, but weâ€™re kind of nerds,â€ Silverman says. â€œWeâ€™re proud of it.â€
Silverman and his team build the hardware that captures imagery for Google Street View, the project that since 2007 has put panoramas of more than 10 million miles of roads, buildings, and the occasional act of public urination online for all to see. The new camera design, the first major upgrade in eight years, started regularly patrolling the streets last month. The data thatâ€™s just starting to come back will strengthen Googleâ€™s digital grip on the world.
As you might expect if you think back to the camera in your 2009 cell phone, Street View imagery is about to get a lot clearer. Look forward to sliding through the world from your couch in higher resolution and punchier colors. But Googleâ€™s new hardware wasnâ€™t designed with just human eyes in mind. The car-top rig includes two cameras that capture still HD images looking out to either side of the vehicle. They’re there to feed clearer, closer shots of buildings and street signs into Googleâ€™s image recognition algorithms.
Those algorithms can pore over millions of signs and storefronts without getting tired. By hoovering up vast amounts of information visible on the worldâ€™s streetsâ€”signs, business names, perhaps even opening hours posted in the window of your corner deliâ€”Google hopes to improve its already formidable digital mapping database. The company, built on the back of algorithms that indexed the web, is using the same strategy on the real world.
The idea behind Street View is nearly as old as Google itself. In 2001, three years after the companyâ€™s founding, CEO Larry Page took a videotape heâ€™d shot driving around the Bay Area into Stanfordâ€™s graphics lab. He asked researchers there to figure out a way to summarize it in images, and they began a project dubbed â€œcrawling the physical web.â€ Its technology was absorbed by Google in 2006, when the companyâ€™s cars first hit the roads ahead of Street Viewâ€™s public launch the following year.
A decade later, Street View cars have snapped more than 80 billion photos in thousands of cities and 85 countries. The companyâ€™s conventional mapping data is even more extensive. But Google still hungers for a better index of the world. Jen Fitzpatrick, the vice president who heads the companyâ€™s maps division, blames that on us. â€œPeople are coming to us every day with harder and deeper questions,â€ she says.
The first time you searched Google Maps or Street View you probably typed in a street addressâ€”perhaps your own. Fitzpatrick says the company now gets tougher queries that require a fresher, more detailed digital model of the world, like â€œWhatâ€™s a Thai place open now that does delivery to my address?â€
She wants her service to handle queries that assume knowledge of what the world looks like: â€œWhatâ€™s the name of the pink store next to the church on the corner?â€ Googleâ€™s push to get us talking with its Siri-style virtual assistant encourages us to be more conversational in our demands. â€œThese are questions we can only answer if we have richer and deeper information,â€ Fitzpatrick says.
Googleâ€™s huge investment in machine learning and AI provides a natural way to get that information. Thanks to recent research inside the maps division, when a Street View car captures photos of a stretch of road, algorithms can now automatically create new addresses in the companyâ€™s maps database by locating and transcribing any street names and numbers. Street View was the first of Google’s product groups to use the company’s powerful custom AI chips, dubbed TPUs.
The team’s system has learned to figure out abbreviations, such as â€œAV.â€ for avenida, by taking hints from other signs in the country where theyâ€™re spelled out in full, and other clues in Google’s maps data. Software has also been trained to recognize business names, and is smart enough to ignore visual trip hazards like the giant Bridgestone logo that might dwarf the name of a tire shop.
Higher quality images coming from the new hardware now atop Googleâ€™s Street View vehicles will allow those systems to extract information like that more reliably. â€œFrom a machine learning perspective, everything gets better,â€ says Andrew Lookingbill, an engineer working on the technology. It will also help his teamâ€™s efforts to build new software even better at understanding the world. Theyâ€™re thinking about trying to automatically recognize different types of business from their appearance and reading finer-grained information like opening hours signs.
Decoding Street View imagery with algorithms can be especially useful in places where roads, cities, and businesses are changing fastestâ€”the less-developed economies where Google and its competitors hope to find their next few billion users. The government of India reported this year that it has recently laid an average of 14 miles of new road every day. Street View went live this summer in Nigerian megacity Lagosâ€”population 21 million. Fitzpatrick says that Googleâ€™s image-scouring algorithms could help translate the new imagery into a significant bump in map quality. Google sells ads inside maps, so new coverage and accuracy can translate into more revenue if they draw new users and usage to the service.
Google wants you to help feed its image-hungry algorithms. The tech industryâ€™s recent interest in virtual reality has made 360 degree cameras relatively cheap. This summer, Google began certifying some cameras as â€œStreet View ready,â€ meaning you can upload your own panoramas through the Street View mobile app to live on the companyâ€™s service. That footage will be processed by Googleâ€™s image recognition algorithms for fresh map data just like its own imagery.
Google is counting on crowdsourcing to make Street View data fresher than it is now. â€œThe expectation is that Google has the world indexed,â€ says Charles Armstrong, a product manager for Street View. â€œBut it never lives up to expectations.â€ Googleâ€™s Street View mobile app rewards individuals who contribute photos with virtual trophies, and it will even suggest local spots to take your camera. More significantly, Armstrong predicts companies, tourist boards, and even governments will soon be driving their own camera-toting cars to make sure the world gets an up-to-date view of their streets and cities.
All the upgrades to Street View could help Google maintain its prime position in digital maps. The company is the most prominent among the handful of leading global mapping projects. The other heavy hitters are HERE, owned by a coalition of German auto companies; TomTom, known for stand-alone GPS units and watches; and collaborative project Open Street Map. â€œEach one measures themselves against the others,â€ says Alyssa Wright, president of the US chapter of Open Street Map. (Appleâ€™s relatively young mapping operation licenses data from TomTom.) In a world where most of us carry GPS-equipped smartphones, maps data is important for much more than just directions. â€œMapping is fundamental to how we build our digital future, from autonomous vehicles to dating apps,â€ says Wright.
Street Viewâ€™s new cameras and Googleâ€™s push for crowdsourced imagery could also lead the company into new privacy controversies. Concern about Google making ephemeral public scenes into permanent internet fixtures has rumbled, and occasionally flared, since Street View began. Germany and Austria are largely invisible on Google Street View, and have been for years, after the company got in trouble for logging Wi-Fi data with Street View vehicles. Google’s fleet only recently returned to both countries. In 2012, Switzerlandâ€™s highest court ordered Google to cut down its cameras to prevent them from peeping over walls and to blur certain places such as womenâ€™s shelters.
Fitzpatrick flicks away the suggestion that higher quality imagery could lead to more privacy concerns. â€œWe havenâ€™t seen or heard of places where there are additional sensitivities,â€ she says. Google will continue to automatically blur faces and license plates on its own Street View images. But it wonâ€™t do that by default on crowdsourced footage, instead leaving it up to users to choose whether to use Googleâ€™s blurring technology when they upload new 360 photos.
How much more could Google extract from Street View using image processing algorithms? A lot.
Earlier this year Stanford researchers, including professor Fei-Fei Li, now chief scientist at Googleâ€™s cloud division, showed they could predict income, race, and voting patterns for US cities with software that logs the make, model, and year of cars in Street View photos. When asked if anything like that was planned at Google, a spokesperson would say only that the company is always looking for ways to use Street View data to improve the company’s platforms, including beyond maps.
Processing Street View images from Google and its users might also help the self-driving cars of fellow Alphabet subsidiary Waymo understand the world. â€œThe team collaborates on things from time to time,â€ is all Fitzpatrick will say about that. But her team has as much to gain in return from Waymo.
Back in that Google parking lot, camera-wizard Silverman confesses that trolling highways in a Street View car sporting one of his teamâ€™s devices isnâ€™t much fun. â€œAfter a day youâ€™re ready to not be a bus driver and go back to engineering,â€ he says. Just as self-driving vehicles would shift the economics of on-demand ride services, not having to pay people to be bored behind the wheel would be a boon to Street View. The companyâ€™s algorithmic index of the physical world may just be getting started.
Write a Reply or Comment:
You must be logged in to post a comment.