https://www.theverge.com/2017/8/29/16219696/google-arcore-augmented-reality-platform-announce-release-pixel-samsung
GOOGLE ARCORE GIVES ANDROID USERS AUGMENTED REALITY WITHOUT TANGO
Last week in San Francisco, Google showed me an app called Oz. Oz is a kind of augmented reality picture book: it places animated characters from The Wizard of Oz into the physical world, as viewed through a smartphone camera. I’d tried it a few months earlier at Google I/O, running on the Tango AR platform, and the content hadn’t changed. But the experience was far more interesting — because for the first time, it could run on a phone that I use every day.
This version of Oz was built on a system called ARCore, which debuts today as a limited preview. As its name suggests, ARCore is Android’s equivalent to Apple ARKit: a baked-in augmented reality platform for developers. Where Tango’s custom hardware requirements have left it languishing on mediocre smartphones, ARCore is less powerful but more accessible. It’s launching on the year-old Google Pixel and Samsung Galaxy S8 phones, supported by Android 7.0 Nougat as well as its recently released successor Android Oreo. An official launch is loosely planned for this winter, when Google promises ARCore will work with 100 million existing and upcoming devices.
Google is under obvious pressure to compete with Apple’s lightweight version of AR, which has produced a small wave of clever experiments since its announcement in June. But the company’s head of augmented and virtual reality, Clay Bavor, describes ARCore as an intentional long-term outgrowth of Tango. “Our approach with Tango was to un-constrain ourselves,” Bavor says. “That really let us learn a lot, figure out what the use cases are, and push forward the technology — out ahead of what would have been possible with standard smartphone hardware.” Google released a couple of consumer products with Tango, but they’ve had little mainstream appeal. Meanwhile, he says, Google was taking key pieces and adapting them for ordinary phones, where they could work without Tango’s two extra cameras.
Now that Google considers ARCore good enough for a wide release, Tango-branded devices — like the Asus ZenFone AR that came out just a few weeks ago — seem to be a thing of the past. “I think Tango fades into the background as more an enabling technology that kind of works behind the scenes,” says Bavor. It’s not technically dead; Google will supposedly keep pushing for new and better cameras based on Tango tech, like a depth sensor. But these would be added to phones as an element of ARCore, not a discrete feature.
As Google describes it, ARCore has three basic components. The first is motion tracking, which estimates a phone’s relative location based on internal sensors and video footage — so you can pin objects in one place and walk around them. The second is environmental understanding, which uses the camera to detect flat surfaces. The third is light estimation, which helps virtual props cast accurate shadows and otherwise fit in with their surroundings. Google is also showcasing a few semi-interactive tricks. In a simple demo app, you can set a little Android mascot down in a virtual forest, where it’ll wave when you hold your phone to its face. And in Oz, the Cowardly Lion jumps in fear if you turn the lights out.
These are the same kind of capabilities you’ll find in Apple’s ARKit, and I haven’t spent enough time with either platform to rigorously compare their quality. But my controlled demo at Google’s offices was one of the best experiences I’ve had with phone-based AR. Objects didn’t jitter when I walked around them, the way I’ve seen even some official ARKit demos do. Props were surprisingly good at popping back into place when I turned the camera away or covered it up, although they couldn’t recover when I lowered the phone and strolled around the conference room.
Android developers can make augmented reality apps without ARCore, just like iOS developers can do without ARKit, and Google isn’t building for every possible use case. ARCore focuses on detecting planes, not human bodies or facial features, although characters in Oz do respond when the camera sees a face. But Bavor promises that for basic AR tracking, Google has optimized ARCore’s performance more than an outside developer could do. “The level of quality, the capability, the things it can do, I think will be several levels above the other solutions out there.”
/cdn.vox-cdn.com/uploads/chorus_asset/file/9135835/arobertson_170829_1953_0001.jpg)
Google is also trying to make it easy to build for ARCore. Experienced developers can use Java/OpenGL, Unity, and Unreal, and people who are new to 3D design can export ARCore objects from Google’s Tilt Brush VR painting app, or the VR modeling tool Blocks, which Google launched last month. Google is also releasing two experimental, AR-focused builds of Chromium: an Android-based web browser using ARCore, and an iOS-based one based on Apple’s ARKit. I used the Android browser to test a limited version of shopping site Wayfair’s furniture preview tool, which exists as a dedicated app for Tango. It wasn’t quick to load, but once it did, it worked about as smoothly as the app-based equivalent.
Not all Tango tools adapt so well to ARCore. One app called Constructor, for example, relies on Tango’s dedicated infrared depth-sensing camera to create detailed 3D meshes. “The environment understanding, as good as it is, is really kind of detecting surfaces to place things on, as opposed to the full 3D structures,” says Bavor. ARCore also has to estimate scale based on the camera feed, while Tango directly measures distance. Things like Wayfair’s furniture previews might be less accurate as a result, although AR director of product Nikhil Chandhok says that “for all the apps that we think that users want,” the difference is negligible.
/cdn.vox-cdn.com/uploads/chorus_asset/file/9135279/oz_5113.jpg)
What do users want? After three years, Tango developers have found some fun and interesting things to do with AR. Bavor says that people “light up” when they place impossible objects into the world, and interior design apps seem like a natural fit. On ARCore, someone even hacked together an app for Google’s complicated espresso machine — if you hold up your phone, you’ll see instructions like “Put your grounds here” or “Don’t touch this piece, it’s hot” overlaid on the camera image.
But the best AR apps might not emerge until other Google products advance, too. Combined with more accurate mapping data, Bavor says, ARCore could let your phone point out specific buildings or street corners. He also brings up Google’s recently announced “visual positioning service,” which is supposed to pinpoint indoor locations within a few centimeters. With VPS, you could conjure an AR prop and come back to it much later, or even leave it for someone else to find.
Google’s augmented reality program could also intersect with its push for visual search. One of the ARCore team’s members is Jon Wiley, formerly the lead designer of Google Search. Now the company’s director of immersive design, he thinks combining ARCore with a visual search tool like Google Lens could pull human-computer interaction more toward the “human” side of the spectrum. If smartphones are going to follow our thought processes and not the other way around, they need to see the world like we do, Wiley says. “Getting the phone and getting the real world to line up is an incredible technical challenge, but it also offers the opportunity to have a much more intuitive interface.”
For an example of how this might work, imagine searching for instructions — say, a guide to that complicated espresso machine — by showing Google a picture of the object. Visual search could identify it automatically, and augmented reality could offer an overlay of instructions, instead of a link to a YouTube video or written manual. “We're working very closely with the Google Lens team, and I see ARCore as one of the many ingredients that will go into experiences like Lens,” says Bavor. “Not anything to announce on that right now, but let's just say we think ARCore is going to make all that stuff more interesting, more powerful, and more useful for people.”
None of this will be happening in the near future, and we shouldn’t assume too much yet about ARCore’s ultimate reach. Google says it’s working with Huawei, Asus, and LG, among other manufacturers, to reach its 100 million goal this winter. But the company promised a similar laundry list of partnerships for the Daydream VR platform, and several of those phone makers still haven’t delivered. That said, ARCore is in a better position than Daydream. It’s being released right away on the popular Galaxy S8, and you don’t need special accessories to use it. ARCore can also benefit from the work iOS developers have done with ARKit, if it’s easy enough to port their apps to Android.
For all the work Google has done on Tango, its biggest opportunity isn’t making Android the most “powerful” AR platform. It’s growing the entire AR space enough that people are constantly using it with services like Maps and Lens, regardless of operating system. “We're here to build great products that a lot of people use, and that likely means for those applications, being where the users are. And that includes iOS,” Bavor says. Augmented reality offers an entirely new way of looking at the world — and no matter what kind of window you’re looking through, Google is designing the tools to interpret what you see.
Update 12:30PM ET: Google tells us that ARCore will run on Pixel phones running Android Nougat, not just the newer Oreo.
:format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/56348639/htc-htc-1.0.0.jpg)
/cdn.vox-cdn.com/uploads/chorus_asset/file/8940751/20170727_htc_u11_red_vladsavov00115.jpg)