WebAR using Tango and ARCore


Fabien Benetou @Utopiah, WebXR developer

Table of content

  • limitations of native AR
  • WebAR, the definition
  • status of the specifications
  • hardware support
  • browsers
  • libraries
  • comparison
  • market penetration
  • deconstruction of an experiment
  • what's yet to come
  • references

limitations of native AR

limitations of native AR (bis)

... and entire thread on that.

limitations of native AR (tris)

WebAR, the definition

(Chromium-WebAR running on Tango)

Starting with Argon Browser, 2011 (project started in 2009) GeorgiaTech

status of the specifications

AR extensions and modifications #254

hardware support

  • Microsoft HoloLens
  • meta2
  • Google Tango
  • iOS 11, iPhone 6> & above
  • ... arguably anything with a camera

hardware support (bis)

hardware support (tris)

aka "Why you pay $3K for the HoloLens yet it can be worth every penny."

Key concepts

  • Motion tracking
  • Room knowledge
  • (re)localisation

Motion tracking

Motion tracking, bis

Feature?

VIO

Structured light/Time of flight

Room knowledge

Relocalisation (ADF, anchor, etc)

browsers: all

browsers: Argon

https://github.com/argonjs/argon-app

browsers: Chromium-WebAR

renamed to WebARon*

browsers: WebXR Viewer

A VR and AR Project by Mozilla

https://github.com/mozilla-mobile/webxr-ios

libraries: AR.js

NFT

+

https://github.com/jeromeetienne/AR.js/

libraries: WebARonARKit

https://github.com/google-ar/WebARonARKit

libraries: WebARonARCore

https://github.com/google-ar/WebARonARCore

libraries: WebARonTango

https://github.com/google-ar/WebARonTango

GoogleTango

libraries: WebARonTango (bis)

  • vrDisplay.getPose()
  • VRPointCloud(vrDisplay)
  • getPickingPointAndPlaneInPointCloud(pos.x, pos.y)
  • vrDisplay.getADFs();
  • vrDisplay.enableADF(uuid);

cf https://glitch.com/edit/#!/webar

libraries: three.ar.js

Abstracting away from WebARonARKit and WebARonARCore

 THREE.ARUtils.getARDisplay().then(init);
 // ...
 arView = new THREE.ARView(vrDisplay, renderer);
 camera = new THREE.ARPerspectiveCamera( /*...*/ )
 THREE.ARView(display, scene.renderer)
 getPlanes()
 hitTest(x,y) with hitpoint
 THREE.ARUtils

libraries: aframe-ar

https://github.com/chenzlabs/aframe-ar

 <!-- Place the ar component on your scene, and the camera will use AR by default. -->
 <a-scene ar>
  <a-sphere radius="0.01" position="0 0.005 -0.5"></a-sphere>
 <a-scene>  

Why A-Frame works cf The future of Virtual and Augmented Reality for the web by Jordan Santell, ColdFront17

libraries: webxr-polyfill

  • class MyARDemo extends XRExampleBase {}

cf https://github.com/mozilla/webxr-polyfill/blob/master/examples/ar_simplest/index.html#L41

libraries: HoloJS

Probably easier to (soon) use BabylonJS + Edge

https://github.com/Microsoft/HoloJS (+ own attempt HoloLensWorkshopMICApril2017)

libraries: playcanvas-ar.js

https://github.com/playcanvas/playcanvas-ar

libraries

Yours?

HYPER-REALITY by Keiichi Matsuda

comparison

  • What problem am I trying to solve?
  • What hardware is actually required to solve it?
  • What's my audience?
  • What hardware does my audience use?
  • Is it a match?

Only then picking a browser then a framework makes sense.

market penetration

features at a cost vs popularity

deconstruction of an experiment

picking on non vertical surfaces + ADF + interaction

what's yet to come

  • anchors and markers might land soon in WebARonARCore, cf PR
  • better tracking
  • better unification: WebXR, WebAR W3C Workshop?

now even networked aframe-ar

references

Particular thank you to Iker from the GoogleWebXR team for constant support.

Augmented reality, discover ARKit, ARCore, Google Tango and HoloLens

VRLab Brussels / GDG Brussels

Videos online!

  • 2pm welcome, casual introduction
  • 3pm presentations
    • ARCore and Tango with the Unity SDK by Srinivas Rao
    • WebAR by Fabien Benetou
    • Beyond WebAR by Trevor Smith, Mozilla
  • 4pm open workshops (AR.js WebXR on ARKit, WebARonTango)
  • 5pm presentation with Q&A
    • Charles Boyle (aka Chuck Knowledge), Signal Garden co-founder (live from SF)
    • Omar Shaikh, Founder of Anstrom Tech (live from SF)
  • 5:30pm wrap up

meetup.com/VR-LAB-Brussels/events/236233748/

http://tinyurl.com/ARCore2017

Unpacked: https://fabien.benetou.fr/Testing/WebARonARCoreDevFestBE2017?action=reveal

Hands-on workshop after lunch (cf schedule), 10 spots only, S8/Pixel/Pixel2 recommended, laptop REQUIRED

ARCore http://arcore.glitch.me/

TangoCore http://webar.glitch.me/