Fabien Benetou's PIM | Tools / VirtualReality

Own motivation

Wiki.VirtualRealityInterface

Own projects

With users

storyboarding tool, private beta beyond https://github.com/Utopiah/StoryboardVR
workshops including VRLab Brussels workshops http://vrlab-brussels.info

With prototype

Note that I have more than 150 prototypes and PoCs on http://vatelier.net/MyDemo/

Google Poly in VR search, edition and share via URL
- http://vatelier.net/MyDemo/aframe-material/boilerplate/searchthenedit.html
- add url save/share (cf discussion with Thomas)
- use Super Hands or similar for edition (scale and move meshes)
- current limitation being lack of WMR Edge support for AFrame materials
  - either fix it (cf PR) or use Firefox + SteamVR for WMR (works just have to make sure dom.vr.enabled is true in about:config)
VR explorable of my ExtendedLayeredModel
- http://jsbin.com/melawic/edit?html,output
scribe to text https://vatelier.net/MyDemo/ApainterOCR/
1. using A-painter to freely write
2. using the camera position to snap a screenshot with the most legible orientation
3. convert the image to text
immersive video player (webm URL in large curved screen with adapted lights) with annotation (e.g. audio recording with seek time)
- https://vatelier.net/MyDemo/LocalVideoPlayer/ working on desktop and mobile
exploring Object.getOwnPropertyNames(AFRAME) as a WebVR experience
- cf WorkshopWebVRAsExplorable#getOwnPropertyNames
- use https://github.com/kfarr/aframe-city-builder with Google Blocks made Scratch Blocks
- see also for visual programing in 2D
  - https://wiki.scratch.mit.edu/wiki/Blocks
  - https://github.com/LLK/scratch-blocks
vrify, Firefox Nightly WebExtensions to add VR interface to your local filesystem
- https://github.com/Utopiah/vrify/
- see also JavaScript#WebExtension
Cube homage
Application to Valve, a bit of a portfolio a la Inside Out (Vive only)
WikiBrainMapping and more generally Cognition
- in fact just 3D for the moment and ... not really mapped either
my silly on-going "VR business card" VR (just added here to force me to fix it)
https://github.com/Utopiah/aframe-triggerbox-component
https://github.com/Utopiah/ThetaWebControl
VirtualRealityInterface
Bobsleigh
VRInceptionSchool
- could provide a room scale version via e.g. https://github.com/caseyyee/aframe-ui-widgets
VRLab
http://vatelier.net
minimalist headset http://www.thingiverse.com/thing:1419991 cf laser cut prototype by Camille
- https://vrlab-brussels.info/wiki/DYIHeadsetFirstWorkshopQuest/DYIHeadsetFirstWorkshopQuest
Cookbook:VRView360
- e.g. (:vrview360:)/pub/360/EteChezPierre.jpg(:vrview360end:)

See also my VR explorations as short Youtube videos

Without prototype

Online museum of tools to explore and think further
- e.g map, boulier, sextan, astrolabe, etc
- not showcasing the tool, instead making the visitor use it to reach a goal
- eventually solving puzzle requiring to combine some
Aframe inVR live coding environment
- keyboard locator https://jesstelford.github.io/aframe-video-billboard/
- virtual keyboard and text editor https://github.com/cjroth/aframe-keyboard
- link traversal as reloading trick in Nightly https://blog.mozvr.com/connecting-virtual-worlds-hyperlinks-in-webvr/
- save camera position http://jsbin.com/xobava/edit?html,output
- modify objects e.g. text editor https://github.com/wmurphyrd/aframe-super-hands-component
- (to try) https://github.com/ngokevin/aframe-sketch
- see also VRInceptionSchool
networked games hub with profile, chat, notification of friends, top scores and component to include and use for experiences
photos and 3D scan memorabilia as a museum inspired timeline
Tron clone using room scale for movement
Google Earth VR with personal content
- cf Trips.Listing
experience feedback
- under or on the side of a webVR experiment, provide an annotation tool
  - e.g. 140 characters with visual selection tool on a predefined format 600x300px, an equivalent of post-its, can be done live (network synchronized, assuming both computers have the minimum spec) with chat or recorded to be sent and checked later
generated and tweakable civilizational collapse virtual worlds based on models e.g. HANDY or Wikipedia:World3 (as learned in "Tout va s'ecrouler") and live updated feeds
BlockBuilder/InventingOnPrinciple/RapidPrototyping in VR
- with live coding, live reloading, easy forking, easy sharing, collaborative debugging
- motivated by own Principle, using JSBin then BlockBuilder, reading "Inadequate Interface" in http://www.wired.com/2016/04/magic-leap-vr/ and earlier discussions with Brian Peirs (RiftSketch) then Pierre
brain imaging interface at FABLab
- https://cpanel.emotivinsight.com/epoc/document.htm
  - required PHP webserver running on localhost, apparently supporting Linux
  - http://www.myndgazer.com/neurofeedback/
- https://github.com/metzel/Emotiv-Browser-SDK-js-example
in VR transaction using Bitcoin (via e.g. http://)bitcoinjs.org) cf vAtelier Gaze2Pay proof of concept

From personal use

AFrame
- lifecycle including https://aframe.io/docs/master/core/component.html#lifecycle-methods and related issue https://github.com/aframevr/aframe-site/issues/242
  1. AFRAME.registerComponent()
  2. Component.init()
  3. Component.update(oldData)
  4. Component.remove()
  5. Component.tick(time)
  6. Component.pause(), Component.play()
  7. scene.hasLoaded
- small demos http://vatelier.net/Category/Aframe
- workshop http://vrlab-brussels.info/wiki/ThirdWorkshopQuest/PostWorkshopNotes
- entire presentation
  - as slides in VR http://vrlab-brussels.info/wiki/Presentations/First
- D3js
- multi-users
- to explore
  - capture for preview https://github.com/ashconnell/three-sixty
  - additional controllers https://github.com/donmccurdy/aframe-proxy-controls
- explored
  - sound https://aframe.io/docs/components/sound.html
    - works except on iPhone, requries user action
  - physics https://github.com/ngokevin/aframe-physics-components or https://github.com/donmccurdy/aframe-extras/tree/master/src/physics
    - mostly works but no texturing of objects, animations also do not work (e.g. impossible to build a proper pinball machine)
  - networking https://github.com/ngokevin/aframe-firebase-component
    - works perfectly
JavaScript#ThreeJS
in VR (motivated by SundayGeekBrunch2016InventingOnPrinciple)
- levels for engines
  - https://docs.unrealengine.com/latest/INT/Engine/Editor/VR/
  - https://labs.unity.com/project/introduction-to-unity-s-editorvr (Tweeted @Unity3D)
  - Project Sansar Creator Preview (testing access requested)
  - https://github.com/Unity-Technologies/EditorVR
- modelling
  - https://www.reddit.com/r/virtualreality/comments/4qnt3l/3d_modeling_in_vr_the_way_god_intended_it/
  - CAD + video chat http://www.middlevr.com/improov/
  - http://threejs.org/editor/ with webVR mode
- live coding environments
  - http://www.primrosevr.com
  - http://toplap.org/live-coding-virtual-reality/
    - http://cognitivevr.co/blog/programming-in-virtual-reality/
  - https://github.com/lukexi/rumpus
  - inDeck in Lua https://github.com/jmiskovic/indeck
  - own homage to Papert in Hubs https://twitter.com/utopiah/status/1264131327269502976
  - few more to expand https://twitter.com/utopiah/status/1304515753278930953
- learning
- texuring
  - V�Artist
- others
  - https://github.com/brianpeiris/RiftSketch
  - https://github.com/richardanaya/aframe-keyboard
  - potentially playcanv.as with https://playcanv.as/p/iUgfldxp/
  - https://news.ycombinator.com/item?id=8411638
  - http://www.archean.io

Timeline

cognitive science studies during SC02 around 2004
- during which I read one of my favourite research paper in cognitive science namely "Where brain, body and world collide"
AR extension Overlay in late 2009
1st note from at least WithoutNotesFebruary11
applied to Oculus in 2013 because... why not
- "At this time, and given your lack of work status in the US, we have nothing to offer you. Please watch our website for potential openings as we grow."
tried Oculus in later 2014 via Hello Bank popup store in Brussels
1st event attended BeVRJanuary2015 then 1st hackathon VRHackatonBrussels2016 part of Portfolio
tried to hire someone to make ThreeJS/PmWiki interface work for me via Job4Bitcoin in March 2015
tought several workshops in 2015 via OpenTechSchool Brussels and at least 3 in 2016 via VRLab Brussels
vAtelier.net started in 2015 motivated by the early 3DVisualization by myself due to the failed hiring for the job
- especially Demos and LessonsLearned
GoldmanSachs early 2016 report considered for ManagingMyInvestments
vrlab-brussels.info with its numerous workshops since early 2016
- actually complained 1 year before during BeVRJanuary2015 about the lack of "home-brew" projects!
2nd hackathon, 2nd "win" at VRHackatonUtrecht2016 in June 2016
1st hardware workshop resulting in 3 new designs in June 2016
1st large (~20 persons without previous setup) multiplayer demonstration during CafeNumeriqueBXL in June 2016

Beyond that see VR

Vive

https://www.reddit.com/r/Vive/
http://www.vronlinux.com/articles/adventures-with-openvr-and-the-vive-on-linux.8
- own related questions https://www.reddit.com/r/Vive/comments/4rhxbd/linux_anybody_running_webvr_experiments_eg_aframe/?st=iqarlur3&sh=189301ed
Aframe
hardware
- https://www.reddit.com/r/Vive/comments/4f5axk/a_comprehensive_guide_on_lighthouse_mounting_so/
- https://vrperception.com/2016/03/08/seated-htc-vive-experiences-with-one-lighthouse-station-is-possible/
- see also the reddit post related to getting extra makers
  - now more officially https://partner.steamgames.com/vrtracking
to test
- https://github.com/LibreVR/Revive/wiki/Compatibility-list
- Archean
- http://www.whirligig.xyz video player (via Michiel)
tested
- ~~Budget Cuts (via Pierre)~~ quite fun, puzzles in VR, always amazing to hide behind the edge of a wall
- ~~own projects~~ works!
- ~~VRTX~~ works but still quite tricky, would have to follow a tutorial first
- ~~The Lab~~ lots of neats demo including a fun arcade style drone themed game
- ~~AltSpace~~
  - ~~own content in AltSpace e.g. http://vatelier.net/MyDemo/AframeAltSpace/~~
- ~~Aframe content with contollers e.g. http://output.jsbin.com/pocacis/~~
design tips
- https://www.reddit.com/r/Vive/comments/4rik1l/a_month_ago_i_left_my_job_at_microsoft_to_focus/d51ln4c?st=iqbohge1&sh=abde0fbc

(:vrview360:)/pub/360/ViveAtHome.jpg(:vrview360end:)

Design & UX

Josh Carpenter
- https://blog.mozvr.com/quick-vr-prototypes/ with template
- SFHTML5 talk https://www.youtube.com/watch?v=ZOaOYTOpwyM
Mike Alger
- GoogleIO2016
- http://vatelier.net/Main/LessonsLearned
the VR Book Human-Centered Design for Virtual Reality
http://vrhig.com
http://www.uxofvr.com

Experience reviews

VRTX
- ✔: conveniently select (edges, faces, etc)
- ✗: low poly load, no scaling of object
DesignSpace
- ✔: preloaded maps, easily add objects
- ✗: table outside of the chaperon
Fantastic Contraptions
- ✔: physics to solve puzzles
- ✗: can't recombine previous part of solutions
NVIDIA FunHouse
- ✔: fluids on virtual HMD, hairs, broken objects
- ✗: nothing particularly fun
SoundStage
- ✔: node based interface
- ✗: links between node staying too long
UnseenDiplomacy
- ✔: non Euclidian space, forced starting point with countdown
- ✗: key elements to solve puzzle outside the chaperone
getspace
- ✔: ?
- ✗: ?
Yours?
- ✔: ?
- ✗: ?

To explore

video
- video compositing with effects and transitions https://github.com/bbc/VideoContext#videocontext
  - and more transitions https://gl-transitions.com/
- effects https://github.com/wizgrav/aframe-effects#aframeeffects
- environment detection https://www.microsoft.com/en-us/research/project/roomalive/
  - with e.g. particle systems https://github.com/IdeaSpaceVR/aframe-particle-system-component
webVR replay
- https://github.com/capnmidnight/replay-telemetry
- https://github.com/mrdoob/webvr-replayer
~~own projects using Aframe in Altspace e.g. https://output.jsbin.com/lexaduq via https://github.com/AltspaceVR/aframe-altspace-component~~
- done, works well
~~https://github.com/stackgl/shader-school~~
- cf Meetup event http://www.meetup.com/VR-LAB-Brussels/events/232559865/
- Shaders
5 take aways from Mobile World Congress 2016 by MediaCom via Melanie
GfK 2016 Tech report via Zineb
Exki bet
- 22 January 2016 Brussels
  - Random stranger after a quick discussion asks me where VR will be in 5 years.
    - My prediction is : in 2 years everybody in Exki Schumann will have tried it at least once, in 5 years most people there will spend at least 30min per month in virtual reality.

Virtual Reality {Tools}