Mozilla Hubs {Tools}

I like to take notes and sketch new ideas. It helps me to organize my thoughts and, hopefully, have more and better ideas.



This wiki holds those notes but manipulating them and the structure is not as natural as post-its. What if we could do it in VR? What if I could invite you in?



Relying on Hubs I tried the following explorations.

Exploring my triage and annotation process

All the following explorations have source code available in their description. Following the Principle of sandboxed explorations all are independant from each other. There is currently no complete integration of all in one system. I planned to do for the most interesting ones related to knowledge mangement i.e. sketching from reMarkable (ideation), sorting Github issues of the project itself (planing), triage of papers for syncing PDFs to reMarkable (research) and finally changing the wiki own structure (groups, tags, etc).

To explore

• real world mapping
  • using a specific set of coordinates, a constant zoom level and thus area
    • each set has a single URL that is a Hubs room connected to 4 other rooms
      • building and roads gets extruded as a scaled gltf, ideally as scene that can't be moved
    • e.g https://www.openstreetmap.org/#map=19/50.84250/4.38919
      • lattitude 50.84250, longitude 4.38919, level 19 = ~300m2
• shared 1 page "browser"
  • render a very long page (e.g. ArcGIS Story) to an image
  • transform that image to a video,
    • using convert to crop and ffmpeg to assemble it,
  • then make interactable buttons where you "stand" on to scroll by seeking in that video

From personal use

My collection of utillity functions for Hubs is available for as a Gist, a WebExtension and an optional module on my own Hubs Cloud instance.

• clone the pinned objects of a room thanks to objects.gltf
• to efficiently test on standalone, assuming adb and remote debugging are allowed
  • start the browser adb shell monkey -p 'com.igalia.wolvic' -v 1
  • activate remote debugging then load the Hubs URL from the desktop appending ?vr_entry_type=vr_now
    • note that microphone prompt might still be needed
  • use scrcpy to visualize the result from the HMD
• remove number of media per room, default to 20 document.getElementById("media-counter").setAttribute("networked-counter", "max", 100)
• to get a usable stream from a PeerTube instance get the m3u8
  • e.g https://video.benetou.fr/videos/watch/HASH becomes https://video.benetou.fr/static/streaming-playlists/hls/HASH/1.m3u8 and can be pasted directly into a Hubs room
    • HASH can be found via the embed code, not the short URL format
  • see Ffmpeg on how to stream a desktop (or desktop of a container) using x11grab
• for faster testing use ?vr_entry_type=2d_now on a room URL to enter in 2D mode, skipping some (but not necessarily all) steps
  • cf Hubs Query String Parameters for more
• Hubs Cloud
  • to run your own client https://hubs.mozilla.com/docs/hubs-cloud-custom-clients.html on your cloud instance
  • disabling then enabling again CDN Workers allows to force S3 deployment from the EC2 instance to hub.html
• scripting from the console works
  • document.querySelector("#avatar-rig").object3D.position.x += 1 translates your avatar 1m along the X axis, other connected can see it
  • by default position, rotation and scale are networked. Everything else should be double checked against NAF schemas
    • object from Spoke are not networked, they are unpacked as a tree of threejs Object3D that be traversed but modification will only be local
• NAF schemas on what components get effectively networked are available via NAF.schemas.schemaDict
• templates of NAF schemas can be modified via NAF.schemas.add()
• window.APP.hubChannel.sendMessage("Hello world!") to send a chat message
• entities with [media-loader] can be quiered via el.components["media-loader"].attrValue.src.match(hash)
• entities with [networked-avatar] can be quiered via el.components["player-info"].displayName.trim()
• adding a media object (video, glTF, mp3, jpg, etc) can be done by adding a new entity to the scene then correctly setting 2 attributes
  • to load the media itself el.setAttribute("media-loader", { src: url, fitToBox: true, resolve: true })
  • to be visible and interactable by all el.setAttribute("networked", { template: "#interactable-media" } )
  • media object will get a partial hash (non unique) that be used to be found back in the scene.
    • if the same URL is used to upload an object it will not be updated, adding an anchor with a different timestamp will force to re-upload it to the server
• networked animation can be achieved via AFRAME.ANIME.default.timeline()
  • using e.g. {targets: star.object3D.position, autoplay: false} then adding animations
  • this also works on #avatar-rig this allowing for smoothly moving the camera and #avatar-pov-node to rotate it
    • using the camera mode allows to remove the interface
• to move a random entity it is important to own it but also stop potential physic movements
  • NAF.utils.getNetworkedEntity(ball).then(networkedEl => { NAF.utils.takeOwnership(networkedEl); networkedEl.components["set-unowned-body-kinematic"].setBodyKinematic(); /* do things */ })
  • ownership can be tested via NAF.utils.isMine(networkedEl)
• scene object in Spoke can be found in Hubs via their threejs Object3D name
  • useful to dynamically position objects in a scene relative to static objects
• avatar can be changed by id using window.APP.store.update({ profile: { ...(window.APP.store.state.profile), avatarId } });
  • for test a random avatar can be used using window.APP.store.resetToRandomDefaultAvatar()
• entities with [media-loader] can get a new source (of the same typp) using el.setAttribute("media-loader", "src", "https://domain.tld/newurl")
• to add Earth-like gravity to an object el.setAttribute("body-helper", { type: "dynamic", gravity: { x: 0, y:-9.8, z: 0 } })

The code snipped search on Gist is experimental. Parsing for hash and URL parameters is only partially encoded.

Other explorations

• Tweets mentioning MobillaHubs
  • e.g. object snapping
• Tweets mentioned byHubs (older account)
• Gists mentioning Hubs
• Scripting Hubs Utils as web extension with some utils

See also

• Hubs extensibility efforts
  • https://github.com/MozillaReality/hubs-plugin-api
  • https://github.com/mozilla/hubs/pull/2562
• other tools
  • Blender
  • JavaScript
  • Extended Reality
  • Virtual Reality
  • Programming
• https://redmonk.com/rstephens/2020/04/30/deserted-island-devops/

Note

My notes on Tools gather what I know or want to know. Consequently they are not and will never be complete references. For this, official manuals and online communities provide much better answers.