This post is part of an ongoing monthly series of posts that will focus on our current efforts in the Museum’s Science Bulletins team to create and test prototypes of Hall-based digital interactions using AR and VR using our scientists’ digital science data, and to share some of the lessons we learn along the way.
In December, it was time to shift our attention from the earth to the stars! Not only did this mean looking at astro data, it meant shifting our attention to data that is spatial in nature (and exploring what it meant to interact with spatial data).
Our resource this time was the Digital Universe, developed by the Museum, which incorporates data from dozens of organizations worldwide to create the most complete and accurate 3D atlas of the Universe (from the local solar neighborhood out to the edge of the observable Universe).
The first challenge was finding an opportunity to leverage within the Hall and then taking a thin slice of the D.U. data to present in an interactive way. For our first astro prototyping, we decided to focus on Constellations.
There are four long information walls within the Lower Level of the Hall of the Universe. One is about Stars, with each section featuring its own constellation. We chose Orion to be our inspiration.
Looking up at the night sky, we experience constellations as a 2-dimensional shape, something flat, as if all the stars were the same distance from us. Their true 3-D nature remains hidden, difficult to observe. If we could shift our perspective, however, such as by moving deep into space, we’d see the constellation distort as we learn that some individuals stars are closer to Earth while others are farther away. With our first prototype for the Hall, we aimed to use AR to let visitors experience that distortion as both an engaging experience and an opportunity for learning.
During the first prototype, visitors viewed a virtual Orion constellation on a Tango handheld device, which they could move forward and backwards to see the constellation’s shape change. We saw the connection between the experience and the constellations, but for the visitors… not so much. Many thought the constellation on their screen was spinning when, in fact, the stars stayed in place – it was only their perspective that was changing. A suspicion I had was that constellations work as an abstraction – to help us make something concrete out of the star field – but that this abstraction can break down when turned into 3-D. A constellation consists of imagined lines – a celestial connect-the-dot. Turning a constellation into a 3-D experience means stretching or shortening the lines, as if the lines themselves were something physical being changed. But of course, the lines are still just imaginary, in this case representing not just connections between the stars but now also the different distances amongst them. All this focus on the lines makes the viewer forgot to notice what’s real, the stars, and their real locations in 3-D space. Making it even harder, perhaps, was asking them to break out of their 2-D framework using a device which reinforced a 2-D view (even though the represented data on the screen was 3-D).
We realized we could go into one of two directions – improve the 2-D experience on the Tango or break out into 3-D with an augmented or mixed-reality experience in a Hololens or a virtual reality in a Vive. 3-D seemed the right way to go but, while that was in development, we adapted something developed earlier to explore a different 2-D option out in the Hall.
To give visitors a less abstract way to understand that the points of lights are objects in relationship to one another, we used Your Face in Space (aka Starface). Using something made in 2014 in BridgeUp Hackathon, Your Face in Space might provide the required context while increasing the level of engagement. With Your Face, visitors using a laptop and its camera to map points around their face and turn those points into a constellation. These points are REAL stars, as seen from earth, in their real relationship, but with lines drawn to show not Orion but your magnificent visage. Once the new constellation appears, the visitor can rotate it to see different perspectives reveals its 3-D nature. While almost no one understood this idea from the first prototype, around half of the visitors got the idea through Your Face in Space. In addition, the older the visitor the more likely they were to understand it. This was a step in the right direction, but we suspected we could get it work even better.
A week later we broke out the Hololens. This time we wanted to learn if experiencing spatial data through an immersive 3D experience is more effective at teaching visitors about constellations sitting in a 3D space, while increasing engagement. Once again, we tested it with all age ranges, inviting them to experience something new and putting them in the Hololens. We invited them first to just look around – noticing the star field around them – and then look for and point to Orion (the pointing is as much to physically engage them as it is to signal us about the orientation of the AR). Once they did we asked the visitor to move around and notice if the shape changes and, if it did, why. This time, almost everyone got the idea – although not everyone – and visitors found it deeply engaging, with dwell times minutes longer than we imagined anyone would be interested.
Some (and far from all) of the key lessons we took away from this round of prototyping:
- VISITORS desire activities. Visitors jumped at the chance to try something interactive and new. “Would you like to try-” is often as far we got before visitors said “Yes!”.
- ASTROLOGICAL data is exciting. Visitors were thrilled to interact with astrological data, and spatial data, sharing their experience with their social group and helping them have their own experience.
- MEDIUM is the message. It was difficult to use Tango’s 2D screen to help visitors realize that constellations sit in 3D space. Switching to the 3D environment of the Hololens felt like a revelation, providing a better fit for the data – immersing visitors in a 3D experience to interact with and better understand spatial data.
Come back soon to see some of what we did, and learned in January.
