Next week at the annual American Alliance of Museums conference in St. Louis, I’ll be presenting with John Durrant, Marco Castro and moderator Lizzy Moriarty to “demystify VR content development and offer attendees the chance to get their hands on some VR tech.” In advance of the session (on Monday, May 8, at 8:45 am) I was asked by the Center for the Future of Museums to share a preview of some of the session, which you can read here or below.
Last fall we launched a new initiative at the American Museum of Natural History in New York City: develop recommendations for engaging visitors with modern science practices by adding digital layers to permanent halls. What this looks like on the ground is working with one of the Museum’s scientists (we have over 200) and then turning their digital specimens (CT scans, genomic data, astronomical observations) into a digital asset we can port into a variety of digital tools to be tested with the public.
For example, we tried various ways of using digital astronomical data to explain the three dimensional nature of constellations. When people look up at the night sky, all the stars seem to lie in a single plane, all the same distance from Earth. In fact, stars occupy a vast three dimensional space—each a different distance from our planet. If you could change your perspective by flying off Earth to somewhere else in space, changing the distance and angle between yourself and each of the stars, you would see Orion “distort”— in other words, the 2D picture we create by drawing imaginary lines from star to star would change shape.
See how long it took me to explain that? We wanted to learn if we could use augmented reality (AR) or virtual reality (VR) to get visitors there, faster. Working with a slice of our Digital Universe database, we created a digital asset that simulates a number of constellations, like Orion. Then we tested a variety of ways for people to interact with this digital simulation of space
1: THE TANGO EXPERIENCE: In our Hall of the Universe (HoU) Visitors viewed a virtual Orion constellation on a Tango handheld device, which they could move forward and backwards, to see the constellation’s shape/line change. Tango is like an iPad with one key difference: it knows where it sits in the space around it. This means, for AR, you can place augmentations in space and then use your Tango to walk around or (in the case of stars) among them.
RESULT: Failure. Visitors did not leave having learned that stars sit in a 3D space. We concluded that was in part because constellations are too abstract (the points in a constellation represent real stars but the lines between are just pretend). But what if we made the experience less abstract, something you’d notice was different if its shape changed, like your face?
2: YOUR FACE IN SPACE: It’ll take too long to explain here, but humor me and presume there’s a good reason why we have a computer app that lets you turn your face into a constellation. We took the app into the HoU and invited visitors to map points around a live image of their face, switch the star names on and off, and then rotate their perspective around the new constellation.
RESULT: Not there yet. On one hand, it seemed to work to ask visitors to use their own face as a metaphor for stars in relationship within a constellation; lowering the level of abstraction was effective. However, many visitors experienced the rotation of the constellation image as due to the constellation itself rotating (which is incorrect), not the visitor’s perspective shifting through space. What if it turned out that visitor’s misunderstandings about stars in 3D space are just being reinforced when shown through a 2D medium? And what if,, instead, we offered them, through a 3D medium?
3: ENTER HOLOLENS: Visitors now viewed a virtual Orion constellation (as well as three smaller constellations) through a Hololens device. (Hololens is an augmented reality headset that enables the wearer to see, and navigate, computer generated images or landscapes.) Walking back and forth, and around, visitors viewed the constellations as existing in a 3D space, with a backdrop of real stars.
RESULT: It worked! While the first iteration failed to communicate the core idea, and the second iteration was successful half the time, the Hololens version worked EVERY time. As the visitor walked around or through the constellation, the stars “moved” at different speeds, depending on their distance from the observer. But could we up the bar, designing the experience to require a visitor wearing Hololens to interact with other visitors, to make it a social experience?
4: ESCAPE THE PLANET: Over a four day design sprint, co-developed with Museum youth learners, we created a prototype of an escape room with an astro-theme: Escape the Planet. (Escape rooms are physical adventure games that require players to solve a series of puzzles.) One of the puzzles required a group of players to use a UV flashlight to find clues in posters that identified one particular constellation. A different player, wearing the Hololens loaded with a new version of the AR Constellation experience, had to look at the name of the closest star to Earth within that constellation (also known as its catalog number) so another player could record those digits and use them to open a padlocked case.
RESULT: Hololens users playing Escape the Planet maintained social contact with the rest of their group, and appeared to have done so more often and with more intensity than during the first three iterations. But was this due to features of the new version of AR Constellation, or due to placing it within a game?
5: STAND ALONE AR: The week after testing Escape the Planet, we took this latest version of the AR Constellation in Hololens back out into the Hall, specifically to watch how users interacted (or not) with the others within their party.
RESULT: Most visitors using the standalone AR said that wearing the Hololens did not affect the way they related with the people around them (in other words, they ignored them and focused on the AR Constellation experience). This is in stark contrast with the Escape the Planet players who not only reported a “heightened desire to cooperate” but expressed a need to share.
And so it goes. Now, a few months later, we are porting a number of our digital specimens into a holdable AR device called a Holocube. Do you think visitors would like to hold a constellation in their hand? It might be time for a new iteration…
