Prototyping Interactive Data Viz: 1. AR SHARK

The following is a deeper dive into one of the projects developed at the American Museum of Natural History in FY17 to help us better understand how to bring the digital work of Museum scientists to visitors through emerging media. Read the top-level findings from the year or carry on below. 

1. AR SHARK

Assets: CT scan of a mako shark, provided by John Maisey and John Denton

Technology: Hololens headset & Tango tablet

What we did: Working with CT scans of a mako shark and consulting directly with Museum scientists John Maisey and John Denton, we produced an animated, interactive digital asset for the Hall of Biodiversity to augment the mako shark model hanging overhead. We developed versions for Microsoft’s Hololens (a headset) and Google’s Tango (a tablet).

We conducted 10.5 hours of public evaluation over five sessions (125 people observed; 64 people interviewed).

Key finding: CT scans, a staple of modern scientific investigations, can form the basis of interactives that appeal to lay audiences, and have strong potential for bringing new life to permanent halls. But snags in the technology and overly complex directions can easily sabotage the user experience.

Other findings:

• We can leverage science assets, but the process is involved. Research-quality CT scans can be turned into interactive, public-facing digital assets. But the process requires considerable production time and technical expertise. Not all of the information is in a scan’s data (e.g. rigging a model so that it moves correctly). And to allow users to interact with huge datasets in real-time, we had to reduce the size–and thus the quality–of the final asset.
• Avoid packing in too much information, too many interactions. For the first version of AR Shark, we produced a “short” audio track introducing users to the mako’s anatomy part by part, and instructing users in how to interact with the specimen. But in the context of an unfamiliar device, this content proved to be far too cumbersome. Users took up to 10 minutes to get through the experience. We pared it down to a few simple voice commands (e.g., “swim,” “bite”). And with later prototypes, we found better ways of keeping the experience simple while still imparting an educational message. Achieving simplicity will be a consistent challenge.
• People love trying new gadgets. The Hololens was an attractor. Visitors wanted to try new technology and were willing to learn how to operate it. They especially enjoyed seeing the shark animated and giving it commands.
• Unfamiliar devices require facilitation. It took considerable facilitation to set up and guide each person through the experience. Language was often a barrier in communication. People had difficulty finding and following the shark through the Hololens’ limited view frame. Users were slow to master gesture commands, so we switched to voice commands in a later prototype. Voice commands seemed easier for users to pick up.
• Familiar devices allow quick experiences. The Tango tablet was more familiar to visitors, and required less facilitation. On the other hand, users spent less time with it; the novelty factor didn’t hold them.
• AR can enhance Hall content. Although we were not able to easily or perfectly overlay content onto an exhibit (i.e., the CT scan onto the overhead mako model), visitors liked seeing the scan in the context of the exhibit. Many turned their Hololens and Tango devices to see if other exhibits were augmented.
• Limited social interaction. The Hololens prototype was a single-user experience, although users often communicated with their social group even as they wore a headset. To be more inclusive, we set up preview screens so that others could see what the user was viewing. Tango allowed for multiple users, but the screen was small; there was limited social interaction around the experience.

Read more about this series of prototypes here or learn more about the other prototypes below:

1. AR SHARK (learn more)
2. CT SCANS WITH HOLOCUBE (learn more)
3. VR WEEVIL (learn more)
4. AR CONSTELLATIONS (learn more)
5. ESCAPE THE PLANET (learn more)
6. AR SCALES OF THE UNIVERSE (learn more)
7. MEAD FESTIVAL 360 VIDEOS (learn more)
8. PALEONTOLOGY 360 VIDEO (learn more)
9. CT MUMMIES (learn more)
10. TREE OF LIFE (learn more)
11. ASTRO BULLETIN GESTURE-BASED INTERACTIVE (learn more)