Augmented Wearables and the Future of Museums

[NOTE: Below is a re-blog of my column on DMLcentral where I first introduced the idea of here/there and me/us. I later revisited and revised the concept – to the more obvious me/we – after the New York Times sent its subscribers copies of Google Cardboard, which you can read here.]

Augmented Wearables and the Future of Museums: HoloLens and Oculus Rift and Google Glass (oh my!)

Wired’s recent cover article, “Microsoft in the Age of Satya Nadella,” is a fascinating piece about how the software giant is aiming to reposition itself to remain dominant in the next operating system revolution. They missed the boat when the rest of us migrated from desktops to mobile devices but this time their sights are focused on the next potential disruptor: augmented reality wearables. And Microsoft’s unanticipated new play in this space was announced this past January: Project HoloLens.

The article rightly grouped HoloLens with Google Glass, Oculus Rift and the largely-unknown Magic Leap. It’s hard to predict which of any of these devices will tip with the public (none are actually available on the consumer market), but it’s clear that if a significant number of users drift from mobile operating systems to augmented wearables, the potential impact on museum-based learning is significant. But what sort of impact?

Unfortunately, these devices get lumped together as if the user experience were all the same. But they’re not. And the different user experiences suggest different areas of impact, in museums and elsewhere. To get my head around all this, I mapped the four devices against two axises – space and people – and created the following chart, which I named, for better or worse, the Mooshme Matrix of Place-based Augmented Devices:

The vertical axis of space charts “here” (the augmentation deepens the user with their surrounding physical space) to “there” (the augmentation transports the user to a different space, often referred to as virtual reality). The horizontal axis of people charts “me” (the augmentation is for a single user OR each user has a personalized experience) to “us” (the augmentation connects the user with the people around them, their social network, and/or other users of the device).

Before I fill the chart, and explore what their placements suggests for the future of museums, allow me to briefly introduce each one.

MEET THE PLAYERS

1. Google Glass has received the most attention. The user wears something similar to a pair of glasses; a small screen visible only to the user appears above one eye, projecting 2D information, primarily text or images. I took a friend’s pair for a tour of D.C. museums last spring and wrote about my experience. I used Glass to take photos of my trip, map point-to-point directions, called up Wikipedia to search topics, and more; it felt like the next step in integrating my daily activities with the world of web-based resources. Google Glass is currently on hold and no longer in public beta.

Okay, Glass. Take photo!

2. Oculus Rift was developed to be an immersive virtual reality headset for gamers. I first tried it in 2013 at a gaming conference. I sat at a chair and strapped the bulky device around my head. The Rift’s screen surrounded my field of vision, like a personal Imax screen. Using a keyboard I first explored a house on a hill – very peaceful – then launched into a military game where I tried to survive a gunfight for more than a few seconds. Midway through the experience I added a headset and, from that point onward, I was no longer at the conference. I was still in my body – as I turned my head from right to left I could look all around my new visual environment – but it was a transporting experience. Oculus Rift was purchased by Facebook in 2014 for more than $2B.

3. Project HoloLens’ promotional video gives a good sense of how Microsoft wants us to think about it: a device that places shared, holographic-style 3D-objects (which can be manipulated) around users wearing the lens. It is slated for a public release later this year, with Minecraft often discussed as the killer feature.

4. Magic Leap is still somewhat of a mystery. Attention from the press built last year when Magic Leap announced $542 million in funding from Google, Qualcomm, Legendary Pictures and more. While we have little to see about the intended user experience, the prominence of the involved players has kept watchers like me riveted. For example, their press release asks us to “imagine being able to generate images indistinguishable from real objects and then being able to place those images seamlessly into the real world.”

ENTER THE MATRIX

I placed the Oculus Rift in the quadrant of there and us. The there is obvious – Oculus’ driving power is its ability to make you FEEL like you are somewhere else, such as climbing the wall in the Game of Thrones. Meanwhile, the social aspects are core to the Oculus design – originally for gamers to play together and now, evident through Facebook’s purchase, with hope to connect non-gamers as well.

Impact on museums: For some, the ability to be social in a virtual environment is a threat to museums: Why travel to meet friends and family to view a dinosaur fossil if you can stay home and interact with them in an immersive environment pretending to be the archeologists on the dig who discovered it? In other words, the science content can be engaged in new and exciting ways, but divorced from the original object and the building that stores and displays it. For others, that’s exactly the point – more engaging and educational experiences enabled by such technology will generate deeper and more informed interest in the original object and reify its value to the increased traffic who then seek it out in person. This suggests the potential for museum-produced immersive experiences outside the museum associated with in-person experiences that tout the value of the original object.  

I placed Google Glass in the quadrant of me and here. Glass is here because in most cases (but not all) the device is used to capture or learn more about the world around you. How do I get from here to there? Take a photo of what I am seeing? Translate the sign before me into a different language? Glass is me because the experience is personalized. To an outside observer, you and I each might be looking at the same painting, but while you’re taking a photo I’m searching online to learn more about it. (Content can be shared with Glass – you could send me your photo and I could email you the link I found – but we might never learn its full social potential unless Google re-releases it in a new form.)

Impact on museums: To the extent that the features of Glass reflect a SmartPhone on steroids, it challenges us to think about how museums can keep up with how people increasingly self-mediate their visits. For example, How powerful are their mobile tools to create and share, and what do visitors require from the museum to meet their expectations? How strong is their desire and ability to seek multiple perspectives beyond the wall copy and how might museums facilitated that engagement? Do we really need to fear selfie sticks? The questions are endless and speak to our general interests and concerns about life in the digital age. And when I compare Glass to a smartphone I am really describing the power to call up everything on the Internet. But what is the Internet? A decade ago YouTube was just about to launch and few heard of this new site called Facebook. A decade from now how will we have changed how we use the Internet and in what ways will that mediate museum visits?

I place Project HoloLens in the quadrant of us and here. HoloLens seems focused on placing “holographic” augmentations in the space around you (here) and using the device to collaborate with other users around or through those augmentations (us). For example, with Google Glass you might experience a virtual tour guide who travels with you on your visit, but it would be a me experience; other Google Glass users couldn’t join you, even if they could access their own private guide. With HoloLens, the guide wouldn’t follow you; you would follow the guide. All device users would see and hear the same thing, like the holographic video of Princess Leia in the first Star Wars film.

Impact on museums: us and here is the sweet spot of museum engagement. It is why people travel to museums in the first place – to have a place-based, shared experience with their friends and family. While Oculus offers a rich new way to contextualize and connect with exhibit content from the comfort of one’s home or classroom, HoloLens suggests an approach that leverages, rather than jettisons, one’s presence within a museum. Because the experience might be shared, rather than personalized, it also might alleviate the privacy concerns that undermined Google Glass.

Then, of course, there is Magic Leap. Until we see a demo it’s hard to say where this device will fit, but I can approach it now as I will anything new in this space, asking the questions that place it within the Mooshme Matrix of Place-based Augmented Devices: To what extent does it allow me to connect with friends and family? To what extent does it transport me to a virtual reality or anchor me where I am?

Digital devices come and go. Some will fail while others succeed in unexpected ways. But the affordances they bring to museum visitors will undoubtedly increase over time and redefine the relationship between museums and their visitors in the near future. Whether or not that relationship leads to a deeper understanding and appreciation for science, art, history (and whatever else a museum may be designed to advance) will depend in large part on how successfully we respond to disruptive digital tools and the habits they generate amongst our visitors.

What do you think?