archive.Joe Lamantia.com

The good peo­ple at Chi Nether­lands just posted video of my talk “Play­ing Well With Oth­ers: Inter­ac­tion Design and Social Design for Aug­mented Real­ity” at the Web and Beyond 2010 here in Ams­ter­dam in June.  It’s couched as a col­lec­tion of design prin­ci­ples for the oncom­ing cat­e­gory of social aug­mented inter­ac­tions made pos­si­ble by the new medium of aug­mented real­ity.  But this talk is also a call to action for all mak­ers of expe­ri­ences for the emerg­ing engage­ment space of every­ware to focus on the human and the humane per­spec­tives as we explore the new inter­ac­tions made possible.

The out­line of the talk is roughly:

1. Overview of aug­mented reality
2. Social inter­ac­tion per­spec­tive on cur­rent AR experiences
3. Def­i­n­i­tion of ‘social aug­mented experiences’
4. Com­mon inter­ac­tion design pat­terns for AR
5. Social ‘anti-patterns’ lim­it­ing design of aug­mented experiences
6. Design prin­ci­ples for social aug­mented experiences

(The audio qual­ity is quite good, and the cam­era­man cap­tured most of the slides nicely — so this is a record­ing worth watching.)

This year’s TWAB fea­tured sev­eral talks on aug­mented real­ity, ubiq­ui­tous com­put­ing and related top­ics; you’ll find record­ings of these on the Chi Ned­er­land Vimeo chan­nel: http://vimeo.com/chinederland

Many thanks to the orga­niz­ers and vol­un­teers for putting on such a well-run event!

TWAB2010: Joe Lamantia – Playing well with others: interaction design and social design for augmented reality from Chi Nederland on Vimeo.

Thanks to all who came to the Muziekgebouw on a lovely early summer day to talk about the emerging engagement space of social augmented experiences for the third edition of The Web And Beyond conference in Amsterdam.

For reference, here’s the session description from the official program:

Augmented reality blends the real world and the Internet in real time, making many new kinds of proximity, context, and location based experiences possible for individuals and groups. Despite these many possibilities, we know from history that the long term value and impact of augmented reality for most people will depend on how well these experiences integrate with ordinary social settings, and support everyday interactions. Yet the interaction patterns and behavior we see in current AR experiences seem almost ‘anti-social’ by design. This is an important gap that design must close in order to create successful AR offerings. In other words, much like children going to school for the first time, AR must to learn to ‘play well with others’ to be valuable and successful. This presentation reviews the interaction design patterns common to augmented reality, suggests tools to help understand and improve the ’social maturity’ of AR products and applications, and shares design principles for creating genuinely social augmented experiences that integrate well with human social settings and interactions.

I’ve posted my slides for the Where 2.0 panel “The Next Wave of AR: Social Augmented Experiences” organized by Tish Shute. After a review of the current state of augmented reality experiences in terms of the social interactions supported (using the metric of ‘social maturity’), it shares 9 principles for creating social AR experiences that people will enjoy and value.

Special points to those who spot the embedded April Fool’s joke…

Radio Johnny (brought to you by the good people of Johnny Holland magazine) just published an interview Jeff Parks recorded with me shortly before the New Year, discussing augmented reality, why it’s of interest for Experience Design, and some of the areas of likely development we’ll see in AR in the near future.

You can download the podcast, check out the show notes, and subscribe to the full Radio Johnny feed here: http://johnnyholland.org/2010/03/08/radio-johnny-joe-lamantia-on-augmented-reality/

Hubris alert: I admit to having grandiose schemes to influence the evolution of an emerging medium, by consistently hectoring the world on the importance of tools for simple content creation…

UXmatters just published Playing Well with Others: Design Principles for Social Augmented Experiences, the latest installment of my column Everyware, which manages to range from Airplane II to zombies, all while continuing the recent focus on augmented reality and experience design.  ‘Playing Well With Others’ suggests AR has two paths to follow as it evolves, and proposes some design principles for creating the social augmented experiences — experiences relying on augmented social interactions as the center of gravity — that lie along one of those two paths.

Here’s an excerpt:

With the exotic, mixed realities that futurists and science-fiction writers have envisioned seemingly just around the corner, it is time to move beyond questions of technical feasibility to consider the value and impact of turning the realities of everyday social settings and experiences inside out. As with all new technologies as they move from the stage of technical probe to social probe, this AR transformation will happen case by case and context by context, involving many factors beyond the direct reach of UX design. However, as a result of the inherently social nature of augmented reality, we can be sure the value and impact of many augmented experiences depends in large part on how effectively they integrate the social dimensions of real-world settings, in real time.

The first four design principles are:

• Default to the Human
• Enhancement Not Replacement
• Build Real Bridges
• Stay Off the Critical Path

Of course, this is just a starting list, and they raise almost as many questions as they attempt to answer.

Some of those follow-up questions include: what other principles are there?

Are there ‘anti-principles’ to be aware of?

What’s the best way to make these principles part of designing augmented experiences?

2009 was a big year for aug­mented real­ity, and there are many pre­dic­tions that 2010 will be even big­ger; with accom­plish­ments com­ing in the form of new tech­nolo­gies, devices, busi­ness mod­els, and ways of hav­ing fun.  But even as we go about build­ing this emerg­ing medium, we’re still rely­ing largely on old-media style cen­tral­ized under­stand­ings of the pro­duc­tion mod­els, form, and con­tent of the aug­mented world.  What hap­pens when we grasp the new social and inter­ac­tion pos­si­bil­i­ties of aug­mented reality?

I’m part of a panel titled The Next Wave of AR: Explor­ing Social Aug­mented Expe­ri­ences that’s address­ing this ques­tion at the Where 2.0 con­fer­ence in San Jose in late March / early April.  We’ve got a good group of speak­ers that includes Tish Shute (Ugo­trade), whur­ley * (whur­leyvi­sion llc),Jeremy Hight (Mis­sion Col­lege, CA), and Thomas Wro­bel (Lost Again).  Our goal is to look ahead at how aug­mented real­ity will soon evolve to include — or be based on! — mean­ing­ful social inter­ac­tions and dynam­ics at small and large group scales.

In the spirit of co-created social aug­mented expe­ri­ences, we’re ask­ing for audi­ence con­tri­bu­tions: in the form of sim­ple sce­nar­ios that describe the future of social AR.  What will it feel like? Who will you inter­act with?  How will these expe­ri­ences change every­day life?

Panel Sum­mary (full descrip­tion on the Where 2.0 site)

This panel will dis­cuss shared aug­mented real­i­ties, con­sid­er­ing some of the essen­tial pos­si­bil­i­ties and chal­lenges inher­ent in this new class of social aug­mented expe­ri­ences. The for­mat is pre­sen­ta­tion of a small set of sce­nar­ios (defined in advance, with audi­ence input) describ­ing likely future forms of shared aug­mented real­i­ties at dif­fer­ing scales of social engage­ment for dis­cus­sion by a panel of lead­ing prac­ti­tion­ers in tech­nol­ogy, expe­ri­ence design, net­worked urban­ism, inter­face design, game design, and aug­mented reality.

Cur­rent aug­mented real­ity expe­ri­ences put who you are, where you are, what you are doing, and what is around you at cen­ter stage. But we can already look beyond the first stage of inter­ac­tions assum­ing a sin­gle user see­ing sim­ple arrows and tags indi­cat­ing POIs, and begin to explore shared (multiuser/multisource) aug­mented real­i­ties.
These social aug­mented expe­ri­ences will allow not only mashups, & mul­ti­source data flows, but dynamic over­lays (not lim­ited to 3d), cre­ated by dis­trib­uted groups of users, linked to location/place/time, and syn­di­cated to peo­ple who wish to engage with the expe­ri­ence by view­ing and co-creating ele­ments for their own goals and benefit.

Share your sce­nar­ios for the Next Wave of AR in the com­ments or else­where (tag nextwaveAR socialAR), and come to Where 2.0 and see the panel!

The most important question facing the augmented reality community – one whose answer will shape the future of AR – is content creation. Put simply, it’s a question of Who can create What kind of content, and How they will create it.  At the moment, a noticeable gap separates those who can create AR experiences from those who cannot.  High barriers to entry in the form of skills, technology, or expense like those in front of AR are acceptable for a new medium at the early stages of development, but in the long run, making it easy for all those people who don’t know a fiduciary marker from fiduciary trust to easily create valuable experiences for themselves and others is far more important to the viability of AR than resolving any of the many conceptual, design, or technological challenges visible at the moment.

In fact, unless the AR community makes it easy for ordinary people to create and share meaningful content broadly, I wager augmented reality will remain a marketer’s overworked dray horse in the near and middle term future. And in the long term, augmented reality experiences will become at best an interface lens [as Adam Greenfield suggests here] supporting specialized visualization needs and a limited range of interactions (with correspondingly limited value), all built around resources originating from elsewhere within the ubiquitous digital experience ecosystem.

I think this is a ‘negative outome’ for AR only because I see so much potential. As a class of experiences, augmented reality has the potential to change our understanding of the world we are immersed in at every moment, but only rarely apprehend in a way that makes informed interaction with people and the environment possible. As Tish Shute noted in her recent interview with Bruno Uzzan, I see the collection of tools, technologies, and concepts affiliated under the banner of augmented reality as the leading ambassador for ubiquitous computing and the weird world of everyware that is rising around us.

Recent developments show progress towards bridging the gap. First is Mobilizy’s proposal of a common markup language – ARML [Augmented Reality Markup Language], based on KML – to the Augmented Reality Consortium.  Setting aside all other questions about ARML, the primary content creation problem I see with this approach is the explicitly geographic frame of reference in KML.  Most people simply do not think in the same terms used by geolocative schemes.  When I ask how far it is to the market, and someone replies “4 minutes north”, they’re not thinking in minutes of latitude….  But rather than attempt to reorient the GIS / GEO location worldview to one that’s more natural in human terms, I think the pragmatic solution is a translation layer in the creation experience that avoids coordinates or other non-natural lcoative schemes, much as domain names overlay or broker IP addresses.  As an example, recall how the travel service Dopplr prompts you to enter the name of a place, suggests likely matches from a library of defined and managed place names, and only then addresses the coordinates associated with that location.

In addition, ARML will need some sort of ability to capture markup that is *not* dependent on geographic reference.  This may seem counterintuitive for a medium that aims to augment reality (which is, after all, a place), but remember that people also orient themselves in terms of other people, time, activity, identifier, etc.  Hanging everything that augments reality off of the geographic skeleton will result in instant reference scheme hackery on an immense scale.  At the least, AR content creation experiences based on ARML will need some means of invoking other reference schemes.

The second development is Layar’s launch of buildAR.com, a public web-based content creation tool that supports map based interaction that extends the model for creation experiences beyond coordinate tagged text data.  BuildAR.com is an early stage tool, but it marks a step toward the evolution towards the goal of reflexivity; the stage of maturity wherein it is possible for people who are unaware of the structure and concepts that define the medium to easily use tools provided within the medium to create experiences.  In McCLuhanesque terms, this effectively entails making provision for using the medium to extend itself.

I’m talking about both direct and indirect creation pathways for augmented content, though the emphasis is on the direct end of the continuum.  Indirect creation could take many forms, such as translating existing geolocative tags or appending ARML metadata to existing digital content items; perhaps social objects like photos, tweets, hotel reviews, or recipes.  Or content that is created as a result of Google Wave, or the instrumentation of urban settings, and our basic economic processes.  (A deep dive into the question of direct vs. indirect content creation pathways would require mapping out the potential augmented content ecosystem of linked data, and assessing each type of data from the cloud of apis / services / sources using tbd criteria.)

Addressing the content creation gap is critical because enabling broad-based creation of augmented experiences will speed up experimentation for all the supporting models that need to evolve: business and revenue, data ownership, technical, conceptual, etc. Evolution is needed here; the early models for content creation include advertiser only (a default in the experimental stage for media where marketers and advertisers are pioneers), subscription based, open source, and nonprofit (academic and otherwise).  None of these yet offers the right combination of convenience and context, the implacable twin giants who rule the domain of value judgments made by digital consumers and co-creaters.

Guidelines for Content Creation Experiences

So what should the AR community offer to close the creation gap?  We’ve learned a lot about what works in broad-based content creation from the evolution of blogging and other mainstream platforms for social interaction.  Without considering it extensively, the guidelines for a content creation experience (mind, I’m not discussing the technical enablers) are:

• No cost of entry: Creating content cannot require spending money (at least for basic capability), as the effort involved is already an investment.
• No cognitive overhead: Creating content cannot require understanding new abstract concepts, mastering tools with low usability, learning complex languages or terminology, etc.
• No maintenance: Creation tools must act like self-maintaining services, i.e. tools that do not require effort or attention
• No accessibility barriers: For global adoption, content creation experiences need to be accessible, which means low-bandwidth, multi-lingual, cross-media, and platform agnostic.

This is a starting list, but it captures the essence of the offerings that have been successful in the past.

In addition to the experience, the content that people create needs to follow some guidelines.

• Addressable: Including findability and searchability, AR content must be fully addressable by a broad spectrum of tools and protocols.  AR will fail at bridging the real and digital if the content people create for augmented experiences  cannot – at least partially – be addressed across this boundary, which is what makes AR an enchanted window rather than a simple browser / UI lens.  This seems like the simplest of these guidelines (after all, what isn’t addressable in a digital space?), but I think in the end it will be quite challenging to realize.
• Interoperable: Content must work across platforms, formats, and browsers, in terms of creation, sharing, and management.
• Portable: Content must be movable or portable for people to make the effort of creation; it cannot be confined to a single storage location, service, tool, owner, etc.  This touches on the familiar questions of data ownership and the commons.

The goal of these suggestions is to push AR toward maturity and broader adoption as quickly as possible, using lessons from the evolution of the Web.  What suggestions for guidelines for content creation experiences and the nature of AR content do you have?

If I am off base in thinking the creation barrier critical at this early stage of augmented reality’s rise above the experimental waterline, then what is more important?

Marshall Kirkpatrick of ReadWriteWeb links to Inside Out: Interaction Design for Augmented Reality (the latest Everyware column) in two recent stories tracking the fast-moving augmented reality space; Augmented Reality: Five Bariers to a Web That’s Everywhere and, and RobotVision: A Bing-powered iPhone Augmented Reality Browser. 

Thanks, Marshall!

And as a bonus, Tim O’Reilly tweeted about Marshall’s article.  I doubt that Tim reads this feed, but it’s always nice to be recognized, even indirectly.

Since I wrote about the user experience of augmented reality less than two weeks ago, the most important development is the arrival of augmented iPhone apps (unofficially for the moment, officially in September).

Why is this so important, when Wikitude and other AR Android apps have been available for almost a year?  Bringing augmented reality to the iPhone changes the cultural assumptions made about AR experiences as a class of offering. Endorsing AR experiences for iPhone users moves augmented reality from the geek realm of Android and Google, to the chic world of Apple.  Culturally, the assumptions we make about the new products and services from Apple and Google are driven largely by the differences in way we perceive the two brands.  Apple is chic, while Google is geek.

Looking Ahead

Connecting the Apple brand to augmented experiences will persuade many people to try out AR.  Yet as I’ve said, and many others as well, getting the user experience of augmented reality ‘right’ is absolutely the critical element to the long term viability of this new class of experiences.  This entails two efforts.

First, designers must refine the experiences offered by all those AR applications based on the four classic interaction patterns known so far – Head-Up Display, Tricorder, Holochess, and X-ray Vision.  Two factors make refinement essential: competition from other AR offerings that reduces the novelty value of your experience, and increased ‘load’ on the UX in the form of actual use for everyday purposes in the complex setting of real life.  Think about trying to choose where to get lunch for the afternoon by sorting through 1500 listings for coffee shops and restaurants while standing on a street corner in the rain in London holding your phone aloft.  The functional aspects of AR experiences just aren’t refined enough to handle the interaction design, visualization, and contextual sensitivity challenges implied. [Prediction: AR usage cases will naturally settle on a set of common scenarios that balance the strengths and weaknesses of each of the four classic patterns.  More speculation on that in a later post.]

Second, designers must address the gaps in the set of concepts now used as the basis for imagining new augmented experiences.  I flagged six ‘missing’ patterns in the range of experiences offered so far; Loner, Second Hand Smoke, Pay No Attention To the Man Behind the Curtain, The Invisible Man!, Tunnel Vision, and AR for AR’s Sake (see the article for details).  I’m sure the very savvy readers of this blog can identify even more.

I hope all the AR innovators, designers, and entrepreneurs working hard on the crest of this breaking wave of technology find ways to take on both of these tasks.  If they can’t refine the existing models and fill in those experience gaps, then neither Apple chic nor Google geek cred will suffice to make augmented reality viable in the long term.  And what could literally be a new way of seeing the world – one with legitimate potential for changing our behavior with regard to urban spaces, the environment, social structures, play, and economics, among just a few spheres of human activity – will remain little more than a camera obscura style curiosity.

I’m very happy to announce that Inside Out: Interaction Design for Augmented Reality – the latest installment of my column Everyware @UX Matters –  is live now.  (Timing is sometimes the writer’s friend, as I was at the Layar event Monday night here in Amsterdam just the day before, and had the chance to talk with some of their team.)

AR is more of a perspective and class of experiences than an instance of new technology, so I wanted to approach the subject from the specific perspective of user experience and interaction design.  Reactions from the augmented reality community are positive so far; Claire Boonstra of Layar, and no less than the inimitable Tish Shute of UgoTrade, have all been kind enough to recommend it.  Thanks to them and to everyone who’s tweeted and posted this one.

As we explore the role augmented reality will play in our gigantic experiment with everyware, we should keep in mind that the map is not the territory.  But there is no denying an effective map will surely help point the way as you try to find your way around a strange new country.