archive.Joe Lamantia.com

I’m pleased to be presenting ‘A Taxonomy of Enterprise Search’ at the upcoming EuroHCIR workshop, part of the 2011 HCI conference in the UK.  Co-authored with Tony Russell-Rose of UXLabs, and Mark Burrell here at Endeca, this is our first publication of some of the very exciting work we’re doing to understand and describe discovery activities in enterprise settings, and do so within a changed and broader framing than traditional information retrieval.  The paper builds on work I’ve done previously on understanding and defining information needs and patterns of information retrieval activity, while working on search and discovery problems as part of larger user experience architecture efforts.

Here’s the abstract of the paper:

Classic IR (information retrieval) is predicated on the notion of users searching for information in order to satisfy a particular “information need”. However, it is now accepted that much of what we recognize as search behaviour is often not informational per se. For example, Broder (2002) has shown that the need underlying a given web search could in fact be navigational (e.g. to find a particular site or known item) or transactional (e.g. to find a sites through which the user can transact, e.g. through online shopping, social media, etc.). Similarly, Rose & Levinson (2004) have identified consumption of online resources as a further category of search behaviour and query intent.

In this paper, we extend this work to the enterprise context, examining the needs and behaviours of individuals across a range of search and discovery scenarios within various types of enterprise. We present an initial taxonomy of “discovery modes”, and discuss some initial implications for the design of more effective search and discovery platforms and tools.

There’s a considerable amount of research available on information retrieval — even within a comparatively new discipline like HCIR, focused on the human to system interaction aspects of IR — but I think it’s the attempt to define an activity centered grammar for interacting with information that makes our approach worth examining.  The HCIR events in the U.S. (and now Europe) blend academic and practitioner perspectives, so are an appropriate audience for our proposed vocabulary of discovery activity ‘modes’ that’s based on a substantial body of data collected and analyzed during solution design and deployment engagements.

I’ll post the paper itself once the proceedings are available.

Endeca invites customers, partners and leading members of the broader search and discovery technology and solutions communities to meet annually, and showcase the most interesting and exciting work in the field of discovery.  As lead for the UX team that designs Endeca’s discovery products, I shared some of our recent work on patterns in the structure of discovery applications, as well as best practices in information design and visualization that we use to drive product definition and design for Endeca’s Latitude Discovery Framework.

This material is useful for program and project managers and business analysts defining requirements for discovery solutions and applications, UX and system architects crafting high-level structures and addressing long-term growth, interaction designers and technical developers defining and building information workspaces at a fine grain, and

There are three major sections: the first presents some of our tools for identifying and understanding people’s needs and goals for discovery in terms of activity (the Language of Discovery as we call it), the second brings together screen-level, application level, and user scenario / use-case level patterns we’ve observed in the applications created to meet those needs, and the final section shares condensed best practices and fundamental principles for information design and visualization based on academic research disciplines such as cognitive science and information retrieval.

It’s no coincidence that these sections reflect the application of the core UX disciplines of user research, information architecture, and interaction design to the question of “who will need to encounter information for some end, and in what kind of experience will they encounter it”.  This flow and ordering is deliberate; it demonstrates on two levels the results of our own efforts applying the UX perspective to the questions inherent in creating discovery tools, and shares some of the tools, insights, templates, and resources we use to shape the platform used to create discovery experiences across diverse industries.

Session outline

Session description

“How can you harness the power and flexibility of Latitude to create useful, usable, and compelling discovery applications for enterprise discovery workers? This session goes beyond the technology to explore how you can apply fundamental principles of information design and visualization, analytics best practices and user interface design patterns to compose effective and compelling discovery applications that optimize user discovery, success, engagement, & adoption.”

The patterns are product specific in that they show how to compose screens and applications using the predefined components in the Discovery Framework library.  However, many of the product-specific components are built to address common or recurring needs for interaction with information via well-known mechanisms such as search, filtering, navigation, visualization, and presentation of data.  In other words, even if you’re not using the literal Discovery Framework component library to compose your specific information analysis workspace, you’ll find these patterns relevant at workspace and application levels of scale.

The deeper story of these patterns is in demonstrating the evolution of discovery and analysis applications over time.  Typically, discovery applications begin by offering users a general-purpose workspace that satisfies a wide range of interaction tasks in an approximate fashion.  Over time, via successive expansions in the the scope and variety of data they present, and the discovery and analysis capabilities they provide, discovery applications grow to include several different types of workspaces that individually address distinct sets of needs for visualization and sense making by using very different combinations of components.  As a composite, these functional and informationally diverse workspaces span the full range of interaction needs for differing types of users.

I hope you find this toolkit and collection of patterns and information design principles useful.  What are some of the resources you’re using to take on these challenges?

The good peo­ple at Chi Nether­lands just posted video of my talk “Play­ing Well With Oth­ers: Inter­ac­tion Design and Social Design for Aug­mented Real­ity” at the Web and Beyond 2010 here in Ams­ter­dam in June.  It’s couched as a col­lec­tion of design prin­ci­ples for the oncom­ing cat­e­gory of social aug­mented inter­ac­tions made pos­si­ble by the new medium of aug­mented real­ity.  But this talk is also a call to action for all mak­ers of expe­ri­ences for the emerg­ing engage­ment space of every­ware to focus on the human and the humane per­spec­tives as we explore the new inter­ac­tions made possible.

The out­line of the talk is roughly:

1. Overview of aug­mented reality
2. Social inter­ac­tion per­spec­tive on cur­rent AR experiences
3. Def­i­n­i­tion of ‘social aug­mented experiences’
4. Com­mon inter­ac­tion design pat­terns for AR
5. Social ‘anti-patterns’ lim­it­ing design of aug­mented experiences
6. Design prin­ci­ples for social aug­mented experiences

(The audio qual­ity is quite good, and the cam­era­man cap­tured most of the slides nicely — so this is a record­ing worth watching.)

This year’s TWAB fea­tured sev­eral talks on aug­mented real­ity, ubiq­ui­tous com­put­ing and related top­ics; you’ll find record­ings of these on the Chi Ned­er­land Vimeo chan­nel: http://vimeo.com/chinederland

Many thanks to the orga­niz­ers and vol­un­teers for putting on such a well-run event!

TWAB2010: Joe Lamantia – Playing well with others: interaction design and social design for augmented reality from Chi Nederland on Vimeo.

Thanks to all who came to the Muziekgebouw on a lovely early summer day to talk about the emerging engagement space of social augmented experiences for the third edition of The Web And Beyond conference in Amsterdam.

For reference, here’s the session description from the official program:

Augmented reality blends the real world and the Internet in real time, making many new kinds of proximity, context, and location based experiences possible for individuals and groups. Despite these many possibilities, we know from history that the long term value and impact of augmented reality for most people will depend on how well these experiences integrate with ordinary social settings, and support everyday interactions. Yet the interaction patterns and behavior we see in current AR experiences seem almost ‘anti-social’ by design. This is an important gap that design must close in order to create successful AR offerings. In other words, much like children going to school for the first time, AR must to learn to ‘play well with others’ to be valuable and successful. This presentation reviews the interaction design patterns common to augmented reality, suggests tools to help understand and improve the ’social maturity’ of AR products and applications, and shares design principles for creating genuinely social augmented experiences that integrate well with human social settings and interactions.

I’ve posted my slides for the Where 2.0 panel “The Next Wave of AR: Social Augmented Experiences” organized by Tish Shute. After a review of the current state of augmented reality experiences in terms of the social interactions supported (using the metric of ‘social maturity’), it shares 9 principles for creating social AR experiences that people will enjoy and value.

Special points to those who spot the embedded April Fool’s joke…

Radio Johnny (brought to you by the good people of Johnny Holland magazine) just published an interview Jeff Parks recorded with me shortly before the New Year, discussing augmented reality, why it’s of interest for Experience Design, and some of the areas of likely development we’ll see in AR in the near future.

You can download the podcast, check out the show notes, and subscribe to the full Radio Johnny feed here: http://johnnyholland.org/2010/03/08/radio-johnny-joe-lamantia-on-augmented-reality/

Hubris alert: I admit to having grandiose schemes to influence the evolution of an emerging medium, by consistently hectoring the world on the importance of tools for simple content creation…

UXmatters just published Playing Well with Others: Design Principles for Social Augmented Experiences, the latest installment of my column Everyware, which manages to range from Airplane II to zombies, all while continuing the recent focus on augmented reality and experience design.  ‘Playing Well With Others’ suggests AR has two paths to follow as it evolves, and proposes some design principles for creating the social augmented experiences — experiences relying on augmented social interactions as the center of gravity — that lie along one of those two paths.

Here’s an excerpt:

With the exotic, mixed realities that futurists and science-fiction writers have envisioned seemingly just around the corner, it is time to move beyond questions of technical feasibility to consider the value and impact of turning the realities of everyday social settings and experiences inside out. As with all new technologies as they move from the stage of technical probe to social probe, this AR transformation will happen case by case and context by context, involving many factors beyond the direct reach of UX design. However, as a result of the inherently social nature of augmented reality, we can be sure the value and impact of many augmented experiences depends in large part on how effectively they integrate the social dimensions of real-world settings, in real time.

The first four design principles are:

• Default to the Human
• Enhancement Not Replacement
• Build Real Bridges
• Stay Off the Critical Path

Of course, this is just a starting list, and they raise almost as many questions as they attempt to answer.

Some of those follow-up questions include: what other principles are there?

Are there ‘anti-principles’ to be aware of?

What’s the best way to make these principles part of designing augmented experiences?

2009 was a big year for aug­mented real­ity, and there are many pre­dic­tions that 2010 will be even big­ger; with accom­plish­ments com­ing in the form of new tech­nolo­gies, devices, busi­ness mod­els, and ways of hav­ing fun.  But even as we go about build­ing this emerg­ing medium, we’re still rely­ing largely on old-media style cen­tral­ized under­stand­ings of the pro­duc­tion mod­els, form, and con­tent of the aug­mented world.  What hap­pens when we grasp the new social and inter­ac­tion pos­si­bil­i­ties of aug­mented reality?

I’m part of a panel titled The Next Wave of AR: Explor­ing Social Aug­mented Expe­ri­ences that’s address­ing this ques­tion at the Where 2.0 con­fer­ence in San Jose in late March / early April.  We’ve got a good group of speak­ers that includes Tish Shute (Ugo­trade), whur­ley * (whur­leyvi­sion llc),Jeremy Hight (Mis­sion Col­lege, CA), and Thomas Wro­bel (Lost Again).  Our goal is to look ahead at how aug­mented real­ity will soon evolve to include — or be based on! — mean­ing­ful social inter­ac­tions and dynam­ics at small and large group scales.

In the spirit of co-created social aug­mented expe­ri­ences, we’re ask­ing for audi­ence con­tri­bu­tions: in the form of sim­ple sce­nar­ios that describe the future of social AR.  What will it feel like? Who will you inter­act with?  How will these expe­ri­ences change every­day life?

Panel Sum­mary (full descrip­tion on the Where 2.0 site)

This panel will dis­cuss shared aug­mented real­i­ties, con­sid­er­ing some of the essen­tial pos­si­bil­i­ties and chal­lenges inher­ent in this new class of social aug­mented expe­ri­ences. The for­mat is pre­sen­ta­tion of a small set of sce­nar­ios (defined in advance, with audi­ence input) describ­ing likely future forms of shared aug­mented real­i­ties at dif­fer­ing scales of social engage­ment for dis­cus­sion by a panel of lead­ing prac­ti­tion­ers in tech­nol­ogy, expe­ri­ence design, net­worked urban­ism, inter­face design, game design, and aug­mented reality.

Cur­rent aug­mented real­ity expe­ri­ences put who you are, where you are, what you are doing, and what is around you at cen­ter stage. But we can already look beyond the first stage of inter­ac­tions assum­ing a sin­gle user see­ing sim­ple arrows and tags indi­cat­ing POIs, and begin to explore shared (multiuser/multisource) aug­mented real­i­ties.
These social aug­mented expe­ri­ences will allow not only mashups, & mul­ti­source data flows, but dynamic over­lays (not lim­ited to 3d), cre­ated by dis­trib­uted groups of users, linked to location/place/time, and syn­di­cated to peo­ple who wish to engage with the expe­ri­ence by view­ing and co-creating ele­ments for their own goals and benefit.

Share your sce­nar­ios for the Next Wave of AR in the com­ments or else­where (tag nextwaveAR socialAR), and come to Where 2.0 and see the panel!

UX Mat­ters just pub­lished Anony­mous Cow­ards, Avatars, and the Zeit­geist: Per­sonal Iden­tity in Flux.  This is the lat­est install­ment of my col­umn on ubiq­ui­tous com­put­ing and user expe­ri­ence, and it takes on the ques­tion of how per­sonal iden­tity is chang­ing is a result of the rise of dig­i­tal tools, ser­vices, and mea­sure­ments for iden­tity.   Iden­tity is a fun­da­men­tal aspect of expe­ri­ence, so it’s crit­i­cal that we under­stand what is hap­pen­ing to this uni­ver­sal ele­ment.  ‘Anony­mous Cow­ards’ is the first of two parts, focused on under­stand­ing how dig­i­tal iden­ti­ties work, and are dif­fer­ent from what we know.  Here’s an excerpt:

Dri­ven by dra­matic shifts in tech­nol­ogy, eco­nom­ics, and media, noth­ing less than a trans­for­ma­tion in the makeup and behav­ior of our per­sonal iden­tity is at hand—what it is, where it comes from, how it works, who con­trols it, how peo­ple and orga­ni­za­tions use and value it. As a direct result of this trans­for­ma­tion, the expe­ri­ence peo­ple have of per­sonal identity—both their own and the iden­ti­ties of others—is chang­ing rapidly. As design­ers of the blended dig­i­tal, social, and mate­r­ial expe­ri­ences of every­ware, we must under­stand the chang­ing nature of per­sonal iden­tity. And now that human­ity itself is within the design hori­zon, it is espe­cially impor­tant for design to under­stand the shift­ing expe­ri­ence of dig­i­tal identity.

The sec­ond part will look at the impli­ca­tions of these changes for our expe­ri­ence of iden­tity.  As I put together my pre­dic­tions for what iden­tity will be like in 10 years, I wel­come input — what do you think?

The most important question facing the augmented reality community – one whose answer will shape the future of AR – is content creation. Put simply, it’s a question of Who can create What kind of content, and How they will create it.  At the moment, a noticeable gap separates those who can create AR experiences from those who cannot.  High barriers to entry in the form of skills, technology, or expense like those in front of AR are acceptable for a new medium at the early stages of development, but in the long run, making it easy for all those people who don’t know a fiduciary marker from fiduciary trust to easily create valuable experiences for themselves and others is far more important to the viability of AR than resolving any of the many conceptual, design, or technological challenges visible at the moment.

In fact, unless the AR community makes it easy for ordinary people to create and share meaningful content broadly, I wager augmented reality will remain a marketer’s overworked dray horse in the near and middle term future. And in the long term, augmented reality experiences will become at best an interface lens [as Adam Greenfield suggests here] supporting specialized visualization needs and a limited range of interactions (with correspondingly limited value), all built around resources originating from elsewhere within the ubiquitous digital experience ecosystem.

I think this is a ‘negative outome’ for AR only because I see so much potential. As a class of experiences, augmented reality has the potential to change our understanding of the world we are immersed in at every moment, but only rarely apprehend in a way that makes informed interaction with people and the environment possible. As Tish Shute noted in her recent interview with Bruno Uzzan, I see the collection of tools, technologies, and concepts affiliated under the banner of augmented reality as the leading ambassador for ubiquitous computing and the weird world of everyware that is rising around us.

Recent developments show progress towards bridging the gap. First is Mobilizy’s proposal of a common markup language – ARML [Augmented Reality Markup Language], based on KML – to the Augmented Reality Consortium.  Setting aside all other questions about ARML, the primary content creation problem I see with this approach is the explicitly geographic frame of reference in KML.  Most people simply do not think in the same terms used by geolocative schemes.  When I ask how far it is to the market, and someone replies “4 minutes north”, they’re not thinking in minutes of latitude….  But rather than attempt to reorient the GIS / GEO location worldview to one that’s more natural in human terms, I think the pragmatic solution is a translation layer in the creation experience that avoids coordinates or other non-natural lcoative schemes, much as domain names overlay or broker IP addresses.  As an example, recall how the travel service Dopplr prompts you to enter the name of a place, suggests likely matches from a library of defined and managed place names, and only then addresses the coordinates associated with that location.

In addition, ARML will need some sort of ability to capture markup that is *not* dependent on geographic reference.  This may seem counterintuitive for a medium that aims to augment reality (which is, after all, a place), but remember that people also orient themselves in terms of other people, time, activity, identifier, etc.  Hanging everything that augments reality off of the geographic skeleton will result in instant reference scheme hackery on an immense scale.  At the least, AR content creation experiences based on ARML will need some means of invoking other reference schemes.

The second development is Layar’s launch of buildAR.com, a public web-based content creation tool that supports map based interaction that extends the model for creation experiences beyond coordinate tagged text data.  BuildAR.com is an early stage tool, but it marks a step toward the evolution towards the goal of reflexivity; the stage of maturity wherein it is possible for people who are unaware of the structure and concepts that define the medium to easily use tools provided within the medium to create experiences.  In McCLuhanesque terms, this effectively entails making provision for using the medium to extend itself.

I’m talking about both direct and indirect creation pathways for augmented content, though the emphasis is on the direct end of the continuum.  Indirect creation could take many forms, such as translating existing geolocative tags or appending ARML metadata to existing digital content items; perhaps social objects like photos, tweets, hotel reviews, or recipes.  Or content that is created as a result of Google Wave, or the instrumentation of urban settings, and our basic economic processes.  (A deep dive into the question of direct vs. indirect content creation pathways would require mapping out the potential augmented content ecosystem of linked data, and assessing each type of data from the cloud of apis / services / sources using tbd criteria.)

Addressing the content creation gap is critical because enabling broad-based creation of augmented experiences will speed up experimentation for all the supporting models that need to evolve: business and revenue, data ownership, technical, conceptual, etc. Evolution is needed here; the early models for content creation include advertiser only (a default in the experimental stage for media where marketers and advertisers are pioneers), subscription based, open source, and nonprofit (academic and otherwise).  None of these yet offers the right combination of convenience and context, the implacable twin giants who rule the domain of value judgments made by digital consumers and co-creaters.

Guidelines for Content Creation Experiences

So what should the AR community offer to close the creation gap?  We’ve learned a lot about what works in broad-based content creation from the evolution of blogging and other mainstream platforms for social interaction.  Without considering it extensively, the guidelines for a content creation experience (mind, I’m not discussing the technical enablers) are:

• No cost of entry: Creating content cannot require spending money (at least for basic capability), as the effort involved is already an investment.
• No cognitive overhead: Creating content cannot require understanding new abstract concepts, mastering tools with low usability, learning complex languages or terminology, etc.
• No maintenance: Creation tools must act like self-maintaining services, i.e. tools that do not require effort or attention
• No accessibility barriers: For global adoption, content creation experiences need to be accessible, which means low-bandwidth, multi-lingual, cross-media, and platform agnostic.

This is a starting list, but it captures the essence of the offerings that have been successful in the past.

In addition to the experience, the content that people create needs to follow some guidelines.

• Addressable: Including findability and searchability, AR content must be fully addressable by a broad spectrum of tools and protocols.  AR will fail at bridging the real and digital if the content people create for augmented experiences  cannot – at least partially – be addressed across this boundary, which is what makes AR an enchanted window rather than a simple browser / UI lens.  This seems like the simplest of these guidelines (after all, what isn’t addressable in a digital space?), but I think in the end it will be quite challenging to realize.
• Interoperable: Content must work across platforms, formats, and browsers, in terms of creation, sharing, and management.
• Portable: Content must be movable or portable for people to make the effort of creation; it cannot be confined to a single storage location, service, tool, owner, etc.  This touches on the familiar questions of data ownership and the commons.

The goal of these suggestions is to push AR toward maturity and broader adoption as quickly as possible, using lessons from the evolution of the Web.  What suggestions for guidelines for content creation experiences and the nature of AR content do you have?

If I am off base in thinking the creation barrier critical at this early stage of augmented reality’s rise above the experimental waterline, then what is more important?