But recently I’ve come to regard that latter statement as a real key to understanding the somewhat confused nature of our nascent field of Human-Centered Computing. Are we mere record-keepers of the state-of-the-art—observing and interviewing people about their relationship to technology for the benefit of techno-historians? Or are we trying to use our observations of the present to help shape the future?

In traditional Computer Science (an outgrowth of the engineering school paradigm), the label “future studies” might actually make sense. My CS Phd colleagues are busy inventing new encryption strategies, or re-architecting the Internet, or optimizing multi-core processing algorithms—all necessary preconditions for technologies which will be ready for prime time in the next 5-1o years. Industry research havens like Bell Labs and Xerox PARC (both still alive but in humbled reincarnations) are no longer the lifeblood of CS innovation they once were. As a result, industry is an important partner in the academic research process, to such an extent that important discoveries made by today’s computer scientists really are quite likely to be translated into future products and thus shape the future.

But the relationship between interaction design and academic HCI(Human-Computer Interaction) is significantly more strained. A company like Microsoft doesn’t need an academic partner to create new interaction strategies; to my knowledge, the Surface tabletop computer was created mainly in-house with some consulting provided by my former employer. In fact, I think I can assert uncontroversially that if you want to explore new ways of visualizing information, create new and compelling interaction techniques, or otherwise revolutionize the way humans and computers interact, you are equally likely to succeed outside the ivory tower as within. Google may have started as a Stanford academic project, but its founders had to leave academe to really make it a success. And as for Apple? They’re notoriously secretive, even and especially within industry—and yet they’ve had more impact on our relationships with machines than any other company in the last 20 years.

The reasons for this are simple enough. In general, new HCI techniques in themselves aren’t feats of engineering, but they are judged based on qualities such as robustness and speed, and thus require a lot of cash to design, test and bring to market. Which do you think would produce a better product faster: hiring a bunch of distracted interns and compensating them with course credit and a small stipend, or turning to a team of seasoned professionals accustomed to fast-paced large-scale projects?  This means that someone with a fantastic concept for a new service or product can (and probably should) get some venture funding and create a start-up to make his or her dream a reality.

There are ways around this conundrum, of course, and I certainly don’t view academics’ efforts as wasted, otherwise I would have stayed in industry myself. The ‘living laboratory’ technique has been used to great effect. There’s a room somewhere in the MIT Media Lab where post-it notes are interactive, and Georgia Tech’s Aware Home is anything but an ordinary house. The ‘Wizard of Oz’ technique (in which a man or woman behind the curtain imbues an interface with more intelligence than is currently possible or pragmatic in order to study the interaction) is also pretty effective, although completely unscalable. [See Neal Stephenson's "The Diamond Age" for my favorite literary example of Wizard of Oz techniques 'in use'] I’ve used these and other techniques with much success—card sorts, paper prototypes, and GOFUS (Good Old Fashioned Usability Studies).Using these techniques a researcher can skip over some present-day technical hurdles and get to the good stuff faster. If we could solve the lag issue, or if we had more advanced voice recognition, here’s what the future would look like.

But for many Human-Centered Computing (HCC) researchers these techniques are just totally impractical for two main reasons: they don’t afford studying social effects, and they ignore one half of the equation: when we bring real people into our labs or research settings, we are studying people from today—with today’s expectations and biases.

So what are we to do? Ethnographic “deep hanging out”, semi-structured interviews and other ‘qualitative’ techniques offer one way out. By getting deep insight into the complex interplay between person, social cohort, technology, and physical environment, we also unlock the information firehose that is people’s everyday lives. If, through studying today’s technology, we can gain future-proof insights into what makes people tick, we can provide solid guidelines for the future and offer a service to humanity, both key goals of any academic institution.  But again, we run up against the difficulty that today’s immutable biases may simply be hard-coded cultural trends that will wear off.

For example, when preparing for my research into privacy effects and online photo-sharing, I read over and over that people prefer to share personal photos via email rather than sharing sites. And indeed, for a certain kind of person even just four years ago, this was technically true. But I found that what was really motivating this was the desire of people in the Kodak Culture (you can generally guess what that means, I hope) to tell different stories through their photos to different audiences, and to share through a medium over which they felt they had control. Email’s reliability and distribution control (ignoring the “forward” button for the moment) allowed people to mimic the kinds of practices they had been used to with in-person or by-mail photo-sharing pre-computer.

But what did I actually find? I generated some ‘design implications’ and got some solid data about people’s motivations and desires with respect to online photo-sharing. I also showed that many of the tenets of ‘Kodak Culture’ which had been observed in the film photography days were still operating, at least in America (cellphone photography had not yet hit our shores) and at least for that moment. But what I still don’t know is the extent to which I was simply observing a cultural moment, and the degree to which the photo-sharing norms and mores of my participants will simply disappear with them and not be carried on in the next generation. In short, my vision of the future was severely constricted by the very things that made my research successful in the short term.

To deal with this, much of our work in HCC proceeds by positing an underlying set of motivations and desires that we hope are future-proof, and then re-imagining them in the context of future or bleeding-edge technology.  Nokia’s Jan Chipchase calls this process “future perfect.” I still suspect there’s a fundamental re-evaluation of the role of the future in HCC waiting to happen, and I’m interested to think what others in the field (or at least in the bivouac) have to say. Have I uncharitably mischaracterized your work, or your vision of the field? Comment below, or email me here.

This age of social data presents us with a new set of tools for self-presentation and peripheral awareness, and countless new ways to be awkward. For example, a former colleague used to update twitter so comprehensively that he even tweeted his bathroom breaks. But such early-stage hiccups are not the big story here – every electronic medium has its own specific “reply all” potential. The real change is the new computational layer mediating our interactions with each other and our environment. Using FourSquare on my iPhone, I can tell people when I’m at a bar (or when I’m actually in the office for a change!). Using Google Reader I can share blog entries or newspaper articles I find interesting, and attach a short note (especially important if irony is involved). Using Last.fm I can share what music I’m listening to right now, and anyone who clicks through to my profile can get a pretty good picture of my music-listening preferences. Facebook lets me share all of the above, as well as update my status and comment on any of my friends’ actions. And of course, there’s Twitter, which I use mainly as a way to stay in touch with techie friends in academia and industry, but which often veers into in-jokes and gossipmongering anyway.

In fact I create so much social data every day, that when setting up my ‘lifestream’ I found myself selecting a subset of all the possible data I could push. I told myself at the time that this was to avoid the andrewmiller.net visitor from feeling overwhelmed, but really it was an exercise in self-curation – the selection of information about myself that I wanted to represent me as if it were my real-time CV. I chose to make this a service-level decision, excluding FaceBook (which is where I discuss emotional, political, and other semi-private topics). But I have on occasion gone in and removed certain tweets that for one reason or another I felt were not appropriate for andrewmiller.net but I didn’t feel like removing from Twitter.

I admit I’m an early test-case, but I think we’re months, not years away from my generation having to deal with these issues. As services like FaceBook open up our homepages to the public (or at least allow us to do so) and as services like Twitter, with a culture of public sharing, reach mass adoption, our virtual and physical identities are going to have to merge. This is one of the central issues I’m trying to unpack in the Human-Centered Computing program here at Georgia Tech, and I’d be interested to hear your thoughts. Comment, shoot me an email, send me a tweet @andrewmiller or whatever other channel you’d like.

In this position paper, we present our initial work in designing and developing Salud!, a web-based platform for supporting health self-management. Salud! will allow its users to track personally-relevant aspects of their everyday life, and provide visualization and analytics tools with which to make sense of the resulting datasets.

Full text [PDF]

Medynskiy, Y., Miller, A., Yoo, J.W., Mynatt, E. Temporal Data in a Health Self-Management Application. To be presented at the Interacting with Temporal Data workshop at CHI 2009.

In this position paper, we present our initial work in designing and developing Salud!, a web-based platform for supporting health self-management. Salud! will allow its users to track personally-relevant aspects of their everyday life, and provide visualization and analytics tools with which to make sense of the resulting datasets.

Full text [PDF]

During the presentation, a representative from LifeScan showed off a blood glucometer that connects to the iPhone to allow immediate measurement and transmission of blood glucose levels. Not only can the peripheral be controlled directly using the touchscreen interface, but it can tap into the iPhone’s always-on connectivity, allowing a caregiver, parent, adult child, or diabetes counselor to track the user’s insulin needs in realtime from anywhere. I owned a Visor, and the ability to extend its capabilities at will was a powerful one. By snapping an MP3 cartridge into it the Visor became a media player; add a camera module and you could take photos and videos on the go.

From a technology standpoint, this announcement marks the rebirth of Handspring’s ambitious but ultimately stagnant SpringBoard system introduced in the late 90s as part of their Visor product, which in turn, was a return to basics by the founders of Palm as that company got acquired, split, merged, and split again into near-obscurity. Most of the difficulties HandSpring encountered were distribution-related, given the niche audience and relatively high costs of stocking physical peripherals, and the digital distribution powerhouse that is the App store has solved those.  Since apps can completely take over the iPhone’s UI, Apple has allowed developers to come very close to the extensibility provided by the original SpringBoard platform. The one thing missing has been the ability to add physical computing elements — sensors and actuators — to allow the iPhone to more actively participate in its users’ world.

This announcement is also important from a research standpoint.  As part of her dissertation work, my colleague Lena Mamykina here at Georgia Tech has shown the benefits of the collection, analysis and discussion of regular blood glucose readings.  Her participants feel more in control of their health and are able to create and test hypotheses about their diet.  For her work, Lena created her own infrastructure, customizing cellphones herself and connecting them to glucometers and her web-based forum.  (you can watch her give a talk about her MAHI project at CHI ‘08 here) Far be it from me to speak for Lena, but it seems to me that the existence of a blockbuster mobile phone that has built-in pathways to connect to sensors could greatly expand the impact of work like this, and smooth the transition from research prototype to commercialized product.

But the porting of a robust prototyping platform (like Arduino, for example) to the iPhone would also have implications in the earliest stages of research. Being able to mock up several different physical design directions and test them with potential users without having to worry that the technical infrastructure is going to come crashing down at any moment is every researcher’s (and prototyper’s) dream.

Of course, Apple has a long history of closing down such opportunities in the name of convenience and simplicity, but we can still hope that this time is different. It seems that the infrastructure for some serious mobile ubicomp is finally taking shape.  Now we have to go out and build it.

A major global consumer bank was looking to extend its mobile relationship with its customers beyond just balance inquiries and ATM finders.

As the User Experience designer on the project, my challenge was to fit all this functionality into a simple interaction model that could be easily adapted for future functionality and worked on a number of displays, including an associated website where consumers could perform more detailed tasks.

A mobile application for managing your financial life.
Platforms: Mobile app, PC website

A major global consumer bank was looking to extend its mobile relationship with its customers beyond just balance inquiries and ATM finders.  The service, to be launched later this year, is a Java app for mobile phones that lets users see their transactions, create and manage a simple budget, monitor and trade stocks, find and redeem coupons, and even send money mobile-to-mobile.  They also wanted to launch the service simultaneously in two pilot markets: one in the US and one in Asia.

As the User Experience designer on the project, my challenge was to fit all this functionality into a simple interaction model that could be easily adapted for future functionality and worked on a number of displays.  They also wanted a companion website which would work in coordination with the bank’s existing website to allow consumers to manage their mobile payments, stock trades, budget and other functions unique to “Mobile Money.”

In order to make sense of it all, I approached this project in true user-centered design fashion: first by visiting the target markets and observing mobile phone use, then creating storyboards based on scenarios rooted in my observations.  From the storyboards I generated a series of paper prototypes, which we user tested in both markets.  I also worked closely with the Creative Director to make sure that the visual look and the wireframes were in sync, and the Solutions Architect to make sure that what we were proposing was feasible.

An iPhone version using our IA and design templates has launched for the Hong Kong market, and the J2ME versions are due early 2009.

In this paper, we present initial findings from the study of a digital photo-sharing website: Flickr.com. In particular, we argue that Flickr.com appears to support—for some people—a different set of photography practices, socialization styles, and perspectives on privacy that are unlike those described in previous research on consumer and amateur photographers.

Full text [PDF]

A study of photosharing theory and practice
written with Professor W. Keith Edwards, Georgia Tech

In this paper, we present initial findings from the study of a digital photo-sharing website: Flickr.com. In particular, we argue that Flickr.com appears to support—for some people—a different set of photography practices, socialization styles, and perspectives on privacy that are unlike those described in previous research on consumer and amateur photographers.

Full text [PDF]

Photo credit: 3blindmice (flickr)

This in some ways is an extension of a process that began with the shift away from software-as-artifact.  When applications were primarily delivered in physical containers and installed into computers, adding functionality to a particular machine required attention and preparation.  This is still true for domain-specific programs like video-editing suites; the user must be prepared to restart the computer at least once, and will often set aside time for the installation process.  Sometimes an IT admin is required to get everything working correctly.

In contrast, the software-as-a-service model championed by Google and others eliminates installation requirements, local configuration (although enabling “offline mode” for these applications does still feel like an installation process) and frees users from ties to a single computer.  Users of Google Docs, for example, can access and edit their documents on multiple computers without separate installs.  I myself have used this feature to take work documents home without lugging my work laptop.

Although software-as-a-service has yet to catch on with many people (just one look at the sales figures of MS Office should be all you need) offerings like Evernote point to an evolution of the concept even before it’s reached mass adoption: software-as-a-platform.

Evernote is certainly not the first “software platform.”  I’d credit that to a service like GMail, which is accessible as a web app, through POP and IMAP interfaces (essentially email APIs), several versions of mobile web app (WAP, static mobile HTML, Javascript-enabled HTML, iPhone-specific HTML), mobile Java app (for phones like Nokia’s E and N series), low-end web interface, and probably a bunch that I’m not aware of. I bet it even works over SMS. All I need to know is that GMail works on any device I’d happen to want to use it with.

And that’s the point. We are in the middle of a (wait for it) modular decomposition inflection point in personal computing.  Instead of the original vision of ubiquitous computing, in which computers are embedded in everyday objects and networked computing recedes into the background, we seem to be working with ever more explicit computing devices.  We have our pocket communicator, perhaps a work computer, maybe a home media server, a DVR, maybe a netbook perhaps? This is starting off in higher-income households, but the proliferation of domain-specific computing devices is infiltrating all levels.

EverNote is a particularly interesting example because while GMail took an existing function (social messaging) and ported it to every platform, EverNote is trying something much more ambitious.  From the way they describe themselves, they seem to be trying to allow you to capture, store and annotate anything that is of interest to you, and access it from any device you may wish to, even those not connected to a network at present. EverNote allows users to upload webpages, photos, to-do lists, etc. and view them on any computer or mobile device. On the iPhone, for example, users can take a photo and upload it on the go, check items off a to-do list, or start a new note. Then everything is synced to all versions, be they the website or the OS-specific downloadable application.  Users of the application can even use EverNote while offline, and then the changes are synced with the central server when the computer has network access again.  The platform is flexible enough that you could write a novel on it if you wanted.

I think EverNote is just the first. All sorts of things that we never thought of as “software” will mingle with functions we thought would never escape the desktop.  Our data will exist on individual devices, but also on servers coordinating the devices.  We’ll be able to get our work (or play) done even when we switch off the wifi (this generation’s “leave the phone off the hook”) and then sync it again when we find a signal.  We’ll be more connected than ever, and we’ll love it. Neal Stephenson was right.

MOSH enables mobile users to upload and download video, images, applications, games, documents and pretty much anything else, from and to a mobile phone.  

I managed the User Experience design from the Alpha launch onwards.  The primary challenge came from our insistence that MOSH be a true multi-platform service, with complete feature parity across mobile & web.  When it launched in 2007, it was certainly the most full-featured mobile website on the market, and continues to be so today.  Keeping all this functionality tidy and accessible when needed was a major challenge that we addressed through constant iteration.  

A mobile content sharing social network  
Platforms: Mobile web, mobile app, PC website

With MOSH (short for Mobilize & Share), Nokia wanted a way for mobile developers to be able to distribute their applications and content more widely and for their users to feel like participants in the process.  What they got was much more powerful than that.  

MOSH enables mobile users to upload and download video, images, applications, games, documents and pretty much anything else, from and to a mobile phone.  MOSH automatically detects the user’s phone and serves up compatible content.  Logged-in users can upload, download and bookmark content on the site, as well as set up profiles, message other users, and see a running activity feed of their friends’ uploads, downloads and bookmarks. MOSH users can also “seek” new content, placing a request to the community which is filled by other users, often through the upload of new content.

I managed the User Experience design from the Alpha launch (90 days after starting the project) through Gamma, for a total of 9 months.  The primary challenge came from our insistence that MOSH be a true multi-platform service, with complete feature parity across mobile & web.  When it launched in 2007, it was certainly the most full-featured mobile website on the market, and continues to be so today.  Keeping all this functionality tidy and accessible when needed was a major challenge that we addressed through constant iteration.  Each phase was completed on an incredibly compressed time scale, leading often to wireframes heading straight to server-side developers, and we made tweaks to the UI literally minutes before pushing it live.

The service continues to evolve and is available on the web at http://mosh.nokia.com and from your phone at http://mosh.nokia.mobi. It is also available as a mobile app and is pre-installed on many Nokia phones.

For the re-launch of TurboNick, Nickelodeon’s online video hub, the Nick team wanted to dramatically increase the amount of video content available online, including full episodes and clips from over 30 properties, as well as integrate games and social features. The result — TurboNick 2.0— is a multiplatform world of video that feels equally at home on the big screen and in a browser window.  

As a part of the User Experience design team, I worked closely with two sets of creative designers (one for web, one for Media Center) to create a unified interaction model and look and feel that transitioned well from the two-foot .com to the ten-foot Media Center experience.  

A super-charged world of video for kids
Platforms: ITV (Media Center Vista) and web

For the re-launch of TurboNick, Nickelodeon’s online video hub, the Nick team wanted to dramatically increase the amount of video content available online, including full episodes and clips from over 30 properties, as well as integrate games and social features. The result — TurboNick 2.0— is a multiplatform world of video that feels equally at home on the big screen and in a browser window.

As a part of the User Experience design team, I worked closely with two sets of creative designers (one for web, one for Media Center) to create a unified interaction model and look and feel that transitioned well from the two-foot .com to the ten-foot Media Center experience. The video is always on screen, with video selection and discovery happening on the left, bookmarking and games on the right, and metadata within the orange video blob.

The whole experience feels very responsive, as the interface elements squish and slide across the screen. While it may feel hectic and busy to adults, this hyperactive design seems to perfectly suit the target audience (8-12 year olds). It has been such a success that the design has changed relatively little in the two years since the launch.

Check it out at http://www.turbonick.com or on your Vista Media Center PC.