In part I of my interview with Yahoo’s Bryce Glass, Bryce explains the basics of Reputation Systems. In the following part II, Bryce digs deeper into the strategic decisions around patterns, like how to determine which pattern to use and who is using reputation patterns best.

6) So let’s say I’m building a site with social networking features. How do I figure out which reputation pattern to use?

My advice would be: start small. Definitely think of which one or maybe two patterns to employ. Consider the spirit and intent of the community that you’re trying to build. What exactly do you hope people will do there? Write reviews? Post videos? Form connections with each other? Consider all of these things, and others, and then try to place your intended community somewhere along that Competitive Spectrum.

And be honest with yourself! I’d posit that the vast majority of consumer-focused social sites are somewhere at or below the ‘Cordial’ level. If you still insist on placing a ‘Top Hunks’ leaderboard on your dating site after all of that? Well… that’s fine, but at least now you’ve made that decision consciously and clearly. (It’s probably still the wrong decision, but…) And, not to re-plug my earlier talk, but more guidance is given there as well on this very question of ‘which pattern?’

7) What is the biggest mistake that designers make when implementing reputation patterns?

I’d say 2 related things: one is employing those more empirical patterns— Points, and Levels, ranked and tracked on Leaderboards— in situations where they’re not appropriate. I feel like I’m belaboring the point, but… if your community values fun, and easy-going interactions with each other and helpfulness? Then don’t destroy that fantastic dynamic by comparing members, one to another. Don’t elevate certain members’ status at the expense of everyone else in the community—’cause resentment, factions and gaming are soon to follow.

And related to this is the mistake of rewarding the wrong types of behavior. Specifically, there’s a tendency to want to reward activity (how many times have I contributed, or how frequently) instead of the quality of those contributions. (Do people like this video? Have they watched it? Responded? Linked to it, or embedded it on their blog? Voted for it, or assigned a rating?) Of course, both are important: you want people who are actively engaged and prolific contributors: but you want those contributions to be quality ones: thoughtfully prepared, formatted along community norms, and above all useful or interesting to the community.

A relevant, and recent, example I could cite is Plurk. Now, I absolutely don’t mean to hate on Plurk. It looks like a fine product (it’s kind of a Twitter-like microblogging platform.) But they’re tracking and displaying some very “official-looking” Karma metrics, and even feature a Leaderboard of Interesting Plurkers. My response to this is two-fold: first is… “why”? What community goals does it further? My guess would be that it’s a desire to promote active, high-use Plurkers to the community, that others might find them and opt to follow them as well.

But the prominent Karma score, and a surface appraisal of how it’s generated, might lead one to believe that Plurk is a competition. And, specifically, a competition won by the amount of stuff you do! (Number of Plurks, number of friends, etc.) Most people can see how badly this could end: if someone really wants to make it onto that leaderboard? They’ll probably try mass-friending and spam-blasts first. (Even if Plurk’s system is smart enough to counter this, the overall effect is still negative.) There is a nod to quality—’Quality Plurking’, however that’s defined—but the emphasis appears to be on Activity. And I’d posit that a karma system for an app like this is somewhat extraneous. It kinda smacks of “wouldn’t it be cool if we…”

I also feel compelled to point out that the particular label they use—’Interesting’—is a loaded one: while very complimentary to those who receive it, it’s can also feel derogatory to those who’re left out. There’s a reason why Flickr has only ever applied the descriptor of interestingness to photos, and not the people that take them—and that reason is that the community folks over there have a wonderful awareness of community spirit, and are sensitive to the effects that labels can have.

8) What is the best example that you know of of a site that implements reputation patterns?

I don’t know if it’s the absolute best, but a site that I’ve praised in the past is Yelp. They’re a review site and they feature a nice variety of reputation indicators. What I like about Yelp is that they seem to have payed close attention to their community, and what motivates people to write reviews, and their reputation system leverages that nicely. It doesn’t work against it.

A really simple example: some of our own research, at Yahoo, indicates that one reason some people may write reviews is just this desire to ‘fill a void’ or provide a review for a product or venue that has none. (I’ve wondered if this isn’t somehow psychologically related to those guys that like to type ‘first!’ into comment fields.) Now of course this isn’t the only thing that motivates someone to write reviews, but it can be a small motivator for some folks.

Yelp must be aware of this tendency, cause they give users a small boon for being the first person to contribute a review for a business. The first review for any establishment will display a ‘first to review’ badge for ever-after. So it’s not a huge thing. They don’t place a lot of importance on it, but there it is: a small and very natural show of appreciation for those users that like to help get the conversation started.

And Yelp does this in a dozen other ways as well. They have specific reputation types that reward funny Yelpers, or helpful ones. They have a special designation (the Yelp Elite — you’ve written about them before, in fact.) So Yelp encourages a wide range of expression from their review-writers: basically, you can be any kind of ‘Yelper’ you want to be, and—as long as the community finds value in your contributions—Yelp has a way of rewarding you. (And Yelp doesn’t ‘rank’ users against each other, or display a leaderboard anywhere on the site.)

And, of course, I’ve already mentioned XBox Live. I think they do a fantastic job. I believe that they employ just about every pattern from the set that we’ve published, and probably a couple others besides. And all for great effect, for a very specific purpose. BUT… they’re a fairly competitive context, so I think they get a lot of leeway to do things that a lot of social community sites should probably not be doing.

9) What is Yahoo’s strategy in getting these out to the community? Are you simply being altruistic? Wouldn’t these help your competitors?

There are a couple of dimensions to my answer here. First, I’d say that the Pattern Library, in general (which has been open since February of 2006, btw) is a good fit for Yahoo!s stated goals of openness and transparency.

Secondly, there’s nothing especially proprietary about the information or opinions embedded in the patterns. Christian, who I’ve mentioned, actually vets all of our public patterns with our Legal team, so if there actually were some sooper-sekrit game-changing reputation business logic in there…? Well, that probably wouldn’t make it outside the firewall.

But, also, these patterns are in large part drawn from examples and experiences of competitors, as well as products that we’ve shipped at Yahoo! So in a way, it’s not so much ‘getting them out’ to the community as giving them back to the community. There is some work involved in these patterns (compiling, researching, refining and writing them out) but the benefits for us are innumerable: the ability to positively influence the community, be seen as thought-leaders in social software. Heck, just taking part in a smarter dialog about the place of reputation systems… it’s all good.

Thanks for the interview, Bryce!

Resources:

• Yahoo’s Social Design Patterns for Reputation
• Bryce’s Designing your Reputation System slide deck
• Bryce’s personal blog

Of all the social software built on the web in the last two decades, none are as important yet as little talked about as reputation systems. Reputation systems have driven the entire business at eBay.com, much of the business at Amazon.com, drives activity at Digg.com, powers the moderation system at Slashdot, etc…and yet for all the millions of words written about web design very few of them have been dedicated to this type of software.

That’s why I’m really excited about the recent release of Yahoo’s social design patterns for reputation systems. The following graphic from the pattern library illustrates what Yahoo calls the “competitive spectrum”, which is a way to classify the activity on your site and helps you to choose which reputation pattern might work best for your community.

My first introduction to the patterns happened by accident. I needed a room for this year’s IASummit and I twittered about not having one. I received a tweet back from Bryce Glass, whom I knew of but didn’t know personally. He was gracious enough to let me share a room with him and over the couple of days I was there Bryce told me about some of the fascinating work he’s doing with Yahoo’s various properties.

Bryce Glass is an interaction designer at Yahoo! He currently works in the user experience group supporting the recently-announced Yahoo! Open Strategy. But in the past year, Bryce was the user experience lead for a Reputation Platform that powers the rep systems for many of Yahoo!s properties. In that time, he had opportunity to work with several different community-oriented sites on Yahoo! on improving their reputation systems, increasing user engagement and generally creating friendlier, more-active communities.

In the talk he gave at the summit, Designing your reputation system, Bryce outlined a framework for designing reputation systems that is probably the best starting point for figuring out how to build your own. In the following interview, he starts from ground zero and explains what reputation is, what a design pattern is, and how to start applying them to your own work.

1) So, Bryce, what is your definition of reputation?

I generally use a fairly coarse-grained definition of ‘reputation’ when discussing these patterns or other work we’ve done with reputation at Yahoo! By my simple definition: one’s reputation in a community is both a history of one’s past actions within that community, and a value judgment about the worth of those actions.

Who makes that value judgment? Ideally, the community itself. So contributions that the community values are good, and those it finds objectionable (or simply has no interest in) are bad. And you, as the author or originator of that content, can be judged according to these same values. Quite simply: contribute to the community and—if the community likes it—your reputation rises.

Obviously, it’s a little more nuanced than that, but…

2) So what is a reputation design pattern?

This family of patterns are a small subset of the Yahoo! Design Pattern Library, which has been out ‘in the wild’ for a couple of years now. I’d suspect that many of your readers are at least passingly familiar with that effort (and, if not, my fellow Yahoo Christian Crumlish has a good video update up on our Developer Network.)

These patterns are notable, however, in that they’re the first social design patterns that we’ve included. I know that this is a direction that Christian wants to take the library in, so expect to see more and more of these in coming months. (Identity representation, participation, more reputation, etc.) But… going back to your question… this family of reputation patterns can be thought of a couple of different ways.

First, and foremost, it’s just an attempt to catalog and describe a number of common ways to represent some one’s status within a community. They are patterns that recur across websites, games or other interactive communities and we’ve made some attempt to be as complete as possible in this release. (Tho, for the sake of finally shipping the patterns, we did omit several—I’m hoping to sneak those out over the next several months.) For the designer of a community, these patterns might he helpful in just identifying and naming a reputation feature which they’ve been considering. And providing some very lightweight guidance on where and when to employ the pattern. So you could almost think of these patterns as a ‘Chinese menu’ of options for showing some one’s reputation on your site.

But this was a concern to us as well: through working with Yahoo! property-owners internally (the folks who make decisions about things like our Finance site, or Yahoo! Local, Maps, Answers, etc.) we understand that sometimes a long list of options is actually less helpful than no guidance at all. So we pushed hard to include at least one meta-pattern, The Competitive Spectrum, that attempts to provide guidance about which of the other patterns to employ. It attempts to provide a selection rationale.

In truth, there’s so much more guidance we could have included (the Spectrum is but one of 15 questions that I covered in a presentation on reputation systems earlier this year at the IA Summit.) But this one central question is an important one for anyone attempting to influence the community spirit of any ‘place’ they’re building online: what type of community will this be? A friendly place, where people cooperate? Or a competitive, cut-throat one with winners and losers?

3) Although reputation is sometimes bestowed on an individual by a community, what steps can an individual take to improve their reputation?

Tho’ it may sound… un-profound (is that a word?)… my advice here is simple. To improve your reputation in a community, become valuable to that community! There are a number of simpler ways to state this as well: Don’t be a jerk! Play nice with others! Do unto others as you’d have done unto you. It was all good advice when our mothers gave it to us in our youth, and is still good now.

But it gets tricky. I’d mentioned how reputation is very contextual. So you have to keep in mind the relative values of that specific community. Maybe your idea of ‘being a jerk’ is, in fact, not at all in agreement with the majority opinion of the community. Some communities on the web are built on ‘jerks’ and they’re fine with that reputation. Perhaps they value being a funny jerk, and you just don’t want to be boring in that community.

So I would say… hang out in a community before you start participating and try to learn its values. Then become an active participant, and try to figure out a way to become a valued participant. And, you know what? If it ends up that the communities values simply don’t agree with yours, then maybe that’s a sign you should find a community to frequent that’s better aligned to your values.

4) What are the biggest hurdles in designing for reputation?

I think it’s probably the number and variety of unintended consequences that little design decisions can have further down the line. I’m fond of the article—so I cite it a lot—but Ben Brown, who founded the dating site Consumating, has a great blog-post about the ‘ill-fated points system’ that they used for that site, and the variety of… um… less-than-ideal behaviors that those incentives gave rise to. Early on, Slashdot struggled with many of these same issues, and they’ve re-jiggered their comment karma system several times through the years.

A lot of these learnings are, in some small way, encoded into these patterns that we’ve released, as well as research that we’ve done at Yahoo!, and products that we’ve released, tweaked, tinkered and learned from.

A big hurdle—and if you can solve this, you’re halfway there to having a well-designed and effective reputation system—is appropriately marrying the incentives that you offer your users to the appropriate set of goals that you have for your community. You want to be sure that you’re rewarding folks for behaving like good citizens, and not just rewarding them for no good reason. (Or for vague and misguided reasons like “to keep them engaged” or “so we can have a leaderboard.”)

5) What is the biggest misconception?

My own belief is that community designers today are a little too enamored of the “allure” of these types of systems. They are, indeed, quite powerful patterns. Microsoft, for instance—I think it’s fair to say that they’ve been quite pleasantly surprised with the success of their XBox Live Achievements program: it’s a huge revenue driver (gamers will actually buy or rent games they have little or no interest in strictly to unlock that game’s achievements!) and a fan-pleaser (witness the dozens of community sites dedicated to tips and tricks about unlocking achievements.)

BUT… and this is a big ‘but’… XBox Live is a very specific context. I would place (parts of) that service to the extreme far-end of the Competitive Spectrum. And it should be pointed out that not everyone who plays on XBox Live universally buys into this ‘competitive’ mindset. (Read the comments here for a nice overview of contra-Achievement viewpoints.)

So, it’s a powerful feature and a real showcase example for the pattern that we’ve labeled Collectible Achievements but you should really think twice before employing it on your own community site. Ask yourself: will it inculcate a certain competitive mindset in the community? Do I want that? Who will be motivated by collecting achievements? Will they be too motivated, and act out in anti-social ways to get them? Who will be turned off by them, and leave the community? Am I okay with that?

So a big misconception, currently, I think is “we should be doing this.” We should have an explicit reputation system, with badges, points, voting, thumbing up and down. All the bells and whistles. I hope that sites will soon start to employ a more measured, more intelligent approach to designing these systems.

Read Part II of this interview: Social Design Patterns for Reputation Systems: An Interview with Yahoo’s Bryce Glass (Part II)

It has long been known that savvy restaurants use a bag of tricks to build buzz and interest. One trick is to seat early customers near windows so that people passing by will think the place is full. This has the effect of making the place seem popular as people usually can’t see the empty seats that are further inside the restaurant.

A second trick restaurants use is to create a line out the door so that people think there is strong demand. This is also often artificial, making us think that many people are waiting to get in. Sometimes they merely create lines by not letting people sit down, making an excuse that the empty seats are “reserved”. Other times they simply don’t let people in. This is often practiced by nightclubs, who rely even more on mystique and exclusivity than restaurants do.

These techniques leverage powerful social behavior. When people are searching for a place to eat, they rely on the behavior of others to help them make their decision. They seek out signs of life…signs that other people are present and already doing something. If they are doing it, it must be worth it, we think. Given the choice between something that nobody has chosen to do and something that many people are doing, it is human nature to gravitate to what others are doing.

On the web, signs of life are extremely important, for several reasons.

• Too much choice
  One reason is the sheer amount of choice we face. As the web continues its torrid growth, we simply have too many web sites to sort through, too many places to buy products from, too many software providers to pick from.
• Black box of use
  Another reason signs of life are important is that web applications are like black boxes. Many applications, like Google Docs, for example, require a login to use. Because of this we simply can’t see what others are doing with the software. We can’t see if they’re using it well or not using it at all.

When sites leverage signs of life well, it provides welcome direction for folks trying to make a decision. A great example of signs of life is the Freshbooks home page, which contains an interface element called “Some of our happy users…”.

The happy users element does many things well. Most importantly, it is authentic. You immediately get the sense that these are real people who actually do like the product. The pictures are decent, but not airbrushed or overly produced. The quotes sound like real people, not infomercial-like. Subtle touches like using people from all over the world and including team size add to the sense that these people are just like you: the intended audience.

In addition to these testimonials, there are many other ways to leverage signs of life. I describe several more in Designing for the Social Web. But though leveraging signs of life in your design is powerful, it must be authentic. You can’t use stock photography and made-up quotes and expect people to react positively to them…people can smell fake a mile away.

While the Freshbooks people aren’t actually standing in line outside the door of a restaurant, they might as well be. They’re having the same effect: showing others there are people here using this software…acting as signs of life on what could otherwise be a desolate home page.

Tucked away as part of the new Open Social initiative launched last week, Google engineers offered an interesting best practices document of social design dos and don’ts.

Social Design Best Practices

The list of best practices are as follows:

1. Engage Quickly – (my interpretation: provide value within 30 seconds)
2. Mimic Look and Feel – (make your widget look like the page it is in)
3. Enable Self Expression – (let people personalize their widgets)
4. Make it Dynamic – (keep showing new stuff)
5. Expose Friend Activity – (show what friends are doing)
6. Browse the Graph – (let people explore their friends and friends of friends)
7. Drive Communication – (provide commenting features)
8. Build Communities – (expose different axes of similarity)
9. Solve Real World Tasks – (leverage people’s social connections to solve real problems)

This list is interesting for several reasons.

One is that we’re clearly seeing a set of practices emerge across all social software that centers around getting people started quickly, allowing for self-expression, engaged in real-life tasks, yet also allowing for flexible discovery and play. On both this site and others concerned with social design, these are the major themes that arise again and again.

Another is how quickly the social networks have changed the way we look at software in just a couple years. The third item on the list “Enable Self-Expression”, for example, would never have existed before the rise of MySpace. Facebook probably had a lot to do with “Expose Friend Activity”, which is a not-so-subtle reference to the news feed feature on that site.

Finally, I’m struck by how only two or three of the best practices are necessarily part of “social networking” software. They could be used in any kind of social software, be it productivity software for groups or even e-commerce sites that help people find the right product. That, to me, is the essence of social design. It isn’t relegated to social networking, even though the rise of social networking is what helped to clarify and refine the ideas. It’s about building software that takes advantage of social connections to provide enhanced value.

Also, note that these best practices are concerned with this particular technology. The Open Social initiative is a set of programming APIs that allows anybody to embed widgets (gadgets) within web pages (called containers). The embedded widgets can access outside services like MySpace, Orkut, and other social networks. As an simple example, I might embed a widget in my blog that shows my MySpace friends and whether or not they’re online at the moment.

Interesting bits aside, I think that the Google folks did a good job of summarizing some major issues in social design.

Add Comment (5)