TagWalk Blog

The Twitter ecosystem has lots of different reputation mechanisms at the moment and I would like to discuss these and then make my own contribution.

User Generated Directories

There are directories like wefollow.com where users add themselves by choosing which categories they belong. This is effective, but I see there are some flaws with this approach for use as a reputation system:

• The user’s motives for listing themselves in the directory is self-promotion.
• It is open to bogus claims. For example: I could tag myself as an #seo expert when I may know very little on the subject.
• Relevance and ranking. What determines which user is higher in a category than another user? Connectedness and number of followers can be used, but my single-malt whiskey followers also boost my rank in the PHP developers category.
• Staleness. The oldest users may well have the most followers and influence. How can a new-comer hope to gain rank?

Twitter Lists

Fairly new on the scene is Twitter Lists. This feature of Twitter allows users to build arbitrary public and private lists and add users to them. This could be viewed as tagging by another name. This is a great usability enhancement to the Twitter service, and they make it much easier to partition friends into subject groups. I imagine a valuable side-effect (and end-game) for Twitter is a new metric: how many lists does a user belong to? Or alternatively “how many people think this user is worth adding to a list?” Emphasis on “worth”, as in, how many users value this user?

It is early days for Twitter Lists. They are clearly a usability win! Whether they get used to build a reputation system is yet to be seen, but I think they are definitely helpful in when screening for news feed bots or spammy accounts who are unlikely to appear on any one’s lists.

My concern around any reputation system built around Twitter Lists is that it costs very little effort to add a user to a list, merely the motivation to do it. Also list membership is black and white; a user cannot be “very in a list” or “a little bit in a list” – a user either belongs to a list or does not. Michael Gray posted about How to Use Twitter Lists To Create Reputation Management Problems which illustrated how a simple act of adding a user to a list can taint a user’s reputation.

Update 14-Jan-2010: MustExist’s List Tag is a working reputation systems which derives from reputation from list membership for a given user name. Given a user’s screen name, it processes their list memberships and produces a tag cloud. For example: this is what is says about timhastings. This is good example of list-based reputation, if I was to suggest an improvement, it would be to allow the discovery of users within any given #hashtag. Thanks to @eric_andersen for the tip.

Wisdom of Crowds

In my opinion, a good reputation system should be derived from user activity and the relationships a user has with other users. I want a system which observes Twitter activity and then auto classifies users based on evidence. Each time somebody talks to me and uses a particular tag, it should increase my score for that tag. The system should be able to differentiate somebody who just talks a lot (self promotion) from somebody who is mentioned a lot (reputable). The number of different talkers using a tag, defines the size of that community.

Reputation emerges from monitoring Twitter activity and aggregating statistics.

Many Wisdom of Crowds systems are well known and very successful. The two best examples I can think of are Amazon’s Recommendation Engine and the Delicious Bookmarking system.

Amazon aggregates sales information, and computes things shopper A would also like to purchase knowing what users with similar purchase histories have also bought. Delicious is far simpler, it relies on users ‘tagging’ bookmarks; if lots of users tag a bookmark as ‘webdesign’, then it probably has a lot to do with ‘webdesign’.

The Delicious model is the closest fit for a Twitter Reputation system. With Delicious, the aggregate value is gained from individuals selfish desire to organise their bookmarks. In our Reputation system, the aggregate value is derived from individuals selfish desire to communicate, it is given a turbo boost when they use #hashtags, hyperlinks and the user names of their friends.

Demonstrating this Approach

This blog is about a project of mine to build a great Reputation and Recommendation system for Twitter. TagWalk analyzes tweets and keeps track of who said what to who. It maintains a data mine of relationships. Each relationship gets strengthened each time it sees a tweet. This is then available to explorer and discover online.

For example, if we take a well known Twitter phenomenon, #followfriday and use this as a basis for computing reputation, we can see some leaders emerge, as well as related hash-tags, and words used.

Our metrics around the volume of tweets and number of talkers helps to define the size of the audience (or niche). It also shows how a new leader can quickly emerge.

I am also fairly certain that a new breed of SEO will evolve which will specialise in Social Media Reputation Optimization (SMRO).

In Conclusion

The number of friends, followers and how many lists you belong to are not enough to build a great reputation system. To be truly great, we must pay attention to the content and flow of the messages between users.

Wisdom of Crowds – it’s the only way to be sure.