Explicit Tagging on the net; Implicit tagging in companies

Collaboration is a hot topic among folks who prognosticate about the future of enterprise search. It's made such a positive difference for internet search at sites like Flickr and among the Facebook-type networking sites of the world, it only makes sense that it should be able to help enterprise search as well.  In his recent book "Everything is Miscellaneous", David  Weinberger talks about the the value of meta-data in organizing content now that everything is in electrons rather than in the physical world of atoms. 

He talks about Flickr as an example of meta-tagging - explicit tagging - improving retrieval. Flickr has 8 million registered users, but on a recent visit to the site, it reported that 5000 pictures had been uploaded in the minute just prior to my visit. That must take an incredible amount of tagging just to break even!

By the way, one benefit of massive meta-tagging, as Weinberger points out, is that companies like Flickr, using some smarts behind the scene, can effectively "learn" to associate "Golden Gate Bridge" and "San Francisco”. This is no doubt similar to the technology that leading enterprise search vendors   are beginning to incorporate around  “fact extraction".

A Problem of Scale

Even though we are fans of explicit tagging as implemented in Flickr, the problem we see with tagging in the enterprise is the significantly smaller base of potential taggers inside the firewall.

Let's consider Cisco, a high tech company with a large store of online content and a highly motivated, highly technical employee base. They have about 60,000 employees; let’s assume that 5% of those employees would actively tag documents given the ability. That means there may be up to 3,000 active collaborators over time. But with an intranet of millions of pages, it's going to be a while before any significant number of pages has useful tags.

The good news is that we do see the solution in the intranet as a more implicit form of 'tagging': document views for a given query. In the same way that someone tagging a picture on Flickr is adding an explicit "vote" associating a picture with a term, a corporate user is entering an implicit  "vote" for a document when he or she opens a document after a search. That is, the user "tags" the document in question with the search term(s) used to find it. When we can find a way to automagically tweak the relevance of a document for a given term without having to do any special handling, then collaborative technology will have found a niche in enterprise search.

Of course, there are always fringe cases: what if a user opens a document and finds the document is totally wrong? Won't that rank a document higher? The answer is yes - but the fact that the boost is a tiny one means the document will only marginally have a better score. And trust us, if you provide a document feedback capability for your users, you'll hear about the bad documents and you can offset the "mistaken" tags with the "thumbs down" votes. We think over time, this implicit tagging will work far better in the corporate environment, even if human-provided explicit tags will continue to be better indicators.

Now to see if any vendors are using that technology now. Do you know of any?