Enterprise Search

My search platform ate my homework

In a recent article on inforword.com, Peter Wayner wrote a nifty piece discussing 11 programming trends to watch. It's interesting in general, but I found one trend really rang true for me with respect to enterprise search.

He calls his 9^th trend Accuracy fades as scalability trumps all. He points out that most applications are fine with close approximations, based mainly on the assumption that at internet scale, if we miss an instance of something today, we'll probably see it again tomorrow. That brought to mind something I'm working on right now for a customer who needs 100% confidence in their search platform to meet some very stringent requirements. The InfoWorld article reminded me of a dirty little secret of nearly all enterprise search platforms, a secret you may not know (yet); but which could be important to you.

Search platform developers make assumptions about your data, and most search platforms do not index all of your content... by design! Don't get me wrong: these assumptions let them produce pretty good accuracy every time; and even 100% accuracy sometimes. And pretty good is fine most of the time. In fact, as a friend told me years ago, sometimes 'marginally acceptable' is just fine.

The theory seems to be that a search index might miss a particular term in a few documents, but any really important use of the term will clearly be indexed somewhere else and our users will get results from these other documents. In fact, some search platforms have picked an arbitrary size limit, and won't index any content past that limit even if it misses major sections of large documents. Google, in fact, is one of the few who actually document this - once the GSA has indexed 2 MB of text or 2.5MB of HTML in a file, it stops indexing that file and 'discards' the rest. This curious behavior works most of the time for most data (although there is an odd twist that will bite you if you feed GSA a large list of URLs or ODBC records). To be honest, most search platforms do this sort of trimming as well; they just don't mention it too often during the sales process.

Now, in legal markets like eDiscovery, it's pretty darned critical to get every document that contains a particular term. It's not OK to go to court and report that you missed one or more critical document because your search engine truncates or ignores some terms or some documents. That excuse might have worked in elementary school or even in high school, but it just doesn't cut it in demanding enterprise search environments.

It may not be a problem for you; just be sure that, if it is a requirement for you, you include it in your RFI/RFQ documents.

 

 

Are you spending too much on enterprise search?

If your organization uses enterprise search, or if you are in the market for a new search platform, you may want to attend our webinar next week "Are you spending too much for search?". The one hour session will address:

• What do users expect?
• Why not just use Google?
• How much search do you need?
• Is an RFI a waste of time?   

Date: Wednesday, November 16 2011

Time: 11AM Pacific Standard Time / 1900 UTC

Register today!

Searching for Sarah at SharePoint Conference 2011

Just noticed one of the most interesting sessions at last May's Enterprise Search Summit is coming to the October Microsoft SharePoint Conference! We blogged about it back in May.

Basically, Booz & Company did an evaluation of SharePoint 2010 search - FAST Search for SharePoint as I recall - versus the Google Search Appliance they had been using. At one point, the search business owner was trying to find the last name of a woman she had met in the firm; and when she searched for 'Sarah', hoping to find her in the directory, the GSA returned 60 men in the result list. Can you guess why? A hint: metadata (check the earlier article, or come to SPC 2011 to find out).

Now in fact, we think the GSA could have been tuned to emulate this OOB behavior by SharePoint; but this is a reminder that not every search platform works great in every environment. Buyer beware!

Ever had a similar experience? Let us know about it!

 

It's not Google unless it says it's Google

A few years back, one of our customers told us that, if he could just license the 'Powered by Google' icon, he was sure most of the users would stop complaining. Not long after that, we heard that our friend Andy Feit, who was VP of search at Verity, hired a marketing research team to compare the quality of search engines when one was "Powered by Verity" and the other was 'Powered by Google'. Andy found that people thought the Google results were significantly better - even though both test cases were, in fact, powered by Verity. The mere presence of the Google icon seemed to make people think the results were better.

At the recent ESS, a woman from Booz & Company talked about their previous enterprise search experience involving Google. A few years back, Booz used FAST ESP on SharePoint 2003 and it simply sucked. Users asked for Google by name. When they upgraded to SharePoint 2007, Booz gave the users what they wanted: they went with a Google Search Appliance. The trouble was that they built a custom interface with a generic search button. Users' responses? "Search still sucks - why don't we just use Google?" even as they were using Google!

This can teach us a number of lessons:

1. Analyze what you need search to do for you before you buy it.

2. Understand how your content and search platform play together.

3, It ain't Google unless you tell your users it's Google.

By the way: in 2010 Booz rolled out FAST Search for SharePoint, and it seems that the results are a bit better now that they understand their search requirements and the nature of their content and metadata.

 

 

Google and the official search blog

A couple of days ago, Google started Inside Search, the 'official Google search blog'. It's not really enterprise search news, but because so many knowledge workers compare the behavior of their internal search platform with the Google public search experience, it may be worth monitoring for those whose job it is to keep enterprise search going.

 

Content owners don't care about metadata

Or do they?

Our recent post about Booz & Company's 'men named Sarah' highlights just how important good metadata can be in order to provide a great search experience for employees and customers.

One of our customers who spoke at the recent ESS 2011 in New York provided some great insights into the problems organizations have getting employee content creators to include good metadata with their documents.

During the ESS talk, they report that content owners don't really seem motivated when asked to help improve the overall intranet site by improving document metadata. However - and this is a big one - when a sub-site owner sees poor results on their own site, they are willing to invest the time to provide really good metadata.

[A bit of background: This customer provides a way to individual site owners within the organization to add search to their 'sub site' pretty much automatically - sort of a 'search as a service' within the enterprise.]

So if you've been thinking of adding the ability to search-enable sub-sites within your organization, but solving the relevance problem is your first task, you might reconsider your priorities!

/s/Miles

Sixty guys named Sarah

We're always on the lookout for anecdotes to use at trade shows, with our customers and prospects, and of course here in the blog, so I have to report that we heard a great one last week at Enterprise Search Summit in New York.

The folks from Booz & Company, a spinoff from Booz Allen Hamilton, did a presentation on their experience comparing two well respected mainstream search products. They report that, at one point, one of the presenters was looking for a woman she knew named Sarah - but she was having trouble remembering Sarah's last name. The presenter told of searching one of the engines under evaluation and finding that most of the top 60 people returned from the search were... men. None were named 'Sue'; and apparently none were named Sarah either. The other engine returned records for a number of women named Sarah; and, as it turns out, for a few men as well.

After some frustration, they finally got to the root of the problem. It turns out that all of the Booz & Company employees have their resumes indexed as part of their profiles. Would you like to guess the name of the person who authored the original resume template? Yep - Sarah.

One of the search platforms ranks document metadata very high, without much ability to tune the weighting algorithms. The other provides a way to tune the relevance; but it also tends to rank people relevance a bit differently - probably stressing documents about people less than the individual people profiles. The presentation was a bit vague about whether any actual tuning that might impact these differences on either platform.

The fact that one of the engines did well, and one did not, is not the big story here - although it is something for you to consider if you're evaluating enterprise search platforms. The real lesson here is that poor metadata makes even the best of search platforms perform poorly in some - if not most - cases.

 

Make your search engine seem psychic

People tell us that Google just seems to know what they want - it's almost psychic sometimes. If only every search engine could be like Google. Well, maybe it can.

Over the years, the functions performed by the actual 'search engine' have grown. At first, it was simply a search for an exact match - probably using punch card input. Then, over time, new and expanded capabilities were added, including stemming... synonyms... expanded query languages... weighting based on fields and metadata.. and more. But no matter what the search technology provided, really demanding search consumers pushed the technology, often by wrapping extra processing both at index time and at query time. This let the most innovative search driven organizations stay ahead of the competition. Two great examples today: LexisNexis and Factiva.

In fact, the magic that makes public Google search so good - and so much better than even the Google Search Appliance - is the armies of specialists analyzing query activity and adding specialized actions 'above' the search engine. 

One example of this many of us know well: enter a 12 digit number. if the format of the number matches the algorithm used by FedEx in creating tracking numbers, Google will offer to let you track that package directly from FedEx. For example, search for 796579057470 and you see a delivery record; change that last 1 to a zero, and you get no hits. How do they know?

The folks at Google must have noticed lots of 12 digit numbers as queries; and being smart, they realized that many were FedEx tracking numbers. I imagine, working in conjunction with FedEx, Google implemented the algorithm - what makes a valid FedEx tracking number - and boosted that as a 'best bet'.

Why is this important to you? Well, first it shows that Google.com is great in part because of the army of humans who review search activity, likely on a daily basis. Oh, sure, they have automated tools to help them out - with maybe 100 million queries every day, you'd need to automate too. They look for interesting trends and search behavior that lets them provide better answers.

Secondly, you can do the same sort of thing at your organization. Autonomy, Exalead, Microsoft, Lucene, and even the Google Search Appliance, can all be improved with some custom code after the user query but before the results show up. Did the user type what looks like a name? Check the employee directory and suggest a phone number or an email address. Is the query a product name? Suggest the product page. You can make your search psychic.

Finally, does the query return no hits? You can tell what form the user was on when the search was submitted - rather than a generic 'No Hits' page. Was the query more than a single term? Look for any of the words, rather than all; make a guess at what the user wanted, based on the search form, pervious searches, or whatever context you can find.

So how do you make your search engine seem psychic? Learn about query tuning and result list pre-processing; we've written a number of articles about query tuning in our newsletter alone.

But most importantly: mimic Google: work hard at it every day.

/s/Miles

 

 

 

 

Enterprise Search Summit DC November 15-18

The new home for the Fall ESS show is the Renaissance Hotel in downtown Washington, DC... so much for ESS-West! The new locale should bring a large number of new attendees and visitors, and a new co-located conference: SharePoint Symposium. InfoToday knows a trend when they see one!

In addition to the usual sessions provided to show sponsors, there are some interesting sessions by Tom Reamy of KAPS Group; Martin White of Intranet Focus; and eDiscovery expert Oz Benamram, CKO of White and Case LLP. Tony Byrne of Real Story Group will also be there, moderating the session I'll be participating in: Stump the  Search Consultant on Wednesday afternoon November 17th.

I really expect the show to have a large number of government folks in attendance, given how hard it's been for these good folks to travel to previous ESS conferences in New York and San Jose. InfoToday reports higher pre-registration this year than in the past; and I'll be happy to find out I'm wrong about most of the attendees being government or government-related folks.

Come by the session Wesnesday afternoon at 3PM; or leave a comment here if you want to get together.

 

 

Google I/I Open for registration!

Google has announced its Google I/O 2010 to be held in San Francisco May 19-20 at the Moscone Center.

I think this is their third such annual event, and it's always been a full two days of information. The good news is the price is $400 per person (until April 15), a bargain really. The bad news? You'll need to bring four or five people from your company to hit all of the sessions in each track!

This conference is VERY technical, VERY good. You get the most from it if you are a developer, you know Java, Ajax, Python, or the other technologies Google uses in its various products. You won't find much in the way of marketing fluff here: in our experience, most presenters are Google developers.

The conference is being held the same week that Gilbane content management conference comes back to San Francisco. Bad timing for them, but good for you: you can probably walk to the nearby Westin at lunch and maybe catch the exhibits.

Last year, attendees received a free phone for development purposes on the Android OpSys; who knows what they might give away this year - besides the expected cool T-shirt!

Register at http://code.google.com/events/io/2010/.