Enterprise Search

Search Quality: You Can't Improve What You Don't Measure

In our latest survey of new newsletter subscribers we found that 29% had no formal metrics for measuring quality of search results.  Search metrics allow you to keep search on the right track and can be a powerful tool for managing your systems.  They are a wonderful source for insights and trends.  We thought we would share a couple that we think work well. Many of these are covered in greater depth in Interpreting Your Search Activity Reports in the Enterprise Search newsletter.

• Count the number of people who use search  
• Count the total number of searches  
• Count the number of zero search results  
• User feedback on top 100 searches  
• Track email complaints about search  
• Measure number of clicks on navigators (navigation menu items)  
• Business Goals  
•    
  • Reduce call volume (normallized for growth in customer base) by enabling self-service from search: results are good enough to reduce calls.
  • Reduce e-mail volume (again adjusted for growth in customer base) by enabling self-service from search: results are good enough to reduce e-mails. 
  • Revenue       
  • Add-on revenue       

A proposed standard for enterprise search

Dieselpoint has announced support for a technology it calls OpenPipeline, which can enhance the task virtually every enterprise search technology uses to get documents into the search index. They will be showing the pipeline at the upcoming Enterprise Search Summit on May 20-21 integrated with their new Dieselpoint Search 4.0, also on display.

The Dieselpoint press release claims:

OpenPipeline provides a common architecture for connectors to data sources, file filters, text analyzers and modules to distribute documents across a network. It is fully functional out of the box and includes an installer, a job scheduler, file scanner and crawlers, doc filters, and point and click interface with drag and drop module installation.

OpenPipeline is compatible with IBM's UIMA (Unstructured Information Management Architecture), and is designed to connect UIMA annotators to other systems.

Document processing can be centralized or parallelized as needed. The transport mechanism is simple, web-services XML over HTTP. RSS/Atom feeds are also possible.

The development philosophy behind OpenPipeline stresses simple, elegant design, and massive scalability. Minimal external dependencies and straightforward plug-in implementation ensure that the learning curve is low.

OpenPipeline can be downloaded without charge from http://www.OpenPipeline.org. It's available under the Apache License.

Making this technology open source makes sense. The core technology for an enterprise search company, their 'secret sauce', is optimizing the index and making search great, not creating new code to parse the latest version of Microsoft Office or of Documentum. By embracing OpenPipeline, presumably we will start to see pipeline stages created by a number of smaller companies and individuals, easing the burden on enterprise search companies. And companies that provide possible sources of data like Content Management Systems, can create a single pipeline stage for their product that could work for every search technology, and be done with it.

To create a searchable index, all search technologies need to create a stream of text. If the source document is a binary file - Microsoft Word, for example - search vendors need to provide some way to read the format and convert it to text. The same is true of content stored in a relational database: each row represents a virtual document which needs to be extracted from the database and turned into a stream of text. This conversion is typically done as one stage of a pipeline. Other stages may include adding metadata, performing entity or sentiment extraction, or even enhanced language processing.

The concept of a 'pipeline' applies directly to many existing search technologies, each with a proprietary method of accessing content. On top of that, no search technology companies have cooperated with competitors to create standards. In the relational database world, standards have made life much better: consider ODBC and JDBC. Because of these standards, developers can write code that can connect to just about any relational database. Not so in search. Maybe this effort will help break the ice. Stay tuned...

As enterprise search users, are you glad to see an open source solution for part of the search puzzle?

The problem with alerts - Google or otherwise

I use Google alerts to keep an eye on current events. Over the weekend I got an alert: "AMEC uses Verity's K2" - Now, since Verity is part of former competitor Autonomy, and because K2 is generally not being actively marketed, I decided to read the article. Sure enough, the content is dated January 2004, but Google Alerts thinks it is brand new. So I have to conclude that either the publisher just changed something on the page, or Google is just finding that document - either way, Google thinks this is news and in reality, it isn't.

Not long after we started SearchButton.com, we met the Google founders Sergey and Larry. Mark Bennett, my co-founder at SearchButton and here at New Idea Engineering, asked about the then-young Google's handling of dates and recency, and the Google guys took the position that date wasn't that important. This has led to a couple of energetic email exchanges over the last few years, but my recent alert illustrates the problem Google - and most other search technologies have - in generating really useful alerts. In fact, this subject was of such relevance to enterprise search owners, we had an article about the importance of dates in the first issue of our enterprise search newsletter in April of 2003.

Updated 2008 Enterprise Search Vendor Roundup

Jan. 10, 2008 - San Jose, CA, USA 

Microsoft announced they were acquiring FAST Search on January 8, forcing New Idea Engineering to amend our January 4th article "2008 Enterprise Search Vendors:  The new 'Fab4 ... and 1/2" (http://www.ideaeng.com/pub/entsrch/2008/number_01/article01.html). The announcement validates our original assessment and reinforces that search is mission critical for corporations, driving Microsoft to invest in a better search technology.

Some Highlights from NIE's 2008 Enterprise Search Vendor Roundup
 
Autonomy IDOL and FAST Search continue to hold the high end. K2 and Ultraseek are finally retiring.
Google's new version 5 appliance has arrived in the enterprise search mainstream.
Endeca is moving from the ecommerce side and had one of the most impressive search demos at ESS West 2007.
Lucene/ Nutch/ Solr (LNS) open source search engines continue to gain customer mindshare.
Microsoft with its acquistion moves in as Tier 1.
IBM and Oracle still not there.
 
Autonomy IDOL and FAST Search continue to hold the high end, evolving into "search platforms" that go beyond traditional drop in applications. The two leaders from earlier this decade, K2 and Ultraseek, are fading.

Google's new version 5 appliance has arrived in the enterprise search mainstream. While the new version won't satisfy every requirement, it addresses many of the earlier integration issues that had held it back. Expect to see the Google logo on a lot more enterprise portals.

Endeca has created some slick administration tools, doing very well in a head-to-head comparison with Autonomy and FAST despite their continued progress in this area.  As the importance of administration continues to increase, we are more enthusiastic about them in the Enterprise space.

Open source tools based on Lucene, including Nutch and Solr (LNS) are increasingly considered by companies, especially in niches that need to micromanage document relevancy and rating. Lucene and its derivatives are increasingly embedded in other software packages and services, to the point that many users won't even realize they're using it.

We had expected IBM to be the next entrant into the "Tier 1" lineup, based on their iPhrase acquisition. To our surprise, when we saw IBM at ESS East 2007, they were featuring one of their older engines, the OmniFind Enterprise Edition. IBM OmniFind is still not one of our new Fab 4 and an 1/2.

Dieselpoint, Intellisearch, Reccomind, ISYS, ZyLAN, Vivisimo, Siderean and Exalead have strong presences in niche markets.
 
To read the full article ... 2008 Enterprise Search Vendors: The New Fab 4 ... and 1/2. http://www.ideaeng.com/pub/entsrch/2008/number_01/article01.html

Is Gartner missing a trend?

The new Gartner 'Magic Quadrant' report for Information Technology, released last month, shows few surprises in the actual vendor chart. But the report goes on to explain that, of the open-source search engines, "none of them are significant enough to threaten the commercial market". They go on to specifically mention Lucene, saying "enterprises don't consider it a significant alternative". We beg to differ.

Gartner does talk about IBM's strong support of Lucene; and they do say that, if IBM invests substantially in the technology, Lucene may reach its potential. However, we see a number of companies already placing their bets on Lucene - although here I am considering the Apache 'Lucene-Solr-Nutch' franchise as a single, related set of tools.  The list of Lucene users we know includes start-up vertical search companies that don't have much money; but we also see some well-funded and growing public companies which are choosing to build their skills in-house for total control over their own search destiny. Netflix, Monster.Com, and Pearson Scott Foresman are just a few of the companies that use Lucene-Solr-Nutch and are incredibly happy with their choice. And more are looking every day.

The open source path may not be right for every company. Lucene is a toolkit, and we tell our customers that "some assembly is required". It is still weak on filters for document formats, it offers weak stemmer support, and has no integrated support for document security. Lucene and Solr don't include a spider/crawler, although Nutch is always available for that. And while there are wrappers for other popular languages, you will probably want some developers who know Java pretty well. But once you have the right skills in-house, it provides pretty good search in a lightweight, portable application.


We agree with Gartner when they say support from a major vendor like IBM would be a major benefit to the Lucene franchise; but we don't think it's necessary. Think about this: Lucene included a parametric search capability months before the Google Search Appliance did. And the Lucene franchise features search term highlighting; completely tunable relevance and a transparent relevance algorithm; and the capability of fine tuning just about everything to work exactly as you want it. It may be a toolkit, but it is sure a pretty good one for many environments.

It's not like Gartner to miss the wave completely; maybe they are just not listening to the same people we've been talking to with in the corporate world.