Enterprise Search

Flashbacks to an earlier user group meeting

As I have been listening to some of the speakers here at FASTForward08, I realized there is an interesting coincidence here. Just about two years ago, I was here in Orlando at a different search technology user group meeting for a different company that had just been acquired by a another player in the space. Was this history repeating itself?

In 2006, it was the very last Verity User Group meeting held at one of the Disney properties here in Orlando and Autonomy was present to tell us all about the future. Why, just weeks before, Mike Lynch had announced that the conversion tools from K2 to IDOL were complete, and all the Verity users had to do was run the tools and everything was rosy. One of our customers had a one-on-one with the Autonomy guys, after which he told me "The conversion is really very easy: just send money". Maybe they have just not sent in enough cash yet, or maybe the exchange rate has dinged them, but I don't think they are finished the process yet, two years later. Your mileage may vary.

So is this a repeat of 2006? Could this be the last FASTForward?

Well, Microsoft is here at FF in numbers - engineers, mostly, maybe a project manager here or there. Alot of folks from Seattle, but some from Europe as well. New owners checking out the property? Maybe. But I think Microsoft is not thinking as FAST as a tear-down, to be replaced by one of their engines. And I don't think it's looking at FAST for SharePoint search only. I think the MS guys are looking to get into enterprise search in a big way, and I think FAST will be in the enterprise search space for a long time. Microsoft did not buy FAST for its customer list - rather, it wants the technology. Consider FAST's heritage: AlltheWeb.com, one of the earliest web search engines. Desktop search, intranet search (hosted as well as installed), web search, all together? Would Microsoft be interested in that?

Stay tuned.

The known, the unknown and searching

This week we're at FastForward08 in Orlando, so you'll see much more FAST-based postings than usual. By the way the FAST blog, fastforwardblog.com, has a number of other items and links that may be of interest.

One of the first banners that greets attendees as they walk into the main foyer says "There are things known and things unknown. The rest we search for". I started thinking about that from the enterprise search point of view -  usually we think of search for things that we need to find - the unknown. In reality, those things we search for are things we individually may not know, but which we expect are known somewhere within in the company. Do we really search our intranets for things that we don't expect to at all?

Sorry if this sounds like a Donald Rumsfeld speech, but I suppose there are always opportunities to discover the things we didn't know were unknown. On the public web,   I suppose stumbleupon helps us find sites of interest based on our behaviors and profiles, but I'm looking forward to how FAST addresses this message this week. Stay tuned.

Is there life after FAST?

It looks like there may be, and it doesn't involve Microsoft in this case.

Ali Riaz, former COO of FAST Search & Transfer, has announced a new enterprise search company called Attivio. Their secret sauce is what they are calling "AIE", which apparently stands for "Active Intelligence Engine". Attivio, which was started by Riaz and a number of former FAST engineers, has started filing patent applications and recently closed a funding round of $6.2M.

The Attivio web site reports that they intend to provide a tool that can combine both structured and unstructured content, and make the Java-based software available via an API or via SaaS.

InfoToday reports that Attivio has used open-source search code from Lucene as well as other open source and commercial technologies and will be contributing some of their work back to the Apache project. Garnter reported last year that Lucene wouldn't be accepted in the corporate market unless a big player got behind it, and we differed in our October 2007 posting, but this is yet another data point that shows the Lucene engine to be quite satisfactory as a foundation for enterprise search that corporations can use.

Enterprise Search and the Microsoft acquisition of Yahoo, what it might mean

Aside from needing to bring the Yahoo yodeler out of retirement to re-record that famous "Ya-hoo-ooo!" for Bill and Co., we're wondering what other changes this merger might bring.  (I guess the yodel would now be "Micro-soahhwww-oft!")

You'll recall Microsoft just swallowed up FAST Search & Transfer a couple weeks ago, a big player in the Enterprise Search market, and now they've grabbed a big public portal search heavyweight.

On the public side, we're assuming that they won't just re-brand Yahoo as MSN Search.  They paid a lot for the Yahoo! brand, presumably they value it.  In fact, maybe some of the Microsoft portal properties could migrate to the Yahoo brand.

And MS's deep pockets and desktops, combined with Yahoo!'s brand, make life a bit more interesting for Google.

And could FAST's advanced ESP technology be used to improve Yahoo's search results?

But our focus here is Enterprise Search.

One early casualty of the MS-Yahoo! merger might be a chill in the search partnership between Yahoo and IBM.  You'll recall IBM and Yahoo released a free search engine last year, which was actually pretty good.  But IBM's OmniFind products are a direct competitor to FAST's ESP product line, so we assume Microsoft will lean towards ESP in the future.  Of course there may be some timetables built into the Yahoo/IBM contract that will have to be allowed to run out.

We would also expect more linkage between FAST's ESP product and the Yahoo public search.  For example, making it a simple option to include a "search the web" radio button in ESP search forms.  This wouldn't be mandatory, just an option, but some companies might like to make it easy for their employees to simultaneously search their private Intranet and the public web at the same time.

For enterprise folks, the MS/FAST acquisition is still the bigger story.

The bigger picture is that now Microsoft and Google are competing head-to-head in quite a few sectors.  They both have public and enterprise search.  Both are providing software infrastructure, desktop and browser tools, document authoring tools, and each is sponsoring a mobile platform for smart phones.  And again,  both have deep pockets.

IBM and Oracle, the other two behemoths, will need to respond if they want to play in these markets.  We still believe that Autonomy is a logical acquisition for Oracle, from a technology and company culture standpoint, and Mike Lynch & co. will find it tougher and tougher to resist.  IBM and Endeca would also seem to be a good pairing, with their mutual emphasis on powering eCommerce sites, and their mutual obsession with meta data (which we totally agree with), and we hear rumors that there may be some financial motivation on Endeca's part, though we haven't been able to confirm that.

We do live in interesting times.