« A paradigm shift in enterprise search | Main | Open Source Search Myth 1 - Enhancements by Committee »

March 11, 2013

Open source search engine - a good idea?

My transition to LucidWorks has been a busy one with little time for other interests and hobbies (like flying!). But a week or so back I spotted a November 2012 post on the LinkedIn Enterprise Search Engine Professionals group asking the question in this post's title. 

Of course, LucidWorks provides deep support for Solr, and markets a Solr-based enterprise search product; but it's my work with enterprise search technology for the last 20+ years that really drives my response. Sadly, my reply was longer than LinkedIn allows.. so I posted a shorter link there and have come back here to reply in full. It's going to take a few posts though, so bear with me if you will. 

First, my response to the poster: Eight years ago open source was cool, but was probably not 'enterprise ready'.  Enterprise search is hard, but years ago the Apache projects (Lucene and Solr) began working to solve the tough issues - ones that were not commercially worth it for the 8 to 10 major commercial enterprise search companies.

Then a funny thing happened: Solr got better and better; and the commercial vendors started merging. Verity got sucked into Autonomy, which got sucked into HP. FAST got sucked into Microsoft. Vivisimo got sucked into IBM. And with every acquisition, the time and money that enterprises had invested in commercial search became totally wasted - when the platform you based your search on got acquired, you had to move to the new engine. A painful, expensive and long process,

As I blogged just a few weeks ago, open source search is now the default SAFE choice for enterprises that need search. You may have to do some coding, or find a skilled expert/team to help; but you own your destiny. Lucid (my company) does sell support for Solr; there are other fine companies, large and small, that do so. We're fortunate enough to employ a good number of the committers - no majority, which is probably best for the community. 

The original poster, an employee of a proprietary search vendor, may have had his reasons. Nonetheless, he listed five reasons he felt that open source search for enterprises was a bad idea - based on a three year old report by my friend Hadley Reynolds - taken a bit out of context. These 'disadvantages' are listed and linked below.  

* Enhancements on community timetable only

* Potentially expensive customization

* Requirement for search development skills in-house or ready-to-hand

* Production functionality may trail in specific features relative to commercial search firms

* Maintenance/system life costs can become significant

In the next several posts, I'm going to address and refute these one at a time. Stay tuned.

 

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d8341c84cf53ef017ee925cdfb970d

Listed below are links to weblogs that reference Open source search engine - a good idea? :

Comments

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.