25 posts categorized "LucidWorks"

February 14, 2013

A paradigm shift in enterprise search

I've been involved in enterprise search since before the 'earthquake World Series' between the Giants and the A's in 1989. While our former company became part of LucidWorks last December, we still keep abreast of the market. But being a LucidWorks employee has brought me to a new realization: commercial enterprise search is pretty much dead.

Think back a few years: FAST ESP, Autonomy IDOL (including the then-recently acquired Verity), Exalead, and Endeca were the market. Now, every one of those companies has become part of a larger business. Some of the FAST technology lives on, buried in SharePoint 2013; Autonomy has suffered as part of HP because - well, because HP isn't what it was when Bill and Dave ran it. Current management doesn't know what they have in IDOL, and the awful deal they cut was probably based on optimistic sales numbers that may or may not have existed. Exalead, the engine I hoped would take the place of FAST ESP in the search market is now part of Dassault and is rarely heard of in search. And Endeca, the gem of a search platform optimized for the lucrative eCommerce market, has become one of three or four search-related companies in the Oracle stable. 

Microsoft is finally taking advantage of the technology acquired in the FAST acquisition for SharePoint 2013, but as long as it's tied to SharePoint - even with the ability to index external content - it's not going to be an enterprise-wide distribution - or a 'big data' solution. SharePoint Hadoop? Aslongf as you bring SQL Server. Mahout? Pig? I don't think so. There are too many companies that want or need Linux for their servers rather than Windows.

Then there is Google, the ultimate closed-box solution. As long as you use the Google search button/icon, users are happy – at least at first. If you have sixty guys named Sarah? Maybe not.

So what do we have? A few good options generally from small companies that tend to focus on hosted eCommerce - SLI Systems and Dieselpoint; and there’s Coveo, a strong Windows platform offering.

Solr is the enterprise search market now. My employer, LucidWorks, was the first, and remains the primary commercial driver to the open source Apache project. What's interesting is the number of commercial products based on Solr and it's underlying platform, Lucene.

Years ago, commercial search software was the 'safe choice'. Now I think things have changed: open source search is the safe choice for companies where search is mission. Do you agree?

I'll be writing more about why I believe this to be the case over the coming weeks and months: stay tuned.

/s/Miles

 

December 18, 2012

Last call for submiting papers to ESS NY

This Friday, December 21, is the last day for submitting papers and workshops to ESS in NY in May 21-22. See the information site at the Enterprise Search Summit Call for Speakers page.

If you work with enterprise search technologies (or supporting technologies), chances are the things you've learned would be valuable to other folks. If you have an in-depth topic, write it up as a 3 hour workshop; if you have a success story, or lessons learned you can share, submit a talk for a 30-45 minute session.

I have to say, this conference has enjoyed a multi-year run in terms of quality of talks and excellent Spring weather.. see you in May?

 

 

December 03, 2012

Why LucidWorks? And Why Now?

Big news for us here at New Idea Engineering. After 16 years as an independent search technology consulting company, we've become part of LucidWorks effective December 1, 2012.

For years we've focused on both the business and technology of search.  We've provided vendor neutral consulting services to large and small organizations. We've worked with search platform companies to help tune their product capabilities and their message; we've helped companies implement enterprise search from 'the usual (vendor) suspects'. We've provided business best practices, data audits, and implementation overview for dozens of companies for most of our time as an independent company.

As you know, the market for enterprise search has changed over the last several years. Verity then Autonomy, FAST, Endeca, Exalead, ISYS, and more have been acquired by large companies with varying levels of success. With these acquisitions the products have morphed to fit into the new owners' world view, we've politely referred to shift in focus as being "distracted". Google, one of the few non-acquired engines, got into the market with a low-cost entry which has enjoyed great acceptance; but as the market changed, Google has started raising its prices for the nifty yellow box.   And while they pursue laudable offerings like phones, tables, Google Glass and self driving cars, cloud computer and simultaneously retool their ad model for the mobile world, it's fair to say that even their enterprise offerings are potentially distracted at times.

Sure, if your company has a typical use case for search, there's an engine or appliance for you.  But so many complex projects we've seen are atypical, almost by definition.  These high-end projects are no longer efficiently served by commercial sector.  Many projects have turned to Open Source offerings, but not out of cost savings as you might think, but out of a desire to have extreme control and flexibility, and not be tied down by vendor meddling and license nit-picking.

Over the same period, more and more people have realized that the need to understand and manage 'big data' is taking off. In fact, search is the interface of choice to find content in big data repositories. 

It's been about 10 years since we did our first project based on Lucene, the basis for nearly all modern open source search engines today. Since then, the capabilities of open source search have increased to the point where we honestly think Solr may be the best search platform available on the market today. 

We didn't call what we did with Lucene back then 'big data', but that's really what it was. Scalable, controllable, flexible, powerful... and open! And free for the taking - and modifying. Just add programmers.

A few years back, Lucid Imagination was started to provide that support, along with training and an easy to use interface that lets business owners - not just developers - use Solr search.  We've called them "the RedHat of Open Source Search".  Now, Lucid Imagination has become LucidWorks, and it is set to be the best way to search web, file, and database content, with extreme control, and of course with big data.

A few months ago we spoke with Lucid CEO Paul Doscher about upping our contract with them, and about where they were going, and it just made sense to us at that time to join a bigger team.

While we're committed to success at LucidWorks, we'll continue to use our blog to discuss all aspects of enterprise search – vendors, tools, technologies, events, and trends.  Unlike our days at past search companies, this one is based on an open platform so we'll be able to share a lot more as we move forward.

We hope you'll find our posts interesting, helpful, and engaging. Let us know how we're doing.

 

October 26, 2012

Deep Solr in London and New York

Last week I had the pleasure of conducting a workshop at the recent Enterprise Search Summit on open source tools including Solr, Lucene, and some of the commercial products based on these tools. To a lesser extent we also covered ElasticSearch, SearchBloxAlcove9, and a few other platforms, as well as a number of open source and commercial tools that support enterprise search.

One thing many of the attendees had in common was that they had been experimenting with Lucene/Solr for a while, but many were skeptical that they were ready do dive into a deep project on their own.

While that sentiment is no problem for me - after all, we provide services around implementing both open source and commercial search to our customers - I know many companies want to have expertise in-house

For those of you who are looking for those skills, you might be interested in a post I just saw from LucidWorks, They are offering a developer course titled  'Everything you always wanted to know about Solr' in both New York City and in London England during November. If you've been experimenting with Solr in-house (or on your own) for a while now, and you're ready to move to the next level, you might give some thought to registering for one of these classes.

It will cover the usual Solr topics, but also replication, sharding, and all the things you need to know to really use Solr in production search. Take a look and see if it's right for you.

 

 

 

 

August 21, 2012

Mind the gap

A few weeks ago, a former client asked me about the 'lay of the land' in enterprise search - which companies were the one to be considered for evaluation. It's something I'm frequently asked, and one big reason why I strive to stay current with all of the leading commercial and open source vendors in the market.

As I pulled together the list, it occurred to me that recent consolidation has led to an odd situation: there is no longer a 'mid-market' in enterprise search.

Under $25,000(US), there are a number of options from free and low-cost open source (SearchBlox and my employer LucidWorks come to mind). 

Google has discontinued its low cost (blue) search appliance, and raised the cost of its regular (yellow) one to apparently be well above $25K.

We also have the old-school major commercial vendors - like FAST (now Microsoft SharePoint Search); Autonomy (now HP); Endeca (now Oracle), and finally Vivisimo (now IBM). Trend or not, these enterprise search products command high initial outlay, often significant implementation costs, and high ongoing 'support' once you've rolled it out. Looks like the mid-market is gone.

So now the question is: What do you get for the difference in price? I'd suggest not much in the way of capability; nothing in terms of scalability; and very very little in the way of flexibility.  I guess it's 'caveat emptor' - buyer beware!

What about some products/projects I haven't mentioned? Well, the focus of my article here is on enterprise search. Great candidates like Coveo are 'windows only' which disqualifies them from my list. I suppose you could consider the GSA as not enterprise ready, but I think appliances make the OS issue irrelevant. I've also omitted mentioning other projects because they have not yet shipped a 'Version 1.0' release - that's testware, no matter who it's from. And I'm sure there are open source projects where a single person is making all the calls - I don't consider that enterprise ready either.

I’ll be looking for the day when the big guys start value pricing their software licenses and help bring the market into line with today’s reality.

If you think I've unfairly represented the market, let me know - I'm not shy about posting comments that differ with my viewpoint.

 

s/Miles