Enterprise Search

Oracle buys Endeca: Is it really just about search?

Is acquiring Endeca 'sour grapes' after losing out on Autonomy? I don't think so. Oracle has had any number of generations of home grown search technology over the years, and all things considered the current Oracle Secure Enterprise Search isn't bad. On top of that, Oracle really just agreed to acquire InQuira in July, and many people think of InQuira as a search platform rather than as a question/answer system so great in customer support. 

We've long considered Endeca as the first really modern platform, created in the late 1990s when it was clear that search was more than just a box on a page. They were just about the first platform to have a fully integrated console that a business user can actually understand. They fit really where relevance means "the document/product matches the user's query, it's in stock, and has the highest profit margin for us - between Monday morning at 8AM UTC through 5PM Pacific time". All with an easy to use GIU.

Consider: Endeca is powered by a fully integrated GUI management console; IDOL is powered by command line tools and configuration files, driven by editors like 'vi'. Autonomy does have more GIU tools now; but they feel like more of an afterthought, lacking the polish and feel of a fully integrated product.

So it's not search that Oracle is picking up: it's the powerful eCommerce capabilities. For years, Endeca have been telling us how great an enterprise search platform it is, and yes, it is pretty cool. But the place it really fits, the place it really shines - and the place where most of its customers are - is in serving eCommerce.

So, viewed as an acquisition to strengthen Oracle's fit in the booming eCommerce market, it seems to me a bit more sense.

What do you think? Let us hear from you!

 

 

 

Autonomy marketing, meet HP

Leslie Owens, the enterprise search analyst over at Forrester Research, has written one heck of an article about a potential Autonomy in the HP era. Her analysis strikes me as being very insightful and, in my opinion, quite accurate. What makes it unique is you just don't see alot of 'analysts' tell it like it really is. Kudos to Ms. Owens!

Technical issues aside, I'm reminded of a story that goes back to the early days of the PC when HP and Apple were just beginning to compete. A popular quip about the difference between HP and Apple went "Where Apple sells sushi, HP sells cold raw dead fish". The implication, of course, being that HP just wasn't good at marketing.

I'll always think of Autonomy as a search technology company. Our first exposure to Autonomy was in 1997  with an early version of the DRE, the predecessor to IDOL. Back then, using vi or emacs to configure a search engine was pretty common; and no one really had grasped the importance of the business side of running enterprise search.

In search, IDOL returns pretty darned good results out of the box, no tuning required. But if you want to tune it, if you have alot of custom work to do, IDOL gets really tough to set up and configure... and it's still done using text editors to create and edit text configuration files. This may be one reason why IDOL projects take so long to complete and require such big teams of consultants. HP probably won't be changing this... they want to grow their consulting revenue!

But now, in the second decade of the not-so-new century, customers expect to use a GUI to configure, manage, and customize enterprise search; and just about all of IDOL is still 'command line based'. I think this is just one of the data points supporting Leslie's remark that IDOL 5 has not had a major update in over 5 years. Sure, they've added dozens of new capabilities... API calls, and the like... but the platform is still a solid 1990s kind of experience. "Powered by vi" was funny in 1998; not so much now.

Nonetheless, Autonomy has been quite effective because their technology is pretty darned good at finding content; and because their sales force has been aggressive in selling the product. HP will love the consulting; but will they be able to move product as successfully as Autonomy had?

What do you think?

 

 

7 Reasons Why the Autonomy Acquisition Makes Sense for HP

With HP's annual meeting coming up this week CEO Leo Apotheker is looking for a way to put some lipstick on his new baby: he wants to acquire Autonomy. PC sales are declining and Apple is eating their lunch in pads AND buying the old HP Cupertino campus. Apparently they will keep the server divisions.Bill and Dave must be beside themselves.

But wait! The knight in shining armor from England is here for the rescue. Autonomy announced just last month they were 'likely to beat expectations' for their fourth quarter. They have enjoyed huge success in eDiscovery.. they claim to 'own search'.. and sponsors its own football team, Tottenham Hotspur FC.

At first blush, the two seem like strange bedfellows. HP, the leader in PCs worldwide, with a well acknowledged reputation for management style - the HP Way. Autonomy has a rep like that of a great professional footballer: skilled and aggressive, known to occasionally feint an injury for his benefit after a rough hit; and perhaps booked for a card a few times every season. But still, a champ and damn good at what he does.

When you think about it, the deal isn't as odd as it may seem. Consider:

1. PC sales are down; they are being replaced by smart devices like Android and iOS devices. WebOS? Too little, too late, too expensive.

2. HP acquired EDS a couple of years back to compete with IBM in consulting services. Among enterprise class search engines, 'consulting services' certainly comes to mind - bring lots.

3.  IT managers, not C-level execs, buy PCs. Chief Risk Officers buy eDiscovery and compliance solutions so he and his fellow executives can sleep well at night. Rarely is there a spending freeze on compliance tools.

4. HP is rumored to be spinning out its PC products, but it seems the server business is staying in Palo Alto. IDOL likes lots of really big servers: HP wins on all three: servers, software, and consulting.

5. Iron Mountain: records management. See (3) above.

6. We used to speculate that SAP might be a buyer for Autonomy at some point in time. Now, the HP chief is Leo Apotheker, who came to HP from... SAP. Coincidence?

7. 'The cloud': HP needs one, HP gets one - enough said.

Still, it may not be a 'made in heaven' match.

1. You may recall that when Microsoft acquired FAST, they soon found some odd accounting issues - something about numbers overly optimistic, and booking revenues prior to firm orders were received. Until you dig deep into the details, there may be no way to know for sure until the deal is sealed.

2. And as mentioned earlier, personnel and policies seem relatively incompatible.

3. HP is along time Microsoft partner, uses SharePoint extensively, and just rolled our FAST search on its public facing web site. That should be interesting.

4. Finally, I seem to recall that when Autonomy bought their larger competitor Verity, part of the rationale for Autonomy being the surviving company was the cost of annual SarBox compliance if they were a US company. HP must be willing to pick up the tab, because I sure can't see Palo Alto moving to Tottenham.

Stay tuned, and let us hear what you think!

 

Full disclosure: I was an HP employee for 10 years, and during my time there was fortunate enough to be the PC support guy for Bill Hewlett, Dave Packard, and Dave Packard Jr. HP is not the same company as it was when they were running the show. I recommend Mike Malone's "Bill and Dave" as a great explanation of what made HP great for 40 years, and successful for 72 years so far.

 

 

HP looking at acquiring Autonomy

The consolidation continues. Since the FAST acquisition by Microsoft a few years back, people have asked who we thought might acquire Autonomy; and we always decided they were almost too big to be a target, that Autonomy would be the surviving company.

It seems that HP, which apparently just rolled out its FAST implementation to its public web site, is the surprising answer.

Quick take: in a way, it makes sense in a few ways:

1. Hardware is getting cheaper and cheaper; being the largest vendor in a shrinking market isn't where you really want to be.

2. HP has long been moving to a services delivery company - look at the 2008 acquisition of EDS. If there is a search technology that typically has huge implementation projects with teams of expensive consultants, it's Autonomy.

3. Company have Chief Risk Officers, who handle compliance, have the budget to buy software to keep fellow executives out of trouble with the government. What better prospects to have?

Wow.. I'm still a bit shocked; I feel like I must have overslept and that I'm having a strange dream. But my dog seems pretty sure it's real.

Stay tuned, and let us know what you think!

/s/Miles

Content owners don't care about metadata

Or do they?

Our recent post about Booz & Company's 'men named Sarah' highlights just how important good metadata can be in order to provide a great search experience for employees and customers.

One of our customers who spoke at the recent ESS 2011 in New York provided some great insights into the problems organizations have getting employee content creators to include good metadata with their documents.

During the ESS talk, they report that content owners don't really seem motivated when asked to help improve the overall intranet site by improving document metadata. However - and this is a big one - when a sub-site owner sees poor results on their own site, they are willing to invest the time to provide really good metadata.

[A bit of background: This customer provides a way to individual site owners within the organization to add search to their 'sub site' pretty much automatically - sort of a 'search as a service' within the enterprise.]

So if you've been thinking of adding the ability to search-enable sub-sites within your organization, but solving the relevance problem is your first task, you might reconsider your priorities!

/s/Miles

Make your search engine seem psychic

People tell us that Google just seems to know what they want - it's almost psychic sometimes. If only every search engine could be like Google. Well, maybe it can.

Over the years, the functions performed by the actual 'search engine' have grown. At first, it was simply a search for an exact match - probably using punch card input. Then, over time, new and expanded capabilities were added, including stemming... synonyms... expanded query languages... weighting based on fields and metadata.. and more. But no matter what the search technology provided, really demanding search consumers pushed the technology, often by wrapping extra processing both at index time and at query time. This let the most innovative search driven organizations stay ahead of the competition. Two great examples today: LexisNexis and Factiva.

In fact, the magic that makes public Google search so good - and so much better than even the Google Search Appliance - is the armies of specialists analyzing query activity and adding specialized actions 'above' the search engine. 

One example of this many of us know well: enter a 12 digit number. if the format of the number matches the algorithm used by FedEx in creating tracking numbers, Google will offer to let you track that package directly from FedEx. For example, search for 796579057470 and you see a delivery record; change that last 1 to a zero, and you get no hits. How do they know?

The folks at Google must have noticed lots of 12 digit numbers as queries; and being smart, they realized that many were FedEx tracking numbers. I imagine, working in conjunction with FedEx, Google implemented the algorithm - what makes a valid FedEx tracking number - and boosted that as a 'best bet'.

Why is this important to you? Well, first it shows that Google.com is great in part because of the army of humans who review search activity, likely on a daily basis. Oh, sure, they have automated tools to help them out - with maybe 100 million queries every day, you'd need to automate too. They look for interesting trends and search behavior that lets them provide better answers.

Secondly, you can do the same sort of thing at your organization. Autonomy, Exalead, Microsoft, Lucene, and even the Google Search Appliance, can all be improved with some custom code after the user query but before the results show up. Did the user type what looks like a name? Check the employee directory and suggest a phone number or an email address. Is the query a product name? Suggest the product page. You can make your search psychic.

Finally, does the query return no hits? You can tell what form the user was on when the search was submitted - rather than a generic 'No Hits' page. Was the query more than a single term? Look for any of the words, rather than all; make a guess at what the user wanted, based on the search form, pervious searches, or whatever context you can find.

So how do you make your search engine seem psychic? Learn about query tuning and result list pre-processing; we've written a number of articles about query tuning in our newsletter alone.

But most importantly: mimic Google: work hard at it every day.

/s/Miles

 

 

 

 

Share your successes at ESS East next May

Our friends over at InfoToday who run the successful Enterprise Search Summit conferences have asked us  to announce that the date for submitting papers to their Spring show in New York in May 2011 has been extended until Wednesday, December 8. You can find out what they are looking for and how to submit your proposal online at http://www.enterprisesearchsummit.com/Spring2011/CallForSpeakers.aspx.

Michelle Manafy, who runs the program again next May, really likes to have speakers who have found creative and successful ways to select, deploy, or manage ongoing enterprise search operations. We've co-presented with several of our customers in the past, and trust me, it's great fun and not bad for your career! And - no promises - the weather at ESS East has been great for just about every year - and we've been there for nearly 6 years now!

A friend told me something years ago that I've always fond helpful; I hope you'll take it to heart: 'Everything you know, someone else needs to know'. Don't worry if your search project isn't perfect; or worry that someone will find fault with what you've done. Trust me: there are many organizations newer to enterprise search than you are, and anything you found helpful will sure be valuable for them as well. And you get to attend al of the sessions, so you might learn more as well! A 'win-win' situation if I've ever seen one!

See you in New York!

/s/Miles

 

 

Autonomy blames the economy: could it be something else?

Good news - bad news on Autonomy today.

First the good news:  Autonomy reports sales of $415M for the first half of the year, in the middle of their predicted range. On top of that, Autonomy announced a $15M deal with an unnamed "major global bank". And they are on target to grow the business 17% compared to last year (Forbes)

Now the bad news: as Avi Rappoport reported on Twitter Friday afternoon, Autonomy shares took a 16% dive during the day. The Financial Times reported the drop was based on lower forecasts and Mike Lynch's remarks "about volatility in customers view of the current macroeconomic situation".

For some time now we have seen Autonomy less as an enterprise search company - even though they claim to "own search" on billboards in the San Francisco area - and more of a 'compliance' play. Steve Arnold has even made the same observation (Autonomy Not (sic) Longer a Search Vendor); and one of the tech guys we know who worked for Autonomy until recently tells us that he has not worked on a pure search project for years unless the customer was also a compliance customer.

Autonomy has a powerful engine that has pretty darned good results right out of the box. And the IDX file that holds data to be indexed provides a really straightforward way to interface to other platforms - probably one reason Autonomy can integrate with new technologies they acquire.

But IDOL is a premium product which often requires lengthy, complex integrations. I can see a few trends that could be causing worries in Cambridge as much as indecisive prospects:

• Very capable open source alternatives
  
• More aggressive pricing by major competitors
• More competition in compliance opportunities
• Competitive products that provide GUI-based tools for managing search

Don't get me wrong: IDOL is a real winner in the right environments when it's configured properly. I'm just wondering if competitors are gaining on Autonomy; and whether customers, wary of price/performance, might start to shy away from big and complex.

 

Document filters webinar July 28 2010

Document filter independent ISYS is hosting a webinar on Wednesday, July 28 at 1PM Eastern to talk  about the role document filters play in successful search indexing and display. You can register now.

Of course, as a search technology company, ISYS has enjoyed great success, particularly among law enforcement where search has to work right at a reasonable price. We've always liked their technology and their approach.

But like every search platform, ISYS needed filters to convert so-called 'binary' formats like Microsoft Office, PDF, or even Photoshop files into a stream of text - after all, today's search platforms primarily operate on words.. in textual format. But ISYS looked at the market at the time, and found that two of their competitors, Autonomy and Oracle, own the best of the filter technologies.

Like any company, they made a 'make or buy' decision, and in their case, making their own filters was the right answer for them, and possibly for you. You see, ISYS decided to start selling their filter technology independent of their search platform, so now you can acquire some really great filtering and viewing technology for just about any search engine, 'off the shelf'. Their customers include other vendors with the need to extract text from various types of content, not just search vendors but also eDiscovery and eCompliance companies and many others who don’t want to pay excessive prices for technology - and who want really great filtering at a reasonable cost.

Then, a few years back, ISYS decided that open source platforms Lucene and Solr - which had no filters - needed them as well. So now you can buy a great filter pack 'off the shelf' with no huge volume commitment - no volume commitment at all! And you can get world class filtering for your open source search project.

Come hear ISYS, the guys from Lucid Imagination, and us here at New Idea Engineering talk about the critical role of filters in your search applications. See you then!

/s/Miles

Enterprise Search Summit 2010 - DC

Even as we prepare for ESS East in New York (ESS NY from now on?), Information Today has issued its call for papers for the first ever ESS-DC to be held in Washington DC November 16-18 2010.

Follow this link to find background on what InfoToday is looking for; or jump right to the submissions page. Don't be shy: everyone who presents papers had, at one time, never done it before. What you know, someone else needs to know!

In our experience, the kind of content InfoToday likes is the information that can help an organization select or manage search and related technologies. Generally, real-world stories about how other companies and organizations have succeeded with search are the ones that attendees appreciate the most. 

We'll also be having a searchdev dinner at ESS DC this year. Details to come late in summer, but plan for it now!

Are you doing search now? Have you been successful getting it going on time and under budget? Tell your story. Submit your idea now!