Enterprise Search

Which social media are you using?

If you read our enterprise search blog, there's some chance that you are involved in enterprise search initiatives in your organization, work for and/or represent an enterprise search vendor, or you're a skilled profession with an interest in search inside the firewall.

That said, may I ask what business-related social media do you use, both for learning about others who are involved with search, and for talking about your experiences/expertise? I'd also like to know what blogs, websites, Twitter accounts, and people you find most interesting with respect to enterprise search. I'm thinking social media sites like LinkedIn, Twitter, Google+, YouTube, blogging sites, web sites and others.

Let me know - either by leaving a comment to this posting or via an email to me at mbk (at) ideaeng (dot) com - and I'll compile a list and post it back here in a few weeks.

I'll compile a summary and post it here in a few weeks.

Thanks in advance for your participation!

 

/s/Miles

 

 

Webinar: What users want from enterprise search in 2012

If you ask the average enterprise user what he or she wants from their internal search platform, chances are good that they will tell you they want search 'just like Google'. After all, people are born with the ability to use Google; why should they need to learn how to use their internal search?

The problem is that web search works so well because, at the sheer scale of the internet, search can take advantage of methodologies that are not directly applicable to the intranet. Yet many of the things that make the public web experience so good can, in fact, be adapted in the enterprise. Our opinion is that, beyond a base level, the success of any enterprise search platform depends on how it is implemented and managed rather than on the core technology.

In this webinar we'll talk about what users want, and how you can address the specific challenges of enterprise content and still deliver a satisfying and successful enterprise search experience inside the firewall.

Register today for our first webinar of the new year scheduled for January 25 : What enterprise users want from search in 2012.

 

 

 

 

 

 

My search platform ate my homework

In a recent article on inforword.com, Peter Wayner wrote a nifty piece discussing 11 programming trends to watch. It's interesting in general, but I found one trend really rang true for me with respect to enterprise search.

He calls his 9^th trend Accuracy fades as scalability trumps all. He points out that most applications are fine with close approximations, based mainly on the assumption that at internet scale, if we miss an instance of something today, we'll probably see it again tomorrow. That brought to mind something I'm working on right now for a customer who needs 100% confidence in their search platform to meet some very stringent requirements. The InfoWorld article reminded me of a dirty little secret of nearly all enterprise search platforms, a secret you may not know (yet); but which could be important to you.

Search platform developers make assumptions about your data, and most search platforms do not index all of your content... by design! Don't get me wrong: these assumptions let them produce pretty good accuracy every time; and even 100% accuracy sometimes. And pretty good is fine most of the time. In fact, as a friend told me years ago, sometimes 'marginally acceptable' is just fine.

The theory seems to be that a search index might miss a particular term in a few documents, but any really important use of the term will clearly be indexed somewhere else and our users will get results from these other documents. In fact, some search platforms have picked an arbitrary size limit, and won't index any content past that limit even if it misses major sections of large documents. Google, in fact, is one of the few who actually document this - once the GSA has indexed 2 MB of text or 2.5MB of HTML in a file, it stops indexing that file and 'discards' the rest. This curious behavior works most of the time for most data (although there is an odd twist that will bite you if you feed GSA a large list of URLs or ODBC records). To be honest, most search platforms do this sort of trimming as well; they just don't mention it too often during the sales process.

Now, in legal markets like eDiscovery, it's pretty darned critical to get every document that contains a particular term. It's not OK to go to court and report that you missed one or more critical document because your search engine truncates or ignores some terms or some documents. That excuse might have worked in elementary school or even in high school, but it just doesn't cut it in demanding enterprise search environments.

It may not be a problem for you; just be sure that, if it is a requirement for you, you include it in your RFI/RFQ documents.

 

 

Webinar: Improving SharePoint search with the FAST indexing pipeline

For those of you still at your desks this short Thanksgiving week, you might be interested in a webinar we'll be doing with our partner SurfRay early next month.

"Everyone knows that great metadata is key to a great user search experience, but what can you do if your existing content falls short? The FAST Search for SharePoint pipeline provides a way to enhance document metadata during the indexing process so your content has better metadata and users will experience better search results.

During the webinar we’ll talk about what the pipeline is, give examples of how it can improve your metadata, and describe some real-world scenarios where having access to the pipeline resulted in better search quality and happier users."

How can the indexing pipeline improve search quality? You'll have to come to the webinar to hear our take, but a hint: you can add and improve metadata to the document during the indexing process - which means better search.

The webinar is planned for Friday, December 9 at 2PM Eastern/11AM Pacific.  You can register for the event now.

Are you spending too much on enterprise search?

If your organization uses enterprise search, or if you are in the market for a new search platform, you may want to attend our webinar next week "Are you spending too much for search?". The one hour session will address:

• What do users expect?
• Why not just use Google?
• How much search do you need?
• Is an RFI a waste of time?   

Date: Wednesday, November 16 2011

Time: 11AM Pacific Standard Time / 1900 UTC

Register today!

Pingar and New Idea Engineering Partnership

I'm happy to announce that our company, New Idea Engineering, has announced a partnership with Pingar, a New Zealand-based company that provides tools to extend and enhance the capabilities of enterprise search. New Idea Engineering is Pingar's first North American reseller.

Pingar markets libraries that provide tools for entity extraction, document summarization, redaction for key documents, autocomplete and a number of other capabilities that organizations can use to improve the user search experience.

In the developer area, Pingar provides access to view the various capabilities in action. For example, you can paste in the text of a document and see the summarization or view the redaction or any of the other Pingar capabilities. Developers can download an API key to test the code yourself. Pingar supports both C# and Java.

We'll be writing more about Pingar in action over the coming months.

 

Oracle buys Endeca: Is it really just about search?

Is acquiring Endeca 'sour grapes' after losing out on Autonomy? I don't think so. Oracle has had any number of generations of home grown search technology over the years, and all things considered the current Oracle Secure Enterprise Search isn't bad. On top of that, Oracle really just agreed to acquire InQuira in July, and many people think of InQuira as a search platform rather than as a question/answer system so great in customer support. 

We've long considered Endeca as the first really modern platform, created in the late 1990s when it was clear that search was more than just a box on a page. They were just about the first platform to have a fully integrated console that a business user can actually understand. They fit really where relevance means "the document/product matches the user's query, it's in stock, and has the highest profit margin for us - between Monday morning at 8AM UTC through 5PM Pacific time". All with an easy to use GIU.

Consider: Endeca is powered by a fully integrated GUI management console; IDOL is powered by command line tools and configuration files, driven by editors like 'vi'. Autonomy does have more GIU tools now; but they feel like more of an afterthought, lacking the polish and feel of a fully integrated product.

So it's not search that Oracle is picking up: it's the powerful eCommerce capabilities. For years, Endeca have been telling us how great an enterprise search platform it is, and yes, it is pretty cool. But the place it really fits, the place it really shines - and the place where most of its customers are - is in serving eCommerce.

So, viewed as an acquisition to strengthen Oracle's fit in the booming eCommerce market, it seems to me a bit more sense.

What do you think? Let us hear from you!

 

 

 

Oracle acquires Endeca

The trend that can trace its immediate roots back to when Microsoft acquired FAST and HP acquired Autonomy continues today as Oracle has announced it is acquiring privetely held Endeca. Forbes reports that Endeca has raised up to $70M; and in July reported its sales are running at $150M annually. Details have not yet been released. More to come on this as it develops.

 

Update: Oracle's slide presentation on the deal at http://bit.ly/qHNCeH

 

 

Searching for Sarah at SharePoint Conference 2011

Just noticed one of the most interesting sessions at last May's Enterprise Search Summit is coming to the October Microsoft SharePoint Conference! We blogged about it back in May.

Basically, Booz & Company did an evaluation of SharePoint 2010 search - FAST Search for SharePoint as I recall - versus the Google Search Appliance they had been using. At one point, the search business owner was trying to find the last name of a woman she had met in the firm; and when she searched for 'Sarah', hoping to find her in the directory, the GSA returned 60 men in the result list. Can you guess why? A hint: metadata (check the earlier article, or come to SPC 2011 to find out).

Now in fact, we think the GSA could have been tuned to emulate this OOB behavior by SharePoint; but this is a reminder that not every search platform works great in every environment. Buyer beware!

Ever had a similar experience? Let us know about it!

 

Autonomy marketing, meet HP

Leslie Owens, the enterprise search analyst over at Forrester Research, has written one heck of an article about a potential Autonomy in the HP era. Her analysis strikes me as being very insightful and, in my opinion, quite accurate. What makes it unique is you just don't see alot of 'analysts' tell it like it really is. Kudos to Ms. Owens!

Technical issues aside, I'm reminded of a story that goes back to the early days of the PC when HP and Apple were just beginning to compete. A popular quip about the difference between HP and Apple went "Where Apple sells sushi, HP sells cold raw dead fish". The implication, of course, being that HP just wasn't good at marketing.

I'll always think of Autonomy as a search technology company. Our first exposure to Autonomy was in 1997  with an early version of the DRE, the predecessor to IDOL. Back then, using vi or emacs to configure a search engine was pretty common; and no one really had grasped the importance of the business side of running enterprise search.

In search, IDOL returns pretty darned good results out of the box, no tuning required. But if you want to tune it, if you have alot of custom work to do, IDOL gets really tough to set up and configure... and it's still done using text editors to create and edit text configuration files. This may be one reason why IDOL projects take so long to complete and require such big teams of consultants. HP probably won't be changing this... they want to grow their consulting revenue!

But now, in the second decade of the not-so-new century, customers expect to use a GUI to configure, manage, and customize enterprise search; and just about all of IDOL is still 'command line based'. I think this is just one of the data points supporting Leslie's remark that IDOL 5 has not had a major update in over 5 years. Sure, they've added dozens of new capabilities... API calls, and the like... but the platform is still a solid 1990s kind of experience. "Powered by vi" was funny in 1998; not so much now.

Nonetheless, Autonomy has been quite effective because their technology is pretty darned good at finding content; and because their sales force has been aggressive in selling the product. HP will love the consulting; but will they be able to move product as successfully as Autonomy had?

What do you think?