If your organization uses enterprise search, or if you are in the market for a new search platform, you may want to attend our webinar next week "Are you spending too much for search?". The one hour session will address:
Date: Wednesday, November 16 2011
Time: 11AM Pacific Standard Time / 1900 UTC
Next week in Washington DC, InfoToday runs their Fall enterprise search conferences - KM World, Enterprise Search Summit, SharePoint Symposium, and Taxonomy Boot Camp.. whew! Monday - Halloween Day! - I am giving a workshop at the conferences with the somewhat vague title 'Enterprise Search Technologies'.
What I'll be talking about is an overview of the platform vendors, with some detail on strengths and weaknesses of the vendors; and a drill down into what you need to do before you call the vendors (if you value your time).
You can still sign up for the workshop for $295US or the entire conference for a bit more; see you in DC in a week!
/s/Miles
Just noticed one of the most interesting sessions at last May's Enterprise Search Summit is coming to the October Microsoft SharePoint Conference! We blogged about it back in May.
Basically, Booz & Company did an evaluation of SharePoint 2010 search - FAST Search for SharePoint as I recall - versus the Google Search Appliance they had been using. At one point, the search business owner was trying to find the last name of a woman she had met in the firm; and when she searched for 'Sarah', hoping to find her in the directory, the GSA returned 60 men in the result list. Can you guess why? A hint: metadata (check the earlier article, or come to SPC 2011 to find out).
Now in fact, we think the GSA could have been tuned to emulate this OOB behavior by SharePoint; but this is a reminder that not every search platform works great in every environment. Buyer beware!
Ever had a similar experience? Let us know about it!
As in much of the IT world, when you build a new system or farm, you need to document everything for posterity - and to save potentially hours of effort when things stop working in the future. SharePoint 2010 is no different.
I just learned of a tool that will document your SharePoint farm(s) for you, greatly easing the problem most of us will face at one point or another. From the folks at Acceleratio Ltd based in Croatia but with US sales and support, the folks who created the Documentation Toolkit for SharePoint understand both the problem - and their audience. Pricing is available per farm ($299 US) or, for consulting firms that maintain multiple farms for their users, for $499.
There is a 30 day free evaluation/download; and using the claim code 'summer2011' you can get 50% of the purchase price through the end of August.
Normally we would have posted this on SearchComponentsOnline.com, another site we run; but it's pretty new, and not as many people know to look there for free and low cost tools for those interested in enterprise search.
Sorry for the late notice - I just discovered it myself today.
Josh Noble, SurfRay consultant and author of Pro SharePoint 2010 Search, will give a webinar on Customizing the Advance Search Page on July 8 at 11AM Pacific, 2PM Eastern. If you're in Europe, it's worth staying up for!
Josh gave a related talk at SharePoint Saturday Sacramento a couple of weeks ago, and he really had some great tips and techniques. Register for the webinar now.
/s/Miles
Or do they?
Our recent post about Booz & Company's 'men named Sarah' highlights just how important good metadata can be in order to provide a great search experience for employees and customers.
One of our customers who spoke at the recent ESS 2011 in New York provided some great insights into the problems organizations have getting employee content creators to include good metadata with their documents.
During the ESS talk, they report that content owners don't really seem motivated when asked to help improve the overall intranet site by improving document metadata. However - and this is a big one - when a sub-site owner sees poor results on their own site, they are willing to invest the time to provide really good metadata.
[A bit of background: This customer provides a way to individual site owners within the organization to add search to their 'sub site' pretty much automatically - sort of a 'search as a service' within the enterprise.]
So if you've been thinking of adding the ability to search-enable sub-sites within your organization, but solving the relevance problem is your first task, you might reconsider your priorities!
/s/Miles
We're always on the lookout for anecdotes to use at trade shows, with our customers and prospects, and of course here in the blog, so I have to report that we heard a great one last week at Enterprise Search Summit in New York.
The folks from Booz & Company, a spinoff from Booz Allen Hamilton, did a presentation on their experience comparing two well respected mainstream search products. They report that, at one point, one of the presenters was looking for a woman she knew named Sarah - but she was having trouble remembering Sarah's last name. The presenter told of searching one of the engines under evaluation and finding that most of the top 60 people returned from the search were... men. None were named 'Sue'; and apparently none were named Sarah either. The other engine returned records for a number of women named Sarah; and, as it turns out, for a few men as well.
After some frustration, they finally got to the root of the problem. It turns out that all of the Booz & Company employees have their resumes indexed as part of their profiles. Would you like to guess the name of the person who authored the original resume template? Yep - Sarah.
One of the search platforms ranks document metadata very high, without much ability to tune the weighting algorithms. The other provides a way to tune the relevance; but it also tends to rank people relevance a bit differently - probably stressing documents about people less than the individual people profiles. The presentation was a bit vague about whether any actual tuning that might impact these differences on either platform.
The fact that one of the engines did well, and one did not, is not the big story here - although it is something for you to consider if you're evaluating enterprise search platforms. The real lesson here is that poor metadata makes even the best of search platforms perform poorly in some - if not most - cases.
Our friends over at Arcovis are hosting a talk "Automating the Top 5 FAST Search for SharePoint Customizations" Wednesday evening, May 11.. Brent Groom, a Senior Engineer at Microsoft with deep experience in enterprise search, is doing the presentation.
The registration site seems to be down right now, but the link to register is http://events.linkedin.com/events/521196/clickthru. You can find information on the seminar/webinar on LinkedIn as well.
You can also attend in person. The session will be held at the Microsoft offices:
Microsoft Corporation
1290 Avenue Of The Americas
New York, NY 10104 US
if you're in New York for the Enterprise Search Summit, and you are in town Wednesday evening, this is only a few blocks form the hotel; show up in person!
I just discovered a great article on a great blog about FAST Search for SharePoint (FS4SP) by Trond Øivind Eriksen of Comperio in Norway. Comperio is a FAST partner, and has been involved in a number of innovative projects involving FAST ESP and now FS4SP.
The article that originally caught my attention is about 'entity extraction' - what Microsoft now calls 'property extraction' in FS4SP. He addresses 'black list' and 'white list' terms that you want to include in the facets/properties you display in results lists; and, even cooler, he provides the example the way God intended things to be run - via scripting (in this case, PowerShell).
Actually I found his blog most helpful; I'm certainly adding it to my 'must read' list. You may find it helpful as well!
/s/Miles
People tell us that Google just seems to know what they want - it's almost psychic sometimes. If only every search engine could be like Google. Well, maybe it can.
Over the years, the functions performed by the actual 'search engine' have grown. At first, it was simply a search for an exact match - probably using punch card input. Then, over time, new and expanded capabilities were added, including stemming... synonyms... expanded query languages... weighting based on fields and metadata.. and more. But no matter what the search technology provided, really demanding search consumers pushed the technology, often by wrapping extra processing both at index time and at query time. This let the most innovative search driven organizations stay ahead of the competition. Two great examples today: LexisNexis and Factiva.
In fact, the magic that makes public Google search so good - and so much better than even the Google Search Appliance - is the armies of specialists analyzing query activity and adding specialized actions 'above' the search engine.
One example of this many of us know well: enter a 12 digit number. if the format of the number matches the algorithm used by FedEx in creating tracking numbers, Google will offer to let you track that package directly from FedEx. For example, search for 796579057470 and you see a delivery record; change that last 1 to a zero, and you get no hits. How do they know?
The folks at Google must have noticed lots of 12 digit numbers as queries; and being smart, they realized that many were FedEx tracking numbers. I imagine, working in conjunction with FedEx, Google implemented the algorithm - what makes a valid FedEx tracking number - and boosted that as a 'best bet'.
Why is this important to you? Well, first it shows that Google.com is great in part because of the army of humans who review search activity, likely on a daily basis. Oh, sure, they have automated tools to help them out - with maybe 100 million queries every day, you'd need to automate too. They look for interesting trends and search behavior that lets them provide better answers.
Secondly, you can do the same sort of thing at your organization. Autonomy, Exalead, Microsoft, Lucene, and even the Google Search Appliance, can all be improved with some custom code after the user query but before the results show up. Did the user type what looks like a name? Check the employee directory and suggest a phone number or an email address. Is the query a product name? Suggest the product page. You can make your search psychic.
Finally, does the query return no hits? You can tell what form the user was on when the search was submitted - rather than a generic 'No Hits' page. Was the query more than a single term? Look for any of the words, rather than all; make a guess at what the user wanted, based on the search form, pervious searches, or whatever context you can find.
So how do you make your search engine seem psychic? Learn about query tuning and result list pre-processing; we've written a number of articles about query tuning in our newsletter alone.
But most importantly: mimic Google: work hard at it every day.
/s/Miles
