Simple and advanced 'whole of site' searching: user behaviour on a government agency intranet. A short case study by Maxine Armitage.
Introduction
This case study analyses user behaviour on a government agency intranet site.
The intranet is 2 years old, the agency's charter is to police compliance with legislation and it employs over 1,000 staff. Staff are mainly clerical officers employed across many divisions within the organisation. Professional staff are mainly legal officers. My observations are based on data gathered over 2 years.
Definitions
The simple search is the search function provided through one text box
It is important to note here that the simple search actually employs complex strategies behind the scenes which make searching and search results more useful. These strategies are:
- use of a thesaurus of synonymous terms. A search for 'OH&S', for example, will also find 'occupational health and safety'. A search for 'adverts' will also find 'ads', 'adverts' and 'commercials'.
- assigning metadata to all information resources. This allows:
- search results to be weighted according to where,and whether, search terms appear in the metadata and in the full text. A search term appearing in the subject metadata is assigned a value of 1.
A lesser value is assigned if the term appears in the title. Lesser still if it appears in the description and the least value if it appears in the text.
Totals of these values determine where resources are ranked in the search results.
The most relevant resources are displayed at the top of the results list. - the display of meaningful search results. Searchers have a very good idea of what they will find when they follow a link.
- search results to be weighted according to where,and whether, search terms appear in the metadata and in the full text. A search term appearing in the subject metadata is assigned a value of 1.
The advanced search is the search function provided through multiple text boxes and pick lists which operate individually or via Boolean operators
In the advanced search the keyword search box operates exactly the same as the simple search described above.
Observations
The use of simple search vastly outstrips the use of advanced search
"A searcher is 13 times more likely to use the simple search interface than the advanced search interface".
A searcher is 13 times more likely to use the simple search interface than the advanced search interface. This has remained remarkably consistent over 2 years with the simple search accounting for 93% of whole of site searches, the advanced search only 7%.
Furthermore, in 36% of the instances of advanced search usage, searchers used only the keyword box. Effectively they were just executing a simple search through the advanced interface and not taking advantage of the powerful functionality offered by the advanced search template.
Failed searches*
17% of simple searches failed, mostly as the result of:
- typos and bad spelling
- lack of understanding of the likely content and purpose of an intranet, eg searching for 'Google'; and 'australia's largest morning tea'
- search logic. When multiple terms are entered, by default, the search engine treats those terms as a phrase. The less precise the phrase, the lower the chances of returning results, eg, 'flex agreement lunch' returned nothing even though the content exists on the intranet.
20% of advanced searches failed, mostly as the result of all of the above and:
- inappropriate use of the multi criteria options
- re-searching through the advanced search after having failed to find results in the simple search.
*A search was deemed a failure if no results were returned.
Advanced interface usage
"The advanced search interface was treated as a form to be filled out as fully as possible ..."
When searchers were faced with multiple search criteria some did their best to fill in lots of boxes even if they didn't understand what they meant. The advanced search interface was treated as a form to be filled out as fully as possible, not as a tool to fix appropriate parameters on a search.
Conclusions
Simple search
"The simple search is by far the most popular search."
The simple search is by far the most popular search. Survey results show that it works well with over 70% of respondents considering the simple search to be 'good' or 'excellent'.
The search log files helped us identify the three main reasons for failure and happily, all three can be remedied to some extent by the intranet manager. This is a much easier strategy than attempting to increase success rates by changing user behaviour.
Typos and bad spelling can be addressed, within reason, through the synonymous terms thesaurus, eg flexable = flexible; and, greivance = grievance. The question is how far do you go? I'm still grappling with 'steakholders'and & 'pornbrokers'.
Searches for non intranet content can also be addressed, via redirect html pages with stacked metadata. For example a Google redirect page with the word 'Google' repeated in the metadata many times ensures that this page is the first returned in the search results.
The search logic can be changed to make 'and' the default operator for multi term entries. The phrase option was originally chosen because it enables more precise searching when more than one phrase is input, for example:
occupational health and safety; sick leave
will return spot on search results. The problem is that people don't generally search like this. As shown in the 'flex agreement lunch'example searchers don't use perfect syntax.
The 'phrase'; search is most effective for searching specialized information with highly structured language. Botanical information is a good example. When looking for 'Acacia longifolia var. sophorae' this is exactly what is required and other permutations of the words will return irrelevant search results.
Whole of site advanced search
"The advanced search is used infrequently, used badly and with less success than the simple search"
The advanced search is used infrequently, used badly and with less success than the simple search. Although the success rate appears high it is deceptive because the administrator tends to favour the advanced search for administrative and site development purposes. The administrator is an experienced searcher.
I suspect that users equate the term 'advanced' with the idea of 'more powerful' (in a superman sort of way) when what it really is, is a 'precise' search (in a 'know what you're doing before you use it' sort of way). Hence the tendency to re-search on advanced after having failed on the simple search.
While multiple options increase the likelihood of success for an experienced searcher they increase the likelihood of failure for the inexperienced.
The two main reasons for failure specific to the advanced search, ie, inappropriate use of the interface and re-searching after failure on the simple search, could be addressed through training. You have to question though, the value of a generic search tool which is not intuitive.
An advanced search, however, is a valuable tool for searchers who share a common understanding and language and to whom the search options make sense. I would advocate an advanced search for specific content within an intranet (if the need is evident) but not a whole of site advanced search.