One interface to rule them all

Prior to summer 2011, Talis Prism 2 provided the main OPAC at The University of Wolverhampton, while Serials Solutions 360 Search (a federated search engine) offered a limited ability to search through our electronic content. Given that Prism 2 was heading towards the end of life, and 360 Search was based on inherently limited federated search technology, we decided that it was the perfect opportunity to implement a more radical change by choosing Summon to unify these two interfaces.

Behind the scenes, it allowed us to set clear boundaries for where our content was catalogued. In the past, everything was catalogued on the LMS database, no matter the format. Maintaining catalogue records for electronic content is a manual and time-consuming task, and can easily become inaccurate or out of date. To avoid significant duplication of data, we chose to make our LMS database exclusively for local physical content, purging it of all electronic content. This content was to be maintained solely in the Serials Solutions KnowledgeBase, which was essential for it to appear on Summon. Removing the electronic content from our local database was a substantial piece of work as we had in the order of 100,000 records; some could be removed automatically, but many more had to be removed by hand.

“… we decided that it was the perfect opportunity to implement a more radical change …”

Our local catalogue was set up to be exported to Summon on a daily basis (initially using a custom script, later replaced by Talis Connect for Summon). One unusual omission from Summon was the lack of title-level records for electronic journals. It was felt that these were vital to include, so initially they were imported into Summon as MARC records. This was done using a Perl script by David Pattern (with a few modifications) which converted the list of our journals available through the Client Centre into MARC format, and then merging them with our main catalogue export. We recently replaced this with an automated method using the 360 MARC Updates service from Serials Solutions. We then had virtually all of our content and resources available through Summon.

“… we chose to make our LMS database exclusively for local physical content, purging it of all electronic content.”

Discovery tools still require a local catalogue service in order to provide details of our local holdings, as well as account functionality. Talis Prism 3 was chosen to provide this, although the theme was modified to provide a more bare-bones experience so that users would not get confused by duplicated functionality. We also modified the navigation within Prism 3 to make returning to Summon easier, for example modifying the breadcrumb navigation to close the window or link back to the Summon results page.

After some initial testing with staff, Summon was finally launched in September 2011. We created our own branded search interface, keeping the single search box that is the default for Summon, and also embedded the search box on our departmental web pages.

The standalone catalogue machines in the learning centres also had to be updated to handle the new system. After a number of iterations, we finally settled on a locked down thin client system running Windows 7. The browser was set up to use a basic proxy to block access to everything except the few sites required for the catalogue to function. Summon and Prism 3 are both hosted services, so we could not just block all external sites. Custom JavaScript was also written for Prism 3 to improve navigation back to Summon after viewing a print record.

Initial feedback

In the months after the launch, we collected as much student and staff feedback as possible. This was done through the feedback link on Summon, and through the frontline staff at the learning centres. Very quickly, two major issues were highlighted:

1. Authentication for electronic articles was problematic.
2. Users searching for specific items were being overwhelmed by the sheer number of results.

The first issue was due to using classic Athens DA as our authentication system. We knew that it would not be optimal for use with Summon, but we had not anticipated such problems. Even if users had authenticated during their browser session, the link from Summon would not detect this, leading to confusion. They would have to find the ‘Athens login’ link which was located in a different place for each resource. This meant that providing instructions for users was tricky. Athens DA was never designed to handle this deep linking, so an alternative method of authentication was needed. This is covered later.

We decided that we could tackle the latter issue immediately by way of an interface redesign. The single keyword search box is ideally suited to discovery, returning a large quantity of results for a vague topic search that can then be narrowed down by way of the facets. This approach does not work as well for specific targeted searches, where many journal or newspaper articles may be a better match for the keywords than the specific book a user may be after. Based on the feedback, students were not using the filters to narrow down their search in these cases, or did not want to go through these extra steps each time.

Three new interfaces were designed that offered the possibility of more specific searching, while trying not to stray too far from the original:

1. Title, author and keyword fields
2. Keyword field with a drop-down format selector
3. Keyword field with ‘Items in the library catalogue’ filter preselected.

These were tested on a group of students, who were asked to perform their search on all three and evaluate which they preferred. Almost 50% of the test group selected interface 1, citing familiarity of the layout and accuracy of the results as their reason.

This interface¹ (see Figure 1) was made default in January 2012, and was a sensible compromise between the ‘OPAC’-style searching that students were used to and the new single search box approach of discovery tools. The embedded search on our website was also updated to use these three fields for consistency. In the month following the change, students were surveyed to see how popular the new interface was. Of the students asked, 62% preferred the new interface to the old one, with only 13% preferring the original. Anecdotal evidence from staff also suggested that students were finding the new interface an improvement.

The Trough of Disillusionment

We had been forewarned by David Pattern (University of Huddersfield) that the implementation of Summon followed the Gartner Hype cycle^2,3. After the Peak of Inflated Expectations at launch, we had plummeted into the Trough of Disillusionment. Every small issue with Summon seemed to be picked up by staff and students.

The first step to combatting this was training for all the frontline staff. The Summon launch coincided with a departmental restructure, making it difficult to train staff beforehand. The half-day session covered searching techniques, use of the filters and a demonstration of all the common authentication pitfalls. Based on the feedback comments, the staff learned a great deal about Summon, and felt better able to help students use it more effectively.

Student skills workshops and inductions were also updated to include a much greater focus on effective use of Summon, which gradually filtered through to the students during the course of the year.

Summon also has a tendency to highlight any inaccuracies in your subscriptions, especially coverage dates for electronic journals which determine the articles that appear as full text. Every ‘problem’ article reported was checked, and the subscription details updated. Gradually, over the course of a year, our holdings data improved to an acceptable level, and reports of inaccessible journal articles tailed off.

“After the Peak of Inflated Expectations at launch, we had plummeted into the Trough of Disillusionment.”

“Speak, friend, and enter”

In early 2012, we also began a full review of our authentication system. Athens DA was no longer fit for purpose as it could not properly support authenticating resources through Summon.

After evaluating all the options, we chose OpenAthens LA as our main Identity Provider (IdP), with EZProxy to provide access to resources through Summon. The combination of these could provide simple authenticated access to all our resources through almost every conceivable route.

This was implemented in August 2012, and has proven to work well with Summon. The number of queries regarding authentication issues appears to have dropped dramatically.

I strongly recommend anyone thinking of implementing Summon to make sure they review their authentication process first, and set up a proxy for all resources.

“I strongly recommend anyone thinking of implementing Summon to make sure they review their authentication process first …”

Statistics

After a year of using Summon, we could finally look at the difference it had made to the use of our electronic and print resources. Naively, we were expecting to see a significant increase in resource usage across the board, but this was not the case. There are numerous other factors that will have affected the usage of our resources. It is possible that the problems with the authentication system have contributed to this, but we will not know for sure until later in 2013 when we have usage statistics for the year since implementing the new authentication system. User behaviour may also be contributing, as more students head to Google as their starting point.

In terms of the usage for individual providers, ProQuest resource usage had risen by a dramatic 528% compared to the previous year, which is not hugely surprising, given that ProQuest is the parent company of Serials Solutions, and so their content is indexed thoroughly. All the other resources saw increases or decreases in usage that were small in comparison. This disparity has raised questions about our content policy for choosing suppliers.

Finally, Figure 2 shows the number of print loans over the past six years, sourced from our SCONUL statistics. We see a gradual decline for five years, followed by a sudden upturn during the first year of Summon. Once again, the introduction of Summon is only one factor that may have affected this, as the upturn also coincides with major refurbishments and new service models within the learning centres, but it is possible that it has made our print stock more discoverable than our old OPAC did. It does suggest that the choice to use Summon as our sole catalogue interface was successful.

The future

The second year of Summon saw us ascend the Slope of Enlightenment, with the service now fully embedded within the department. New students were introduced to Summon during their induction, and training formed an important part of all the workshops run.

This is not to say we have solved every problem. We continue to have issues with the quality of e-book records in the KnowledgeBase from some providers, which causes delays, and authentication is an ever moving target as browsers and operating systems change.

“Users rarely look beyond the first page of results …”

Future work involves analysing how students interact with Summon in more detail, making use of Matthew Reidsma's excellent Summon stats package⁴. Preliminary analysis has already revealed a few interesting results. Users rarely look beyond the first page of results, even when the default was changed to ten results per page. We also found that around 60% of searches ended with no results being viewed. Three quarters of these were followed by another search, many with only a small change to the search terms. Some of these may be students just looking for the shelfmark, and some are spelling mistakes. However, some may be due to the way Summon handles word fragments, especially author initials, which fail to match if the Summon record includes the full name. A cursory look at the data shows a significant number of these searches, as students copy and paste from reading lists. Fewer than 20% of these searches made use of filters to refine their search, slightly lower than the global figure of 23% for all searches.

The figures also appear to support our decision to add search fields to our interface, with ‘title’ being the most used, and fewer than 4% resorting to the advanced search. Hopefully, further analysis will give us a greater insight into the way students search Summon and will highlight areas for improvement, or topics that require greater attention during inductions and workshops.

There are still those who have concerns over the use of Summon as the main interface, suggesting that in some circumstances it would be more appropriate and thorough to search a subject-specific database directly. Implementing Summon to be both an OPAC and discovery tool was always going to be a slight compromise compared to multiple specialized tools but, in our opinion, the benefits to the students of having a single consistent interface outweigh this. Summon also implements a database recommender to promote relevant resources, and now it is up to information literacy training to educate students on the most appropriate method to find information.

“… now it is up to information literacy training to educate students …”

The past two years have been an epic journey, certainly longer than we originally expected. We initially purchased Summon as a natural replacement for 360 Search, but it has ended up impacting on many of our workflows and decisions within the department. Our experience shows that, although some compromises have to be made, it is possible to offer a single discovery interface to students rather than maintaining a separate OPAC, since, like all universities, our goal is to improve the student's experience.