Discovery skills versus evaluation skills


Should academic libraries be obsessing so much about teaching the discovery of resources? Should we turn more attention to teaching the evaluation of resources ? Who was Professor Rudolph G Briggs of the Department of Psychotechnology and did he really have to die in the pursuit of information literacy ?


I’ve been reading the JISC & SCONUL Library Management Systems Study which has been out for around three months. This has a great horizon scan as well as several other “stand alone” chapters about factors influencing the choice of future library management systems.

I found myself nodding vigorously when I read about the existing business model for academic libraries:

The business case for the library is predicated on the assumption that the library is the authoritative source of information, and presents optimum access to the best and most appropriate resources in the most efficient way. This raises a dynamic tension between ‘reliable’ and ‘suspect’ sources and questions about the nature of authority.

The subtext of the report is to question whether this business model is still relevant. What if most of our users want “good enough”, not “best”? Can we really claim that it is libraries that deliver optimum access in the most efficient way? What about what the report calls the “gorilla in the room” – Google ?


Academic libraries seem to be focusing so much energy and attention on discovery tools. Tweaking them, trying to get them to play nicely together, and even turning our attention to the usability of our websites. Vendors are trying to sell us ill-defined “vertical searches” that try to emulate Google’s simplicity but claim to eliminate Google’s noise.

A lot of energy in information literacy classes seems to go into ensuring our students understand our discovery tools. In the last couple of years, some libraries have been trying approaches like starting with Google as an entre to showing their own discovery tools. ( ‘Path of least resistance: from Google to Databases’ ).

I think we have all heard tales of academics who declare a blanket ban on Google searching or using Wikipedia. (White bread for young minds says University of Brighton professor ). If you have ever staffed a reference desk you have no doubt stood across from a first or even third year student who seems to have only ever used Google to find information for assignments. (eg. check out the answer to question 12 here on Kate’s Uni blog )


I wonder whether we should shift our focus much, much more toward the evaluation of resources, rather than the discovery of them. The student who only ever uses Google, the academic who bans it and the librarian who only teaches about resources “owned” by the library all have one thing in common – they are focusing on the source and not the resource. They are privileging the means of retrieval, rather than what is retrieved.

Surely it would make more sense to teach students how to work out whether an item is authoritative, well researched and/or suitable for citing in an academic work (these factors do not always overlap). Instead of a small section of an information literacy class, what if evaluation of resources was made central? What if, like Iris Jastram of Carleton College, we tried “teaching databases backward” – starting with a resource and looking at what elements were retrieved, how and why ?

Knowing how to assess whether a retrieved resource is fit for the purpose would “inoculate” our students against using inappropriate discovery tools. Not getting quality resources? Move on to a different discovery method. The starting point is knowing what is a quality resource.

Well, do we have the skills to evaluate what is retrieved and pass that information on to our students? I don’t know.


It’s no longer a case of looking for a “.edu” suffix at the end of the web address or whether the item is cited by other sites. Laura Cohen and Trudi Jacobson, librarians at the University of Albany, SUNY, illustrated this when they created a fake web page for an information literacy class back in 1997 ( see. The Customer is Always Right Part 2 ). Their completely fictitious Dr Rudolph G Briggs, who wrote an equally fictitious seminal work on the subject of Internet Addiction is today still cited on several online health sites . In fact, Professor Briggs was only killed off this year , presumably when Laura Cohen retired.

We need to know a bit more about the mechanics of how websites are published than the average user if we want to be able to teach how to spot a bogus site. We should know, for example, why phishing works and how to spot a fake site. We need to understand Search Engine Optomisation and how websites are manipulated to rank higher on sites like Google. We need to understand the commercial model that led Google to set up Google Scholar and Google Books and how this limits what is retrieved. We need to understand new scholarship and where media like podcasts, blogging and collaborative scholarly tools fit in each discipline, instead of automatically presuming that tools dubbed “scholarly” by vendors will produce better results.

I’m sure that almost all academic libraries are designing information literacy classes that include elements of “evaluating your resources” – usually in sequence with “define question, choose your sources, evaluate, cite”. I’m concerned that some of the items we currently ask students to consider when assessing resources are beyond their life skills when they enter university. Where does the work sit in relation to the body of literature in the discipline?. Is the publishing body a reliable source? Does it have a biased point of view? Does it have the depth of research that makes it valuable? Is the research design sound ? This is sustained and demanding academic work and involves skills that people come to university to learn.

Another study by JISC, Great expectations of ICT: how higher education institutions are measuring up released early June, suggests that students do not already use these methods to check the validity of their sources:

… other than crosschecking online information with references, journals and libraries, students do not see that there are any more rigorous ways in which students ensure information is credible. This cross checking would be complex and time consuming, so they are comfortable using their online sources, therefore, and do not believe that their learning suffers.


What I have a hard time imagining is *how* to design a series of information literacy classes to bridge the gap between uncritically using Google and a thorough intellectual assessment of scholarly resources. How do we break it into small, engaging components that build on each other to give those critical thinking skills? It is much easier to teach students the mechanics of searching, of discovery.

Is it encroaching on what academics should be doing as part of their course? Should schools have already taught them this by the time they set foot in our libraries? It’s definitely beyond our traditional brief, but given that we no longer have a monopoly on the best discovery tools, is it time we sold the library as a place that has value because there are smart people who can give you personalised help to evaluate your information needs and the resources you find?

8 thoughts on “Discovery skills versus evaluation skills

  1. Kathryn, if you were speaking to me via Skype you would be eliciting a vigorous amen. This is where professional practice needs to go. Discovery is great and many folks in the US can do that. Few can evaluate what they find.

    This may explain why I am seemingly out of step with American library practice as of late.

  2. Brilliant! I have been saying similar things for at least the past two years! Part of the problem is that librarians don’t always feel comfortable teaching evaluation. In their obsession with teaching searching (especially their databases) they leave aside the need to be able to evaluate what the students find. By banging on about complex search strategies, they end up terrifying the participants and making it more likely they will end up in Google. There is nothing wrong in going to Google : depends on the search need, and ensuiring they have the knowledge to use other sources as appropriate. Same goes for Wikipedia.
    In a session I gave for librarians in Bradford a few weeks ago I asked them which of the SCONUL Seven Pillars framework
    they thought was the most important to teach in the future. I was very pleased (and slightly surprised) that they picked the evaluation one so all is not lost!
    Web 2.0 technologies give us new ways to teach : how about using Wikipedia against other sources or top see the way information is built up ? How about delicious for collecting resources for evaluation or for sharing?See our book “Information Literacy meets Library 2.0”

What do you think? Let us know.