Recognizing Relevance

Part Three of a Five-part series

Carl Heine, Ph.D.

When I first began observing Internet searches, I was amazed how often people of all ages failed to see relevant information in their search results. It was like watching prospectors mine for gold with their eyes shut—passing by valuable nuggets right on the surface, unearthing treasures only to cast them aside. After watching hundreds of searchers abandon pertinent results, I now consider it a common occurrence. It is an unnecessary waste of time that with training may be avoided.

Here are some typical examples:

•  A high school student looking for information about the number of Broadway show openings per year finds a web page that answers the question but fails to notice a table listing openings by year. As a result, she backtracks and continues to browse.

  [See if you do any better—try this challenge yourself]

• A teacher queries ‘information fluency half day workshop materials’ to find participant materials during one of our Half Day Power Searching workshops. Instead of selecting the top return (the right one), he opts for the fourth return instead and links to the Full Day set of materials.
• Middle school student are asked to identify which of three web pages contains facts that support a research question. Only 28% do so correctly.
• Junior honors students perform the same task with 36% accuracy, only slightly better than guessing.

In this series of five things today's digital generation cannot do, the first two problems occur prior to searching (inability to turn a question into a query, and not knowing where to search). Failing to notice relevant information is the first mistake made while searching. As to what goes wrong at this point, several factors must be considered.

• At the root level, recognizing relevant information in snippet and web pages requires sufficiently developed reading skills. To oversimplify, knowing whether information is relevant depends on comprehension; comprehension depends on ability to read. Fifth graders who read at grade level cannot be expected to make sense of materials written for graduate students. Except for aligning readings more closely with abilities, addressing problems with cognition, reading or comprehension is well beyond the scope of this article and demands more extensive intervention than is practical within the context of teaching digital search skills.

• If individual abilities are not the problem, factors inherent in Web page design may be. Hypertext (as opposed to linear text), column width, font size and color, screen resolution and graphics that distract from the content are among web-dependent factors that may prevent recognition of relevant information. Moreover, answers on the Internet occur in unpredictable places: snippets, standard text, hypertext, captions and tables—virtually anywhere on a Web page. Targeted training may help searchers overcome a host of Web-related Readability problems.

• Assuming reading skills are adequate and web desgin elements are not a problem, haste may be the ultimate culprit when relevant information is missed. Watch young people search. You will see haste, with time per page measured in mere seconds. If the eye does not lock on an answer almost immediately the answer is missed. Active reading strategies (described below) are intended to make searchers slow down, think more critically and scan more carefully. Readers who spend time to determine the relevance of information save time in the long run.

In my experience, overlooking relevant information is not typically a problem of cognitive ability or the fault of web page design. The problem is simply scanning too fast.

Speed alone does not equal efficiency. Even though computers and the vast resources on the Internet encourage a culture of speed—there remains an individual limit to how fast each of us scan and comprehend text. Exceeding your limits decreases efficiency and leads to frustration.

What you can do to help  

If you suspect your students are regularly missing important information, investigate to find the cause of the problem.

Observe your students search

The assessment article in this Kit, "What are they thinking?" includes ideas on how to do effective observations and talk-alouds. Internet Search Challenges quickly reveal performance mistakes. Watch students play the Gold Rush tutorial games in this Kit. This will help you judge their ability to scan and comprehend. For a more thorough diagnosis enlist the aid of a reading specialist. Digital information training isn't the time for focused reading remediation but it does provide teachable moments for modeling good reading and scanning practices.

To help students must diagnose their reading problems. Attempt to isolate the cause of the problem to one of these areas: ability to read, usability of the information or reading too fast. In the context of digital information skills training, a different intervention is recommended for each of these problems, starting with the most common, reading or scanning too fast.

1. Haste—reading too quickly

In most cases, missing relevant information is caused by reading to fast or ineffective scanning. Encourage students to slow down by reading more deliberately. Good readers tend to employ active comprehension strategies, see Michael Pressley whose questions for students have been adapted below for use when reading online.

Questions to ask yourself:

1. Have I read the texts I expect my students to read? This may not be practical if students are conducting open-ended searches. Especially in this case, demonstrate how to use the questions intended for students. 2. Are the texts appropriate in terms of readability and content? This may be difficult to judge except in the case of teacher-made web quests or materials drawn exclusively from subscription databases. For ways to determine readability, refer to the paragraph on Readability at the end of this article. 3. Do the texts answer the question(s) I posed or support the objectives for this lesson? 4. How do I model active comprehension strategies for my students? Model searching for your students. Conduct an interactive search demonstration as you look for information. Try using the prompts and techniques found listed below.

Questions for students:

1. What information am I looking for? Active comprehension starts with knowing why an article is being read, this is the way expert searchers begin. 2. What do I expect to find in this text? Make predictions about the reading. Assume that EVERY page has an answer—or at least a valuable clue—that will move the search forward. Answers in web articles are found in unpredictable places. Unlike textbooks, answers are not clustered near the ends of chapters or highlighted. Look for answers in hyperlinks, captions, tables, graphics, snippets and in the body of web articles. The interactive game GOLD RUSH #1 reinforces this idea using snippets. Before you abandon a web page learn something to assist your search: find a better keyword, add some information about the topic or realize that different keywords are needed because you found nothing relevant to your search. 3. Where are my keywords located on this page? What sections are they in? What are the headings?  Scan or use the FIND command to locate keywords. Your keywords brought you to this page—find out what they are doing there. Have students play GOLD RUSH #2 to find terms in paragraphs more quickly using effective scanning. 4. What, if any, scanned parts look the most promising? Start by reading selectively the sections where your keywords are found. 5. Do my keywords make sense in the context of this article? (Does this reading make sense to me?) Either Gold Rush game may be used to check for comprehension. These games will encourage students to read more carefully for meaning. 6. What if nothing I scanned looks promising? If nothing looks promising, return to your search results and a) view another web page or b) search using alternate keywords.

Using strategies 1 – 6 will improve comprehension and help searchers recognize important information. The remaining active comprehension strategies, 7 – 12, promote higher-order thinking, optimal search refinement (the theme of Resource Kit vol. 1.3) and web page evaluation (the theme of vol. 1.4): 7. What ideas in the text are familiar? Are the facts believable? 8. What ideas and words are unfamiliar? Can I visualize the meaning? Does the context or hypertext provide clues to unfamiliar vocabulary? 9. Are any of the words here better keywords? 10. Were my predictions about what I would find correct? 11. Do new ideas conflict with what I know? Do I need to revise my prior thinking? 12. What insights and/or questions do I have after reading this text?

2. Usability—managing the computer screen display

Even capable readers can have trouble reading online when web pages are poorly designed for usability. Text may be too small, columns may be too wide, screen colors may be distracting or make it difficult to read text. Hypertext presents unique challenges, as do animations and unrelated graphics. For tips on reading hypertext, see Help with Hypertext. A series of Flash Video Tutorials in this issue addresses common problems such as these and how to overcome them. For more information, refer to Browser Literacy, also in this issue.

3. Readability—adjusting the grade level

Reading remediation falls outside the scope of digital information skills training. But before you call in a specialist, ask yourself this question. Is it possible the web pages your students are struggling with are simply too challenging for them? To determine the reading level of selected materials, use an online diagnostic tool or copy and paste a passage into MS Word, as described below.

Online: Test the readability of a Web page using tools such as juicystudio. Copy and paste the URL of the Web page you want to check into the search box to calculate page readability. Explanations of the scales and algorithms are provided to help interpret the results. Juicystudio calculates that our current 21cif.imsa.edu portal page is at the 8th grade level on the Flesch Kincaid scale. To find more online readability tools, search using keywords: Flesch Kincaid Gunning Fog.

Manually using MS Word: Copy and paste the text to be assessed into a Word document. Press F7 to check the entire passage, or mouse over (select) just the text to be diagnosed and press F7 (same as Tools > Spelling and Grammar). The display lists the readability information. Using this technique on this paragraph, a rating of grade 9 is obtained. (Reading level is grade 10.3 for the entire article).

In the next issue: Helping young people refine searches by finding more powerful keywords in search results.