Finding Relevant Results

Finding Relevant ResultsIn the September 2006 edition, Dan Balzer demonstrated that search results depend on which search engines are used. This assessment activity applies the same concept to Blog Search engines. Using three different engines, Technorati, Google Blog Search and Blogpulse, we captured the top ten results for the query: 'search strategy" -job (on 12/04/2006).

We searched for blogs on the topic of effective online search strategies. By using "quotes" we were able to narrow the results to that exact phrase and by eliminating job (-job) we prevented the job search pages, of which there are many, from coming to the top.

This assessment is aimed at students' abilities to recognize relevant information in snippets. The task requires them to:

• understand what content (concepts) they are searching for

• read the top ten results from three blog searches

• rate the results using the following scale:

2 - this result contains relevant information

1 - this result MAY contain relevant information

0 - this result does not appear to contain relevant information

$book icon\$ Materials Needed: Links and Student Worksheet

Reproducable Student Answer Worksheet (Word document only)

Google Blog Search Results (available as Word document)

Technorati Results (available as Word document)

Link to Blogpulse Results (available as Word document)

Instruct students to use the Worksheet to rate how relevant they think each result is, in terms of the query. Provide them with worksheets and links to the three sets of search results.

Depending on the time available, divide your group into three parts and assign each group a different set of results to rate.

Table 1 provides an ANSWER KEY which may be used as a basis for assessing students' task performance. This key is not intended to be used for grading a student's performance.

	Table 1: Answer Key
	Technorati	Google Blog Search	Blogpulse
	1,993 total results	4,076 total results	195 results
1.	1	1	1
2.	1	0	0
3.	2	1	0
4.	0	1	0
5.	0	0	2
6.	2	2	2
7.	0	0	2
8.	1	1	2
9.	2	1	0
10.	2	0	0
total	11	7	9
Number Key: 2 - this result contains relevant information 1 - this result MAY contain relevant information 0 - this result does not appear to contain relevant information

discussion icon Debrief the activity by having students discuss what they found:

• Which search engine produced the greatest number of relevant results?
• What information did they use to determine if a result was relevant or not?
• Do you or your students disagree with our findings (Table 1)? If so, why?
- (Note that many of the relevant Blogpulse pages are written in Arabic)
• What did you discover about blog search engines?
- (see bulleted list at bottom, for starters)
• How can you use this approach to reading to improve your own search strategy?
- (e.g., choose only the pages rated as 2 to start with...)

If there is no time for discussion, collect and evaluate the worksheets. Scores that are significantly different than the ANSWER KEY may indicate a misunderstanding of the initial search task or a failure to recognize relevant results, both of which require further intervention.

Several lessons may be learned from the comparison of blog results. Although this activity focuses on assessment, it may also serve as a starting place for helping students discover that:

• blog search engines return different numbers of results. The reason is that three databases contain different records. Blogpulse contains the least;

• blog search engines return different types of results. While there is some overlap among the results, there are also returns that are unique for each search engine;

• if operators are not used, results are typically less well-focused than results from a webpage search engine;

• compared to a webpage search engine (Yahoo, etc.), results are fresher. Blogs are crawled more frequently than web pages, up to several times a day compared to weeks apart.