The Rocky Road to Assessment: The Creation of an Assessment Tool for an Information Literacy Credit Course
Angela Dunnington and Mary Lou Strong, Southeastern Louisiana University

Abstract
The article discusses the online assessment survey developed by the Sims Memorial Library Reference Department Instruction Team to measure student learning outcomes of students enrolled in an eight-week one-credit hour freshman level course.  The authors, who are instructors of the course, examine the planning, implementation, and evaluation of the pre- and post-test survey instrument and discuss the lessons learned from undergoing this process.  

The Rocky Road to Assessment: The Creation of an Assessment Tool for an Information Literacy Credit Course

Introduction and Background 
In the summer of 2004, Sims Memorial Library of Southeastern Louisiana University developed an online assessment survey designed to measure learning of selected library research knowledge and skills of students enrolled in an eight-week one-credit hour freshman level course. Grounded in twenty-four measurable outcomes identified from the Association of College and Research Libraries (ACRL) Information Literacy Competency Standards for Higher Education, the instrument was developed through the collaborative efforts of the Reference Department Instruction Team (The Team). This article will examine three distinct processes in the creation of the instrument: planning, implementation, and evaluation. The authors, who are instructors of the course, will also discuss some of the challenges encountered in developing and implementing a pre- and post-test assessment instrument. 
Southeastern Louisiana University is a public, four-year institution with an enrollment of approximately 15,160 students.1 Southeastern currently offers seventy-eight degree programs: four at the Associate Level, fifty-two at the Baccalaureate level, twenty-two at the Masters level, and one Doctoral program.2 Sims Memorial Library provides a multi-faceted library instruction program for Southeastern students, faculty, and staff. Librarians provide “one-shot” instructor-requested sessions for classes at all levels and within all disciplines. Sims Library also demonstrates its commitment to information literacy education through its Library Science (LS) 102: Introduction to Information Research course. LS 102 is an eight-week, one-credit hour course that is designed to help students effectively and efficiently locate, evaluate, and use information. The course is taught during the fall, spring, and summer semesters totaling approximately fifty-eight sections per year. The course is open to all students and is required by approximately half of the University’s majors as part of their curriculum. It is typically taught by library faculty members within the Reference Department.  
Literature Review 

The literature on the use of pre-test and post-test methodology to measure student learning in information literacy for-credit courses is limited. In summer 2004, when the Team initiated its project to create a learning outcomes assessment instrument, few articles on this topic were available. One study by Mollie Lawson (1999) employed pre-test/post-test methodology to measure student learning in an information literacy credit-bearing course.  The instrument contained fifteen content-based questions. The pre-test was given on the first day of class and the post-test questions were included on a final exam. Thus, the students may have been particularly motivated to correctly answer the post-test questions. Comparison of pre- and post-test scores showed significant improvement. Based on this data, Lawson concluded that “it can be shown that a credit hour course can increase the knowledge and skills of library clients.”3  
In recent years, several additional studies have been conducted in conjunction with the assessment of information literacy credit-bearing courses. Nancy Goebel, Paul Neff, and Angie Mandeville (2007) created a pre-test/post-test instrument to evaluate learning among second-year students in for-credit discipline-specific information literacy courses.4 In their article, they discussed the development of the instrument, data collection methods using a locally developed database system, and plans for future refinements. Like the current study, the test was geared toward the ACRL Information Literacy Competency Standards as well as the library’s objectives and mission statement. However, the information literacy courses involved were discipline-specific in nature and directed to second-year as opposed to freshman level students. 
Likewise, Jon R. Hufford (2010) used pre- and post-tests to assess student learning outcomes in a for-credit library research course at Texas Tech University. A total of 176 students completed both surveys. The findings suggested that the average score of students taking both surveys improved from pre-test to post-test. However, the author expressed disappointment with the post-test scores as the gains were lower than anticipated. As with the current study, some students even scored lower on the post-test than the pre-test on several questions. The author discussed ways to improve both the instrument and the instructional methods used. Hufford emphasizes that librarians should share their experiences in conducting assessment because “librarians who want to improve their information literacy program through assessment can benefit immensely from the experiences of their colleagues at other institutions.”5  
In his seminal article, Donald Barclay (1993) challenges library teaching faculty to engage in the evaluation of library instruction.6 He contends that evaluation is necessary to meet calls for accountability, for the justification of programs, and for improvement of instruction.  Barclay urges teaching faculty to undertake assessment of their programs even under circumstances where library faculty have few resources and even less time to conduct a perfectly designed study employing complicated statistical analysis. He cogently advises teaching librarians to “set your sights lower and do the best you can with what you have.”7Although Barclay’s evaluation project did not involve the assessment of an information literacy for-credit course, the study provides important insights into the assessment process. 

Developing the Assessment Instrument
The initiative for this project began as the university focused on assessment in preparation for a comprehensive evaluation and accreditation by the Southern Association of Colleges and Schools (SACS) in 2003-2005. The library recognized the need to address the issue of assessment in its instruction programs and to measure student learning outcomes. Prior to the creation of this instrument, the effectiveness of the LS 102 program was measured using simple pre- and post-surveys focused solely on student confidence levels. To assess whether actual learning was taking place, the Reference Department Instruction Team decided to develop a series of content-based questions. The Team elected to use pre- and post-test surveys to measure the skill sets of LS 102 students at the beginning of the course and at its completion. 
In determining what knowledge and skills to measure, the Team looked to the LS102 course objectives which are: 
•	Students will understand the organization, tools, and vocabulary of an academic library.
•	Students will utilize databases to find and retrieve information.
•	Students will identify relevant sources using critical thinking skills. 
•	Students will create a bibliography of relevant sources. 
•	Students will understand the need for developing life-long learning skills.
The Team also analyzed the ACRL Information Literacy Competency Standards for Higher Education and corresponding performance indicators in relation to the course objectives.8 Through this process, twenty-four outcomes were identified for measurement. Not all outcomes set forth in the ACRL Standards are conducive to measurement by pre- and post-test methodology or are relevant to course objectives. For example, Standard One, Performance Indicator 3, Outcome b “considers the feasibility of acquiring a new language or skill (e.g., foreign or discipline-based) in order to gather needed information and to understand its context,” an objective that exceeds the scope of the LS 102 course.9  
Each Team member was responsible for drafting one survey question based on each of the assigned ACRL standards and performance indicators shown in Table 1. Relevant performance indicators and outcomes selected from the ACRL Standards are set forth in Appendix A. After much discussion and revision, seventeen content-based questions were created, along with three confidence level questions, for a total of twenty pre- and post-test questions. The pre- and post-test assessment instruments were identical.  

Table 1
ACRL Information Literacy Competency Standards used to create survey questions	
ACRL Standard(s) 	Performance Indicators 	
Determine the extent of information needed. [ACRL 1]	1c, 1d, 1e, 2b, 2d, 4b	
Access the needed information effectively and efficiently. [ACRL 2]	2a, 2b, 2c, 2d, 2e, 3a, 3b, 4a, 4b, 4c, 5c, 5d 	
Evaluate information and its sources critically and incorporates selected information into one’s knowledge base. [ACRL 3] 	2a, 2c	
Use information effectively to accomplish a specific purpose. [ACRL 4]   	1a	
Understand the economic, legal, and social issues surrounding the use of information, and access and use information ethically and legally. [ACRL 5]     	1d, 2f, 3a	

Implementation and Administration of the Instrument
The online assessment survey (shown in the Appendix B) was created using Microsoft FrontPage and was stored on the university’s web server. It was designed to gather statistical information in order to compare pre- and post-test results. With the assistance of the University’s Center for Faculty Excellence, the results were collected and maintained using Statistical Package for the Social Sciences (SPSS) software. Icons were created on the computer desktops in the instructional classrooms linking to the online survey. The pre-test was administered during the last ten to fifteen minutes of the first LS 102 class meeting for each section. The post-test was administered during the class prior to the final exam. The surveys were administered during the fall 2004 and spring 2005 semesters. 
Evaluation and Analysis of Results 
A total of 1063 students took the pre-test, while 759 students participated in the post-test. Some of the disparity in numbers can be attributed to students who dropped the course or failed to attend class on the survey administration date. Also, several instructors failed to give the survey due to class time constraints or they simply forgot.  The survey answers were anonymous and no attempt was made to match the pre- and post-test responses of individual students. Changes in the performance of the respondent group as a whole were examined.  
In reviewing the results of the confidence level questions, the data suggest that the confidence levels of students improved with respect to the use of the library and its resources.  For example, in question two of the survey, the percentage of students who reported that they felt “very confident” (the highest confidence level) in choosing the best search tools for locating specific types of information sources rose from 15.7 percent in the pre-test results to 29.4 percent in the post-test results.  
The results from several of the seventeen content-based questions showed improvement from pre- to post-test. For example, question ten centered on the ability of students to determine the correct use of the Boolean operator “OR.” The correct responses by student participants on question ten rose from 28 percent to 41.9 percent from pre- to post-test. However, performance on many of the content questions did not significantly improve and, in some cases, did not improve at all. In question fourteen, student participants were asked to identify a journal citation.  87.1 percent of respondents identified the correct answer in the pre-test and 88.5 percent of students answered correctly in the post-test, resulting in an increase of only 1.4 percent. Moreover, a large percentage of students (87.1 percent) answered question fourteen correctly on the pre-test without any instruction. Question six was designed to measure the ability of students to narrow topics. The percentage of correct answers actually dropped from 43.7 percent to 40.4 percent on the post-test. 
In reviewing the overall results, the most perplexing aspect of our experience is while the data suggested an increase in the percentage of students who felt more confident about their information research skills, the scores of students on individual content questions did not significantly improve on most questions. Does this mean students are not learning content in our credit course?  Anecdotally, we do not believe this to be the case based on comments from students about how much they have learned, including the occasional exuberant cry that “Academic Search Premier is a freshman’s best friend” spoken by a student in the throes of a beginning composition course, and on our own experiences as teachers. 
Lessons Learned: Plans for Improvement of Assessment Process 
	The assessment process clearly has been a learning experience. In trying to make sense of the inconclusive results, the authors have reflected on the data and examined recent literature on assessment of student learning to identify flaws within the instrument and its implementation.  Several conclusions have been reached as a result of this evaluative process.  
	First, the composition of the group responsible for creating the instrument should be restructured. The Team should be reduced from a large group of nine librarians with varied degrees of interest to a smaller working group during the initial phases of the project. Obtaining consensus on the content and wording of each question proved difficult when dealing with nine individuals. An Assessment Working Group, comprised of four reference/instruction librarians, has been formed to create a revised instrument and to address problems identified with the initial assessment process.   
	Second, analysis of the results suggests that several of the questions are ambiguous or poorly worded. Joseph R. Matthews has cautioned about the problems inherent in using “locally developed questions [which] are often not subjected to rigorous analysis to screen out use of jargon, or the answer might be indicated in another item.”10 The quality of the instrument can be improved by crafting better questions and using focus groups to pre-test the questions. Also, efforts need to be made to refine specific questions to ensure that only one answer is correct and that the correct answer is not too obvious. For example, question fourteen (shown below) is designed to assess whether students can identify information sources from a citation.    
Q14.  Keeling, Richard P.  "Binge Drinking and the College Environment."  Journal of American College Health, 50(5): 197.
The above citation refers to:
o	a chapter from a book
o	an excerpt from an encyclopedia
o	an article from a journal
o	not sure

A high percentage of the students answered the question correctly in both the pre-test (87.1 percent) and post-test (88.5 percent) results. One problem with the question is that the answer was too obvious. The publication title used in the question was Journal of American College Health and the answer was “an article from a journal.” More rigorous analysis of the survey including pre-testing the questions through focus groups in all likelihood would have identified this question and others as problematic.   
	Third, the issue of the variation between the number of students taking the pre-test and post-test is a serious one and must be addressed. A total of 1063 students took the pre-test, while 759 students participated in the post-test. Thus, almost a third of the students failed to take the post-test. Without any means to identify the respondents, it is impossible to compare the results of the pre- and post-tests with confidence. In the Hufford study, only 176 of 310 students took both the pre- and post-test surveys. However, the author was able to identify the students who had not taken both tests through WebCT  (its course management software) and was able to omit the data relating to those students.11 In any future assessment, a method to identify respondents who have not participated in both pre-test/post-test surveys must be devised. 
	At Southeastern, the disparity in numbers appears to have resulted from two factors: the number of students who withdrew from LS 102 and the failure of instructors to administer the post-test. As a result of new university policies limiting student withdrawals from classes, the authors have found that the number of withdrawals from LS 102 has substantially decreased in recent semesters. Measures also must be found to encourage better compliance by instructors in administering the post-test survey. One solution may be to explore the idea of administering surveys electronically through course management software (e.g., Blackboard or Moodle) as a course requirement.  
	Fourth, the mechanics of assessment data collection proved to be overly complicated.  For instance, a single folder had been created in FrontPage to receive results from the pre- and post-tests surveys. When responses were imported into SPSS and Excel for later analysis, the data had to be sorted into pre- and post-categories by examining the dates on which the surveys were taken. Better ways to store data and to separate “pre-” from “post-” results must be devised.  Examining online survey systems such as SurveyMonkey could provide a time-saving approach to collecting and analyzing assessment data.  

Conclusion
While the results of the initial attempt at assessment of student learning through locally developed pre- and post-test surveys were imperfect, the authors believe that the project constitutes an important first step. Barclay suggests that “some hard evaluation data, even if the data may be less than perfect, are better than either no data at all or soft data obtained from anecdotal observation and surveys of student satisfaction.”12 As we attempt to obtain more reliable data on how well the for-credit course is meeting student needs, we believe it is important to educate ourselves further about survey methodology to improve the validity and reliability of the instrument. Looking forward, an Assessment Working Group, comprised of four reference/instruction librarians, has been created. The Assessment Working Group has begun its work by reevaluating the ACRL Information Literacy Competency Standards for Higher Education in light of our course objectives and reviewing the current assessment literature. Assessment of student learning outcomes is crucial to the success of any instructional program. In that spirit, we make our findings available so that others can learn from our experiences.   
Notes
1Southeastern Louisiana University, Office of Institutional Research and Assessment, “Common Data Set 2009: Enrollment and Persistence,” http://www.selu.edu/admin/ir/cds/2009/enroll.html 

2Southeastern Louisiana University, Office of Institutional Research and Assessment, “Information on Majors,” Southeastern Louisiana University,” http://www.selu.edu/admin/ir/factbook/majors_info.html 

3Lawson, Mollie D., “Assessment of a College Freshman Course in Information Resources,” Library Review 48, no. 2 (1999): 77.  
4Goebel, Nancy, Paul Neff, and Angie Mandeville, "Assessment Within the Augustana Model of Undergraduate Discipline-Specific Information Literacy Credit Courses," Public Services Quarterly 3, no. 1/2 (January 2007): 165-189. 

5Hufford, R. Jon, “What Are They Learning? Pre- and Post-Assessment Surveys for LIBR 1100, Introduction to Library Research,” College and Research Libraries 71, no. 2 (March 2010): 150. 

6Barclay, Donald, "Evaluating Library Instruction: Doing the Best You Can with What You Have,” RQ 33, no. 2 (1993): 196. 
7Ibid.,196.
8ACRL, “ACRL Information Literacy Competency Standards,” ACRL, http://www.ala.org/ala/mgrps/divs/acrl/standards/informationliteracycompetency.cfm 
\9Ibid. 
10Matthews, Joseph R., Library Assessment in Higher Education (Westport: Libraries Unlimited, 2007), 76.
11Hufford, “What Are They Learning,” 146.  
12Barclay, “Evaluating Library Instruction,” 196.  

Appendix A: ACRL Information Literacy Competency Standards used to create survey questions.  

Standard One: The information literate student determines the nature and extent of the information needed.

Performance Indicator 1: The information literate student defines and articulates the need for information. 

Outcomes: 
c. Explores general information sources to increase familiarity with the topic
d. Defines or modifies the information need to achieve a manageable focus 
e. Identifies key concepts and terms that describe the information need 

Performance Indicator 2: The information literate student identifies a variety of types and formats of potential sources for information. 

Outcomes: 
b. Recognizes that knowledge can be organized into disciplines that influences the way information is accessed

Performance Indicator 4: The information literate student reevaluates the nature and extent of the information need. 

Outcomes: 
b. Describes criteria used to make information decisions and choices

Standard Two: The information literate student accesses needed information effectively and efficiently.

Performance Indicator 2: The information literate student constructs and implements effectively-designed search strategies. 

Outcomes: 
a. Develops a research plan appropriate to the investigative method
b. Identifies keywords, synonyms and related terms for the information needed 
c. Selects controlled vocabulary specific to the discipline or information retrieval source
d. Constructs a search strategy using appropriate commands for the information retrieval system selected (e.g., Boolean operators, truncation, and proximity for search engines; internal organizers such as indexes for books)
e. Implements the search strategy in various information retrieval systems using different user interfaces and search engines, with different command languages, protocols, and search parameters.

Performance Indicator 3: The information literate student retrieves information online or in person using a variety of methods. 

Outcomes: 
a. Uses various search systems to retrieve information in a variety of formats
b. Uses various classification schemes and other systems (e.g., call number systems or indexes) to locate information resources within the library or to identify specific sites for physical exploration 

Performance Indicator 4: The information literate student refines the search strategy if necessary.

Outcomes: 
a. Assesses the quantity, quality, and relevance of the search results to determine whether alternative information retrieval systems or investigative methods should be utilized
b. Identifies gaps in the information retrieved and determines if the search strategy should be revised
c. Repeats the search using the revised strategy as necessary

Performance Indicator 5: The information literate student extracts, records, and manages the information and its sources. 

Outcomes: 
c. Differentiates between the types of sources cited and understands the elements and correct syntax of a citation for a wide range of resources 
d. Records all pertinent citation information for future reference 

Standard Three: The information literate student evaluates information and its sources critically and incorporates selected information into his or her knowledge base and value system.

Performance Indicator 2: The information literate student articulates and applies initial criteria for evaluating both the information and its sources. 

Outcomes:
a. Examines and compares information from various sources in order to evaluate reliability, validity, accuracy, authority, timeliness, and point of view or bias 
c. Recognizes prejudice, deception, or manipulation 

Standard Four: The information literate student, individually or as a member of a group, uses information effectively to accomplish a specific purpose.  

Performance Indicator 1: The information literate student applies new and prior information to the planning and creation of a particular product or performance. 

Outcomes: 
a. Organizes the content in a manner that supports the purposes and format of the product or performance (e.g., outlines, drafts, storyboards) 
Standard Five: The information literate student understands many of the economic, legal, and social issues surrounding the use of information and accesses and uses information ethically and legally. 

Performance Indicator 1: The information literate student understands many of the ethical, legal and socio-economic issues surrounding information and information technology. 

Outcomes: 
d. Demonstrates an understanding of intellectual property, copyright, and fair use of copyrighted material 

Performance Indicator 2: The information literate student follows laws, regulations, institutional policies, and etiquette related to the access and use of information resources. 

Outcomes: 
f. Demonstrates an understanding of what constitutes plagiarism and does not represent work attributable to others as his/her own 

Performance Indicator 3: The information literate student acknowledges the use of information sources in communicating the product or performance. 

Outcomes: 
a. Selects an appropriate documentation style and uses it consistently to cite sources 

Appendix B: Pre- and Post-Test Information Literacy Survey Questions and Results  
Fall 2004 and Spring 2005 (correct answers are highlighted)

Directions:  Please mark the response that best fits each statement or answers each question.  When you have finished filling out this form, click on the Submit button at the bottom of the page.

Q1.  How confident do you feel in selecting, developing and narrowing a topic for research?
o	very confident
o	confident
o	somewhat confident
o	not confident
		 
Pre-test (Q1)	
Frequency	
Percent	

very confident	
228	
21.4	

confident	
463	
43.6	

somewhat confident	

337	

31.7	

not confident	
35	
3.3	

Total	
1063	
100.0	

	Post-test (Q1)	
Frequency	
Percent	

very confident	
267	
35.2	

confident	
306	
40.3	

somewhat confident	

167	

22.0	

not confident	
19	
2.5	

Total	
759	
100.0	

Q2.  How confident do you feel in choosing the best search tools for locating specific types of information sources?
o	very confident
o	confident
o	somewhat confident
o	not confident
		          
Pre-test (Q2)	
Frequency	
Percent	

very confident	
167	
15.7	

confident	
422	
39.7	

somewhat confident	

399	

37.5	

not confident	
75	
7.1	

Total	
1063	
100.0	

	Post-test (Q2) 	
Frequency	
Percent	

very confident	
223	
29.4	

confident	
310	
40.8	

somewhat confident	

200	

26.4	

not confident	
26	
3.4	

Total	
759	
100.0	

Q3.  How confident do you feel in evaluating sources to determine the type of information, such as popular, scholarly, or trade?
o	very confident
o	confident
o	somewhat confident
o	not confident

Pre-test (Q3)	
Frequency	
Percent	

very confident	
176	
16.6	

confident	
436	
41.0	

somewhat confident	

319	

30.0	

not confident	
132	
12.4	

Total	
1063	
100.0	

	Post-test (Q3) 	
Frequency	
Percent	

very confident	
173	
22.8	

confident	
278	
36.6	

somewhat confident	

222	

29.2	

not confident	
86	
11.3	

Total	
759	
100.0	

Q4. The library catalog is used primarily for locating: 
o	courses offered at Southeastern
o	books for sale
o	materials owned by the Sims Library
o	all books published about history
o	periodical articles  

Pre-test (Q4) 	
Frequency	
Percent	

materials owned by the Sims Library	
871	
81.9	

periodical articles	
113	
10.6	

all books published about history	
45	
4.2	

courses offered at Southeastern	
27	
2.5	

books for sale	
7	
.7	

Total	
1063	
100.0	

	Post-test (Q4) 	
Frequency	
Percent	

materials owned by the Sims Library	
642	
84.6	

periodical articles	
51	
6.7	

all books published about history	
47	
6.2	

courses offered at Southeastern	
18	
2.4	

books for sale	
1	
.1	

Total	
759	
100.0	

Q5.  You are writing a persuasive essay on the process to sainthood of Mother Teresa of India.  Which of the following is most appropriate for locating background information for this topic?
o	New Catholic Encyclopedia
o	New York Times
o	www.google.com
o	not sure

Pre-test (Q5)	
Frequency	
Percent	

New Catholic Encyclopedia	
670	
63.0	

www.google.com	
269	
25.3	

not sure	
113	
10.6	

New York Times	
11	
1.0	

Total	
1063	
100.0	

	Post-test (Q5)	
Frequency	
Percent	

New Catholic Encyclopedia	
518	
68.2	

www.google.com	
164	
21.6	

not sure	
66	
8.7	

New York Times	
11	
1.4	

Total	
759	
100.0	

Q6.  Your research topic is: “food and culture in literature.”  Which of the following would be the most manageable focus for this topic? 
o	leave the topic as is
o	broaden to a larger topic: food or culture 
o	narrow to be more specific: food symbols and literature
o	dietary restrictions and religion 
o	not sure

Pre-test (Q6)	
Frequency	
Percent	

narrow to be more specific: food symbols and literature	
465	
43.7	

leave the topic as is	
308	
29.0	

not sure	
112	
10.5	

broaden to a larger topic: food or culture	
107	
10.1	

dietary restrictions and religion	
71	
6.7	

Total	
1063	
100.0	

	Post-test (Q6) 	
Frequency	
Percent	

narrow to be more specific: food symbols and literature	
307	
40.4	

leave the topic as is	
272	
35.8	

broaden to a larger topic: food or culture	
76	
10.0	

not sure	
52	
6.9	

dietary restrictions and religion	
52	
6.9	

Total	
759	
100.0	

Q7.  Your research topic is: “Would raising the minimum wage alleviate child poverty?”  Which combination of keywords do you think would provide the most relevant search results?
o	minimum wage and child poverty
o	poverty and employment
o	children and poverty  
o	not sure

Pre-test (Q7) 	
Frequency	
Percent	

minimum wage and child poverty	
770	
72.4	

poverty and employment	
171	
16.1	

children and poverty	
96	
9.0	

not sure	
26	
2.4	

Total	
1063	
100.0	

	Post-test (Q7) 	
Frequency	
Percent	

minimum wage and child poverty	
568	
74.8	

poverty and employment	
104	
13.7	

children and poverty	
73	
9.6	

not sure	
14	
1.8	

Total	
759	
100.0	

Q8.  You have been given a research topic on the hiring practices of minorities in the workplace.  Which would be the best plan for coming up with a search strategy?
o	use background information from reference works on the topic of hiring practices and minorities in the workplace
o	look at statistical database sources
o	view relevant newspaper articles from local newspapers
o	all of the above
o	not sure 

Pre-test (Q8) 	
Frequency	
Percent	

all of the above	
764	
71.9	

use background information from reference works on the topic of hiring practices and minorities in the workplace	

131	

12.3	

look at statistical database sources	
94	
8.8	

not sure	
59	
5.6	

view relevant newspaper articles from local newspapers	

15	

1.4	

Total	
1063	
100.0	

	Post-test (Q8) 	
Frequency	
Percent	

all of the above	
544	
71.7	

use background information from reference works on the topic of hiring practices and minorities in the workplace	

105	

13.8	

look at statistical database sources	
61	
8.0	

not sure	
34	
4.5	

view relevant newspaper articles from local newspapers	

15	

2.0	

Total	
759	
100.0	

Q9.  Which would be an appropriate keyword to use in a search on the topic of alcoholism?
o	substance abuse
o	drugs 
o	alcohol
o	moderate treatment for alcoholism
o	all of the above 
o	not sure

Pre-test (Q9) 	
Frequency	
Percent	

all of the above	
461	
43.4	

substance abuse	
276	
26.0	

alcohol	
246	
23.1	

moderate treatment	

62	

5.8	

not sure	
16	
1.5	

drugs	
2	
.2	

Total	
1063	
100.0	

	Post-test (Q9) 	
Frequency	
Percent	

all of the above	
376	
49.5	

substance abuse	
170	
22.4	

alcohol	
168	
22.1	

moderate treatment	

36	

4.7	

not sure	
8	
1.1	

drugs	
1	
.1	

Total	
759	
100.0	

Q10.  Which of the following searches in a research database would likely result in the greatest number of search results retrieved?
o	cigarette and smoking
o	cigarette or smoking
o	cigarette not smoking
o	not sure

Pre-test (Q10)	
Frequency	
Percent	

cigarette and smoking	

705	

66.3	

cigarette or smoking	

298	

28.0	

not sure	
39	
3.7	

cigarette not smoking	

21	

2.0	

Total	
1063	
100.0	

	Post-test (Q10) 	
Frequency	
Percent	

cigarette and smoking	

404	

53.2	

cigarette or smoking	

318	

41.9	

not sure	
22	
2.9	

cigarette not smoking	

15	

2.0	

Total	
759	
100.0	

Q11.  In the library catalog, the search terms truck$ and transportation will retrieve results:
o	that contain the words "truck" or "transportation"
o	that contain the words “truck” or “trucks” or “trucking” and “transportation”
o	where the word "transportation" is in the title
o	where the word “truck” is considered more important
o	not sure

Pre-test (11) 	
Frequency	
Percent	

that contain the words "truck" or "trucks" or "trucking" and "transportation"	

520	

48.9	

that contain the words "truck" or "transportation"	

253	

23.8	

not sure	
168	
15.8	

where the word "transportation" is in the title	

65	

6.1	

where the word "truck" is considered more important	

57	

5.4	

Total	
1063	
100.0	

	Post-test (Q11) 	
Frequency	
Percent	

that contain the words "truck" or "trucks" or "trucking" and "transportation"	

427	

56.3	

that contain the words "truck" or "transportation"	

137	

18.1	

not sure	
103	
13.6	

where the word "truck" is considered more important	

51	

6.7	

where the word "transportation" is in the title	

41	

5.4	

Total	
759	
100.0	

Q12.  If your keyword search legalization and marijuana" in the Academic Search Premier research database retrieves 60 search results, how would you choose the best three articles for your topic?
o	look through the first 10 search results, because those are the most relevant
o	look for the articles that are short and to the point
o	first skim the titles of the articles, and then read the summaries of those that seem relevant
o	not sure

Pre-test (Q12) 	
Frequency	
Percent	

first skim the titles of the articles, and then read the summaries of those that seem relevant	

694	

65.3	

look through the first 10 search results, because those are the most relevant	

280	

26.3	

	

 	
look for the articles that are short and to the point	

41	

3.9	

not sure 
Total	
48
1063	
4.5
100.0	

	Post-test (Q12) 	
Frequency	
Percent	

first skim the titles of the articles, and then read the summaries of those that seem relevant	

463	

61.0	

look through the first 10 search results, because those are the most relevant	

214	

28.2	

look for the articles that are short and to the point	

45	

5.9	

not sure	
37	
4.9	

Total	
759	
100.0	

Q13.  In the above search, what keywords would you use in order to narrow your search results?
o	(marijuana or pot) and legalization
o	legalization and marijuana and cancer
o	legalization or marijuana or cancer
o	not sure

Pre-test (Q13)	
Frequency	
Percent	

(marijuana or pot) and legalization	

662	

62.3	

legalization and marijuana and cancer	

239	

22.5	

legalization or marijuana or cancer	

85	

8.0	

not sure	
77	
7.2	

Total	
1063	
100.0	

	Post-test (Q13) 	
Frequency	
Percent	

(marijuana or pot) and legalization	
421	
55.5	

legalization and marijuana and cancer	

233	

30.7	

legalization or marijuana or cancer	

79	

10.4	

not sure	
26	
3.4	

Total	
759	
100.0	

Q14.  Keeling, Richard P.  "Binge Drinking and the College Environment."  Journal of American College Health, 50(5): 197.
The above citation refers to:
o	a chapter from a book
o	an excerpt from an encyclopedia
o	an article from a journal
o	not sure

Pre-test (Q14) 	
Frequency	
Percent	

an article from a journal	
926	
87.1	

not sure	
53	
5.0	

a chapter from a book	
48	
4.5	

an excerpt from an encyclopedia	
36	
3.4	

Total	
1063	
100.0	

	Post-test (Q14) 	
Frequency	
Percent	

an article from a journal	
672	
88.5	

not sure	
33	
4.3	

	

 	
a chapter from a book	
22	
2.9	

an excerpt from an encyclopedia
Total 	
32
759	
4.2
100.0	

Q15.  When doing research for papers, speeches, and projects, it necessary to properly acknowledge authors whose work has been used in your end product.  There are many documentation styles used by a variety of academic disciplines.  Which of the following are standard style guides?
o	MLA Handbook of Writers of Research Papers
o	Publications Manual of the American Psychological Association
o	Chicago Manual of Style
o	Turabian Style
o	all of the above  

Pre-test (Q15) 	
Frequency	
Percent	

MLA Handbook of Writers of Research Papers	
740	
69.6	

all of the above	
282	
26.5	

Chicago Manual of Style	
18	
1.7	

Publications Manual of the American Psychological Association	
20	
1.9	

Turabian Style	
3	
.3	

Total	
1063	
100.0	

	Post-test (Q15) 	
Frequency	
Percent	

MLA Handbook of Writers of Research Papers	
519	
68.4	

all of the above	
207	
27.3	

Chicago Manual of Style	
17	
2.2	

Publications Manual of the American Psychological Association	
12	
1.6	

Turabian Style	
4	
.5	

Total	
759	
100.0	

Q16.  An important part of the publication process for articles in scholarly publications is the peer review or refereed process. 
o	True
o	False

Pre-test (Q16)	
Frequency	
Percent	

TRUE	
910	
85.6	

FALSE	
153	
14.4	

Total	
1063	
100.0	

	Post-test (Q16)	
Frequency	
Percent	

TRUE	
653	
86.0	

FALSE	
106	
14.0	

Total	
759	
100.0	

Q17.  Internet Web sites always contain the most up to date and accurate information.
o	True
o	False

Pre-test (Q17)	
Frequency	
Percent	

FALSE	
882	
83.0	

TRUE	
181	
17.0	

Total	
1063	
100.0	

	Post-test (Q17)	
Frequency	
Percent	

FALSE	
633	
83.4	

TRUE	
126	
16.6	

Total	
759	
100.0	

Q18.  Using synonyms is a useful method to help you find the appropriate search terms for an index / database.
o	True
o	False

Pre-test (Q18)	
Frequency	
Percent	

TRUE	
849	
79.9	

FALSE	
214	
20.1	

Total	
1063	
100.0	

	Post-test (Q18)	
Frequency	
Percent	

TRUE	
645	
85.0	

FALSE	
114	
15.0	

Total	
759	
100.0	

Q19.  The purpose of a bibliography is to provide complete, concise information on the origin of sources used in a research paper. 
o	True
o	False

Pre-test (Q19)	
Frequency	
Percent	

TRUE	
988	
92.9	

FALSE	
75	
7.1	

Total	
1063	
100.0	

	Post-test (Q19) 	
Frequency	
Percent	

TRUE	
698	
92.0	

FALSE	
61	
8.0	

Total	
759	
100.0	

Q20.  Copying and pasting text from a print or electronic source (e.g., Internet site) into your document without proper citation is an example of plagiarism.
o	True
o	False

Pre-test (Q20)	
Frequency	
Percent	

TRUE	
1031	
97.0	

FALSE	
32	
3.0	

Total	
1063	
100.0	

	Post-test (Q20) 	
Frequency	
Percent	

TRUE	
731	
96.3	

FALSE	
28	
3.7	

Total	
759	
100.0	

Angela Dunnington is an Associate Professor and Library Science 102 Coordinator at Southeastern Louisiana University in Hammond, Louisiana.  Angela has been serving as coordinator for the library’s one-hour credit course, Library Science (LS) 102: Introduction to Information Research for seven years.  After receiving her MLIS from Louisiana State University, Angela has held Reference/Instruction Librarian positions at Louisiana State University in Shreveport and the University of Texas at San Antonio prior to joining Southeastern Louisiana University’s faculty in January 2002. She may be contacted at adunnington@selu.edu.
 
Mary Lou Strong is an Assistant Professor and Reference/Instruction Librarian at Southeastern Louisiana University in Hammond, Louisiana. In addition to her reference duties, she teaches multiple sections of the library’s one-hour credit course, Library Science (LS) 102: Introduction to Information Research.  She received a BA in English from Louisiana State University, a JD from Louisiana State University Paul M. Hebert Law Center, and her MLIS from the University of Oklahoma.  She may be contacted at mary.strong@selu.edu.