“Opportunities in Disguise”: The Continuing Evolution of an Authentic 
Information Literacy Assessment
Janet Pinkley and Debra Hoffmann, California State University, Channel Islands

Abstract
This article outlines the continuing evolution of the authentic assessment process for information literacy instruction at the John Spoor Broome Library at California State University, Channel Islands (CI), with particular focus on the library’s 2013 assessment project.  The goal of these continually evolving processes is to assess the library’s value in ways that allow librarians to easily translate assessment findings to the campus community and to have actionable results that improve the library’s service to students.   Librarians at CI continue to adjust and improve their authentic assessment process with the following goals in mind:  to utilize information gleaned from assessment efforts to create opportunities that positively impact and support learning through targeted, assignment-specific library instruction;  to clearly define the role of the library and librarians in the student learning process; to translate and articulate assessment findings in meaningful ways to the campus community; and to have actionable results that improve the library’s service to students.   
 Keywords: assessment, information literacy, continuous improvement, library instruction, collaboration, academic libraries

Introduction
This article outlines the continuing evolution of the authentic assessment process for information literacy instruction at the John Spoor Broome Library at California State University, Channel Islands (CI), with particular focus on the library’s 2013 assessment project.   As the youngest campus in the California State University system, Broome Library benefits from CI’s youth in that there is no standard “way it’s always been done” in terms of assessment, allowing librarians the freedom to explore a variety of assessment options.  The library makes an effort to avoid getting caught up in counting tick marks and tallies just for the sake of counting, and caution is used when identifying connections within assessment data in order to avoid the assumption of causation for correlative relationships.  Librarians at Broome Library strive to create strategic, purposeful, and translatable assessment practices, with student learning at the center of these efforts.  The goal of these continually evolving processes is to assess the library’s value in ways that allow librarians to easily translate findings to the campus community and to have actionable results that improve service to students.   This article will be of practical use to librarians involved in assessment and continuous improvement efforts at their institutions, and will add to the literature on collaboration, assessment, and continuous improvement initiatives in academic libraries.
Background
	In 2005, the California State University (CSU) system partnered with Educational Testing Services (ETS) to pilot the ICT Literacy Assessment.  According to ETS, “The ETS ICT Literacy Assessment is designed to measure proficiency in digital information management necessary for a modern academic and economic environment”(Educational Testing Services, 2014, 1). The pilot assessment was a four-hour test with a sample size that was pre-determined based on the size of the campus student population.  Upon completion of the pilot, CI librarians gleaned no usable data regarding student information competency levels or how the library impacted student learning, information we were hoping to have learned from the assessment.  As a result, librarians began a process of critical reflection to take stock of library and institutional values and how these values could be assessed in a way meaningful way (Hoffmann & LaBonte, 2012).
	In 2006, two librarians applied for and received a three-year grant from the CSU to partner with faculty from CI’s Composition program to assess information literacy among their students.  Composition was a natural fit for this project as Composition’s freshman courses were already targeted by the library for its information literacy (IL) program.   Additionally, the Composition program already had a robust portfolio system for student evaluations that were collaboratively assessed by Composition faculty using a rubric based on their course outcomes.   As part of the grant, the library developed a rubric to assess four IL outcomes utilizing student work already submitted to compositions portfolio system. Through a comprehensive process that included examining existing rubrics from other institutions, grant participants created a rubric that aligned with the three campus General Education outcomes for IL, which resulted in four evaluation criteria (See Appendix A).  Each criteria would be evaluated and ranked at one of three proficiency levels: emerging, proficient, and advanced (Hoffmann & LaBonte, 2012). With the hope of furthering the library’s authentic assessment efforts, partnering in the grant helped inform the library about the information competence of incoming and outgoing students, allowed librarians to reflect on the impact of its IL program, and helped identify weaknesses in order to improve (Hoffmann & Wallace, 2008).     

Literature Review
As a profession, academic librarianship has long sought to find concrete ways to identify and articulate its value within institutions of higher education.  One of our primary professional organizations, the Association of College and Research Libraries (ACRL), “promulgates standards and guidelines to help libraries, academic institutions, and accrediting agencies understand the components of an excellent library” (2016, para. 1). While ACRL does offer a variety of standards related to value, the organization does not attempt to tie the components of an excellent library back to student learning.   Additionally, although the guidelines present in ACRL’s standards are foundational pieces for many libraries, the academic library community still lacks consensus on best practices for assessment, evaluating library instruction, training future instruction librarians, or measuring the impact of library services such as information literacy instruction on student learning (Sobel & Sugimoto, 2012).  
This lack of consensus is further reflected in the recent rescinding of the Information Literacy Competency Standards for Higher Education by the ACRL Board of Directors (2016), at the 2016 American Library Association in Orlando, which has sparked many questions and debates within the academic library community.  Discussions related to the rescission of the Standards that have appeared on professional listservs and blogs, such as the Information Literacy Instruction Discussion and Inside Higher Ed’s Library Babel Fish, reflect the contention and ongoing debate related to ACRL’s decision.  Since the development of the ACRL Information Literacy Standards in 2000, many libraries have relied on them as a strategic component of their information literacy efforts.  Librarians have utilized the Standards to effectively participate in their campus assessment efforts and to help demonstrate and communicate the value of information literacy in relation to institutional outcomes. 
Academic libraries quest to communicate their value within institutions of higher learning predate the 2000 Information Literacy Standards.  In her article, Outcomes Assessment in Academic Libraries, Wallace (2001) recognizes the evolution of assessment measures in academic libraries which have shifted over decades, moving away from quality towards outcomes and impacts.  Wallace tracks the history of early library assessment and notes that while the 1970’s focused on inputs, and the 1980’s emphasized output and performance measures, the 1990’s literature reflects clear consensus that libraries needed to demonstrate positive effects on institutional outcomes.  These shifts from quality to outcome based measures within academic libraries align with developments in accreditation processes.  Sims’ 1992 book, Student Outcomes Assessment: A Historical Review and Guide to Program Development, documents the development of student outcomes assessment and recognizes large shifts in accreditation as well.  Sims notes that the accreditation process itself dates back to the early 1900’s with its inception based firmly in defining a college (Wallace, 2001).  As this process evolved and grew, little attention focused the educational learning outcomes, though a national study conducted by the Council on Postsecondary Accreditation (COPA) and completed in 1978 revealed that educators were strongly in favor of a shift in accreditation to more emphasis on the assessment of educational outcomes (Wallace, 2001).  As a result of that study, the six accreditation agencies took a more critical look at their criteria and began incorporating the inclusion of outcomes assessment as part of institutional effectiveness standards (Wallace, 2001). 
In her 1998 article, Defining and Measuring the Library’s Impact on Campuswide Outcomes, Lindlauer asserts that much of the library literature related to accreditation up to that point either focused on measures of effectiveness and quality to inform performance value or relied on various data gathering methods and measures to inform performance value.  Lindlauer (1998) also identifies two primary challenges librarians encounter in articulating the impact of services as it relates to institutional outcomes:   libraries often do not consider the larger campus context in identifying measures and evidence; and second, libraries often fail to align the evidence derived from assessment with language and ideas used in campus-wide planning.   
In 2007, Sanders examined the accreditation standards of all six regional accrediting organizations for their use of information literacy as part of their standards.  Sanders (2007) found that three of the organizations referred specifically to information literacy, while the other three alluded to library instruction despite not mentioning information literacy specifically.  Further, in another study, Sanders (2008) found that although the ACRL standards have been influential, librarians may have found more buy-in from campus faculty and administrators if they leveraged the attention given to information literacy by accreditation organizations, as information literacy has a role in critical thinking and is therefore given significant recognition by accreditation organizations.  This should be viewed as an opportunity for librarians, not only in terms of faculty collaboration, but also in the broader context of accreditation and participation in the campus community. More recently, Sobel and Sugimoto (2012) found that libraries are increasingly integrating information literacy into higher education standards.  As a result, the profession has seen accreditation standards and assessment data being leveraged to ensure that library outcomes for information literacy are aligned with institutional goals for student learning (Becher, 2013).
Due to information literacy’s growing role in accreditation and assessment at the institutional level, libraries have worked to find an effective method for IL assessment at the library and the IL session level.  There have been some attempts to address the need for standardized tools to assess IL in higher education, including Standardized Assessment of Information Literacy Skills (SAILS), iSkills, and Research Readiness Self-Assessment (RRSA), to name a few.  Part of the challenge of widespread acceptance of such tools is that there have been very few multi-institutional studies related to their effectiveness.  One larger-scale study that did focus on the use of SAILS was performed by librarians at Hunter College.  The study examined multiple institutions and discovered that although SAILS can be a valuable assessment tool in many respects, there were limitations present.  Ultimately, SAILS was found to be most effective when there was institutional support for a variety of factors, including data analysis, recruiting random samples that accurately reflect student population, utilizing results to impact information literacy planning, and the ability to correlate data with institutional data (Lym et al., 2010).  
Only a handful of tools attempt to serve as a standardized measure for information literacy assessment, with varied success, which means that many institutions are using their own methods to assess institutional information literacy goals.   Most recently, Graf and Harris (2016) took information literacy assessment one step further by not only attempting to gauge student learning, but also by seeking ways to incorporate critical consciousness and self-reflection into assessment.   The authors conclude that these significant components help librarians “more carefully evaluate what it is we profess to value in the first place” (Graf & Harris, 2016, p. 45). Academic library assessment has shifted over the years but it is clear that librarians continue to seek ways to fulfill their institutions’ need to assess the information literacy of students and the role the library plays in teaching information literacy skills.
Methods
In 2013 librarians at Broome Library undertook a large-scale assessment project focused on assessing student research products. With campus WASC accreditation and Library program review on the horizon, librarians revisited previous information literacy assessment efforts as part of the library’s continuous improvement process.  Similar to their work with the CSU grant project in 2006, instruction librarians created and applied a rubric to 114 samples of student work, a mix of research papers and research posters (See Appendix B). Each sample was scored by two librarians.  The scoring rubric was a four-criteria/4-point likert scale rubric based on CI’s two General Education outcomes for information literacy.  Each student paper or poster was rated on a 4-point Likert scale by two librarians and averaged. Scale scores reflect: 1 = Initial; 2=Emerging; 3=Developing; 4= Highly Developed.
Although the assessment process aligned with the one developed by Broome Library in 2006 (Hoffmann & LaBonte, 2013), upon reflection, librarians felt modifications were needed in order to yield the most revelatory data.  The first modification was that the likert proficiency categories were expanded from the existing three (emerging, proficient, and advanced) to four: initial, emerging, developing, and highly developed.   This is because the samples ranged from freshman papers to senior capstone papers and posters. Additionally, in the 2006 grant assessment project, librarians and faculty had worked collaboratively to adapt and adjust assignments to provide more insight to the evaluators, while our 2013 assessment focused entirely on existing, faculty created assignments.  The 2013 effort also aimed to expand the levels and courses it examined.  Papers and research posters (where applicable) were assessed from the Fall 2012 semester for three freshman composition courses (COMP 105), two upper division, interdisciplinary,  general education courses, and three capstone senior research courses.  The final difference from the 2006 assessment project was that before ranking any products, librarians participated in a normalizing process.   This norming process was accomplished through two, three-hour norming sessions, where librarians independently scored the same set of student sample papers, chosen at random.   After scoring the papers, librarians reviewed their scores together, identifying any score discrepancies.   Discussion followed in an effort to reconcile inconsistent scores and to provide additional context for scoring.  The norming process was designed to insure that librarians understood the rubric that they’d be using and to build consensus among those rating the student work.  At the end of these two norming sessions, librarians were confident in their ability to apply the rubric to student work and produce consistent scoring.    
Results
In total, librarians evaluated and assessed 114 papers and 18 posters, with each piece of student work being evaluated by two librarians.  The breakdown of the items was 75 papers from three COMP 105 courses, 26 papers from two upper-division, interdisciplinary general education (UDIGE) courses, 13 papers and 18 posters from three capstone courses.
	Scores for student papers in freshman Composition 105 courses ranged from 1.03 – 1.49, which is firmly in the “Initial” range. Scores for papers from UDIGE courses ranged from 1.03 – 1.96, again ranging from “Initial” to nearly “Emerging.”  For both Composition and UDIGE papers, the most commonly occurring scores from raters were 1’s and 2’s.  Scores for senior Capstone work (which included both the papers and the research posters) ranged from 1.87 – 3.05, indicating a range of scores from “Initial” to “Developing.”  Here, the most common score from raters was 2 (Emerging), followed by 1 (Initial).  Scores of 3’s and 4’s were given less often by raters.  (For a breakdown of scores by outcome and course, please see the Appendix C)
Discussion and Conclusion
We discovered early in this process that viewing the assignment details prior to assessing the product was critical in determining what the students were tasked with and if there was opportunity for students to demonstrate information literacy skills in the assignment or course learning outcomes. In fact, products from one course were eliminated from the original sample because the assignment contained no research component. We quickly discovered a “gap” in their assessment process-- while the library and the university value information literacy as a student learning outcome (as evidenced by the university’s three IL outcomes), IL is often not a course-level learning outcome and thus, nearly impossible to assess by examining student products alone.  Happily, we found this “gap” to be an opportunity to reach out to faculty regarding course learning outcomes and information literacy.   Since 2015, instruction librarians have hosted three half-day workshops for faculty entitled, “Sustainable Information Literacy: Facilitating the Information Literate Classroom.” These sessions are opportunities for faculty to work collaboratively with librarians to incorporate high impact information literacy practices into their curriculum and a way for the library to positively impact student learning. 
Additionally, the “random” nature of the assessment project’s course/product selection lacked context, which created challenges as we worked to make meaning out of the data.   For example, assessment data reflected that out of the three senior capstone courses assessed, products from the course that did not request an information literacy session from the library were the ones that received the highest assessment scores from raters. This appears to suggest that students in the course without a library session had more highly developed information literacy skills than students who had a library session.   
Upon deeper examination, we realized that this discrepancy was due to a lack of “context” available to us in the assessment process, and did not accurately reflect students’ previous exposure to library sessions through existing scaffold approaches. Broome Library has a robust IL program for students that targets certain disciplines and “gateway” courses (freshman composition, UDIGE courses, etc.). In looking at the data more closely, we discovered that the Sociology capstone course is the one course that did not request a library session, but assessed at a high score.  The library typically teaches IL sessions to Sociology students at numerous places in their academic careers at CI, so it is probable that students in this course had numerous IL sessions prior to their senior capstone course. By contrast, Political Science requested an IL session for their senior students, but librarians actually teach very few IL sessions for Political Science students prior to their senior capstone.   So although the data is important, without context it is not useful in and of itself.  Assessment librarians recognized an opportunity going forward to reach out to all librarians who teach for these targeted disciplines to be included earlier in the assessment process to participate in course selection for assessment and to provide additional context during the assessment process. 
We also found it challenging to accurately rate one aspect of CI’s student learning outcome for information literacy (3.1 a), the information literate student accesses needed information effectively and efficiently. The evaluation for this outcome has three primary components: search strategy, timeline, and variety/sufficiency of sources.  While the end product does allow a clear depiction of variety/sufficiency of sources, it gives little insight into the strategy developed and employed by students, nor the timeline used to implement the strategy.  Our goal is to be able to assess the product and the process, but without a reflective component in addition to the end product, we are unable to truly assess the process.  There are a variety of methods that can be used to capture student understanding and mastery of the research process, but the most authentic would be the addition of a research journal/log that the student completes as they work on their project, or the addition of a reflective component to the research assignment, like the one developed by librarians for use by Composition faculty for their students during the earlier grant project (see Appendix D).  Components such as these would provide opportunities for assessment librarians to gain insight into the search strategies used by students, the variety of methods employed, the number of sources consulted, their access points for information, and even gauge a student’s understanding of the research process as a whole, while also assessing the end result to determine utility and integration.  Although a simple solution on the surface, it will take a collaborative approach with faculty to incorporate these methods into student research projects, which will likely take time for buy-in, development, and implementation.      
Librarians at CI continue to adjust and improve their authentic assessment process with the following goals in mind:  to utilize information gleaned from assessment efforts to create opportunities that positively impact and support learning through targeted, assignment-specific library instruction;  to clearly define the role of the library and librarians in the student learning process; to translate and articulate assessment findings in meaningful ways to the campus community; and to have actionable results that improve the library’s service to students.   
Acknowledgements
California State University, Channel Islands Instruction Librarians who participated in the assessment:  Linda Carroll, Matthew Cook, Monica Pereira, Elnora Tayag, and Laura Worden.

References
Association of College and Research Libraries. (2016). Guidelines, standards, and frameworks.    Retrieved from http://www.ala.org/acrl/standards
Becher, M. (2013). Instruction coordinators and higher education accreditation: A study of awareness and assessment documentation use.  The Journal of Academic Librarianship, 39, 573-581.
Board of Directors, Association of College & Research Libraries. (2016, June 25). ACRL board takes action on information literacy standards. [Web log post]. Retrieved from http://www.acrl.ala.org/acrlinsider/archives/12126
Educational Assessment Services. (2014) ICT literacy assessment: An issue paper from ETS.  Retrieved from http://www.ets.org/Media/Tests/Information_and_Communication_Technology_Literacy/0202heapaper.pdf
Graf, A. J. & Harris, B. R. (2016).  Reflective assessment: Opportunities and challenges.  Reference Services Review, 44(1) 38-47.
Hoffmann, D. & LaBonte, K. (2012). Meeting information literacy outcomes: partnering with faculty to create effective information literacy assessment.   Journal of Information Literacy, 6(2), 70-85.
Hoffmann, D. & Wallace, A. (2008).  Information competence assessment using first year and upper division writing samples.   In S. Hiller, K. Justh, M. Kyrilldou, & J. Self (Eds.), Proceedings of the 2008 Library Assessment Conference: Building Effective, Sustainable, Practical Assessment, Seattle, WA (pp. 473-484). Washington D.C.: Association of Research Libraries. Retrieved from http://libraryassessment.org/bm~doc/proceedings-lac-2008.pdf
Lindlauer, B. G. (1998).  Defining and measuring the library’s impact on campuswide outcomes.   College & Research Libraries 59(6), 546-570.
Lym, B., Grossman, H., Yannotta, L., & Talih, M. (2010). Assessing the assessment: How institutions administered, interpreted, and used SAILS.  Reference Services Review 38(1), 168-186.   
Sanders, L. (2007).   Regional accreditation organizations’ treatment of information literacy: Definitions, collaboration, and assessment.  The Journal of Academic Librarianship 33(3), 317-326.
Sanders, L. (2008).  Perspectives on accreditation and information literacy as reflected in the literacture of library and information science.  The Journal of Academic Librarianship 34(4), 305-313.
Sobel, K. & Sugimoto, C. R. (2012). Assessment of learning during library instruction: Practices, prevalence, and preparation.  The Journal of Academic Librarianship, 38(4), 191-204.
Wallace, V. E. (2001). Outcomes assessment in academic libraries: Library literature in the 1990’s.  College & Undergraduate Libraries, 8(2), 59 – 71.

Appendix A: Original Rubric
Information Literacy Assessment Product
SCORE SHEET

Reviewer:			

Product Type:

Annotated Bibliography:		Yes	No	Type:

Original Assignment Information	

Semester/Year:   		Professor:   			COMP     UDIGE    OTHER:

1. 	The Information literate student applies new and prior information to the planning and creation of a particular product or performance (Grant Outcome).

X	Rater cannot determine.

1	Emerging.  Student identifies insufficient and/or inadequate information sources.  Student identifies insufficient or inadequate concepts and terms that lead to limited information.  Student develops ineffective research questions lacking focus and clarity.  

2	Proficient.  Student identifies sufficient and somewhat varied information sources.  Student identifies some concepts and terms that lead to somewhat appropriate information.  Student develops research questions with limited focus and clarity.  

3	Advanced.  Student identifies extensive and varied information sources in numerous formats.  Student clearly identifies key concepts and terms that lead to the appropriate information.  Student develops research questions that are focused, clear, and complete.  

2. 	The information literate student accesses needed information effectively and efficiently (CSUCI GE Outcome).

X	Rater cannot determine.

1	Emerging.  Student creates an ineffective search strategy using limited and/or inappropriate research methods.  Student develops an unrealistic or inadequate timeline for implementation of the search strategy.  Student gathers insufficient and/or inappropriate sources of limited variety.

2	Proficient.  Student creates a search strategy using somewhat varied and appropriate research methods.  Student develops a realistic timeline for implementation of the search strategy.  Student gathers sufficient and somewhat varied sources.

3	Advanced.  Student creates a thorough search strategy using a variety of appropriate research methods.  Student develops a flexible timeline that allows for implementation and revision of the search strategy.  Student gathers numerous and varied sources in multiple formats.

3.  	The Information literate student evaluates information and its sources critically (CSUCI GE Outcome).

X	Rater cannot determine.

1	Emerging.  Student fails to or is unaware of how to evaluate sources for relevance, accuracy and credibility.

2	Proficient.  Student evaluates sources for relevance, accuracy and credibility.

3	Advanced.  Student uses critical thinking to evaluate sources for relevance, accuracy and credibility to establish his or her own authority.

4. 	The information literate student compares new knowledge with prior knowledge to determine the value added, contradictions, and other unique characteristics of information (Grant Outcome).

X	Rater cannot determine.

1	Emerging.  Student unsucessfully or inadequately compares new information to prior information.

2	Proficient.  Student sufficiently compares new information to prior information.

3	Advanced.  Student uses critical thinking to compare new information to prior information and create a heightened understanding of the research.

5. 	The information literate student synthesizes main ideas to construct new concepts (Grant Outcome).

X	Rater cannot determine.

1	Emerging.  Student poorly summarizes and fails to synthesize the main ideas from the information gathered to develop his or her own interpretation.

2	Proficient.  Student proficiently summarizes and struggles to synthesize the main ideas from the information gathered to develop his or her own interpretation.

3	Advanced.  Student clearly summarizes and synthesizes the main ideas from the information gathered to develop his or her own interpretation.

6. 	The information literate student acknowledges the use of information sources in communicating the product or performance (Grant Outcome).

X	Rater cannot determine.

1	Emerging.  Student exhibits little control over information and fails to integrate it into his or her research.  Student presents the information in a format inappropriate to the purpose and audience of the assignment.

2	Proficient.  Student exhibits proficient control over information with competent integration into his or her research.  Student presents the information in a format somewhat appropriate to the purpose and audience of the assignment.

3	Advanced.  Student expertly controls information and integrates it seamlessly to advance his or her research.  Student presents the information in a format highly appropriate to the purpose and audience of the assignment.

7. 	The information literate student explains the economic, legal, social, and ethical issues surrounding the use of information (CSUCI GE Outcome).

X	Rater cannot determine.

1	Emerging.  Student demonstrates little or no understanding of intellectual property and fair use of copyrighted materials.

2	Proficient.  Student demonstrates a working understanding of intellectual property and fair use of copyrighted materials.

3	Advanced.  Student demonstrates a comprehensive understanding of intellectual property and fair use of copyrighted materials.

Hours of Information Literacy Instruction:

Describe Type of Information Literacy Instruction:

Reviewer Comments:
Appendix B: Revised 2013 Rubric

GE Goal 3:  Graduate information literate students
3.1 Students will be able to find, critically evaluate, and make use of appropriate and relevant information.  
	Initial 1	Emerging 2	Developing 3	Highly Developed 4	
The information literate student accesses needed information effectively and efficiently	Student creates an ineffective search strategy using limited and/or inappropriate research methods.  Student develops an unrealistic or inadequate timeline for implementation of the search strategy.  Student gathers only one or two sources, non-relevant, or inappropriate sources..	Student creates a search strategy using somewhat varied and appropriate research methods.  Student develops a realistic timeline for implementation of the search strategy.  Student gathers more than one or two sources which are somewhat varied. 	Student creates a search strategy using somewhat varied and appropriate research methods.  Student develops a realistic timeline for implementation of the search strategy.  Student gathers sufficient and somewhat varied sources.	Student creates a thorough search strategy using a variety of appropriate research methods.  Student develops a flexible timeline that allows for implementation and revision of the search strategy.  Student gathers numerous and varied sources in multiple formats.  Student is satisfied with the sources found or, went back, or knows to go back to search process. 	
The Information literate student evaluates information and its sources critically.	Student fails to or is unaware of how to evaluate sources for relevance, accuracy and credibility.  Student is unable to develop or apply provided frameworks to evaluate information.	Student evaluates some sources for some combination of bias, relevance, accuracy, and credibility, or student applies a provided framework to evaluate individual sources information. 	Student evaluates all sources for bias, relevance, accuracy and credibility.  Student begins to evaluate sources.	Student uses critical thinking to evaluate sources for relevance, accuracy and credibility to establish his or her own authority.  Student evaluates across sources, and reflects on strengths and weaknesses of the sources as a whole.	
The Information literate student make use of appropriate and relevant information.	Task called for research and only opinion is represented in the final project.  Student does not integrate sources from a variety of perspectives and disciplines.  Student demonstrates little or no understanding of intellectual property and fair use of copyrighted materials. 	Student integrates secondary sources from a variety of perspectives or disciplines.  Student demonstrates a working understanding of intellectual property and fair use of copyrighted materials.  	Student integrates secondary sources from a variety of perspectives or disciplines, and begins to use them to support their own perspective and conclusions.  Student demonstrates a working understanding of intellectual property and fair use of copyrighted materials. research. 	Student integrates secondary sources from a variety of perspectives and disciplines, and seamlessly incorporates primary source material, their own study, data gathered research or unprocessed data.  Student demonstrates a comprehensive understanding of intellectual property and fair use of copyrighted materials. 	
3.2  Students will be able to consider the social, political, legal, and other landscapes to pose new questions, integrate information, and contribute to information dilemmas (big data, privacy, intellectual property, tracking, identity and personal data, information surrogacy) in our society.  	
	Initial 1	Emerging 2	Developing 3	Highly Developed 4	

[Intentionally left blank]	Student is able to articulate an information dilemma from media reports, readings, or other information networks.	Student gathers appropriate secondary sources from a variety of perspectives or disciplines to consider the multiple facets of the information dilemma.  More than just a two sided pro/con approach.	Student gathers appropriate secondary sources from a variety of perspectives or disciplines to consider the multiple facets of the information dilemma.  More than just a two sided pro/con approach.  Adds in own ideas to the dialogue.	Student gathers appropriate secondary sources from a variety of perspectives or disciplines to consider the multiple facets of the information dilemma.  More than just a two sided pro/con approach.  Adds in own ideas. Poses solutions or considerations that move the dialogue.	

Reviewer:

Product Type:     Paper	  Poster

Original Assignment Information and/or SLOs:

Semester/Year:			Professor:				COMP   UDIGE   CAPSTONE: 

Hours of Library Instruction: 		Describe type of IL instruction (including any IL instruction by professor):

What we may want to know: 

Do the library’s information literacy instruction sessions impact the achievement of the CI GE Goals for information literacy (3.1- 3.2)?  
What other factors are helping or hindering achievement?
 
Appendix C: Results

ENGL 105 A
	High	Low	Mean	Mode	IL	
Goal 3.1 a	2	1	1.3	1	Yes	
Goal 3.1 b	2	1	1.26	1	Yes	
Goal 3.1 c	2	1	1.21	1	Yes	
Goal 3.2	2	1	1.09	1	Yes	

ENGL 105 B
	High	Low	Mean	Mode	IL	
Goal 3.1 a	2	1	1.49	1	Yes	
Goal 3.1 b	2	1	1.38	1	Yes	
Goal 3.1 c	2	1	1.31	1	Yes	
Goal 3.2	2	1	1.04	1	Yes	

ENGL 105 C
	High	Low	Mean	Mode	IL	
Goal 3.1 a	2	1	1.45	1	Yes	
Goal 3.1 b	2	0	1.09	1	Yes	
Goal 3.1 c	2	0	1.03	1	Yes	
Goal 3.2	2	1	1.03	1	Yes	

UDIGE
ART/ENGL 432
	High	Low	Mean	Mode	IL	
Goal 3.1 a	3	1	1.86	2	No	
Goal 3.1 b	3	1	1.67	2	No	
Goal 3.1 c	4	1	1.62	1	No	
Goal 3.2	4	1	1.52	1	No	

UDIGE
UNIV 349
	High	Low	Mean	Mode	IL	
Goal 3.1 a	3	1	1.97	2	Yes	
Goal 3.1 b	4	1	1.66	2	Yes	
Goal 3.1 c	4	1	1.66	1	Yes	
Goal 3.2	4	1	1.59	1	Yes	

Capstone
ENGL 499
	High	Low	Mean	Mode	IL	
Goal 3.1 a	4	2	3	4	Yes	
Goal 3.1 b	4	1	2.57	3	Yes	
Goal 3.1 c	4	2	3	4	Yes	
Goal 3.2	4	2	2.57	2	Yes	

Capstone
POLS 499
	High	Low	Mean	Mode	IL	
Goal 3.1 a	4	1	2.12	4	Yes	
Goal 3.1 b	4	1	1.85	3	Yes	
Goal 3.1 c	4	1	1.88	4	Yes	
Goal 3.2	3	1	3.06	2	Yes	

Capstone
SOC 499
	High	Low	Mean	Mode	IL	
Goal 3.1 a	3	2	2.41	2	No	
Goal 3.1 b	3	2	2.53	3	No	
Goal 3.1 c	3	1	2	2	No	
Goal 3.2	3	1	2.18	2	No	

Appendix D: Reflective Annotated Bibliography Assignment

Reflective Annotated Research Bibliographies

Instructors: you may want to begin the search process by asking your students to reflect upon the following:
•	What is my research question?  (OR:  “have I developed a research question based on my assignment?”)
•	What would a GOOD resource for this assignment/research question look like?  Where would I look for good resources for my question or assignment?
•	Who is my audience? (Instructor? Peer?)  What type of resources will be accepted by my audience?

Students: create an annotation (length of annotation to be determined by instructor) including the following components:
•	Citation of the work, using either MLA, APA style (ie: author, title of the work, date of publication, publisher, page numbers, etc.)
•	Main focus or purpose of the work—what is this resource that I’ve found? (ie: is it a journal article, web site, press release, etc.)  What is the scope or purpose of the work? What makes it a GOOD resource for my assignment/research question?
Students: you may want to consider the following questions when creating your annotations:
o	Does currency matter when choosing resources for my topic?
o	Who is the intended audience for this resource? Is it discipline –specific or written for a general audience?
o	What is its usefulness or relevance of the resource to my research topic?

o	Are there economic, social or ethical considerations related to this resource- ie: is it from a subscription database or free on the Internet?  Is the author of the resource an authority on the topic? Can I detect author bias in the resource?
Students, once you’ve created an annotation, assess the annotation in light of your research question:  
•	How will I use (or not use) this resource to address my research question or assignment?
•	In light of this first resource, what will my NEXT resource look like?  Do I need to refine/adjust my research question?  Do I need to locate a different type of resource to address my research question or assignment? 

Students, repeat the above process of creating annotations for your additional resources, continuing to reflect upon the research process as you continue to develop or refine your research question.  
Sample Annotation**

Finneran, K. (2001). What's food got to do with it? Issues in Science and Technology 17, 24-25. 

In this editorial, Finneran questions why many people on both sides in the debate over the safety of genetically engineered food base their arguments on speculation, rumor, and emotion rather than scientific research. He references an article by Harvard biologist Richard Lewontin. Lewontin discusses an anti-genetic engineering physicist whose arguments are based on Hindu scripture instead of lab results and pro-genetic engineering scientists who advertise "Golden Rice" (a genetically engineered variety of rice rich in beta carotene) as a benefit for victims of malnutrition who lack vitamin A, even though many people suffering from malnutrition are too weak to properly metabolize the beta carotene into vitamin A. 

Kevin Finneran is editor-in-chief of Issues in Science and Technology, a policy journal sponsored by the National Academy of Sciences, the National Academy of Engineering, and the University of Texas at Dallas, making him someone qualified to write about forming opinions on scientific matters.

This editorial serves as a cautionary reminder that sensible decisions on important issues must be grounded in fact and not influenced by vague fears, unrelated beliefs, unwarranted enthusiasm, or knee-jerk emotional reactions.

Students, consider the following questions as you reflect upon the research process:
•	Am I satisfied with the amount and quality of my resources?  Do I have enough variety and breadth of resources to successfully complete my research question or assignment?
•	Was the search process I used adequate to complete my assignment? What could I have done differently to yield more useful/relevant resources or to make the search process more efficient? 

**Annotation example from Florida Gulf Coast University Library Services http://library.fgcu.edu/RSD/Instruction/handouts/writing%20an%20annotated%20bibliography.pdf