research_process

The project you undertake must be well connected to eLearning delivery, eLearning assessment (or evaluation) or both and must clearly show connections to major themes in this course. – Brian Newberry

 


Using TRAILS as a Library Skills Activity and Assessment Tool

Executive Summary

The Cerritos College library instruction program has several facets:

  • Individual interaction with a student/researcher either in person or virtually via chat, email, or phone.
  • One-shot library orientation sessions for a specific class either in the library computer lab or in the classroom.
  • Library Skills Workshops offered thought the Success Center.
  • For-credit library skills courses.
  • Online research guides.

Over the years the librarians at Cerritos College have used a variety of methods to determine the usefulness and success of our instruction program. According to Scott Lanning, Professor of Library Media at Southern Utah University, assessment is the means of discovering the quality of library services and programs. It’s how libraries determine what students already know and what they have learned. It’s is the way to demonstrate the value of the library to administrators, faculty, and students. And it can help us improve library services (Lanning, 2014). At Cerritos College we know assessment is important. We have used pre- and post-tests, satisfaction surveys, use statistics, learning outcome rubrics, test scores, and librarian evaluations forms, to name a few. However, none of our assessments adequately addresses the usefulness of our online research guides. And these guides are typically the research starting point for Cerritos College students taking online classes.

I would like to create a set of online activities that will be available 24/7 so that students have access online and information literacy skills assessment can be ongoing with continuous feedback to the librarians. These activities would be available via the Cerritos College Library website and also via a link on each of our research guides. Although these exercises would be geared towards online learners and generally self-administered on the students own initiative, they could also be used in library skills classes, workshops, or orientations as a pre- and post-test at the recommendation of the librarian. In all settings, students would be able to test their research savvy by completing the activities and would receive immediate feedback online. The assessments would give students a sense of what they need to know in order to undertake a successful research project. But equally important, the scores would help the Cerritos College librarians determine the information literacy competencies of our students and adjust our library instruction program as needed.

For this project I set up a prototype page using a free web-based assessment tool developed at Kent State University Libraries called TRAILS – Tool for Real Time Assessment of Information Literacy Skills. TRAILS features multiple-choice questions that target a variety of information literacy skills based on various grade standards. The assessment questions are based on the American Association of School Librarians’ Standards for the 21st-Century Learner and those from the Common Core State Standards Initiative. This web-based system was developed to provide an easily accessible and flexible tool for school librarians and teachers to identify strengths and weaknesses in the information-seeking skills of their students (TRAILS, 2014). In other words, the system was developed so that librarians can use and customize for their own library.

I explored the TRAILS assessment tool and found that it was really quite simple to use it at Cerritos College. After setting up an account, I simply had to choose which assessments I wanted to use and create a module for each skill. I was then provided with a link for each assessment. The skills categories I selected are:

  1. Develop topic.
  2. Develop, use, and revise search strategies.
  3. Identify potential sources.
  4. Evaluate sources and information.
  5. Recognize how to use information responsibly, ethically, and legally.

I chose to make the assessments available via LibGuides, the research guide CMS we use at the Cerritos College Library. You can see the guide I created, Library Research: Skills & Exercises, here.

http://libraryguides.cerritos.edu/LibraryResearchSkillsExercises

Since I’m initially planning to use this as a self-administered assessment, I chose not to get the pre-generated random codes for each student. If it were to be used in a skills class or orientation the codes would be useful to allow me to see individual performances.

While the page I created is live now, I have not added the link to the library home page or the research guides yet so it has not been tested. I need to get to campus to unveil it to the librarians before making it available to the public.

TRAILS in Action

TRAILS is geared towards K-12 information literacy skills, yet it is appropriate for many community colleges, where between 60 and 80 percent of students are in basic-skills classes (Oldham, 2010). TRAILS currently has over 13,000 registered users representing all 50 states and more than 30 countries. Over 600, 000 students have taken the assessments (Eschedor Voelker, Schloman, & Gedeon, 2013).

As assessments are completed by students, results are immediately available in the administration module where the librarian administrator can detailed reports on performance for each item. Results are reported for each item by number and percentage of students choosing each possible response, with correct responses highlighted. Librarians can compare their students’ results with benchmarks established by TRAILS.

However the TRAILS team notes that the student score provided by TRAILS is not meaningful in and of itself. One cannot say that the numeric score is a definitive measure of a student’s information literacy knowledge. What the score does provide is a measure of attainment relative to other students who have taken the same assessment. The report may be used to identify areas of strengths and weaknesses in students’ understanding of specific concepts related to information literacy, which will assist in targeting instructional efforts (Kent State University Libraries, 2014).

There are many advantages to librarians for using online assessments such as TRAILS in conjunction with research guide and library skills classes. Besides giving students an opportunity to exercise and expand their skills it tells the librarians if our instruction has had a positive effect on our patrons. As a leading expert in the information literacy field has said, without assessment there’s really no way of knowing if our learners learned anything from the instructional experience or if the instruction was just a waste of everyone’s time (Kaplowitz, 2014). We are certainly not in the education field to waste time.

Librarians across the country have embraced TRAILS as an opportunity for an objective measure of information literacy. School librarian Patricia L. Owen notes that librarians have long attempted to produce evidence of the library’s impact on student learning and achievement. TRAILS, with its standardized format and robust report and statistics output, has allowed her to capture a large amount of information about student learning quickly, enabling her to get a thorough picture of her students strengths and weaknesses (2010). Other librarians have used the results to leverage more opportunities to work with classroom teachers and the ability to provide useful concrete data to school administrators (Eschedor Voelker, Schloman, & Gedeon, 2013).

Conclusion

I look forward to implementing TRAILS at Cerritos College in Fall 2015. This project helped me understand what it does, how to implement it, and how I can use it in my college library. However, during my research I found many other online library skills assessment tools that look equally promising. HeLIOS (Hemingway Library Information Online Skills) in particular looks very comprehensive and is designed to prepare students for college level research (Christensen, Morgan, & Kinikin, 2013). A project at James Cook University in which the librarians created and embedded information literacy modules in Blackboard, which were made mandatory by the course instructors, looks very promising as well (Johnston, 2010). In the end, this project gave me not only hands on practice with the TRAILS system but also a lot to think about regarding the implementation of online tutorials and assessments at Cerritos College.

References

American Association of School Librarians. (2007). Standards for the 21st century learner. Retrieved from http://www.ala.org/aasl/sites/ala.org.aasl/files/content/guidelinesandstandards/learningstandards/AASL_LearningStandards.pdf

Christensen, J., Morgan, F., & Kinikin, J. (2013). An online information skills tutorial. School Library Monthly. 29(5). Retrieved from http://www.schoollibrarymonthly.com

Common Core State Standards Initiative. (2014). English language arts standards. Writing. Grade 11-12. Research to build and present knowledge. Retrieved from http://www.corestandards.org/ELA-Literacy/W/11-12/

Eschedor Voelker, T.J., Schloman, B.F., & Gedeon, J.A. (2013). Pathways for success:  the evolution of TRAILS and transitioning to college. Informed Transitions: Libraries Supporting the High School to College Transition. 209-216. Retrieved from http://digitalcommons.kent.edu/libpubs/15

Johnston, N. (2010). Is an online learning module an effective way to develop information literacy skills? Australian Academic & Research Libraries. 41(3), 207-218. Retrieved from http://www.tandfonline.com.mms02.cerritos.edu:2048/toc/uarl20/current

Kaplowitz, J.R. (2014). Designing information literacy instruction: The teaching tripod approach. Lanham, MD: Rowman & Littlefiled.

Kent State University Libraries. (2014). TRAILS: Tools for real-time assessment of information literacy skills. Retrieved from http://www.trails-9.org

Lanning, S. (2014). Reference and instruction services for information literacy in school libraries. Santa Barbara, CA: Libraries Unlimited.

Oldham, Jennifer. (2010). Community college students mired in basic-skills classes. The Hechinger Report. Oct 8. Retrieved from http://hechingerreport.org/content/community-college-students-mired-in-basic-skills-classes_4397/

Owen, P.L. (2010). Using TRAILS to assess student learning: a step by step guide. Library Media Connection. May/June. Retrieved from
http://www.librarymediaconnection.com/pdf/lmc/reviews_and_articles/featured_articles/Owen_May_June2010.pdf


Project Description and Requirements

The project you undertake must be well connected to eLearning delivery, eLearning assessment (or evaluation) or both and must clearly show connections to major themes in this course.

The project you do must result in a product or products that you can share with the class via your blog. This can include sharing by creating a link from your blog to your product or by attaching your product to the blog in some way. I do not envision a product that would be contained by just posting text on your blog.

The product you create must represent a significant body of work, worthy of a graduate student and the very high standards of the instructor.

The product you create must be grounded in the literature base of the field, meaning that you must clearly connect what you do with relevant citations and discussion of your project with scholarly articles and other appropriate academic sources.

In addition to the product, you must create a short executive summary, up to 3 pages in the form of a text document shared on  your blog, that describes your project, including providing information that describes the context of your work, for example, explaining why you think the project you are doing is worthwhile to you, detailing the content area, the grade level, and other considerations that both make the project you take on useful to you, and also that influence design choices that you make in doing the project. Your executive summary should also describe your development process, discuss your results and provide your bibliography.


Other Discussion Topics This Week

1. What did this class miss? What topic related to delivery or assessment of online learning should have been included but wasn’t.

The course packed a lot into 10 weeks and I can’t imagine what might have been missed.

In a way I wasn’t prepared for this particular course, especially since it was my first graduate course in over 30 years. I entered the program hoping to work toward the Ed Tech Certificate. When I found out it isn’t currently being offered I decided to work towards the e-Learning Certificate instead. I admit I really didn’t understand the difference. But I later looked a little more closely at the website and saw that “The study of eLearning at CSUSB will help you master the theoretical, human and technological aspects of creating effective online learning environments.” The key term here is “theoretical”! I had imagined I’d be taking mostly hands-on, techie courses in the program, so all the e-learning, delivery, and assessment theories and philosophies that were covered in this class were a bit overwhelming to me, especially since I have no formal education training. It was all brand new.

But in the end I really enjoyed the course content, format, assignments, and instructor. I learned a lot, read a lot, and had fun exercising my research and writing skills. This course was enough for me in every way, but most importantly I was able to relate all of it to my work with students in the library .Thank you!

2. If you were going to do a research study about eLearning delivery and/or assessment, what would your study be and why?

The literature of the library profession is awash with studies. I’m sure any idea I come up with has been done already. However, conducting a survey of my own library users would interest me a lot. I’d like to find out how the Cerritos College library can provide the best possible support to our e-learners. How should we deliver content and incorporate it into the curriculum? Is the material we currently provide used and useful? What format is best? We have use statistics and have done satisfaction surveys but by adding a level of assessment and evaluation I would hope to get a better idea of what our patrons need. If I do this, I agree with Daniel that it would have been useful in this class to get more information, guidance, and practice developing valid and reliable tests.

Advertisements