Assessing programming and outreach
This chapter discusses the importance of assessment in programming and outreach and methods in which the diversity of University Library’s staff and collections and the success of its diversity initiatives have been or will be measured.
Unfortunately, many academic librarians have little background in formal assessment methods (or instruction or event planning, for that matter) and as is the case in most enthusiastic endeavors, assessment is generally an afterthought rather than a part of the planning process. We at IUPUI University Library are no exception and we find ourselves in the uncomfortable position of ‘playing catch-up’ in this area, particularly as budgets continue to tighten and some may believe that diversity initiatives in the library constitute a luxury rather than a necessity.
‘Constructing and implementing diversity initiatives involves continuous experimentation, assessment, modification and innovation. Assessment of diversity is an ongoing lifetime process of improvement’, states Love (2001). As institutions of higher education attract more and more diverse students and faculty who may benefit from library diversity initiatives and as budgets tighten, necessitating accountability for time and money, assessment has become more important than ever.
Librarians must determine what needs to be assessed and how to go about it. In terms of diversity, the library’s collections, resources, and staff can and should be assessed on an ongoing basis. Initiatives and programs, including those aimed at library staff, as done by Texas A&M University (Yang and White, 2007), and those focused on students, staff and faculty at a university such as IUPUI should also be assessed to determine the effectiveness of promoting and reflecting diversity.
But just because assessment should be done, does not mean that it is always easily done. One of the goals identified by our Diversity Council in 2009 was to evaluate the diversity of our collections. Several members of the Council performed literature searches to identify best practices for such an evaluation, as conventional collection evaluation techniques were deemed unsuitable or too labor intensive, but they located very little on the subject at the time. And as work on the International Newsroom and other projects took center stage, the task of evaluating the collection was set aside. In the interim, some practical options for conducting a collection assessment have become known to us.
One possible method, which might still be a tedious task even with our electronic library systems, would be to compare lists of monograph titles purchased prior to the library’s Diversity Council activities to a list of those purchased after librarians had been exposed to these initiatives and become more aware of the University Library’s goals for increasing diversity. We have also recently become aware of an online resource, Guide to Reference (http://www.guidetoreference.org), which is ‘a selective guide to the best reference sources, organized by academic discipline and published by the American Library Association’ that seeks to identify core reference titles in a wide variety of disciplines. The lists available here could serve as an excellent guide to inventorying our own collection to determine if the library owns the best possible sources focused on diversity issues and diverse groups of people. Given the continuing focus on diversity, no doubt other such tools will be developed and best practices will begin to appear in the literature. But in the meantime, it is also useful to invite faculty members, particularly those from diverse groups, to discuss their opinions about the diversity of the library’s collection, how they use it, if it fulfills their own research needs, and how best to improve weak areas. They will no doubt be invaluable in identifying seminal or recent works in many areas.
While work needs to be done on evaluating our collection, we are planning a few different methods to assess the effectiveness of the International Newsroom (see Chapter 5). Before the Newsroom was installed, digital cameras were strategically placed around the room as part of another assessment project. These cameras permit unobtrusive observation of how many people occupy the space and how they use it and had, in fact, been used by a SLIS faculty member to study the use of the library’s Academic Commons computer clusters a few years ago. Although the Newsroom opened in August 2009, we have retained camera footage only since December 2009 and now need to develop a strategy to effectively evaluate the information we are gathering.
Initially, students appeared to use the space simply as another study area, but since the large library tables were removed from the space and replaced with more casual chairs and seating for the fall 2010 semester, there appear to be more students viewing the television screens, even while they seem to be working on their laptop computers. We plan to use samples of earlier security camera footage as a baseline and compare it to samples of this semester’s footage to determine the impact, if any, that the change in seating has had on the use of the Newsroom. We will be able to identify peak use times, which televisions the students gravitate to, and if they appear to be performing other tasks while watching the broadcasts, if they ignore the televisions altogether, or if they are only watching the programs.
Another assessment method came about very quickly and serendipitously, proving once again the necessity of being an agile group that can respond to opportune moments. We had been under the impression that no class had visited and used the International Newsroom as part of their course work. As it happened, one instructor had brought a class in during fall 2009 semester without contacting library staff and somehow without attracting much attention. In fall 2010, she spoke with a librarian she had approached to work with her introductory International Studies course. The librarian consulted Mindy Cooper about recent changes to the International Newsroom and options for a class of 25 students to view and listen to the broadcasts. We had recently removed the large library tables, which had been the anchors for the headphone plug-in jacks in front of each television, and had not yet reinstalled the jacks in smaller tables. Therefore, we had turned up the volume slightly on the televisions, making it possible for those in close proximity to listen without headphones or dial into a specified FM frequency on small radios. For this class, each student was supplied with a radio to use during class and told they could keep it to use in the Newsroom in the future. After the class, each student was given a survey to complete on a half sheet of paper and asked to respond to questions about which televisions they had watched, the helpfulness of the programming in completing their assignment, and their overall impressions of the International Newsroom. (See Appendix E).
Although the 22 student responses we received from the survey hardly constitute a large sample, they will prove useful to begin a focused evaluation of the television programming and furniture configuration. We plan to improve the survey and include a question about the listening method used by each student to determine which option is preferred. It is also possible to create this as an online survey that the faculty member or librarian can load into the university’s course management system and permit the student to complete the survey in the context of the class. This would also allow the faculty member to receive the feedback and attach completion of the survey to points or a grade. The results of this class survey included:
The least watched broadcast was TV2 – Headlines Today (India). The difference in the number of students who watched TV2 compared to the other channels was not significant; this leads us to believe that most of the students wandered about the room and at least glanced at all the televisions during their class period.
While over 95 percent of the students watched news programming, more than half also watched sports and/or popular programming. Comments from some of the students indicated they also viewed a documentary program and a ‘soap opera’. TV4 – KBS World (Korea) broadcasts soap operas at the hour of the morning the class was in the Newsroom. One student commented ‘love TV4!’ so it seems the idea of popular programming is appealing to students and may prove to be an excellent means of learning about other cultures.
While the surveys have been an unexpected and significant starting point, feedback from the International Studies instructor, Dr Dawn Whitehead, who is also Director of International Curriculum, has been absolutely invaluable. We asked her a series of questions about how she used the International Newsroom and its resources in developing the assignment for her course. We also asked her thoughts on the changes we had made in the physical space. She kindly allowed us to relate her responses verbatim below.
For many of our students the English programming is also a great change. For students who are interested in engaging with international issues and news, this offers an opportunity for them to listen and watch international programming without the language barrier. This should open up the reading room to more students. It offers students an opportunity to learn about other parts of the world that they may not have studied in their classes even if they have studied at least one world language. Instead of being limited to one channel, they now have the opportunity to watch several channels from different corners of the world.
I think the use of stations for listening is a preferred option. You are more mobile, which is nice, but a few of the students had problems with static. With the headphones in the jacks, they all had good quality of sound, but they had to cluster together, which made it more of a shared experience than an individual experience.
Last year, the emphasis was really on interpreting the actions of the news presenters, looking at images and trying to discover what issues were covered and how this coverage varied from country to country. This year, they were able to watch and listen and try to discern what was covered differently in different countries and why this may be as well as what wasn’t covered in some places. It was more of a specific comparison than last year. Last year they had to draw conclusions based on what they saw versus what they heard this year.
I thought the mixture of news and other programming was great. Is it possible to add an African station? That would be a nice addition, if possible, and it would provide students with an opportunity to learn about an area of the world where there are a lot of negative assumptions about the use of technology and communication.
Dr Whitehead’s comments indicate that we were wise to remove the large library tables and increase our soft seating to create a more relaxed and comfortable atmosphere. Also, that by switching from broadcasts in foreign languages to primarily English language newsfeeds that the Newsroom would be more inclusive, rather than less so. We will also need to find a solution to provide quality audio for the televisions very soon. The radios, although inexpensive and highly portable, do not seem to provide the needed fidelity and turning up the volume any higher on the televisions would be too distracting to others in surrounding areas. This indicates that we will have to come up with a means of providing more plug-in jacks for headphones – a challenge because they require a power source and cannot be moved from location to location. In the interim, we may be able to supply the small radios to faculty and their classes who come to the Newsroom, but could not sustain the bulk cost (approximately $2.50 each) indefinitely. Although it is possible that having students clustered around a television and watching together would offer additional opportunities for shared experiences and ease of discussion, depending upon the size of the class and how many others are in the space.
If other classes wish to visit the Newsroom – and we certainly hope that by Dr Whitehead’s example and word of mouth advertising they will – we will certainly require a protocol for reserving the space for class visits or at the least create signage with a schedule. Students are rather accustomed to the latter type of arrangement as this is their experience with many computer labs on campus that serve as both classroom spaces and as open labs. Dr Whitehead’s assignment for her students was exactly the sort of class activity we had in mind for the International Newsroom and now that she has blazed the trail, it may provide impetus for librarians to approach their faculty and develop assignments that would encourage students to use the International Newsroom for course work and personal enrichment.
The enthusiasm exhibited over popular culture programming on some of our television channels may imply a need to rename our space, since the coverage now goes beyond exclusively news programming. We will continue to evaluate our service provider to see if there are ways to change programming quickly to accommodate specific requests. Also, we have been trying to locate a reliable and affordable package that includes broadcasts in English from the African continent, especially in light of the university’s recent partnership with Moi University in Eldoret, Kenya.
Administer a survey to students in all classes that come to the International Newsroom. This will require a mechanism for librarians and/or faculty to alert the Diversity Council or Reference Desk to prepare the space, post signs advising users of a class visit, and having copies of the survey available.
Ask faculty for feedback on their class’s use of the Newsroom and its resources, possibly by automatically e-mailing them a link to an online survey via Survey Monkey (University Library subscribes to this service, but there are also free versions).
Petition the library’s website editors for a link on the homepage to information about the International Newsroom, including links to the surveys, additional library resources, etc. Add a similar link to the Faculty Support page. Use the website statistics logs to determine how often these links are used.
Offer workshops for librarians, faculty, and the Center for Teaching and Learning on how to incorporate the Newsroom into instruction, provide sample assignments or activities that exhibit how the International Newsroom can be utilized in support of campus diversity, the RISE initiative (see Chapter 4), study abroad programming, the Principles of Undergraduate Learning (see Chapter 1), and information literacy. Record the number of workshops offered, attendance at the workshops, which marketing venues used to advertise the workshops were most effective, and how many usages of the Newsroom resulted directly from the workshops.
As useful as the information gleaned from surveys and questionnaires can be, qualitative feedback regarding the changes and impact that our diversity initiatives have made provides the richest information, particularly as reflected in the lives and work of our staff, as evidenced by the following comments from an original Diversity Council member:
I used to be very angry at the library at the racial inequalities in hiring practices. I noticed that there were very few minority librarians, and only slightly more minority full-time non-librarian staff. I thought this was due to racism on the part of the library. Since serving on the Diversity Council, I have come to realize that it is more complicated than that. I believe now that the library wants to hire minority staff, but that for a variety of reasons, not many minority staff apply for open positions. Part of the problem is that not many minority individuals go to library school. I am pleased that this is being addressed in a profession-wide manner with scholarships and cohort programs for minorities. The Diversity Council made a small contribution to this effort with its Diversity Scholars program and perhaps also with its READ posters. I still think it is important to hire minority staff, but I have more patience with the library in its efforts now.
I think my service on the Diversity Council has energized me in my day-to-day work. I have really enjoyed my service and would like to be on the council again at some point in the future. Between my work on the International Newsroom and my work putting together the GLBT grant, I feel like I’ve made a real contribution to diversity at the library, and that’s given me a sense of pride and accomplishment.
I think diversity is a most relevant issue and a major driving force in any educational institution. As a college library, our community continuously broadens to include, educate and empower individuals from many backgrounds. Their multiple talents benefit us all. The creed of UL Diversity Council mirrors my personal beliefs so I felt compelled to join it.
I decided to join the council because diversity is all about celebrating individuality. And individuality is what makes people interesting. Taking part in the council seemed like a good way to help encourage students, faculty, and staff to embrace each other’s differences, thereby creating a more engaged, creative, and open-minded community.