logo_web_blk_on_trans (1K)

2002 ASSESSMENT ACTIVITIES                                      Prepared February 2003

 

2002 was an active year in assessing user needs; their library and information use patterns; and in gaining a better understanding of how online information resources and services impact their work.  During Spring Quarter, assessment information was drawn from four initiatives:

 

  • Undergraduate Survey (makeup for 2001 survey which had faulty student sample)
  • LibQUAL+ 2002 Survey
  • In-Library Use Survey
  • Faculty and Graduate Student focus groups

 

In general, results from these assessment efforts show that our community of faculty and students are moving rapidly to remote use of online information resources – indeed, it is the preferred method for finding and using information needed for work   Self-reliance and the ability to perform library-related work without library staff mediation are of high importance to users.  Undergraduates continue to use libraries as a place to do work but faculty and graduate student use of physical facilities continues to fall, most notably in those units that have large serial collections.  Libraries total circulation statistics, including in-house use, continue to show steady decline.  During the 2001-02 year, there were a million fewer items used in-house compared to 1995-96.

 

 

Survey data and focus group transcriptions are currently available in the Assessment shared-docs file on Tahoma and will soon be available on Staff Web.  Tabular data shown below is selective. 

 

 

UNDERGRADUATE SURVEY

 

The Spring 2002 Undergraduate Library Survey was a rerun of the 2001 survey due to a faulty sample used by the Office of Educational Assessment.  Several questions dealing with information literacy were changed from 2001, otherwise the survey was the same.  The 497 surveys returned represent 24.9% of the 2000 surveys distributed.  Key results are summarized below. All data shown in tables is in percent with 1998 results in parenthesis.  


Importance of Libraries

 

Libraries remain important for the work of undergraduate students.  Indeed, the importance of libraries increased between 1998 and 2002 while the importance of the World Wide Web declined.  The table below shows changes between 1998 and 2002 (1998 in parenthesis) with the percentage marking importance on a 1 (lowest) to 5 (highest) scale.

 

Source of information

 5 ( % )

4  (%)

MEAN

UW Libraries

49.7 (44.0)

29.2 (29.7)

4.20 (4.06)

Other libraries

  8.2 (8.9)

13.9 (14.9)

2.44 (2.59)

World Wide Web

34.8 (41.2)

33.0 (31.8)

3.84 (3.98)

Departmental resources

24.3 (28.1)

28.8 (30.7)

3.51 (3.65)

Other students/faculty

31.2 (28.6)

32.0 (27.8)

3.68 (3.52)

 

 

Frequency and Type of Use

 

Results showed a decline in the frequency of in-person library visits from about 70% of respondents who visited at least weekly in 1998 to 60% in 2002.  However, the frequency of use from a residence computer nearly doubled from 21% at least weekly in 1998 to 40% in 2002. 

 

 

Method of using libraries

More than once per week

Weekly

Visit in person

32.8 (38.8)

27.6 (31.5)

Use campus computer (not in library)

18.1 (13.0)

17.9 (21.1)

Use residence computer

18.7 (8.8)

21.3 (12.1)

 

 

Among those who visit the Libraries at least weekly, there was a steep decline in the frequency of those looking  for journals and books.  Use of library computers rose substantially compared to 1998 (large computer lab in OUGL opened after the 1998 survey). 

 

 

Activity

More than once per week

Weekly

Look for a journal

  4.0 (6.5)

  6.2 (12.3)

Look for a book

  6.2 (8.4)

  6.6 (12.3)

Look for other material

  3.4 (4.1)

  6.2 (5.7)

Consult library staff

  1.4 (2.5)

  3.6 (5.3)

Photocopy

  6.6 (10.3)

14.1 (24.1)

Use library computers

34.8 (19.4)

22.5 (23.9)

Print from library computer

17.9

13.7

Work individually

30.4 (34.9)

25.6 (29.7)

Work in groups

11.1 (8.1)

14.1 (15.2)

 

When asked what library services were needed weekday evenings and weekends, nearly 75% needed a place to work, 67% a computer to use, while access to the collection was marked by only 15% and reference assistance by 20%.  While the frequency of in-person visits dropped, satisfaction with library hours of opening rose with 87% of respondents very satisfied with hours on weekday evenings.

 

 


Computer Use

 

OUGL Commons is the preferred campus location for student computer.  The frequency of use was highest from residences – 21% daily from campus residences and 43% from off-campus residences. 

 

SITE

Daily

Weekly

OUGL Computing Commons

19.1 (14.1)

29.4 (24.7)

Mary Gates (Suzzallo CRC)

  7.8 (9.5)

22.7 (22.1)

Microlab - Health Sciences

  0.6 (1.3)

  2.6 (3.6)

Other UW Libraries

  9.9 (4.3)

15.9 (13.2)

Department computing lab

15.1 (18.6)

12.9 (14.2)

Campus residence

20.5 (10.4)

  2.8 (1.9)

Off campus residence

43.1 (45.5)

  8.0 (11.1)

 

The ubiquity of undergraduate computing use is shown in the data below, especially in the use of email and the Web. 

 

Activity

Daily  %

Weekly %

Monthly %

Read or send email

87.3 (80.7)

  8.7 (12.7)

  1.0 (1.8)

Search UW Libraries catalog*

  4.0 (4.3)

30.6 (30.1)

46.3 (40.3)

Search lib provided bib database

  0.8

18.1

39.0

Find library full-text sources

  2.4 (1.0)

20.3 (11.8)

41.0 (29.5)

Do course assignments on Web

27.2 (14.1)

43.7 (37.1)

16.3 (26.8)

Look for information on the Web

50.1 (25.8)

34.8 (42.6)

  8.7 (22.5)

Surf the Web

47.7 (22.4)

28.4 (33.0)

12.7 (23.1)

Use application software

38.2 (34.2)

38.6 (34.3)

10.1 (14.0)

Listen to sound/view images

34.2

21.1

14.9

*1998 combined library catalog and bibliographic database into one question

 

37% of undergraduates report that they dial directly into the modem pool or use UWICK, while 28% use another Internet Service Provider.  28% also use high speed cable modem or DSL.

 

Information Literacy

Students ranked finding information needed for their work as most essential toward successful completion of course work this year.  Comparing these results to those of faculty responses to a similar question in the 2001 survey, agreement between the two groups was closest in defining research topics and finding information resources and furthest apart in the importance of evaluating and using information effectively.


 

 

Relevance to topic and usefulness for supporting their argument were most often considered when evaluating information sources for course-related work.  Availability of electronic full-text was also seen as important.  Author’s credentials and recommendations by instructors and librarians ranked lowest.

 

How often do you consider the following when

Most often

 

Least often

No Reply

 

evaluating information sources:

5

4 

3

2

1

   %

MEAN

Relevance to your topic

71.8

18.5

  4.6

  1.4

  0.6

  3.0

4.54

Author’s credentials

  9.9

22.3

33.2

20.9

10.5

  3.2

2.94

Instructor or librarian recommendations

16.9

29.0

30.2

11.7

  8.5

  3.8

3.27

Publication date of source

21.5

35.4

24.1

12.1

  3.4

  3.4

3.53

Usefulness for supporting your argument

55.1

26.6

11.9

  1.4

  1.6

  3.4

4.25

Availability of electronic full-text

31.4

29.2

21.5

  9.7

  4.6

  3.6

3.66

Source is scholarly

22.9

33.0

24.5

10.3

  5.2

  4.0

3.50

 

 

Satisfaction

 

Satisfaction with specific services increased from 1998 with the largest increase in library computer access.

 

SERVICE

High

Satisfaction

 

 

 

Not Satisfied

No Reply %

Don’t Use

Mean

 

5

4

3

2

1

 

 

Reshelving

36.7 (19.0)

35.9(34.6)

23.0(35.8)

 3.4  (8.1)

 1.0 (2.4)

22.1 (21.9)

4.04 (3.60)

Course reserves

29.6 (22.5)

42.0(36.2)

23.5(30.4)

 2.0  (7.5)

 3.0 (3.4)

18.5 (19.4)

3.93 (3.67)

Library instruction

24.0 (11.3)

40.2(27.5)

26.3(41.6)

 6.5 (12.9)

 3.0 (6.8)

32.0 (28.8)

3.76 (3.24)

Staff assist in library

27.8 (25.0)

42.0(35.2)

24.9(27.6)

 3.2  (8.8)

 2.2 (3.3)

17.5 (15.3)

3.90 (3.70)

Staff assistance remote

23.2

34.3

31.5

 7.9

 3.1

48.9

3.67

Photocopy

24.6 (17.7)

37.7(38.0)

23.9(30.0)

 8.6 (11.1)

 5.2 (7.7)

18.3 (10.2)

3.68 (3.45)

ILL/Article delivery

25.7 (16.8)

35.1(32.8)

29.4(38.8)

 6.4 (7.6)

 3.4 (4.1)

46.7 (53.1)

3.73 (3.51)

Lib computer access

30.2 (13.9)

40.2 29.1)

21.0(34.4)

 6.5 (16.0)

 2.2 (6.7)

 6.8  ( 8.6)

3.90 (3.28)

Libraries Web site

36.1

39.7

19.7

 3.1

 1.3

10.3

4.06

 

Overall satisfaction with the Libraries, as well as satisfaction with specific services and hours rose in every category compared with 1998.  The mean overall satisfaction score of 4.22 moved closer to that of graduate students and faculty (4.26, 4.33 respectively).

 

 

                                                Very Satisfied                                                   Not Satisfied

 

5 (%)

4 (%)

3 (%)

2 (%)

1 (%)

Mean

Library collections

32.6 (26.3)

47.4 (47.8)

17.2 (22.7)

  2.1 (2.4)

  0.6 (0.8)

4.09 (3.96)

Library services

34.7 (23.1)

52.6 (53.3)

11.2 (21.2)

  1.3 (2.1)

  0.2 (0.3)

4.20 (3.97)

Overall

33.3 (23.1)

56.1 (55.4)

10.0 (19.9)

  0.4 (1.1)

  0.2 (0.5)

4.22 (3.99)

 

 

Priorities 

 

Delivery of full-text to the desk-top became the top priority (similar to faculty and grad students) with electronic course reserves next.  There were sharp drops in priorities dealing with training in using library and Web resources (from 48% to 18%) and in providing consultation on how to do library research (40% to 21%). 

 

LIBRARY PRIORITIES

%

Deliver full-text to your computer

54.9 (34.4)

Maintain quality of Libraries print collections

24.5  (28.8)

Provide electronic access to older journals

40.0

Increase library hours

33.8  (41.9)

Add more computers in the library

45.9  (54.5)

Provide course reserve materials electronically

49.5  (50.8)

Provide training in using library/Web resources

17.5  (47.5)

Add group study/seminar rooms

33.4  (35.5)

Provide consultation on how to do library research

20.9  (39.8)

Provide quieter study/work areas in libraries

32.6

 

 

 

LIBQUAL+2002

 

The Libraries continued its participation in the ARL/Texas A&M LibQUAL+ pilot.  This was the 3rd year for the survey and more than 170 libraries participated, including 43 ARL ones.  Most UW results could not be compared with our 2001 data due to methodological and sampling problems.  However, comparisons can be made with other ARL results as well as where possible with our 2000 results.

 

The table below shows the “gap” (difference between perceived level of service and minimum service expectations) for the questions with highest importance ranking on the 2002 LibQUAL+ survey.  The more positive the number, the more the perceived level of service exceeded the minimum expectations. UW faculty and grad student gaps were more positive than the ARL averages (except for library hours for grad students) with undergraduate scores showing less differentiation but generally higher.    

 

 

 

Fac

2002

ARL

2002

Fac

2000

Grad

2002

ARL

2002

Grad

2000

UG

2002

ARL

2002

UG

2000

ACCESS TO INFORMATION

 

 

 

 

 

 

 

 

 

Complete journal runs

  -.17

-.72

-.13

.11

-.16

-.20

.65

.64

.35

Timely ILL/Art delivery

   .77

 .19

 .43

.93

 .36

 .58

.85

.67

.55

Comprehensive print collections

   .21

-.36

  .32

.62

 .22

 .02

.68

.72

.79

Convenient business hours

  .16

 .29

  .11

-.19

.12

-.04

.78

.34

.58

AFFECT OF SERVICE

 

 

 

 

 

 

 

 

 

Knowledgeable staff

  .44

.13

  .72

.36

.28

.50

.68

.43

.59

Courteous staff

  .90

.59

1.19

.80

.69

.57

.62

.64

.63

Staff who understand user needs

  .59

.29

  .88

.58

.47

.55

.62

.58

.76

Respond to user questions readily

  .55

.36

  .82

.79

.50

.62

.68

.54

.56

PERSONAL CONTROL

 

 

 

 

 

 

 

 

 

Electronic resources to office or home

-.08

-.10

-.48

.18

.07

-.06

.45

.38

.25

Library website to locate info myself

 .28

-.06

 

.52

.33

 

.81

.63

 

Easy to use access tools

 .19

-.05

 

.54

.30

 

.60

.44

 

Info accessible for independent use

 .41

 .09

 

.60

.36

 

.66

.55

 

 


 

 

Personal Control was the most “important” dimension for faculty and graduate students and one where both the mean perceived level of service and gap were higher than the mean for ARL libraries.

 

PERSONAL CONTROL 2002

UW Fac

ARL

Fac

UW

Grad

ARL

Grad

UW

UG

ARL

UG

Perceived Level of Service Mean Score (scale of 1 to 9) and gap between minimum and per.

N=

133

N=

7218

N

=128

N=

8253

N=

139

N=

7985

Making electronic resources available from my home and office

7.07

-.08

6.92

-.10

7.10

.18

6.85

.07

.6.74

.45

6.79

.38

Library website enabling me locate information on my own

 7.35

.28

7.05

-.06

7.69

.52

7.30

.33

7.04

.81

7.22

.63

Easy to use access tools that allow me to find things on my own

 7.18

.19

6.88

-.05

7.31

.54

7.07

.30

6.77

.60

6.88

.44

Making information easily accessible for independent use

7.41

0.41

7.00

.09

7.36

.60

7.10

0.36

6.80

.66

6/96

0.55

 

 

LibQUAL+ scores are more difficult to analyze due to the complexity of the survey, differences between institutions, and the smaller number of respondents.  The most apparent change since 2000 is the smaller positive gaps on the “affective” questions for faculty.  Yet, upon closer examination the difference is due to significantly higher minimum expectations – the mean scores for perceived level of service were all higher in 2002 than 2000. 

 

Overall quality of services and satisfaction with treatment and scholarly support are shown below ( a nine point scale is used) UW faculty and graduate student scores rank significantly higher than the ARL average, especially for satisfaction with scholarly support.  

 

LibQUAL+

Fac

2002

ARL

2002

Fac

2000

Grad

2002

ARL

2002

Grad

2000

UG

2002

ARL

2002

UG

2000

Satisfaction with treatment

7.78

7.34

 

7.45

7.23

 

7.09

7.09

 

Satisfaction with support

7.29

6.60

 

7.32

6.83

 

6.98

6.83

 

Overall quality of services

7.58

7.00

7.17

7.39

7.04

7.13

6.99

7.05

6.88

 

 

IN-LIBRARY USE SURVEY

 

During Spring Quarter 2002, the Libraries conducted in-library use surveys at all UW Seattle libraries.  Surveys were distributed to those entering the library during designated 2 hour blocks on weekdays (2) and weekends (2) for a total of four sessions.  During these 4 sessions, 6018 surveys were distributed and 3878 completed surveys returned (64% return rate).   Undergraduate students comprise the largest user group.

 

User Groups

Number

Percent

Percent UW only

Undergraduates

2304

59.4%

65.1%

Graduate Students

  986

25.4%

27.8%

Faculty/Staff

  251

  6.5%

  7.1%

Non-affiliated

  312

  8.0%

 

Did not state

    25

  0.6%

 

 

Health Sciences, Odegaard Undergraduate, and Suzzallo-Allen libraries accounted for 74% of the surveys distributed and 68% of those returned.  User group composition varied by library. 

 


Unit Users – All Units by Group

 

(Percentage of total users in each unit from each group)

 

Undergrad

Grad

Faculty/Staff

Other

Total

 

#

%

#

%

#

%

#

%

#

%

Allen North

221

52.7%

98

23.4%

27

  6.4%

76

18.0%

422

10.9%

Allen South

199

68.8%

50

16.8%

27

  9.1%

24

  8.0%

300

  7.7%

Architecture

  37

36.6%

50

49.5%

  7

  6.9%

  7

  6.9%

101

  2.6%

Art

  87

82.1%

13

12.3%

  1

  0.9%

  5

  4.7%

106

  2.7%

Chemistry

  65

69.9%

19

20.4%

  8

  8.6%

  1

  1.1%

  93

  2.4%

Drama

  44

75.9%

  7

12.1%

  0

  0.0%

  9

15.0%

  60

  1.5%

East Asia

  30

54.5%

23

41.3%

  2

  3.6%

  0

  0.0%

  55

  1.4%

Engineering

  65

44.8%

44

30.3%

10

  6.9%

27

18.5%

146

  3.8%

Fish-Ocean

  23

46.9%

12

24.5%

12

24.5%

  2

  4.1%

  49

  1.3%

Forest Resources

  26

50.0%

20

38.5%

  1

  1.9%

  5

  9.6%

  52

  1.3%

Foster Business

174

65.2%

73

27.3%

  8

  3.0%

12

  4.5%

267

  6.9%

Health Sciences

123

20.0%

359

58.5%

83

13.5%

54

  8.7%

619

16.0%

Math Research

    6

13.0%

20

43.5%

11

23.9%

  9

19.6%

  46

  1.2%

Music (inc ListCent)

  60

47.6%

50

39.7%

11

  8.7%

  5

  4.0%

126

  3.3%

OUGL

1091

84.9%

95

  7.4%

26

  2.0%

84

  6.5%

1296

33.4%

Physics-Astronomy

  27

55.1%

14

28.6%

  7

14.3%

  1

  2.0%

  49

  1.3%

Social Work

  27

30.0%

39

43.3%

  9

10.0%

16

17.6%

  91

  2.3%

 

 

What Users Did During Library Visit

 

In-Library Survey results confirmed much of the data provided by the triennial survey on library use.  Undergraduates visit primarily to study/work and use computers, while faculty generally look for material.  Graduate students do all three.

 

 

Undergrads

Grads

Faculty/Staff

Other

Total

Ask for Help

  9.3%

14.9%

19.2%

17.6%

12.0%

Looked for/used library material

21.8%

44.4%

59.6%

53.5%

32.6%

Borrowed/returned material

11.2%

27.5%

23.6%

15.4%

15.3%

Made copies or used copy center

18.1%

26.3%

25.2%

22.4%

21.0%

Studied/worked alone

58.5%

49.1%

19.6%

39.1%

52.0%

Studied/worked in groups

13.1%

13.3%

  2.0%

  7.1%

12.0%

Used a computer

63.6%

58.0%

35.2%

37.8%

58.2%

Used a printer

16.1%

14.9%

  2.0%

  6.4%

14.1%

 

 

FOCUS GROUPS

 

Two focus groups were held in Spring 2002, based around the topic of the impact of online library resources on faculty and student work.  The faculty and graduate students groups were comprised primarily of people from health sciences and the sciences. Some of  the comments that arose independently in each group included:

 

  • Google is initial search starting point for many searches
  • Bibliographic database interfaces too complex
  • Color is critical; color printers in wide use
  • Electronic journals are great and foster ubiquity of library research – any place, any time
  • Fewer visits to the library equals greater efficiency
  • Personal connection to library staff is important

 

 

The following quotations from focus groups are organized loosely by topic and represent common discussion threads.  They tell the story of the impact of online resources and information technology in the words of our faculty and graduate students.

 

The Information Environment Is Complex

 

I’d like to use Inspec more.  I avoid it because I have problems with the search interface. And I know there are articles there that should be coming up, but I’m not finding them.  And I’m finding hundreds of garbage items.  The librarian keeps saying, “Well, sit down with me and I’ll show you how to do it.”  But I can’t remember how to do complicated things from one day to the next. (Professor, Physics)

 

I start with a search engine and I typically use Google.  I find that it’s very well designed and very effective.  For the searches I’ve done it’s been very fast.  Google isn’t medical but it sure comes up with a lot of convenient references.  I’ve got material going through Google that I never did actually find through PubMed much to my surprise.  So I tend to go to Google first, find articles and convenient MeSH terms and then I’ll come back in.  Google’s just so much easier, so much more intuitive.  (Professor, Pathology).

 

Rather than start with a search engine, I pick up a few key articles from colleagues or whatever, and then build a citation tree forwards and backwards.  Using ISI Web of Science is incredibly useful there.  I think I would pretty well not know how to do research without it if it were to disappear.  (Faculty)

 

My frustration is that usually when I call my favorite reference librarians I am feeling humiliated, like I should be able to do this and can’t.  And I’m tired of explaining why I can’t.  I just want to get the job done.  (Graduate student)

 

I’m not keeping up with a lot of this stuff.  I’m getting old.  I don’t know what the new search engines are, I don’t know what’s going on at the library in terms of how someone or something can help me find the material I’m looking for in a faster easier format.  I need someone to tell me that, I don’t have time to read a 10 page document on how to do it. (Faculty) 

 

 

Online Resources Save Time

 

I wish all my journals were online. It’s not so much that I dislike trekking to different libraries, it’s just the whole process of getting there; tracking down the journal; finding a working copy machine, especially on weekends.  It’s just so time-consuming and I have so little time, I have no time. (Graduate Student, Technical Communications)

 

It’s a huge luxury to be able to work at home.  With our teaching and research loads that’s where we get our good work done.  Instead of having to spend my day finding articles and getting them copied, I can go home and just work.  It’s wonderful. (Faculty)

 

 

Online Resources And Information Technology Effect Teaching

 

 

I teach research to masters degree students and I gravitate towards e-journals.  If the journal is in the library then I don’t tend to use it.  I worry that the journals that are e-journals will get used and the ones that aren’t won’t get used.  Is that a good thing or a bad thing?  (Professor, Nursing)

 

I have put all my lectures into PowerPoint over the last 10 years.  And each year, due to the fact that I’m gathering new information, I’m changing those lectures.  They do not remain static. Because slides are static you keep them in a tray and you can bring them out next year and give the same lecture.  As you are putting them into PowerPoint, you begin to realize that there are some vacancies you need to fill in – got to go back and look for the information to put it into PowerPoint.  .  You go OK, I want an illustration of this, go to the Web, grab a picture and put it in there. (Faculty)

 

 

Online Resources Make Researchers More Productive

 

I can do more work from home and I can work at odd hours. When libraries aren’t open I can still take a look at a reference.  If I’m lying down to fall asleep and it dawns on me that, “Yeah, I need to check that”, and I can get up and do it in 5 minutes and have an answer. (Graduate Student)

 

I find that it has changed the way I do library research.  It used to be a stage process: Initial trip, follow-up trip, fine-tuning trip.  Now it’s a continuous interactive thing: I can follow-up anything at any time.  While I’m writing, I can keep going back and looking up items or verifying information.  (Graduate Student, Psychology)

 

 

If one person finds a really interesting paper we all have it within 15 minutes. And it moves like wildfire through the lab. Because the PDF file is sent all around by email and we all print and we are all reading it. It’s great.  (Faculty)

 

 

The Personal Connection Is Important

 

So I need someone to tell me those things and to give me a quick info. That’s why Terry [Jankowski] is so blessedly useful.  She’ll give me that information very quickly.  Just getting those tidbits at the right time can make a very, very large difference in how quickly we can access useful information.    That’s one thing to chorus loud and clear on the tape, “Thank God for librarians.” There’s no better place for human intervention in information science. (Faculty)

 

I am hoping that the reference librarians I know by name never disappear from the face of the earth.  Nothing can quite replace the human contact when you get stuck (yes, yes from group). (Graduate Student)

 

 

Conclusion

 

Online information resources and information technology have helped change the way in which teaching, learning and research is done at the University of Washington.  The pace of change is accelerating and the impact on libraries and users is profound.  While more information is needed on user outcomes, it is clear that the substantial shift in library use from physical to virtual will intensify as more material in different subject areas is made available to the desktop.  The library as place remains critical to undergraduates.