So What? Using Outcome Measurement to Assess the Impact of Library Programs

Rhea Rubin was both the organizer and speaker. [Code 437 from PLA Conference Handout page.] The book is available from ALA, wrote the book under contract, and does not receive royalties. [These are notes based on the speaker's remarks, and not my comments...which will be clearly noted.]

Crime rates have gone down in many cities, a recent newspaper article said that police departments are reporting to citizens what they are doing differently to reassure the need for police services. In addition to the statistics, the are trying to a "here's the effect of our work." So what is that result? They are reporting a reduction in crime and tying it all together. For example, neighborhood policing may be a reason. Others may not be reporting a reduction because of vacancies. This is outcome measurement. Outcome measurement is the reporting of the effect of our efforts.

United Way pioneered in this area. For example homeless shelters are supposed to report what happens to the people who use the shelter. In libraries, do people who get correct health information have a better health outcome? Perhaps they ask different questions of their doctor or other health care professional.

The first handout to look at should say Outcome Measurement Key Concepts [this one is not on the web...yet, but it will be]. We are used to dealing with inputs like money, staff, volunteers, etc. Answers the question which of our inputs do we use to achieve a goal. Outputs are a measure of what we do. It is an answer to the question "How much did the library do?" Outcomes are what happens to a user because we had inputs and deployed them to help a user. "What difference did our program make to the customer?" Inputs and outputs are about efficiency. These are also very much staff perspective numbers. Outcomes are from the user's perspective. What are key results.

How can we ever know the outcome from the general circulation of books. Outcome measurement does not work very well for a broad and general category. It is best used with relatively small programs and services which you can then look at as a whole. Outcome measurement is something you do at one point at a time and may not ever do it again, or perhaps five years later to measure change. To try this look for a program which has been designed to address the needs of a specific group of users. Look for a program that affects a specific person rather than producing a product. It is critical to pick a program where the people come more than once.

In Planning for Results they suggest specific kinds of statistics to measure: output measures; process measures (combines input and output measures), and outcomes.

Outcome kind of question:

• And what will you do with that information?

Ask not only what do users need, but what will they do with that information. Takes anecdotes and uses them as the basis for analyzing our work.

May sometimes use output measures instead of outcome measures. The latter measures the impact not just the efficient. A program can be more effective if not efficient.