Measuring excellence is difficult.
Output statistics are not useless, but they cannot tell us about quality, excellence, or if there is a match between community need and services.
Interesting examples of use of data from the afternoon. One branch was looking at cutting. Circ per hour was same first hour or last hour, but all circ last hour was from one person, where there were many the first. Therefore closing last hour inconvenienced only one person. Or rearranged collection so easier to find, but while circ went up, reference questions went down (and patrons were more satisfied).
Cost benefit analysis has several advantages quantify monetary value of library services. How do you chose the economic value. Technical ideas: consumer surplus valuation; contingent valuation (how much would you pay for ... ) or Willingness to pay.
Select specific services delivered to specific audiences.
One advantage is that it is well known in the business community. It can be used over time for specific services or products. Cost can be high since it involves surveying the community. It is possible that you may choose and area to study, when you could be missing other areas which the community values more than you think.
Larry White
Estimated cost of performance assessment in Florida in 2000 was $16 million, but the state only gave $32 million in state aid, and therefore 1/4 to 1/2 of state aid was wasted.
Outcomes based performance measurement takes a long time to generate results...years.
Return on investment. Everyone hopes a high return on investment.
ROI is used by business. Happy stories and smiley kids did not work. Created an ROI. First year 6 to 1. Took a buck from commissioner, promised return. I created this (showed stats). Showed ROI, then told a story.
Combination of cost avoidance and return revenue wise. Used data from geneology/history room list, took local folks and multiplied by figures by tourism for local. Then estimated distant customers for overnight stays. Showed library accounted for $500,000 in tourism revenue. Then cost avoidance (average cost per book times number of circ -- because then people did not have to purchase).
Data mining for libraries gets to be an important role in the community. Need to use a combination of numbers and words in a creative fashion.
Can use it to justify what you want to do and to save what you want to do. It is scalable.
Lack of consensus in value and use. It is usually used defensively (preserving library funding) and reactively. We wait for disasters to tell the story.
Joe Matthews
Summer Reading Program
In Portland OR, 75,000 in program, raise $60K, fly family of 4 to Disney Land. Only 30% complete. Real outcome is improvement in reading level. Cost is high. Can be done with third party agency for confidentiality of kids.
Encourage to start thinking about outcome side of things do they spend more time reading, do they do better in school. One place does a survey of caregivers about perception of outcomes.
Statistics
Context, trends, history.
Age of collection as a stat.
Unintended consequences of performance measures
- assessment process consequences (survey fatigue)
- tactical process consequences
- strategic process consequences
Tactical consequences: operational (how you work); systems (can create an 'us v. them'; can look at forest and forget the trees); financial (ten to one return speech -- loaned out to economic development dept.); service ("new Coke"); organizational impacts (can bring good things).
Strategic consequences occur over a long period of time. Operation supported an unethical behavior to support the need to constantly increase circulation. Problem where assessment drives the mission rather than the mission driving the assessment.
Final of the day: Management Frameworks (Joe Matthews)
Three Rs: Resources, Reach, and Results.
Resources: how much money do we need to achieve
Reach: who do we want and where
Results: What do we want and why
Choose only two or three measures. It is important to think about customer segments. (other than demographics) they come with different needs and with different expectations.
Performance Prism (see handout) used in England and New Zealand and Australia
Balanced Scorecard
Financial perspective: saving time, reducing costs, gnerating new revenues, share of the pie
(see hand out....)
Building a Library Balanced Scorecard: Mission and Vision Statements; Strategy -- how to reach the mission; select performance measures; set targets or goals; projects or initiatives
Strategic plans often do not use the word strategy. Most use one of two approaches: conserative, reachable, or the scientific wild ass guess approach, but you may want to have a BHAG.
The scorecard is usually created by a small group with the results shared with the stakeholders.
No comments:
Post a Comment