evaluation-frameworks-review.pdf

(2208 KB) Pobierz
Excerpts from
Review of
Evaluation
Frameworks
prepared for
the Saskatchewan Ministry of Education
by Barbara Kahan, Kael Consulting
March 2008
© Saskatchewan Ministry of Education
TABLE OF CONTENTS
N
OTES
........................................................................................................1
E
XECUTIVE
S
UMMARY
..................................................................................2
Underlying Values and Principles .........................................................2
Concepts ...............................................................................................2
Approaches ...........................................................................................2
Benefits, Challenges, Suggestions For Success ..................................2
Incorporating Evaluation into an Organization......................................3
I
NTRODUCTION
............................................................................................4
Purpose of this Review .........................................................................4
Methods ................................................................................................4
Context ..................................................................................................5
Contents ................................................................................................5
G
UIDING
P
RINCIPLES
...................................................................................7
Values and Principles............................................................................7
Evaluation Standards ............................................................................8
Evaluator Ethics ....................................................................................9
Research Ethics ....................................................................................9
Organizational Practice Guidelines.......................................................9
E
VALUATION
C
ONCEPTS
............................................................................11
Defining Evaluation .............................................................................11
Evaluation Types.................................................................................13
Other Evaluation-Related Concepts ...................................................19
O
VERVIEW OF
A
PPROACHES
......................................................................23
Steps to Guide Evaluation Practice ....................................................23
Summary of Major Evaluation Approaches ........................................26
Results-Focused Evaluation ...............................................................28
Utilization-Focused Evaluation............................................................39
Collaborative Evaluation .....................................................................40
Balanced Scorecard............................................................................45
Appreciative Inquiry.............................................................................49
Other Approaches ...............................................................................54
Summary Note on Evaluation Approaches .........................................55
E
VALUATION IN
P
RACTICE
..........................................................................56
Benefits ...............................................................................................56
Challenges ..........................................................................................56
Suggestions for Success ....................................................................57
Incorporating Evaluation into an Organization....................................60
R
EFERENCES
............................................................................................63
N
OTES
This review was prepared for the Saskatchewan Ministry of Education in 2008 to
assist the development of a guiding framework for the Saskatchewan Ministry of
Education’s evaluation activities.
The Saskatchewan Ministry of Education holds the copyright to this document.
Results were based on several key informant interviews and a literature review.
The assistance of people in a number of ministries, in particular the Saskatchewan
Ministry of Education, is gratefully acknowledged.
Excerpts from the original document are included here with permission of the
Saskatchewan Ministry of Education.
Discussion of the specific Saskatchewan context and a possible Saskatchewan
Ministry of Education framework are omitted from the current document.
Review of Evaluation Frameworks (March 2008)
1
E
XECUTIVE
S
UMMARY
Underlying Values and Principles
Every framework is based on a set of underlying values and principles. Making
values and principles explicit increases the likelihood they will be integrated into
practice and promotes a reflective approach, resulting in greater understanding
of, and ongoing revisions to, values and practice. Some documents translate
values and principles into sets of guidelines, ethical principles or standards. For
example, the Joint Committee on Standards for Educational Evaluation’s most
recent draft of
Program Evaluation Standards
(2007) focuses on four categories:
feasibility; propriety (ethics); accuracy; and utility (usefulness).
Concepts
Evaluation concepts are understood in very different ways, making it important
to define them in order to ensure high quality communication. The concepts
include evaluation itself, the various types of evaluation (ranging from process
and outcome evaluations to meta-evaluation), and other evaluation-related
concepts, such as: baseline, stakeholder, qualitative, quantitative.
Approaches
Many evaluation approaches exist. The literature suggests that no one approach
is best for all situations as each approach has its own particular set of strengths
and cautions. Often two or more approaches are combined when conducting an
evaluation. Each approach has an associated set of steps designed to guide
evaluation processes and activities; there is a large degree overlap in the
suggested steps, although the nature of the methods and tasks related to each
step vary. Major evaluation approaches include: results focused (goals based,
goals free, theory based); utilization focused; collaborative; balanced scorecard;
appreciative inquiry.
Benefits, Challenges, Suggestions For Success
Considering evaluation-related benefits, challenges and suggestions for success
increases the likelihood of an evaluation framework successfully guiding an
evaluation. Potential benefits range from increased capacity to improved
outcomes. Challenges range from lack of resources to measurement issues.
Examples of suggestions for success include planning for evaluation from the
start of a project, reviewing processes as well as outcomes, and using a variety of
methods and designs.
Review of Evaluation Frameworks (March 2008)
2
Incorporating Evaluation into an Organization
Examples of guidelines for incorporating evaluation into an organization include
the following: take context into account; ensure adequate resources; foster an
evaluation-friendly culture; integrate evaluation into ongoing processes and
activities; define roles and responsibilities.
Review of Evaluation Frameworks (March 2008)
3

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika:

Zgłoś jeśli naruszono regulamin