Cense

Research.Design.Impact.

Evaluation’s Hidden Value: Clarifying Your Strategy

by

Evaluation is more than just the assessment of merit, worth, and significance of a program, product, or service. Although these are the primary functions of evaluation, it can serve far more than this when designed and implemented in the appropriate manner.

Evaluation focuses attention on the things that your organization builds, has built, and implements and the purpose and role(s) they serve. In doing so, evaluation serves as a primary support tool for strategy. In order for an evaluation to do its job, it must be aligned with the purpose of the thing it is evaluating.

For example, if an evaluation is being asked to assess behaviour change, the focus of the program or stimuli (e.g., app, message, etc..) for that change is a core part of the strategy. By matching the program components with the activities and intended outcomes, evaluators can help determine the degree of alignment and assess (and advise on) the likelihood of success before any data is collected. It is through the data — the evaluation itself — that this alignment and the assumptions behind it are tested.

Theory into practice

The process of aligning the planned activities with the outcomes is part of what a Theory of Change is designed to do. To illustrate, consider a program designed to promote some kind of behaviour change (e.g. engaging in a new activity, doing more or less of something, etc..). For a change to take place, there needs to be a certain logic in how the intervention (i.e., the activity or service) is set up based on some established design principles.

We also know from the wealth of knowledge of behavioural science, that there are many different ways to promote change and that some are more efficacious than others. A skilled evaluator can help determine whether the approach being used is likely to produce the desired change by building out a Theory of Change and then matching that with the appropriate outcomes.

This is what the heart of a strategy is all about: aligning the resources, intentions, and actions together to produce an outcome.

Like the image above, evaluation helps peel back the top coat to reveal what is happening underneath. It helps take the mystery out of why programs work or don’t work and what the reasons are why people do things (or don’t).

It’s one of the reasons why we bring strategy, evaluation, and design work together. By connecting them, you get far more performance out of your program than if you just keep the cover on.

To learn more, contact us and we’ll be happy to show you what’s under your hood.

Image credit: Nathan Van Egmond

Theory of change: An introduction

by

Is the above tree alive and growing or dead and ready to be made into furniture? How does something like a tree connect to providing a swing, becoming a coffee table, or supporting the structure of a home? That is based partly on a theory of change about how a tree does what it does. That might sound strange, but for more sophisticated things like human service programs, linking what something does to what it achieves often requires a tool for explanation and a Theory of Change can serve this need well if used appropriately.

Theory of Change is described as “a comprehensive description and illustration of how and why a desired change is expected to happen in a particular context.” It has taken hold in the non-profit and philanthropic sectors in recent years as a means of providing guidance for program developers, funders, and staff in articulating the value of a program and its varied purposes by linking activities to specific behavioural theory.

Matthew Forti, writing in the Stanford Social Innovation Review (SSIR), suggests a Theory of Change (ToC) contain the following:

To start, a good theory of change should answer six big questions:
1. Who are you seeking to influence or benefit (target population)?
2. What benefits are you seeking to achieve (results)?
3. When will you achieve them (time period)?
4. How will you and others make this happen (activities, strategies, resources, etc.)?
5. Where and under what circumstances will you do your work (context)?
6. Why do you believe your theory will bear out (assumptions)?

Unlike a program logic model, which articulates program components, expected outputs, and outcomes a ToC explains how and why a particular set of actions is to produce a change and the assumptions that underpin it all. ToC can be used with a program logic model or be developed independently.

What a ToC is meant to do is allow you to explain in simple language the connection between a program’s purpose, design, and execution and what it produces in terms of benefit and impact. While it may draw on theories that have been published or tested, it may also be unique to the program itself, but in all cases, it is meant to be understandable to a variety of stakeholders and audiences.

Creating a Theory of Change

A strong ToC requires some understanding of behaviour change theory: what do we know about how change happens? It can’t simply end up with “and then change happens”, it must have some kind of logic that can be simply expressed and, whenever possible, tied to what we know about change at the individual, group, organization, system, or a combination. It’s for this reason that bringing in expertise in behaviour change is an important part of the process.

That is one of the points that Kathleen Kelly Janus, also writing in the SSIR, recently made as part of her recommendations for those looking to better the impact of creating a ToC. She suggests organizations do the following:

  1. Engage outside stakeholders
  2. Include your board and staff
  3. Bring in an outside facilitator
  4. Clearly define the outcomes that will spell success
  5. Track your results rigorously.

Inclusion, consultation, and collaboration are all part of the process of developing a ToC. The engagement with diverse stakeholders — particularly those who sit apart from the program — is critical because they will see your program differently. Outsiders will not get caught up in jargon, internal language, or be beholden to current program structures as explanations for change.

Defining the outcomes are important because change requires an explanation of the current state and what that changed state(s) look like. The more articulate you can be about what these outcomes might be, the more reflective the ToC will be of what you’re trying to do. By defining the outcomes better, a ToC can aid a program in developing the appropriate metrics and methods to best determine how (or whether) programs are manifesting these outcomes through their operations.

Supporting strategy

A ToC is best used as an active reference source for program managers, staff, and stakeholders. It can continually be referred to as a means of avoiding strategy ‘drift’ by connecting the programs that are in place to outcomes and reminding management that if the programs change, so too might the outcomes.

A ToC can be used as a developmental evaluation tool, allowing programs to see what they can do and how different adaptations might fit within the same framework for behaviour change to achieve the same outcomes. Alternatively, it can also be used to call into question whether the outcomes themselves are still appropriate.

By making a ToC accessible, easy to read and to understand the key is to make it visual. Employing someone with graphic design skills to help bring the concepts to life in visual representation can provide a means to clarify key ideas and getting people beyond words. It’s easy to get hung up on theoretical language and specific terms when using words; where possible use visuals, narrative, and representations. Metaphors, colour, and texture can bring a ToC to life.

A ToC, when developed appropriately, can provide enormous dividends for strategy, performance, and evaluation and help all members of an organization (and its supporters and partners) understand what it is all about and how what it does is linked to what it aims to achieve. The ToC can serve your communications, strategy development, and evaluation plans if done well and appropriately facilitated, particularly for complex programs. It doesn’t solve all your problems, but few things will help you understand what problems you’re trying to solve and how you might do it than a good Theory of Change.

If you need help building a Theory of Change, contact us and we can help you develop one and show you how it can support your strategy, innovation, and evaluation needs of your programs and organization as a whole.