Cense

Research.Design.Impact.

Seeing Innovation Differently: The Role of Evaluation

by

See what you do through new eyes

We begin a series called the Innovators Secret Advantage by starting with a simple, but powerful premise: perspective is everything. 

This is based on a phrase that we use as our guide at Cense: 

“The real voyage of discovery consists not in seeking new landscapes, but in having new eyes.”

Marcel Proust

Evaluation is most commonly seen as a means to assess the merit, worth, and significance of a program, service, or product. Evaluation can focus on implementations and learning, how a program works, outcomes and outputs, or some combination of these things. This is how most see evaluation. 

However, for innovators — those looking to develop and apply something new into a situation — evaluation offers so much more when approached as a tool for innovation. This is less about the method and much more about how we treat the methods. This is where we need our new eyes. 

Evaluation = Feedback

What evaluation offers to innovators is feedback, which plays a central role in complex systems. These are the kind of systems that most human services exist to various degrees. Complexity creates conditions for innovators where it’s difficult to know what is happening with a high degree of certainty. Feedback allows innovators to ‘take the temperature’ of what they are ‘cooking up’ to see if they need to make changes. These changes might look like doing more of something, adding things, or perhaps removing something. It’s very much like making a soup. 

Evaluation is the means of obtaining feedback and doing it systematically. It allows innovators to avoid bias, mis-perceptions in the data, and overcome blind spots that get created. Innovators are focusing on their product, service, or program itself, but with a slight shift in perception toward feedback systems, and they can gain much more in the way they approach their work.  

Evaluation is the means of providing fuel for innovation, particularly as an idea moves from concept to prototype. Data, systematically collected and focused, along with the analysis and sensemaking that comes with it, plays a vital role in determining how ready an innovation is for deployment or how appropriate it is for scaling upward or outward. 

Seeing With New Eyes

Seeing with new eyes begins by asking some questions. The first three questions foster evaluative thinking, not just new ways of seeing: 

  • What is going? 
  • What is new?
  • What does it mean?

The next question is: What are you hiring innovation to do for you? The answers to these questions are what can inspire that new way to see the work being done. 

The final set of questions to give you those new eyes come from mindful reflection. Organizational mindfulness is not some new age meditation technique, but a scientifically-supported approach to paying attention across the organization to learn what is being done and how it connects with purpose and values. 

Engaging these questions will set the stage for using evaluation as a means to help innovations fulfill their true promise. In the weeks ahead, we’ll look at how this is done and what your new eyes can be trained to look at. 

Evaluation: The Innovator’s Secret Advantage

by

I have a secret….

Innovators are a hopeful lot. They are looking to find an advantage by creating something new that will improve their current condition or at the very least prevent their current situation from deteriorating. Sometimes you need to change to keep things the same. 

Evaluation is one of the hidden (maybe secret) advantages from those innovators who are looking to sustain their work over the long haul. For those who are looking to innovate in human systems, evaluation might be the only thing that allows an idea to scale beyond its current implementation. 

Why? How? And what does this look like? We will begin a series looking at the Innovators Secret Advantage and how evaluation can and does play a role in supporting good design, measuring impact, guiding strategy, and informing the development of new ideas. Over the next few weeks we’ll be introducing ten points to illustrate how evaluation can take your ideas forward and create the impact on your clients, the public, your market, and the world. 

Sound good? Stay tuned and the secrets will be revealed. Through case studies, practical examples, evidence, and theory we’ll illustrate how to use evaluation to support and amplify your innovation with humans, products, services, and policies. It’s time that evaluation’s advantages are accrued to everyone, not just those ‘in the know’. 

And, if you’re in the area, our Principal, Cameron Norman, will be speaking on this topic at the upcoming Service Convention Sweden on November 28th, 2018. Get a jump on what’s to come!  

Image Credit: Sai De Silva on Unsplash

Evaluation’s Hidden Value: Clarifying Your Strategy

by

Evaluation is more than just the assessment of merit, worth, and significance of a program, product, or service. Although these are the primary functions of evaluation, it can serve far more than this when designed and implemented in the appropriate manner.

Evaluation focuses attention on the things that your organization builds, has built, and implements and the purpose and role(s) they serve. In doing so, evaluation serves as a primary support tool for strategy. In order for an evaluation to do its job, it must be aligned with the purpose of the thing it is evaluating.

For example, if an evaluation is being asked to assess behaviour change, the focus of the program or stimuli (e.g., app, message, etc..) for that change is a core part of the strategy. By matching the program components with the activities and intended outcomes, evaluators can help determine the degree of alignment and assess (and advise on) the likelihood of success before any data is collected. It is through the data — the evaluation itself — that this alignment and the assumptions behind it are tested.

Theory into practice

The process of aligning the planned activities with the outcomes is part of what a Theory of Change is designed to do. To illustrate, consider a program designed to promote some kind of behaviour change (e.g. engaging in a new activity, doing more or less of something, etc..). For a change to take place, there needs to be a certain logic in how the intervention (i.e., the activity or service) is set up based on some established design principles.

We also know from the wealth of knowledge of behavioural science, that there are many different ways to promote change and that some are more efficacious than others. A skilled evaluator can help determine whether the approach being used is likely to produce the desired change by building out a Theory of Change and then matching that with the appropriate outcomes.

This is what the heart of a strategy is all about: aligning the resources, intentions, and actions together to produce an outcome.

Like the image above, evaluation helps peel back the top coat to reveal what is happening underneath. It helps take the mystery out of why programs work or don’t work and what the reasons are why people do things (or don’t).

It’s one of the reasons why we bring strategy, evaluation, and design work together. By connecting them, you get far more performance out of your program than if you just keep the cover on.

To learn more, contact us and we’ll be happy to show you what’s under your hood.

Image credit: Nathan Van Egmond

Innovation’s Single Biggest Question

by

There is not a bigger question for innovators — social, product, service, policy — than What are you hiring this [ ] to do for you? 

Let’s break this down a little and then explain why we ask that at the beginning of any engagement and all throughout from the first meeting to the final run of the evaluation data. The question gets us to shape what and how we might design our innovation while the answer is about the ways we tell what kind of value it generates and the impact it produces.

What’s in a question?

The start of the question is about you, the aspiring innovator. This highlights the role of the creator and reminds us that we are generating this ‘thing’ by procurement, by design, or by simply encouraging something to be made. Without us (i.e., you), nothing changes.

The active use of hire is about the reality that we are paying for innovation through our time, our energy, our focus, our social (and often political) capital, and our money. All of these could be spent elsewhere. Design is an investment and it’s purposeful. Asking this big question gets us to pause and think deeply about what we’re putting into innovation and what we’re looking to get out of it.

The [ ] is the thing you’re hiring — the proposed service, experience, product, policy, or ecosystem — and is what you’re purposefully bringing about. This is your idea manifest into something real.

The last part is ‘to do for you‘ is active: it’s about ensuring that you’re clear about what purposes it serves. No matter how beneficial your planned innovation might be for others, you are ultimately asking it to serve a role, fill a need, for you. It is you that wants to solve a problem, build a market, or prevent something from happening, and this requires some clarity to innovate well.

What’s in an answer?

Innovation is not just creating things, it’s about evaluating the things we create. If our novel products, services, experiences, and policies don’t generate value for people, they aren’t really innovations. It’s just stuff.

An evaluation perspective on the question asked above might look at things like:

-The roles people played in the innovation process, including the skills they used, experiences they had, and the insights that they gained along the way. This learning is what feeds into our understanding of how an innovation develops along with the people and organization it is a part of. All of that is part of the innovation dividend or ROI.

-The resources used as part of the ‘hiring‘ process like money, time, human resources, and other capital; all can help look at the value of the initiative to see if the costs and benefits make sense.

-What ‘things’ are produced — the prototypes, their functioning, their benefits, and weakness — as well as provide a means to document the iterations, the steps taken, the new ‘offshoots’ that might emerge, and the resulting products, services, experiences, and policies.

-Lastly, the innovation needs to fulfill some requirements or expectations and evaluation looks at what it does in the world, for whom, under what conditions, and what other impacts might emerge unintentionally. This helps assess risks, benefits, and find new opportunities for further development and innovation.

Better questions, better answers

Innovation is what will drive much of the future value of your organization. It’s what allows you to build, grow, adapt, or sustain what you’re doing because even if you don’t feel a need to change, everything is changing around you and sometimes you need to change just to stay where you are.

By asking this one simple question you might find answers that will lead you to much better innovations to shape and create that future.

We help our clients ask this question. If you want our help, contact us and we’ll gladly help you ask better questions for better answers.

Evaluation As Part of An Innovation Value ROI

by

Follow us for a moment. We’re going to talk about design, innovation, evaluation, and how they all go together.

Design is really the discipline — the theory, science, and practice — of innovation. That means that if you are innovating, you’re designing. Innovating is about adding value through introducing something new to a situation — it might be entirely new, a twist on an existing idea, or an old idea placed in a new context. Innovation and design are about taking ideas and purposefully transforming things into making those ideas real in the form of services, products, or experiences.

Design and innovation are all about creating value.

Thus, understanding the value of design is partly about the understanding of valuation of innovation. At the root of evaluation is the concept of value. One of the most widely used definitions of evaluation (pdf) is that it is about merit, worth, and significance — with worth being a stand-in for value.

Understanding value

Value can only be understood by asking the right questions because it’s a relative question as many people will see the worth of something different from others.

One of the big questions professional designers wrestle with at the start of any engagement with a client is:

“What are you hiring [your product, service, or experience] to do?”

What evaluators ask is: “Did your [product, service, or experience (PSE)] do what you hired it to do?”

AND

“To what extent did your PSE do what you hired it to do?”

“Did your PSE operate as it was expected to?”

“What else did your PSE do that was unexpected?”

“What lessons can we learn from your PSE development that can inform other initiatives and build your capacity for innovation as an organization?”

In short, evaluation is about asking:

“What value does your PSE provide and for whom and under what context?”

Value creation, redefined

Without asking the questions above how do we know value was created at all? Without evaluation, there is no means of being able to claim that value was generated with a PSE, whether expectations were met, and whether what was designed was implemented at all.

By asking the questions about value and how we know more about it, innovators are better positioned to design PSE’s that are value-generating for their users, customers, clients, and communities as well as their organizations, shareholders, funders, and leaders.

This redefinition of value as an active concept gives the opportunity to see what the return on investment — time, money, energy, commitment — can yield an organization in real-time. This means that value is fluid, dynamic and that can be generated on an ongoing basis. It’s not just what you report at the end of the fiscal year or project.

Imagine reporting real-time value at your next stakeholder, staff, or shareholder meeting? Imagine knowing what you’re creating now and having that focus your efforts on what you could create in the near and long-term future?

Evaluation is how you do it. It’s in the name itself.

If you’re looking to hire an evaluation to better your innovation capacity, contact us at Cense. That’s what we do.