Designful innovation

Expanding Perception Through Evaluation

2018-12-08 by cense

One of Evaluation’s greatest contributions to innovation is its ability to expand the vision of the innovator and attend to matters of perception. What we look at it is not always all we consciously see, nor is what we see the sum of all we perceive. Humans have many conscious and unconscious biases in what they perceive and how they interpret what they perceive. Evaluation can help add clarity to that perception and expand it.

Evaluation can help distinguish reality from illusion. In this latest in our series on Evaluation: The Innovator’s Secret Advantage we look at the secret of perception and how evaluation can help change what is seen (and reveal what is not) in the journey of innovation.

Seeing More

The image above is of a jail modeled on the Panopticon, an ancient architectural design to allow someone to see all that is going on at any one time. While its use within prisons is not one that’s particularly comforting or attractive, the model or concept of creating a design that enables a vantage point to see what is happening within an innovation context is a useful metaphor.

Evaluation can enable innovators to see what is happening from a single vantage point at considerable depth or, as we will discuss, provide an alternative means by seeing the phenomenon from multiple perspectives. Both of these have advantages for innovators looking to assess the performance of a product or service in areas of high complexity and uncertainty.

It is by reflecting on the vantage point (perspective) that we can determine whether what we are seeing is real or just an illusion and take the appropriate action based on that assessment.

Seeing Differently

Consider one of the most famous of these visual illusions: the Young Girl / Old Woman Illusion (below). The history of this illusion has been traced back to the late 19th century and has been replicated many times since then to be considered a staple of any introductory psychology text.

What’s interesting about this illusion is that you can’t hold both the image of the young girl and the old woman in your mind’s eye at the same time. You can only see one, not both even if you can switch back and forth. The same situation can occur within an innovation where it is difficult to see two ideas at the same time. By asking the right questions, an evaluation can help innovators to see what others see, what they default to, and whether or under what conditions do they see something different (e.g., seeing something as a product or a service).

Other illusions are more about perceptual shifts that, once made, are difficult or impossible to ‘unlearn’. These can be found in such things as the FedEx logo and its embedded arrow (check the link if you’ve never seen it). Another is the often common hidden ‘8’ within the 8 of diamonds in a deck of cards (see below).

Eight of diamonds playing card, isolated on white background.

Evaluation for innovation involves asking questions about a program, product, or service that includes taking a perceptual view of many different people and from different situations. It’s a systems-oriented perspective that considers what a particular phenomenon looks like from a particular point of view.

Seeing What’s Hidden in Plain Sight

Another famous psychological illusion is the one conceived of by Daniel Simons and his colleagues called The Monkey Business Illusion. This selective perception task is designed to show how your attention and focus can blind you to other things going on. The video below provides an updated version of the original video (available via the link above) to illustrate how we can miss things that are right in front of us based on what we attend to.

The Monkey Business Illusion

Innovators are often focused on the core aspects of their innovation: the product, the process, and the intended outcomes. This focus is often what sets successful innovators apart from others, yet it can also be a liability. An evaluator can help an innovator (an individual, a team, an entire organization) to see the full picture.

How? The means to do this is first achieved by asking evaluative questions that look at what is going on, what is new, and what meaning is derived from various activities.

Taking the Temperature (and Other Innovation Measures)

There is a useful, if not morbid, fable about the frog in the boiling pot. In that fable, the frog fails to notice that the temperature of the water continues to rise because the change is so gradual. The frog ends up boiling to death because of this inability to see that he is being boiled alive because the changes are so hard to detect. A good evaluation designed for innovation provides means to collect data akin to a thermostat calibrated to the particular conditions, situations, and product or service necessary.

This could include monitoring key inputs and outputs, tracking sales or engagements over time, or looking at resource levels like personnel and how they respond to change. The latter example is a good one, particularly with larger organizations where the duties assigned to a single person could be absorbed by others if that individual left the role. However, as time moves on and the extra work is normalized it is possible that the pattern is repeated over time where those who are left eventually shoulder such a burden that it makes the work impossible, but also the performance degrades.

These are the kinds of situations where collapse is likely. We’ve seen this with leadership when many people start leaving at the top or when those at the front face of an organize leave in large numbers. The cost in capital, focus, lived experience, and working knowledge can cripple an organization over the long haul. But like the frog in the pot, it might not be until just before the boil that it is noticed.

Evaluating for Illusion & Reality

Evaluation for innovation recognizes that the attention and focus of those leading the charge is precious and that leaders are likely to miss key things without the right tools to draw their attention to the whole system and the parts. Evaluation can be hired to cover some of these perceptual gaps and illuminate things like the present temperature or noticing gorillas, patterns in data like the 8 of diamonds or discerning the best moment to see a young girl or an old woman.

In the next piece in this series, we’ll look at how evaluation can uncover layers of impact that go beyond seeing what is in front of us to looking far past it. If evaluation for innovation is something that you need help with, connect with us; we can help you see things differently.

Title image credit: Cameron Norman

Mindfulness: Seeing the Constellations of Innovation

2018-12-03 by cense

In a world filled with increasing number of signals and lots of noise, it can be difficult to achieve focus and determine what to pay attention to. In this third piece in a series, Evaluation: The Innovator’s Secret Advantage, we look at one of the bedrocks of sustainable innovation that is the best representation of something shockingly simple, enormously powerful, but not easy: mindfulness.

Before embarking on an introduction to mindfulness for innovation, let’s dispel some myths about what it is not. It is not religious, spiritual, or some new-age trend, nor is it meditation. While it can and often is be affiliated with all those things, mindfulness is simply the practice of paying attention to what is around you and to yourself in the process. It is about conscious awareness of the present moment and non-judgemental attention toward the thoughts, feelings, experiences that arise from that experience.

An Exercise in Mindfulness

To many, the idea of mindfulness seems like an odd start to the conversation about innovation and evaluation, but the closer you look at what mindfulness is all about, the more it becomes clear how important it is to what innovators and evaluators both do.

At a recent talk on innovation and evaluation as part of Service Convention Sweden, Cameron Norman took the audience through a short exercise in mindfulness that looked like this:

Close your eyes (or lower your eyelids a little). [This reduces the amount of distraction from visual stimuli]
Put your feet flat on the ground and keep your back straight [This also reduces the amount of ‘signal’ coming from the body by getting into a more optimal position for sitting]
Breathe easy and relaxed and simply pay attention to what’s going on around you and what you are thinking. [Breathing easy avoids the issues created by holding one’s breath, creating a whole realm of problematic stimuli]

There are variants of this exercise that include focusing on the breath (which is a common technique for mindfulness-based stress reduction) and emphasis on quieting the mind, yet the outcomes are similar: an increased awareness of things that were — up until that moment — unaware and that is where mindfulness comes into evaluation.

Evaluation serves innovation best when it goes beyond the simple assessment of outputs, outcomes, and process document. Evaluation can be a mechanism for focusing attention on the work of innovation and its context. In the example above, the audience was asked: what did you notice? The answers ranged from hearing the HVAC system, noticing their breath, and realizing how much noise takes place during a talk.

In previous workshops, participants have reported bodily sensations (e.g., getting hungry), temperature changes, physical discomfort (e.g., back getting stiff from sitting), a wandering mind, and often forms of judgement about not ‘doing it well’ (for which there is no ‘well’, but rather ‘practice’). Just like evaluation for innovation, there is only ‘useful’ and not ‘useful’, not ‘good’ and ‘bad’.

Linking Mindfulness to Evaluation

The idea of applying mindfulness to organizations is not new and has actually been well-researched. Introducing and practicing the concept of organizational mindfulness has been shown to be strongly correlated with what high reliability in organizations, meaning that they continually produce desirable results, consistently. Innovation is difficult to do and doing it repeatedly and consistently is even more so. What organizations that have fostered mindfulness in the way they work have done is create a mechanism for paying attention that is systematized and implemented consistently.

Mindfulness is not a one-off exercise for those who adopt it into their work, but closer to a way of being. What it does is provide the means for organizations to not only focus on what’s in front of them, but to pay attention to the subtle signals that take place around that focus. (see two articles on our sister site Censemaking that link organizational mindfulness to social innovation and to developmental evaluation)

Mindfulness also cultivates curiosity. Curiosity is what draws an innovator and evaluator to consider what additional things might be happening (which we will explore later in this series) beyond just the intended outcomes. It is what leads us to innovation in the first place. By regularly creating space for mindful reflection into the innovation space, we nurture and cultivate curiosity.

Doing the Work

What does mindfulness practice look like for innovation? As mentioned earlier: we are dealing with something simple, not easy. A place to start is to follow some of these practices:

Ask evaluative questions, which include one that focuses on paying attention. We’ve discussed three of the most useful questions for promoting evaluative thinking in a previous post.
Create regular reflection space within the organization at each level (program, division, organization). This includes setting aside deliberate time regularly to reflect on what the data is telling you about what is happening, drawing on some of the questions above. This means instilling quiet, uninterrupted time for members of an organization to think and reflect. This involves unplugging from networks and thinking. It may involve tools (e.g., whiteboards, notebooks, cards) or movement (e.g., going for a walk or run), but it must be focused. Switching back and forth from email or other demands won’t work.
Bring reflections together. Individual reflection is important as is the chance to discuss those insights or experiences with others. Socialize the process of reflection by establishing a sharing culture. This builds much on what Donald Schon proposed as part of his work on attractor mapping, by enabling innovators to determine where the action is in the face of not always knowing. This means attending to the system that an innovation is a part of.
Reserve judgment and avoid labels. The rush to judge something is what kills curiosity and mindfulness. Ever notice that once something is labeled as ‘good’ or ‘bad’ we cease to ask the kind of questions of it that get deeper into its core? Mindfulness is about being open and then assessing utility.
Use learning as an outcome. What mindfulness does is encourage identification and insight into patterns. What comes from that identification and discussion is learning and is a genuine and important outcome for innovators. Document what is seen, heard, discussed, and concluded.

The biggest barrier that we see in our work is time. It isn’t that this takes a long time, although it does require some investment of it, rather it is that organizations are reluctant to prioritize this work and make it a regular part of their practice. Doing it occasionally has some benefit, but making it part of the organizational culture is really what will transform everyday work into something that has potential beyond the original purpose.

By instituting mindfulness into the work of your organization, you are more likely to see the constellations and quiet of night and not just the blue sky of the daytime.

Photo by Teddy Kelley on Unsplash

Seeing Innovation Differently: The Role of Evaluation

2018-11-26 by cense

We begin a series called the Innovators Secret Advantage by starting with a simple, but powerful premise: perspective is everything.

This is based on a phrase that we use as our guide at Cense:

“The real voyage of discovery consists not in seeking new landscapes, but in having new eyes.”
Marcel Proust

Evaluation is most commonly seen as a means to assess the merit, worth, and significance of a program, service, or product. Evaluation can focus on implementations and learning, how a program works, outcomes and outputs, or some combination of these things. This is how most see evaluation.

However, for innovators — those looking to develop and apply something new into a situation — evaluation offers so much more when approached as a tool for innovation. This is less about the method and much more about how we treat the methods. This is where we need our new eyes.

Evaluation = Feedback

What evaluation offers to innovators is feedback, which plays a central role in complex systems. These are the kind of systems that most human services exist to various degrees. Complexity creates conditions for innovators where it’s difficult to know what is happening with a high degree of certainty. Feedback allows innovators to ‘take the temperature’ of what they are ‘cooking up’ to see if they need to make changes. These changes might look like doing more of something, adding things, or perhaps removing something. It’s very much like making a soup.

Evaluation is the means of obtaining feedback and doing it systematically. It allows innovators to avoid bias, mis-perceptions in the data, and overcome blind spots that get created. Innovators are focusing on their product, service, or program itself, but with a slight shift in perception toward feedback systems, and they can gain much more in the way they approach their work.

Evaluation is the means of providing fuel for innovation, particularly as an idea moves from concept to prototype. Data, systematically collected and focused, along with the analysis and sensemaking that comes with it, plays a vital role in determining how ready an innovation is for deployment or how appropriate it is for scaling upward or outward.

Seeing With New Eyes

Seeing with new eyes begins by asking some questions. The first three questions foster evaluative thinking, not just new ways of seeing:

What is going?
What is new?
What does it mean?

The next question is: What are you hiring innovation to do for you? The answers to these questions are what can inspire that new way to see the work being done.

The final set of questions to give you those new eyes come from mindful reflection. Organizational mindfulness is not some new age meditation technique, but a scientifically-supported approach to paying attention across the organization to learn what is being done and how it connects with purpose and values.

Engaging these questions will set the stage for using evaluation as a means to help innovations fulfill their true promise. In the weeks ahead, we’ll look at how this is done and what your new eyes can be trained to look at.

Evaluation’s Hidden Value: Clarifying Your Strategy

2018-11-20 by cense

Evaluation is more than just the assessment of merit, worth, and significance of a program, product, or service. Although these are the primary functions of evaluation, it can serve far more than this when designed and implemented in the appropriate manner.

Evaluation focuses attention on the things that your organization builds, has built, and implements and the purpose and role(s) they serve. In doing so, evaluation serves as a primary support tool for strategy. In order for an evaluation to do its job, it must be aligned with the purpose of the thing it is evaluating.

For example, if an evaluation is being asked to assess behaviour change, the focus of the program or stimuli (e.g., app, message, etc..) for that change is a core part of the strategy. By matching the program components with the activities and intended outcomes, evaluators can help determine the degree of alignment and assess (and advise on) the likelihood of success before any data is collected. It is through the data — the evaluation itself — that this alignment and the assumptions behind it are tested.

Theory into practice

The process of aligning the planned activities with the outcomes is part of what a Theory of Change is designed to do. To illustrate, consider a program designed to promote some kind of behaviour change (e.g. engaging in a new activity, doing more or less of something, etc..). For a change to take place, there needs to be a certain logic in how the intervention (i.e., the activity or service) is set up based on some established design principles.

We also know from the wealth of knowledge of behavioural science, that there are many different ways to promote change and that some are more efficacious than others. A skilled evaluator can help determine whether the approach being used is likely to produce the desired change by building out a Theory of Change and then matching that with the appropriate outcomes.

This is what the heart of a strategy is all about: aligning the resources, intentions, and actions together to produce an outcome.

Like the image above, evaluation helps peel back the top coat to reveal what is happening underneath. It helps take the mystery out of why programs work or don’t work and what the reasons are why people do things (or don’t).

It’s one of the reasons why we bring strategy, evaluation, and design work together. By connecting them, you get far more performance out of your program than if you just keep the cover on.

To learn more, contact us and we’ll be happy to show you what’s under your hood.

Image credit: Nathan Van Egmond