Cense

Research.Design.Impact.

Expanding Perception Through Evaluation

by

Creating an Innovation Panopticon?

One of Evaluation’s greatest contributions to innovation is its ability to expand the vision of the innovator and attend to matters of perception. What we look at it is not always all we consciously see, nor is what we see the sum of all we perceive. Humans have many conscious and unconscious biases in what they perceive and how they interpret what they perceive. Evaluation can help add clarity to that perception and expand it. 

Evaluation can help distinguish reality from illusion. In this latest in our series on Evaluation: The Innovator’s Secret Advantage we look at the secret of perception and how evaluation can help change what is seen (and reveal what is not) in the journey of innovation. 

Seeing More

The image above is of a jail modeled on the Panopticon, an ancient architectural design to allow someone to see all that is going on at any one time. While its use within prisons is not one that’s particularly comforting or attractive, the model or concept of creating a design that enables a vantage point to see what is happening within an innovation context is a useful metaphor. 

Evaluation can enable innovators to see what is happening from a single vantage point at considerable depth or, as we will discuss, provide an alternative means by seeing the phenomenon from multiple perspectives. Both of these have advantages for innovators looking to assess the performance of a product or service in areas of high complexity and uncertainty. 

It is by reflecting on the vantage point (perspective) that we can determine whether what we are seeing is real or just an illusion and take the appropriate action based on that assessment. 

Seeing Differently

Consider one of the most famous of these visual illusions: the Young Girl / Old Woman Illusion (below). The history of this illusion has been traced back to the late 19th century and has been replicated many times since then to be considered a staple of any introductory psychology text. 

Young Girl/Old Woman Illusion

What’s interesting about this illusion is that you can’t hold both the image of the young girl and the old woman in your mind’s eye at the same time. You can only see one, not both even if you can switch back and forth. The same situation can occur within an innovation where it is difficult to see two ideas at the same time. By asking the right questions, an evaluation can help innovators to see what others see, what they default to, and whether or under what conditions do they see something different (e.g., seeing something as a product or a service). 

Other illusions are more about perceptual shifts that, once made, are difficult or impossible to ‘unlearn’. These can be found in such things as the FedEx logo and its embedded arrow (check the link if you’ve never seen it). Another is the often common hidden ‘8’ within the 8 of diamonds in a deck of cards (see below). 

Eight of diamonds playing card, isolated on white background.

Evaluation for innovation involves asking questions about a program, product, or service that includes taking a perceptual view of many different people and from different situations. It’s a systems-oriented perspective that considers what a particular phenomenon looks like from a particular point of view. 

Seeing What’s Hidden in Plain Sight

Another famous psychological illusion is the one conceived of by Daniel Simons and his colleagues called The Monkey Business Illusion. This selective perception task is designed to show how your attention and focus can blind you to other things going on. The video below provides an updated version of the original video (available via the link above) to illustrate how we can miss things that are right in front of us based on what we attend to. 

The Monkey Business Illusion

Innovators are often focused on the core aspects of their innovation: the product, the process, and the intended outcomes. This focus is often what sets successful innovators apart from others, yet it can also be a liability.  An evaluator can help an innovator (an individual, a team, an entire organization) to see the full picture. 

How? The means to do this is first achieved by asking evaluative questions that look at what is going on, what is new, and what meaning is derived from various activities. 

Taking the Temperature (and Other Innovation Measures)

There is a useful, if not morbid, fable about the frog in the boiling pot. In that fable, the frog fails to notice that the temperature of the water continues to rise because the change is so gradual. The frog ends up boiling to death because of this inability to see that he is being boiled alive because the changes are so hard to detect. A good evaluation designed for innovation provides means to collect data akin to a thermostat calibrated to the particular conditions, situations, and product or service necessary. 

This could include monitoring key inputs and outputs, tracking sales or engagements over time, or looking at resource levels like personnel and how they respond to change. The latter example is a good one, particularly with larger organizations where the duties assigned to a single person could be absorbed by others if that individual left the role. However, as time moves on and the extra work is normalized it is possible that the pattern is repeated over time where those who are left eventually shoulder such a burden that it makes the work impossible, but also the performance degrades. 

These are the kinds of situations where collapse is likely. We’ve seen this with leadership when many people start leaving at the top or when those at the front face of an organize leave in large numbers. The cost in capital, focus, lived experience, and working knowledge can cripple an organization over the long haul. But like the frog in the pot, it might not be until just before the boil that it is noticed. 

Evaluating for Illusion & Reality

Evaluation for innovation recognizes that the attention and focus of those leading the charge is precious and that leaders are likely to miss key things without the right tools to draw their attention to the whole system and the parts. Evaluation can be hired to cover some of these perceptual gaps and illuminate things like the present temperature or noticing gorillas, patterns in data like the 8 of diamonds or discerning the best moment to see a young girl or an old woman. 

In the next piece in this series, we’ll look at how evaluation can uncover layers of impact that go beyond seeing what is in front of us to looking far past it. If evaluation for innovation is something that you need help with, connect with us; we can help you see things differently. 

Title image credit: Cameron Norman

Developmental Evaluation Trap #4: Organizational culture

by

In this latest post in a series on Developmental Evaluation (DE) and its traps, we look at the innovator’s trap of culture. In a quote widely attributed to Peter Drucker

Culture eats strategy for lunch

Culture is basically “the way things are done around here”, which includes the policies, practices, and people that make what happens, happen. Culture can be a trap when we fail to prepare for the change in how we do things that comes with adopting a developmental approach.

Just as we might fear success (discussed in an earlier post), we may also not prepare (or tolerate) it when it comes. Success with one goal means having to set new goals. It changes the goal posts. It also means that one needs to reframe what success means going ahead.

Many successful sports teams face the problem of reframing their mission after winning a championship: what it takes to win the first time is different when the motivation, context, and opportunities shift because of that victory. The same thing is true for organizations.

Building a culture of innovation

We’re not naïve enough to think that we can simply offer you a few bullet points and expect you to change your culture overnight. It is a big, important job. However, there is much research on behaviour change and organizational development that we’ve drawn on to help get you started.

  1. Manage deadlines differently. Deadlines drive much of our organizational behaviour and strategy, yet research has shown that deadline-driven work is less satisfying, that people’s decision-making gets impaired under the stress of urgency, and that priority isn’t always given to the more important tasks. Consider developmental milestones, which are similar to what we use in other areas of life. It’s a mindset that looks at tasks, accomplishments, and expectations in an evolving context with adaptive targets.
  2. Planning your stops as well as your starts. Developmental Evaluation can not only help you build out your innovation, but it can also determine what needs to stop. This means making hard choices about what kind of activities you might continue, what needs to be stopped, and what new things you wish to build out. The Tamarack Stop Start Continue tool provides a practical means to guide an organization through the process of making these strategic decisions. Deciding what you will stop is just as important as creating innovations.
  3. Measure performance compassionately. New research has emerged to show that intensive performance measurement can actually hamper the performance of an organization. Collecting too much data or making it onerous to do, not disclosing the purposes of the data, not using data collected, failing to engage those involved in the program in the design of the data collections, or neglecting to collect context-rich data that can help understand the mediators and moderators of performance are ways to alienate people from evaluation and limit its use. A strong DE approach seeks to engage users, making for more effective evaluation overall.
  4. Give data back. Show those involved in the program and the evaluations associated with it what the data reveals and what decisions are made from it. By making evaluation data transparent, you build trust, but also a culture of innovation by showing the purpose of the data and the ways in which it is used to make decisions. This kind of accounting to your users and stakeholders is part of the culture development for innovation.

By getting the right data, using it respectfully, and making it visible are ways in which the culture of an organization can change through developmental evaluation and lead to the kind of system shifts that allow great ideas to emerge and flourish.

Interested in learning more about what DE is and how implementing evaluation in this manner can support your innovation efforts? Contact us to find out more.

Developmental Evaluation Trap #3: Fearing Success

by

What if you tried to innovate and succeeded? This fourth in a series of posts looking at Developmental Evaluation traps explores that question and the implications that come from being good at innovating.

A truly successful innovation changes things — mindsets, workflows, systems, and outcomes. Some of these changes are foreseeable, some are not and they are rarely ever uniformly positive. Take the strange situation that many non-profits face: most will put themselves out of business if they truly succeed in their mission.

Or consider the corporate manager who is looking to get her or his staff engaged. The benefits of staff engagement are many, including contributing more to the organization, but that also means they will ask more of the organization and require more.

The trap with Developmental Evaluation (DE) is that it provides a window into the organizations’ innovation strategy and disrupts the way in which it learns, adapts, and designs its products. It tests assumptions, meaning that some myths that an organization holds might be challenged. DE emplores organizations to use evidence to support decisions, not just precedent.

DE also provides a means to account for the work that is performed in the quest to innovate (not just what is produced), which sounds like an enormous perk for those who are used to being judged solely on production, unless you really didn’t want to put the work in and are content to simply produce the most marginal of products.

What fear looks like

The most problematic form of fear is self-sabotage.  Self-sabotage refers to the behaviours we engage in — individually or as a group or organization — that keep us from taking the actions we believe will lead to success. The reasons might have to do with social norms. If what you’re doing is normal, then it probably isn’t innovation. Innovating means deviating from what others are doing.

Risk is another fear-factor. Doing something different requires risk-taking and that means holding yourself up to possible criticism, disapproval from others, or financial costs (among others).

Further complicating things is that risk-taking and social behaviour are often linked together.

Fear also manifests in actions not taken. It may manifest in not pursuing opportunities or pursuing only those that are safe. It may be ignoring problems or asking the kind of questions that focus attention away from what is important, to what is easy. Often we see organizations seek to collect lots of data on something unimportant because it’s safer than collecting data on something that matters but is much riskier.

Fear can further manifest in things like over-researching a topic (never having ‘enough’ data’), deferring decisions, blaming the timing (“It’s just not the right time to innovate”), or complexity (“it’s too complex”).

Practical fear-fighting

How do guard against your own fear of success? There are some things we can do to better utilize lessons gained through approaches like DE to assist innovation efforts.

  1. Innovation therapy. Mental health professionals often go through and remain in some form of professional therapy or guided reflection to enable them to attune to their biases, work through their personal issues, and hone their skills with another professional. The same process of ongoing, guided, reflective practice and ‘treatment’ is beneficial to organizations as well because it creates that same social norm to talk about fears, challenges, and ambiguities. Create space for reflection, discussion, and problem-solving in your regular meetings. Make the time for innovation.
  2. Create outcomes from processes. DE is a great tool for showcasing the work that goes into an innovation, not just the innovation itself. Regularly collect the data that goes into the process — ideas generated (quality and quantity), rough sketches, prototypes, and hours worked, to name a few suggestions. These can help show what goes into a product, which is an end in itself. This makes something that seems abstract more tangible.
  3. Make change visible. Make work visible through evaluation and visual thinking – including the ups, downs, sideways and showcase where along the journey you are along a process (e.g., use a timeline)
  4. Create better systems, not just different behaviour. Complex systems have path-dependencies — those ruts that shape our actions, often unconsciously and out of habit. Consider ways you organize yourself, your organization’s jobs and roles, the income streams, the system of rewards and recognition, the feedback and learning you engage with, and composition of your team.  This rethinking and reorganization are what changes DNA, otherwise, it will continue to express itself through your organization in the same way.

These are all part of the developmental evaluator’s toolkit, which is basically a core part of innovation itself. DE can be a means to help you confront your fears, not deny them. Try these out and see how you can avoid the trap of self-sabotage and to become better innovators and learners.

Want to learn more about how to do DE or how to bring it to your innovation efforts? Contact us and we’d be happy to help.

 

 

Building a developmental mindset

by

Build your developmental mindset

Innovating in human systems requires more than ideas, resources, clever strategy or even new thinking; it requires a new mindset.

A mindset is a concept that psychologist Carol Dweck has explored in-depth and described a thinking orientation to the world. It’s not set in stone, but rather a lens through which we see our situations and the possibilities and challenges that they present to us. While Dweck has identified particular mindsets that shape how individuals use their brains, we’d like to explore another mindset here: a developmental one.

Over at Censemaking, Cameron Norman considered the role of the developmental mindset and how organizations engage with developmental evaluation. In that post, Norman argues that developmental evaluation is not possible without having the right mindset in place and we couldn’t agree more.

Witnessing evolution

The critical step toward a developmental mindset is considering viewing programs as a part of living systems. If a program involves a high variety of information and feedback, interacts with other programs and social forces, and exists in a space where there are ongoing disruptions and change, then it is what we would consider complex. If it is complex, then it fits the kind of scenario argued in Censemaking.

To take an evolutionary or developmental mindset requires first viewing programs as dynamic, rather than static. This perspective involves asking different evaluative questions. Static programs allow for questions like:

  • How effective is [the program] at achieving [specific target]?
  • How well does [the program] fit the industry-standard benchmark?
  • What is the impact of [the program] on [specific behaviour]?
  • How did [program component] contribute to [specific outcome]?

These questions allow for answers that have verifiable correlates with expected outcomes, a theory of change and industry benchmarks.

Dynamic programs require asking more sophisticated, nuanced questions that might produce less precise answers, but are more useful in helping a program to develop and evolve. These include:

  • What outcomes are generated through [the program] at this given moment?
  • How have the outputs and possible outcomes of [the program] differed over time?
  • How is [the program] interacting with its social system and what new behaviours and activities are generated through those interactions?
  • What new, beneficial attractors (clusters of organized activity) have established that are attributed in part to the involvement of [the program]?

Seeing opportunity in complexity

A review of the questions above might seem less satisfying in some ways, but they also reveal possibilities. It is why developmental evaluation (DE) is known as evaluation for innovation. DE provides the feedback necessary to understand the role(s) that a program is playing within a system and what comes from the interaction between people, programs, and systems. Thus, DE is a valuable part of the innovation process by allowing organizations to test out ideas and see what kind of influence they have in the world and make adaptations and modifications to it as the conditions, situations, and requirements evolve.

DE is about evaluating programs as part of living systems. This approach recognizes that programs exist in dynamic systems akin to the phrase ‘the river I stand in is not the river I step in’ from Heraclitus. This mindset recognizes that your market competition will change as you change and react to your actions, just as your organization might have to react to their actions.

DE doesn’t solve the problem, but it does allow for new opportunities to be discovered. By asking different questions to see things different, you can see these opportunities, too.

To learn more about developmental evaluation and how the right approach to generating feedback and learning about your programs can help your organization contact us for a consultation.

Image credit: “Mindset” by Steve Davis is licensed under CC BY 2.0

Engagement for ideas

by

Engage everyone in idea creation

Among the biggest success factors in any organizational change initiative is engaging staff and stakeholders in participating in the change process; this is true for idea generation through the design cycle to implementation and evaluation. How do we overcome the challenge of disengagement to produce productive creative, innovative ideas in our organizations?

Beyond brainstorming

Ideas are usually the starting point for any change initiative. Ideas produce the raw content of problem identification and the seeds for solution generation. Brainstorming is often the means people first think of as a way to generate ideas and explore concepts. However, there are some substantial problems with this approach and some ways around it.

Brainstorming has been widely criticized for good reasons. Among them: it favours those voices who think (and speak) quickly, speak early, often and loudly. Those early suggestions drive the conversation for what comes next, creating a path dependency that’s hard to escape once initiated. If you’re a quieter, perhaps more contemplative person, you’ll find that you are either late to the conversation or not included. From Jungian Personality Theory, this indicates a bias towards extroversion over introversion, which excludes about 40 per cent of the population according to some estimates.

This approach also favours what Min Basadur would classify as an ‘generator’: someone who’s work style preferences favour idea generation. Those from the other quadrants in the Basadur Profile, particularly ‘conceptualizers’ and ‘optimizers’ (ones who’s work style preference leans toward processing and organizing information) are less likely to respond quickly.

Ideas in private and public

Another problem with many ideation strategies is that many really useful ideas are a bit heretical or outlandish. While design thinking writers often refer to the need to generate ‘wild ideas’ without criticism or judgement, the truth is that’s a lot harder to do in practice, particularly when there are power dynamics in the room, reputations, and the real fear that comes with change and challenges to established practices. The public nature of the ideation process favours transparency, but induces self-censorship and produces disengagement for many and hyper-engagement for a few.

A solution is to have those involved in the process generate ideas independently and anonymously contribute them via a suggestion box (digital or analog). Keep the suggestion box open for a defined period — we recommend no more than one week or as short as three days. This allows time for those who are more reflective to mull through their ideas and contribute them, while those who are more quick to generate ideas will not be affected.

These are then collated independently by someone neutral to the problem and solution set and organized thematically. This has the advantage of potentially embedding some of the wildest ideas within a small set of other ideas that might seem far less threatening. If you create a category of ‘wild ideas’ the risk is that the entire category will be dismissed. A more public, but elaborate, means of doing this can be found in the CoNEKTR Model described elsewhere.

This process can be repeated over the design cycle at different stages – anywhere diverse perspectives and feedback is needed.

Overcoming biases to better ideas

The bias toward ‘rapid ideation’ in design thinking systematically excludes people in favour of a well-meaning intention of trying to avoid participants ‘over-thinking’ a problem. While that might happen, it still prevents many from engaging in the process fully because of personality, work preferences, cognitive style and social pressures.

The strategy listed above is a simple, but highly effective means of getting lots of ideas and engaging your entire team in the process. Try it out. You might be surprised what ideas come from it all.