Cense Ltd.

Inspiration, Innovation and Impact

Developmental Evaluation, Learning and Innovation

by

What does it mean to undertake a developmental evaluation (DE)?

DE is simply another way to say evaluation for learning and innovation. That’s really all it is in practice, although anyone looking to do DE might want to look a little closer to see if it really is the right approach for them.

In this post we look a little closer to help you understand what it means to do evaluation for innovation and what developmental evaluation has to do with it.

Understanding Innovation


Innovation is a word that can mean much and requires clarification when it’s used because of this. When we speak of innovation, we refer to this definition of learning transformed into value through design.

Developmental evaluation is a concept that was first put forward by Michael Quinn Patton through many books, articles, and other teachings. It’s now become a popular approach within social innovation and the public sector.

DE is often referred to as a means to strategic learning because rather than assume the conditions of whatever is being evaluated (e.g., program, policy) are stable, it assumes some level of complexity. This is dramatically different from what most other forms of evaluation do where the assumption is the ‘thing’ being evaluated is stable – it’s the users of that thing that might change.

Because DE is focused on learning it assumes that there will be some kind of change with the actual process of learning itself on the way from one state to the next. 

Understanding Design’s Role in Innovation

What DE does not do well is account for the role of design in supporting innovation. If we come back to the definition of innovation presented earlier, design is really the means by which innovation occurs. In most of the literature on DE the matter of design is not discussed much, if at all. 

With much of DE the assumption is that those doing DE know how to take what they learn and transform it into value. From our experience, this is an assumption that rarely holds up. The reasons are many, but most central to this is that most people think of design as beginning from a standing start when in practice much of what do when we innovate takes place from a moving position.

We sometimes refer to this as developmental design, but more appropriately expand how we see this relationship between DE and design as something called Design-driven Evaluation. In this case, design and evaluation are intimately connected and the processes of using both to mutually inform one another are embedded in the innovation process. 

Understanding Evaluation’s Role in Design & Innovation

Evaluation is the means to provide the necessary feedback to contribute to innovation. Evaluation determines the quality of the design in achieving its purpose. If your design is meant to change behaviour, evaluation is the means to assess whether that happens and to what degree.

Evaluation also is the means to document the circumstances in which this takes place to help you assess whether the implementation of your innovation goes as intended. Without evaluation, your innovation is a wish, not a result. Evaluation is the means to determine what the results of your innovation are. 

Evaluation is also the means to assess the quality of your design work. It’s one thing to develop something and put it out in the world, but is it any good? Is it fit for purpose? Many innovations fail to deliver value because they don’t provide the means to learn about the quality of the design itself. They don’t achieve a positive impact because they were never designed to in the first place. 

To innovate is to learn and to learn is to evaluate. Design is what ties them all together. 

Next time you’re looking to embark on a new program of activity, create a new product, it’s worth considering how these all go together and whether DE alone will help you to get there. 

For help in generating powerful design-driven approaches to innovation and evaluation, contact us. We can help

Multimedia Visualizations For Strategy and Impact

by

Multimedia visualizations are powerful ways to convey a lot of information in a small space and with little time. The use of a visual — a map, a drawing, a picture, sketch, or collage — provides layers of information beyond words. When dealing with complex situations, visualizations provide clear communication pathways.

Unlike some of our other posts, this is not pointing us to a specific tool or technique, rather a general approach to thinking about and engaging with content.

Let’s look at the benefits of adding more visuals to your work to help your strategy, evaluation, and program design efforts.

1. Relationships. One visuals can help you with understanding proportions, distance, and other relative characteristics. It is useful to position things next to one another – whether that is choice alternatives, network connections, or even contrasts. Seeing things in relation to one another on a visual canvas provides our brains with new information that is difficult or impossible to gain from more abstract thinking without visuals.

2. Colour and Texture. How something looks and feels can tell us a lot about what it means. By adding colour to something we apply a simple lens that can convey meaning. Colours like green invoke environmental imagery, red is well-known to have many meanings in different cultures, while blue can be relaxing. Texture can do the same thing. To illustrate, consider New Zealand-based Icebreaker who makes wool-based clothing and uses texture-rich images of sheep’s wool to highlight how its products are natural in composition. Use of texture in the imagery gives a sense of integrity, warmth, and a feel to fabrics even when the buyer might only see them online.

3. Variety. As more people work remotely and digitally than ever before, the visual landscape for many of us has changed in most workdays. We lack contrast in our environments and also in our digital screens. By increasing the visual constraint and variety by diverging from text and boxes toward photographs, graphic images, videos, or some other form of visual media we draw attention. When so much feels the same, few things are more valuable than attention.

4. Abstraction and Narrative. Words offer us a restricted set of options due to the need for us to be linear in our the way we speak. With a visual we can better draw abstract themes from an object and use things like metaphor to describe scenarios that allow us to transcend words.

There are many ways to use visuals and software tools like online whiteboards such as Miro and Mural, visual organizing tools like Milanote, and tools such as Jamboard or Trello to showcase information in ways that extend beyond text.

Consider bringing more details into your next project and discover what visuals can do for creating richer, more vibrant ideas and creative opportunities to your work.

The Art of Gathering

by

Innovation work from identifying problems in need of solutions, generating ideas, creating things, and simply providing the inspiration and mutual support for doing the work often requires gathering.

Whether it is face-to-face, online, or some hybrid option the same general rules apply for what makes a good gathering when it comes to the up-front work.

Priya Parker has made the art of gathering a focus of her life’s mission and work and has pulled together some brilliant resources for those of us who are looking to bring people together.

These opening questions are the starting place by asking some fundamental questions about the purpose of gathering (something too often forgotten about).

Here are Priya’s recommended questions to help guide you in gathering:

  • Who is this for? What is it that this person loves to do? 
  • What are the various gifts or talents or skills of the people invited? 
  • How do you actually want to spend the time? 
  • How will you create a “moment of focus”? 
  • If this is a digital gathering, what are simple ways you can have guests bring something or wear something that connects them to the purpose, the person of honor and each other?  

Among the most valuable assets engaged in any innovation effort is time and attention and by asking these simple questions we can best use and respect both.

Perspective Taking Circles

by

The power of perspective is one of the things that differentiates high output and impact innovators from others. It’s easy to get lucky or have good timing, but it’s another to create value when those things don’t happen.

One of the ways we do this is by engaging in some perspective-taking. This simple exercise and question set can help build some of the ‘thought muscles’ that can help encourage us to see, imagine, and engage things differently for creative benefit.

The Exercise

This works best when physically in the same space and can work online as well. What you need is to create a space where people can re-position themselves against a central object that can be almost anything except a ball (because it looks the same from every angle). Place your participants around the object with a comfortable space to sit or stand.

You can do this virtually using a dynamic space like Kumospace or some other virtual reality-like environment. It can also work using a board like Miro or Mural with some designed object in the middle, but it is more awkward.

The idea is that everyone has a chance to literally see things from a different point of view.

This can be done as an observation exercise but is more enhanced when it is combined with drawing. Asking people to draw what they see — no matter what kind of skills or abilities participants have with sketching — is a great means to engage people in thinking more deeply about their perspective.

Once individuals have had time to observe and reflect on what they see, the next step is to have everyone share their perspective. This is where drawings are useful as people can speak to what they drew as drawing focuses us on certain elements and provides a means to account for those perspectives. It also allows others to point to the drawing and make specific, not general comments.

It’s that specificity that is key to illuminating and articulating differences of perspective.

Uses

The role of this method is to reveal how where we sit in a system — even a small one of people interconnected around a shared experience of an object — can have remarkably different perceptions of the same thing in the same space.

It begins to build cultural practices around creating space for exploring and sharing perspectives within an organization and can serve as a base for better organizational design and learning.

It’s simple, engaging, and revealing in its method.

Evaluating Learning for Innovation

by

Learning is at the heart of innovation. Our definition of innovation is contingent upon it:

Learning transformed into value, by design

Traditional approaches to learning tend to focus on two things: recall and performance. Thus, we look at knowledge-based tests (consider most of what we took in grade school and college or university) or performance assessments (reviewed accomplishment on a specific task or problem set).

What is often absent from both of these is a consideration of the context and the learner themselves. It’s one thing to be exposed to information or demonstrations of how to do something, yet it’s entirely another thing to actually learn it.

Assessment of Learning: Questions to Ask

Our assessment of learning is inspired by this approach and informs the questions we ask about learning:

  • Exposure. What level of exposure did we have to the material or experience? How did this exposure treat our sense of understanding of the problem and the context? What salient information did we attend to during this exposure? How (or were) we attentive to what was presented to us and what was the context in which this was presented?
  • Connection. How is what we are exposed to related to what we’ve already experienced? In what ways does this new material fit with what we have previously learned? Where does this material or experience fit in shaping my goals, intentions, or general direction for myself or my organization?
  • Reinforcement. What activities allow me to apply what I’ve been exposed to in new ways and in practical ways repeatedly (including fitting with what I already know, whether a theory or practice-focused material)? What systems are in place to allow me to integrate what I’ve been exposed to into my current understanding of the situation? What activities have been set out to enable recall and re-visiting materials in the near and longer-term?
  • Application. What opportunities do I have to apply what I’ve learned into shaping a new understanding of a problem or steps toward addressing it? What opportunities are present to reflect and integrate this application of knowledge or skill into my existing repertoire of practice or understanding?

Without these, there is a decent chance we will undergo some form of performative learning — doing activities that look like learning but aren’t really effective. Taking workshops on topics that are not relevant to your work or set up too distant from the problems you’re seeking to tackle is one way. Filling your days with reading articles, watching webinars, taking courses, listening to podcasts, and viewing videos on some material without a means to organize and integrate that into your experience.

You will retain some, but probably not enough to warrant your time and energy. Furthermore, this kind of ‘filling’ of your attention actually serves to decrease the likelihood that you’re able to use this because of the sapping of energy from your brain. The more we expose ourselves to, the more energy is required and eventually we’ll be depleted enough that things won’t stick.

Ask yourself these questions above next time you’re looking to really, truly learn as an individual, team, or organization.

If you’re looking to learn more, do more with what you have, please contact us and we’d be happy to discuss ways we can help you be a better learner and innovator.

Better learning comes from better systems, by design. we can help you build them.