Cense

Innovation to Impact

Evaluating Learning for Innovation

by

Learning is at the heart of innovation. Our definition of innovation is contingent upon it:

Learning transformed into value, by design

Traditional approaches to learning tend to focus on two things: recall and performance. Thus, we look at knowledge-based tests (consider most of what we took in grade school and college or university) or performance assessments (reviewed accomplishment on a specific task or problem set).

What is often absent from both of these is a consideration of the context and the learner themselves. It’s one thing to be exposed to information or demonstrations of how to do something, yet it’s entirely another thing to actually learn it.

Assessment of Learning: Questions to Ask

Our assessment of learning is inspired by this approach and informs the questions we ask about learning:

  • Exposure. What level of exposure did we have to the material or experience? How did this exposure treat our sense of understanding of the problem and the context? What salient information did we attend to during this exposure? How (or were) we attentive to what was presented to us and what was the context in which this was presented?
  • Connection. How is what we are exposed to related to what we’ve already experienced? In what ways does this new material fit with what we have previously learned? Where does this material or experience fit in shaping my goals, intentions, or general direction for myself or my organization?
  • Reinforcement. What activities allow me to apply what I’ve been exposed to in new ways and in practical ways repeatedly (including fitting with what I already know, whether a theory or practice-focused material)? What systems are in place to allow me to integrate what I’ve been exposed to into my current understanding of the situation? What activities have been set out to enable recall and re-visiting materials in the near and longer-term?
  • Application. What opportunities do I have to apply what I’ve learned into shaping a new understanding of a problem or steps toward addressing it? What opportunities are present to reflect and integrate this application of knowledge or skill into my existing repertoire of practice or understanding?

Without these, there is a decent chance we will undergo some form of performative learning — doing activities that look like learning but aren’t really effective. Taking workshops on topics that are not relevant to your work or set up too distant from the problems you’re seeking to tackle is one way. Filling your days with reading articles, watching webinars, taking courses, listening to podcasts, and viewing videos on some material without a means to organize and integrate that into your experience.

You will retain some, but probably not enough to warrant your time and energy. Furthermore, this kind of ‘filling’ of your attention actually serves to decrease the likelihood that you’re able to use this because of the sapping of energy from your brain. The more we expose ourselves to, the more energy is required and eventually we’ll be depleted enough that things won’t stick.

Ask yourself these questions above next time you’re looking to really, truly learn as an individual, team, or organization.

If you’re looking to learn more, do more with what you have, please contact us and we’d be happy to discuss ways we can help you be a better learner and innovator.

Better learning comes from better systems, by design. we can help you build them.

Inspiration As Outcome

by

Anyone working in design or consulting knows that no amount of advice will guarantee action with your clients or colleagues. Great, thoughtful designs are important to success yet are not guarantees for adoption.

What is critical above all is that designs inspire people. Without inspiration and excitement, there is no action.

Designer Bruce Mau came to this realization during his teaching when he was asked by trainees how it was that he designed and while he was able to speak to the production and philosophy behind his work, he was stuck with how to describe the process of the work. It was in reflecting upon this process that he came to realize that without inspiration design was dead. Prototypes would not get realized. Design culture within an organization would not change and the kind of transformation and innovation requested by clients and communities would never materialize.

For our clients, we are recommending that inspiration be considered the primary outcome of any project. No matter what kind of process is undertaken to create something — whether it was co-design, expert-driven, or some other model — the end result must be that what you produce must inspire people.

That will not guarantee success, but without it there is near guarantee of failure.

Consider inspiration as the precursor to any kind of innovation adoption or impact and you may find the correlations between those projects where people felt inspired, saw a future vision and felt motivated to act and their overall success rate in being implemented is close to 100 percent.

Narration Interrogation

by

Police (and some parents) know the secret to spotting a lie in a story: ask someone to repeat that story backwards. As it turns out, it’s a lot easier to concoct a false story going forward than it is backward because of the way we logically connect events in our heads.

This same technique can be used to help spot gaps in logic. Even if we’re not lying to one other (or ourselves) we may find some parts of the story that don’t quite make sense. This gap in logic is not uncommon because as humans we often will fill in the story because of how we are wired for narrative coherence as a species.

Narrative interrogation is a way that we can walk through the story of our program or service to help us identify the key elements that are present in most good stories and how or whether we have them organized (or have them at all). Unlike real interrogation, this is not aggressive or adversarial — rather a way to explore stories through inquiry.

Story Elements

Some of the key elements in a good story are:

  1. Actors. These are your protagonists (the leads), the supports, and the chorus — those in the background. Ask yourself who the main actors are in each scene of the story (e.g., who has the problem that needs solving? What are they looking for? What is their motivation?). This is where using personas can be helpful to fill in the details about these characters.
  2. Relationships. How are the actors related to each other? Are they working collaboratively or competitively and do they need each other? Are there roles that individuals fill? Are there special qualities to their relationship (e.g., power, partnership, etc.).
  3. Setting and Structures. Where are things taking place? Do people need a particular service or product in a specific setting or context? Articulating these will also help you to frame the way in which system structures shape the interactions between the actors and help contribute to or facilitate the problem (or solutions).
  4. Time. Determining when things are to happen and how that temporal aspect shapes everything is important. Does timing matter? Does the amount of time matter? is the problem and solution one that is highly dependent on when something happens or not?
  5. Arc. The last piece is creating some form of coherent story arc between them all. Tying them together helps us understand who is involved, what they are interested in or seeking, why they have the challenges they do, how they are going about things now (and how we could change that with an intervention of a product or service) and the ways in which that will be affected.

Together, this starts to generate a theory of change and helps us connect what we’re seeking to do through our innovation (service, product, policy) and what is needed by those we are aiming to serve.

Using the Method

Stories are told by people, not objects, so this is one method where speaking with individuals is key. Involve those for whom the story matters in the telling of that story. This might be customers or clients, service operators, managers, or founders; it depends on what story you are looking to hear. The aim is usually not to capture everything, rather keep it focused on a specific aspect of your innovation. It might be in use (customers or clients), the development (product team) and marketing, or in understanding the purpose relative to the organization (e.g. senior management).

Using an open-ended approach — free-form — ask people to speak about the topic using a story lens:

  1. Start with the beginning: what is the first thing someone needs to know. This might be the choice to start the project, the moment the ‘problem’ appeared that required a solution, or even the backstory. This is something that the storyteller determines on their own.
  2. Focus. Encourage the person to speak in a manner that focuses on the purpose, however, ask points of clarification when it is unclear what the connection is at different parts. Good stories often involve non-sequiturs and so do poor ones; it’s important to know which one it is.
  3. Reflect back. Once the story is told, re-cap the logic of the story from front to back and
  4. Go backwards. This is the ‘interrogation’ part of sorts. Ask people to retell the story backward from the end. For example, ask what happened right before the conclusion of the story and then what happened before that and before that. It’s similar to the reverse of A Day in the Life method.

What you might find is that the story has different descriptors, relationships, or emphasis when told backward. These allow us to see different configurations of the issues that are associated with the story. It’s not that the person is necessarily lying or keeping anything from you, it’s about the limitations of narrative in that it only works with one set of issues connected logically at a time. Going backward allows us to see things differently, expanding our view.

The interviews and conversations with those involved should be informal and relaxed and can go into as much depth as you want. Generally, this is an approach that makes for a good ‘coffee conversation’ of about 30 minutes. It also can be done remotely, if necessary. It can be done internally by staff associated with the project or externally by an outsider. If the story involves highly sensitive subject matter or material, it is best to use an outsider to the project.

Learn more from your program, your people, and your work with this simple, powerful method for design exploration and research.

We help with storytelling through data. Contact us if you want to implement this with your organization.

After-Action Review: Learning Together Through Complexity

by

Complexity science is the study of how systems behave when under conditions of high dynamism (change) and instability due to the number, sequencing, and organization of actors, relationships, and outcomes. Complex systems pose difficulty drawing clear lessons because the relationship between causes and consequences are rarely straightforward. To illustrate, consider how having one child provides only loose guidance on how to parent a second, third or fourth child: there’s no template.

An After-Action Review is one such way to learn from actions you take in a complex system to help shape what you do in the future and provide guidance on what steps should be taken next.

The After-Action Review (AAR) is a method of sensemaking and supporting organizational learning through shared narratives and group reflection once an action has been taken on a specific project aimed at producing a particular outcome — regardless of what happened. The method has been widely used in the US Military and has since been applied to many sectors. Too often our retrospective reviews happen only when things fail, but through examining any outcome we can better learn what works, when, how, and when our efforts produce certain outcomes.

Here’s what it is and how to use it.

Learning Together

An AAR is a social process aimed at illuminating causal connections between actions and outcomes. It is not about developing best practice, rather it is to create a shared narrative of a process from many different perspectives. It recognizes that we may engage in a shared event, but our experience and perceptions of that event might be different and within these differences lie the foundation for learning.

To do this well, you will need to have a facilitator and a note-taker. This also must be done in an environment that allows individuals to speak freely, frankly, and without any fear of negative reprisals — which is a culture that must be cultivated early and ahead of time. The aim isn’t to point blame, but to learn. The facilitator can be within the team or outside the team. The US Military has a process where the teams undertake their own self-facilitated AAR’s.

To begin, gather those individuals directly involved in a project together in close proximity to the ‘end’ (e.g., launch, delivery of the product, etc..). As a group, reflect on the following three question sets:

  1. The Objectives & Outcomes:
    • What was supposed to happen?
    • What actually happened?
    • Practice Notes: Notice whether there were discrepancies between perceptions of the objective in the first place and where there are differences in what people pay attention to, what value they ascribe to that activity (see below), and how events were sequenced.
  2. Positive, Negative, and Neutral Events:
    • What created benefit / what ‘worked’ ?
    • What created problems / what ‘didn’t ‘work’?
    • Practice notes: The reason for putting ‘work’ in quotes is that there may not be a clear line between the activities and outcomes or a pre-determined sense of what is expected and to what degree, particularly with innovations where there isn’t a best practice or benchmark. Note how people may differ on their view of what a success or failure might be.
  3. Future Steps:
    • What might we do next time?
    • Practice notes: This is where a good note-taker is helpful as it allows you to record what happens in the discussion and recommendations. The process should end with a commitment to bringing these lessons together to inform strategic actions going forward next time something similar is undertaken.

Implementing the Method

Building AAR’s into your organization will help foster a culture of learning if done with care, respect and a commitment to non-judgemental hearing and accepting of what is discussed in these gatherings.

The length of an AAR can be anywhere from a half-hour to a full-day (or more) depending on the topic, context, and scale of the project.

An AAR is something that is to be done without a preconceived assessment of what the outcome of the event is. This means suspending the judgement about whether the outcome was a ‘success’ or ‘failure’ until after the AAR is completed. What often happens is successes and ‘wins’ are found in even the most difficult situations while areas of improvement or threats can be uncovered even when everything appeared to work (see the NASA case studies with the Challenger and Columbia space shuttle disasters).

Implementing an AAR every time your team does something significant as part of its operations can help you create a culture of learning and trust in your organization and draw out far more value from your innovation if you implement this regularly.

If your team is looking to improve its learning and create more value from your innovation investments, contact us and we can support you in building AAR’s into your organization and learn more from complexity.

Design for Living: A Day in the Life

by

Product and service developers can easily be fooled into thinking all they need to focus on is the moment of engagement with their product. The design method “A Day in the Life” can help us put our potential audience (customer, client, or “user”) into a clearer perspective.

A Day in the Life is a simple activity that seeks to catalogue the activities and contexts that your audience might engage in within a typical day to help shed light on the life circumstance and situations that could influence your product.

Begin at the Beginning

Let’s illustrate this simple method with an example: education and training. When we design for education and training, the actual service might be a class, webinar, or workshop. However, the total experience of learning may involve much more than that.

Rather than assume your service starts at the moment people sit down (in person, at the computer etc..) go back to the start of their day.

Start with imagining a ‘user’ — be as specific as possible about this person with as much detail as you can provide that reflects a ‘typical’ or a particular (e.g., specific segment) service or product user.

Then ask: What happened the moment they woke up?

This question tells you a lot and invites other questions: Did they get a good sleep? What were the conditions that they slept in? What time did they wake up?

This matters because one of the assumptions behind your education and training service might be that people are attentive, able to listen and process the material, participate when necessary, and able to codify what’s learned into their brain and apply that later to whatever problem is at hand.

If you want your service to be useful, it needs to fit the circumstances of your user. If your participants didn’t sleep well, had to get up early to commute, are living in a state of fear or violence, or have no good place to sleep at all they are already facing some challenges before they start.

Continue the Story

The first question will lead you to a series of other questions that continue with: What happened next?

You continue this story as you progress through the day in the life of your participant up to and through the actual service event you’re involved in. After that? Continue the story through to the end of the day.

Along the way you will identify such things about your audience like:

  • Demographics
  • Social life and network
  • ‘Touchpoints’ with other systems and services
  • Preferences
  • Social and psychological circumstances.

These are imaginations of sorts based on what you think is a ‘typical user’. To increase the likelihood of reflecting the experience of a diversity of users it is best to conduct some background research to ensure you are reflecting the true characteristics of your audience. This method also works for identifying qualities about non-typical or non-users to help you understand why they might not use or desire your product or service.

Putting it into Practice

This exercise is best done as a group and can be conducted within 2 hours comfortably with more time for more granular exploration. It is meant to be participatory, engaging and allow for some creative reflection.

Materials include:

  • Whiteboards or large flipchart paper
  • Markers
  • Sticky notes
  • Stickers (optional)

Over the course of a morning or afternoon, you can bring your team into a place of greater understanding of your users — current and potential — and help set the context for your service. If we consider our example of education and training, the lessons we learn from this might be that we break programming into different chunks, change the distribution model, provide additional or alternative means to access content, or perhaps follow-up with reminders and tips to aid memory or application.

This simple, engaging and powerful method will help you tell better stories about your product or service and those of the people you wish to influence and serve.

A Day in the Life is one of the methods that we teach as part of the Design Loft Experience pop-up held as part of the annual American Evaluation Association annual conference each year. It’s one of many methods we use to help our clients understand the bigger picture and gain new insights into their work. Want help implementing it? Contact us — this is what we do.