Cense

Innovation to Impact

After-Action Review: Learning Together Through Complexity

by

Complexity science is the study of how systems behave when under conditions of high dynamism (change) and instability due to the number, sequencing, and organization of actors, relationships, and outcomes. Complex systems pose difficulty drawing clear lessons because the relationship between causes and consequences are rarely straightforward. To illustrate, consider how having one child provides only loose guidance on how to parent a second, third or fourth child: there’s no template.

An After-Action Review is one such way to learn from actions you take in a complex system to help shape what you do in the future and provide guidance on what steps should be taken next.

The After-Action Review (AAR) is a method of sensemaking and supporting organizational learning through shared narratives and group reflection once an action has been taken on a specific project aimed at producing a particular outcome — regardless of what happened. The method has been widely used in the US Military and has since been applied to many sectors. Too often our retrospective reviews happen only when things fail, but through examining any outcome we can better learn what works, when, how, and when our efforts produce certain outcomes.

Here’s what it is and how to use it.

Learning Together

An AAR is a social process aimed at illuminating causal connections between actions and outcomes. It is not about developing best practice, rather it is to create a shared narrative of a process from many different perspectives. It recognizes that we may engage in a shared event, but our experience and perceptions of that event might be different and within these differences lie the foundation for learning.

To do this well, you will need to have a facilitator and a note-taker. This also must be done in an environment that allows individuals to speak freely, frankly, and without any fear of negative reprisals — which is a culture that must be cultivated early and ahead of time. The aim isn’t to point blame, but to learn. The facilitator can be within the team or outside the team. The US Military has a process where the teams undertake their own self-facilitated AAR’s.

To begin, gather those individuals directly involved in a project together in close proximity to the ‘end’ (e.g., launch, delivery of the product, etc..). As a group, reflect on the following three question sets:

  1. The Objectives & Outcomes:
    • What was supposed to happen?
    • What actually happened?
    • Practice Notes: Notice whether there were discrepancies between perceptions of the objective in the first place and where there are differences in what people pay attention to, what value they ascribe to that activity (see below), and how events were sequenced.
  2. Positive, Negative, and Neutral Events:
    • What created benefit / what ‘worked’ ?
    • What created problems / what ‘didn’t ‘work’?
    • Practice notes: The reason for putting ‘work’ in quotes is that there may not be a clear line between the activities and outcomes or a pre-determined sense of what is expected and to what degree, particularly with innovations where there isn’t a best practice or benchmark. Note how people may differ on their view of what a success or failure might be.
  3. Future Steps:
    • What might we do next time?
    • Practice notes: This is where a good note-taker is helpful as it allows you to record what happens in the discussion and recommendations. The process should end with a commitment to bringing these lessons together to inform strategic actions going forward next time something similar is undertaken.

Implementing the Method

Building AAR’s into your organization will help foster a culture of learning if done with care, respect and a commitment to non-judgemental hearing and accepting of what is discussed in these gatherings.

The length of an AAR can be anywhere from a half-hour to a full-day (or more) depending on the topic, context, and scale of the project.

An AAR is something that is to be done without a preconceived assessment of what the outcome of the event is. This means suspending the judgement about whether the outcome was a ‘success’ or ‘failure’ until after the AAR is completed. What often happens is successes and ‘wins’ are found in even the most difficult situations while areas of improvement or threats can be uncovered even when everything appeared to work (see the NASA case studies with the Challenger and Columbia space shuttle disasters).

Implementing an AAR every time your team does something significant as part of its operations can help you create a culture of learning and trust in your organization and draw out far more value from your innovation if you implement this regularly.

If your team is looking to improve its learning and create more value from your innovation investments, contact us and we can support you in building AAR’s into your organization and learn more from complexity.

Simple Systems Scoping

by

Systems thinking done broadly allows us to take into consideration the various factors — structures, activities, relationships, interconnections — that can influence our organization, market, and domain of inquiry.

One of the fundamental qualities of systems is that they have boundaries. For example, when we consider an organization as a system we need to place boundaries on that organization such as who to include (e.g., all employees? full-time vs part-time? paid staff vs volunteers? settings or sites? etc..). There is no correct choice, just a useful one. Your boundary choices are to reflect what you are seeking to understand and how you’re seeking to act.

But how do you tell? We share with you a remarkably simple, but powerful way to scope your systems and determine if you have set the right boundaries.

Two Criteria

If you set your boundaries of inclusion in the system and find that you are lost and struggling to identify, map, or monitor the various interconnections, actors, actions, and outcomes within a system because there is too much to focus on then that is a sign you have bounded your system too loosely.

If you’re continuously finding yourself trying to explain what happened in the system by things outside of the boundaries, then you have bounded your system too tightly.

That’s it.

It may take some experimentation to get your boundaries right, however these two criteria can tell you if you’re on the right track or not.

Systems-informed strategy, mapping, and evaluation can be complicated, but understanding the boundaries does not have to be. This simple strategy has consistently allowed organizations to focus on what matters and avoid getting lost. The key is to make sure you have the ‘just right’ amount of detail and focus to allow you to make a meaningful sense of things and guide your action.

If you want help implementing a systems strategy for innovation and change in your organization or network, contact us.

Acting in Complex Times

by

The complexities and complications of circumstances tied to the COVID-19 pandemic represent a hyper-exaggerated version of situations organizations find themselves in moments of disruption due to economic, social, and technological shifts. It is a moment of innovation.

What makes the current situation distinct is that the issues are global. Usually we have safe refuge in a new market, region, or setting, but now we don’t. How can we develop or implement strategy when things are continually changing for us and our partners, suppliers, and customers or clients?

Smallest Visible Systems (SVS)

Systems thinking helps us to understand how things are connected and structured, while complexity science can help us appreciate the challenges associated with how to understand the nature of the problems that present themselves. What both can teach us is that in times that are truly unprecedented in their complexity and scope it can be difficult to know what to do and how to act.

The temptation with systems approaches to strategy is to look at the whole system, but that is dependent upon whether we can see the boundaries of the system to help us understand the range of activities we need to consider in developing a model to guide us.

  1. Coherence is what we are seeking. In order to achieve coherence, we need to take some kind of action (often called a probe) and then see what that does. This helps us to examine how the system is behaving and how an action generates reactions and where (or whether) coherence forms. Coherence is basically a way of saying that things go together with some manner of alignment where
  2. Seeing or creating coherence is about meaning and meaning is context-dependent. What is meaningful for us depends on our circumstances, but it also provides us with a means to focus our attention amid the various signals we’re getting. Various patterns, relationships, interconnections and signals that we see that align together and create something meaningful are coherent.
  3. Coherence also provides us with a language to communicate. When you observe coherence it begins to create a language you can use to communicate to others about what you’re seeing. When we look at what is happening at a societal level, its difficult to find what coherent narratives are actionable. At a smaller level, we might find them and this allows us to communicate more fully with others and this will allow us to scale and grow our learning.

This is the smallest visible system (SVS) in which you can make a difference. Once you can act wisely on this system, you can expand the boundaries and scope to work larger.

Acting on Systems

What this means for action is this:

  1. Pay attention to what is going on around you. Ask yourself: what is important and meaningful to me?
  2. Be systematic, but not rigid, in how you pay attention. This could mean looking at sales numbers, social trends, meeting minutes and observations from everyday life. If you’re working in teams, ask people about what they are paying attention to and what has meaning for them. What things are they organizing their work or life around? Reflective journaling can help, too. This is data.
  3. Gather the meaning. Bring together those things that offer some coherence to see how they make sense for what you are doing, seeking to become, or what you wish to accomplish. This is a social process called sensemaking. By guiding yourself through the data it’s possible to see patterns and what is called emergent properties — new forms of order arising from what might seem unordered.
  4. Start to act on this new coherence narrative and then repeat the cycle from step 1.

This will help you to determine what is useful and not useful for you in whatever context you are operating in. It’s a simple, but powerful means to start the journey toward a greater understanding of your present situation and help you see how and where you can act, whether that is in a time of massive upheaval or something merely disruptive.

Keep safe and know there’s more that you can do than you realise.

If you need help in setting this process up, implementing it, and making sense of it all, reach out. This is what we do.

The Amazing Spidergram

by

Illustrating action points within a complex system challenges evaluation users. Time to call on your friendly neighbourhood spidergram for help.

Visualizing complex systems is a challenge within strategy, foresight, and evaluation because each component of the system is interconnected with others. Influence on one of these is likely to influence others.

From an action standpoint, it’s easier to focus on one or two small parts of the system than tackle the entire system all at once. How do we reconcile this and provide a means to see the parts of the system without being reductionistic and neglecting the relationship with the whole?

One answer is: look to the spiderweb.

A spiderweb is a good entry-level metaphor for helping people see places they can take action within a system by creating distinctions between the parts (nodes, intersections) and the whole (branches, webs). There are two related, but different spiderweb models worth noting.

Spider Diagrams / Mindmaps

Spider diagrams (or spidergrams or mindmaps) are ways to connect ideas together through the branch-and-thread model akin to a spider’s web (hence the name). These are often called mindmaps and have been shown to facilitate learning about complex topics.

They enable the development of relationships between ideas and possible causal or associative pathways between ideas, concepts, or other data- or evidence-informed concepts. They also enable us to cluster related concepts together to identify sub-systems that may be more amenable to our intervention within the larger whole.

Spidergrams / Radar Diagrams

The other spider-related metaphor that is available to innovators and evaluators is the spidergram, sometimes called a Radar Chart or Spider Chart. These allow for the display of data collected along a scale presented alongside others that use a similar proportioned scale.

This hub-and-spoke model of data allows users to see a variety of performance indicators presented along a similar set of axes related to a common goal.

What this allows for is a view of performance across a variety of metrics simultaneously and can recognize how we make progress on one area often at the expense of another. Strategically, it can enable an organization to balance its actions and foci across a variety of key indicators at the same time.

This can be used with quantitative data such as the financial data above or social data, too.

Spidergrams/charts can also overlay data within the same domain (see example) providing even more depth into recurring or separated data points within the same topic or subsystem.

The web of engagement

What makes these tools powerful is that they display a lot of data at the same time in a manner that can facilitate engagement with a group of people tasked with making decisions. Visualizing data or systems brings the benefit of literally getting people’s focus on the same page.

People around a table can then literally point to the areas they are interested in, concerned with, do not understand, or wish to explore assumptions about.

Complex systems introduce a lot of data and a lot of confusion. Sensemaking through the use of visuals and the discussion that they encourage is one of the ways to reduce confusion and get more from your data to make better decisions.

If you’re stuck in the web of data and complexity, call on your friendly neighbourhood spidergram.

Want to learn more about how this can assist your innovation efforts? Contact us and we’ll gladly swing over and help (without the costume).

Photo by Jean-Philippe Delberghe on Unsplash. Spidergram by

Living History for Developmental Evaluation

by

Developmental Evaluation (DE) is an approach to supporting the evolution and development of an innovation. This means not only helping guide where an innovation is going but understanding where it is now and where it has been. Just like human development, an innovation is as much a product of its past as it is the present decisions and that past can help inform the strategy for moving forward. But how do we properly account for the past and present context in understanding what comes next? This is where The Living History method comes in.

The Living History method comprises a set of data collection, sensemaking, and design strategies that come together to reflect on how a project comes into being, develops a focus, and evolves up to the present context. It helps provide the ‘backstory’ to the present situation and provides a means of developing a baseline – another key part of a DE.

Why does knowing this matter? Taking a Living History of a program helps surface the path dependencies of a program or innovation. Path dependencies can include those habits of mind that form, patterns of behaviour, and general routines that can develop within a complex environment and consciously or unconsciously shape the context in which decisions are made. By understanding what these patterns are, we can better prepare to address them as the program unfolds.

It’s helpful to the biases and mental models that we hold in guiding our decisions to better account for them both when things work well, but also to help understand how things didn’t work out.

To borrow from the image above: the Living History method helps you use the data collected from preparing the climb, starting up to base camp, and setting the groundwork (the trials, failures, and victories) to initiate the climb as well as the data from the actual ascent to the summit.

Introducing the method

The Living History method is something we developed to ‘backfill’ much of the information that is necessary to understand the present context of a program. Many evaluators do this kind of work with informational interviews, document reviews, and site visits, however, the means in which this data is collated and made sense of isn’t always systematic, nor is it done in a manner that recognizes complexity (which is the space in which DE is meant to be employed).

A Living History involves building that historical narrative that has been constructed by program developers (founders, staff, stakeholders, etc..) and the activities that contribute to this narrative. It draws together these ‘stories’ from the data into an understanding of the present. This present space is where the existing narrative and current construction of the program (‘what is happening now?‘) comes together. It’s also space where strategy is developed and initiated to shape what is to come (‘where are we going and how are we going to get there?‘).

Below is an image that demonstrates this process.

Taking a Living History

A Living History can encompass many different data gathering techniques. Four of the most useful are:

  1. Document review. A thorough review of foundational documents that might include things like the initial application(s) for funding, ‘pitch decks’ used to generate funds or interest, program and policy manuals, staff reviews, board meeting minutes, evaluations, and annual reports (to boards, funders, stakeholders, etc.). Some programs may have an early program logic model or Theory of Change that can be examined. These can give insight into the planning structure, theories, and reasons for establishing the program.
  2. Interviews. Connecting with the program founders, board members (past and present), investors (and funders), as well as leaders involved in the initiative is critical. Having conversations with these people will help surface both explicit and tacit knowledge held by those that helped shape the program. These interviews can be critical in understanding the logic, political context, and social dynamics that underpin the direction of the organization. It is useful to determine not only the direction taken, but the directions considered. The reason is that these ‘unexplored paths’ may linger in the minds of people and serve as a source of tension or inspiration in moving forward.
  3. Site visits. It is helpful to understand that programs – even those that are virtual such as a website or app — originate in the physical world somewhere. Site visits, when possible, help gather information about organizational culture, physical resources, and relational barriers and facilitators. Take the example of the organization that starts up as a maker space in a garage and then evolves to a large corporate high-rise office. Is the culture that was established at the start-up maintained at a different scale or abandoned? We’ve seen this with organizations that began as a start-up, grew enormously, and then abandoned the culture that was created at startup for a more conservative corporate one despite holding on to the belief that things had not changed. Only by pointing out (seeing) the manner in which things physically changed (e.g., open space vs. closed spaces, size and shape of offices) was this able to be pointed out. Further, such visits can help determine developmental milestones such as large investments of capital.
  4. Timeline. A timeline is a simple method of connecting activities — actions + decisions — together in a visual means. This helps determine and organize data about what was done, but also when it was done. This can make an enormous difference in understanding things like time lags and exploring cause-and-consequence connections between actions, reactions, and other effects.

The Role of Sensemaking

Living History is not complete without sensemaking. Sensemaking is required because, like most interesting stories, innovating through complexity is messy and full of contradictions, gaps in logic, and remarkable things that can only be understood by sitting together with all the data in one place. These emergent properties — coherence that forms from diverse sources of information as part of the process of (self) organization — are what make a program a ‘whole’ and not just the ‘sum of the parts’.

Sensemaking is collaborative, discursive, and involves debate, reflection, and a process of drawing conclusions and forming hypotheses based on data and experience. It is the process of taking the strangeness of seeing things in chunks and putting them together. What comes from this process, which is a key part of Developmental Evaluation and design, is a ‘sense’ of what happened, what is going on, and what options might work next.

It helps you understand where you and your program stand now.

Moving forward (and backward)

The Living History method doesn’t end with sensemaking, it also is carried forward.

Developing a project dashboard for innovation evaluation is a helpful means to pull together the data collected from the past, the present, and then populating it as the program evolves. The Living History method views history as being co-constructed as the program evolves and thus thrives on having new data about the present (which is always a second away from being the past) added to the overall corpus of data.

New data, when combined with what has already been collected, can provide new insights into the bigger picture. As data is added and time passes, it is possible that new patterns will emerge, which can be insightful for making decisions. This data, collected now and going forward, will later be recombined as part of an ongoing sensemaking process.

A Living History can also provide a means to document the activities that have taken place up to the present moment by systematically, although retrospectively, capturing (mostly existing) data to show what has happened before, which can be useful for organizations and programs who are starting their evaluations later than they would have liked to.

Consider the Living History approach with your evaluation. By looking at the past you might find yourself better prepared to go forward. We can help.

Photo by Aaron Benson on Unsplash