Cense

Research.Design.Impact.

The Amazing Spidergram

by

Illustrating action points within a complex system challenges evaluation users. Time to call on your friendly neighbourhood spidergram for help.

Visualizing complex systems is a challenge within strategy, foresight, and evaluation because each component of the system is interconnected with others. Influence on one of these is likely to influence others.

From an action standpoint, it’s easier to focus on one or two small parts of the system than tackle the entire system all at once. How do we reconcile this and provide a means to see the parts of the system without being reductionistic and neglecting the relationship with the whole?

One answer is: look to the spiderweb.

A spiderweb is a good entry-level metaphor for helping people see places they can take action within a system by creating distinctions between the parts (nodes, intersections) and the whole (branches, webs). There are two related, but different spiderweb models worth noting.

Spider Diagrams / Mindmaps

Spider diagrams (or spidergrams or mindmaps) are ways to connect ideas together through the branch-and-thread model akin to a spider’s web (hence the name). These are often called mindmaps and have been shown to facilitate learning about complex topics.

They enable the development of relationships between ideas and possible causal or associative pathways between ideas, concepts, or other data- or evidence-informed concepts. They also enable us to cluster related concepts together to identify sub-systems that may be more amenable to our intervention within the larger whole.

Spidergrams / Radar Diagrams

The other spider-related metaphor that is available to innovators and evaluators is the spidergram, sometimes called a Radar Chart or Spider Chart. These allow for the display of data collected along a scale presented alongside others that use a similar proportioned scale.

This hub-and-spoke model of data allows users to see a variety of performance indicators presented along a similar set of axes related to a common goal.

What this allows for is a view of performance across a variety of metrics simultaneously and can recognize how we make progress on one area often at the expense of another. Strategically, it can enable an organization to balance its actions and foci across a variety of key indicators at the same time.

This can be used with quantitative data such as the financial data above or social data, too.

Spidergrams/charts can also overlay data within the same domain (see example) providing even more depth into recurring or separated data points within the same topic or subsystem.

The web of engagement

What makes these tools powerful is that they display a lot of data at the same time in a manner that can facilitate engagement with a group of people tasked with making decisions. Visualizing data or systems brings the benefit of literally getting people’s focus on the same page.

People around a table can then literally point to the areas they are interested in, concerned with, do not understand, or wish to explore assumptions about.

Complex systems introduce a lot of data and a lot of confusion. Sensemaking through the use of visuals and the discussion that they encourage is one of the ways to reduce confusion and get more from your data to make better decisions.

If you’re stuck in the web of data and complexity, call on your friendly neighbourhood spidergram.

Want to learn more about how this can assist your innovation efforts? Contact us and we’ll gladly swing over and help (without the costume).

Photo by Jean-Philippe Delberghe on Unsplash. Spidergram by

Living History for Developmental Evaluation

by

Developmental Evaluation (DE) is an approach to supporting the evolution and development of an innovation. This means not only helping guide where an innovation is going but understanding where it is now and where it has been. Just like human development, an innovation is as much a product of its past as it is the present decisions and that past can help inform the strategy for moving forward. But how do we properly account for the past and present context in understanding what comes next? This is where The Living History method comes in.

The Living History method comprises a set of data collection, sensemaking, and design strategies that come together to reflect on how a project comes into being, develops a focus, and evolves up to the present context. It helps provide the ‘backstory’ to the present situation and provides a means of developing a baseline – another key part of a DE.

Why does knowing this matter? Taking a Living History of a program helps surface the path dependencies of a program or innovation. Path dependencies can include those habits of mind that form, patterns of behaviour, and general routines that can develop within a complex environment and consciously or unconsciously shape the context in which decisions are made. By understanding what these patterns are, we can better prepare to address them as the program unfolds.

It’s helpful to the biases and mental models that we hold in guiding our decisions to better account for them both when things work well, but also to help understand how things didn’t work out.

To borrow from the image above: the Living History method helps you use the data collected from preparing the climb, starting up to base camp, and setting the groundwork (the trials, failures, and victories) to initiate the climb as well as the data from the actual ascent to the summit.

Introducing the method

The Living History method is something we developed to ‘backfill’ much of the information that is necessary to understand the present context of a program. Many evaluators do this kind of work with informational interviews, document reviews, and site visits, however, the means in which this data is collated and made sense of isn’t always systematic, nor is it done in a manner that recognizes complexity (which is the space in which DE is meant to be employed).

A Living History involves building that historical narrative that has been constructed by program developers (founders, staff, stakeholders, etc..) and the activities that contribute to this narrative. It draws together these ‘stories’ from the data into an understanding of the present. This present space is where the existing narrative and current construction of the program (‘what is happening now?‘) comes together. It’s also space where strategy is developed and initiated to shape what is to come (‘where are we going and how are we going to get there?‘).

Below is an image that demonstrates this process.

Taking a Living History

A Living History can encompass many different data gathering techniques. Four of the most useful are:

  1. Document review. A thorough review of foundational documents that might include things like the initial application(s) for funding, ‘pitch decks’ used to generate funds or interest, program and policy manuals, staff reviews, board meeting minutes, evaluations, and annual reports (to boards, funders, stakeholders, etc.). Some programs may have an early program logic model or Theory of Change that can be examined. These can give insight into the planning structure, theories, and reasons for establishing the program.
  2. Interviews. Connecting with the program founders, board members (past and present), investors (and funders), as well as leaders involved in the initiative is critical. Having conversations with these people will help surface both explicit and tacit knowledge held by those that helped shape the program. These interviews can be critical in understanding the logic, political context, and social dynamics that underpin the direction of the organization. It is useful to determine not only the direction taken, but the directions considered. The reason is that these ‘unexplored paths’ may linger in the minds of people and serve as a source of tension or inspiration in moving forward.
  3. Site visits. It is helpful to understand that programs – even those that are virtual such as a website or app — originate in the physical world somewhere. Site visits, when possible, help gather information about organizational culture, physical resources, and relational barriers and facilitators. Take the example of the organization that starts up as a maker space in a garage and then evolves to a large corporate high-rise office. Is the culture that was established at the start-up maintained at a different scale or abandoned? We’ve seen this with organizations that began as a start-up, grew enormously, and then abandoned the culture that was created at startup for a more conservative corporate one despite holding on to the belief that things had not changed. Only by pointing out (seeing) the manner in which things physically changed (e.g., open space vs. closed spaces, size and shape of offices) was this able to be pointed out. Further, such visits can help determine developmental milestones such as large investments of capital.
  4. Timeline. A timeline is a simple method of connecting activities — actions + decisions — together in a visual means. This helps determine and organize data about what was done, but also when it was done. This can make an enormous difference in understanding things like time lags and exploring cause-and-consequence connections between actions, reactions, and other effects.

The Role of Sensemaking

Living History is not complete without sensemaking. Sensemaking is required because, like most interesting stories, innovating through complexity is messy and full of contradictions, gaps in logic, and remarkable things that can only be understood by sitting together with all the data in one place. These emergent properties — coherence that forms from diverse sources of information as part of the process of (self) organization — are what make a program a ‘whole’ and not just the ‘sum of the parts’.

Sensemaking is collaborative, discursive, and involves debate, reflection, and a process of drawing conclusions and forming hypotheses based on data and experience. It is the process of taking the strangeness of seeing things in chunks and putting them together. What comes from this process, which is a key part of Developmental Evaluation and design, is a ‘sense’ of what happened, what is going on, and what options might work next.

It helps you understand where you and your program stand now.

Moving forward (and backward)

The Living History method doesn’t end with sensemaking, it also is carried forward.

Developing a project dashboard for innovation evaluation is a helpful means to pull together the data collected from the past, the present, and then populating it as the program evolves. The Living History method views history as being co-constructed as the program evolves and thus thrives on having new data about the present (which is always a second away from being the past) added to the overall corpus of data.

New data, when combined with what has already been collected, can provide new insights into the bigger picture. As data is added and time passes, it is possible that new patterns will emerge, which can be insightful for making decisions. This data, collected now and going forward, will later be recombined as part of an ongoing sensemaking process.

A Living History can also provide a means to document the activities that have taken place up to the present moment by systematically, although retrospectively, capturing (mostly existing) data to show what has happened before, which can be useful for organizations and programs who are starting their evaluations later than they would have liked to.

Consider the Living History approach with your evaluation. By looking at the past you might find yourself better prepared to go forward. We can help.

Photo by Aaron Benson on Unsplash

Developmental Evaluation Traps

by

Developmental evaluation (DE) is a powerful tool for supporting innovation in complex systems. 

Developmental evaluation (DE), when used to support innovation, is about weaving design with data and strategy. It’s about taking a systematic, structured approach to paying attention to what you’re doing, what is being produced (and how), and anchoring it to why you’re doing it by using monitoring and evaluation data. For innovators, this is all connected through design.

DE helps to identify potentially promising practices or products and guide the strategic decision-making process that comes with innovation and provides evidence to support the innovation process from ideation through to business model execution and product delivery.

As introduced over on Censemaking, DE is also filled with traps. 

Over the next couple weeks, we’ll be providing some guidance on how to navigate these traps and what your organization can do to steer clear of them and harness the great potential of this important approach to innovation strategy and evaluation for your organization.

Image credit: 200 pair telephone cable model of corpus callosum by J Brew used under Creative Commons license

The Experimentation Myth

by

Innovation Growth Lab: Making innovation and growth policy work from Nesta UK on Vimeo.

The world of social innovation is filled with wonderful things and, like any creative space, has its share of myths. One of these is the role of experimentation. One view is that social innovation operates in such a space of complexity that the use of methods such as randomized controlled trials are not only inappropriate, but harmful.

The alternative perspective is one held by folks like Geoff Mulgan, the Chief Executive of Nesta UK. According to Mulgan (discussed in the video embedded above):

the only way you figure out what works is by doing experiments using control groups and seeing who benefits from an intervention.

This latter position is also a myth. Like any myth, it makes complexity much easier to handle and accept because it takes away the need to invest energy in determining ‘what is going on’.

Alas, that ‘what’s going on’ is at the crux of understanding impact in the realm of social enterprises, because we humans have a tendency to resist standardization in how we behave. This is not to suggest that there is no place for experiments, because as Nesta UK and others have shown, it can be done. But to suggest that the only way to find out whether something works is experimentation is taking things too far.

To be fair, Mulgan was speaking on matters of public policy, which have enormous consequences and his comments after the quote above about understanding potential impact before scaling are spot-on, but there are different paths to understanding how impact is achieved and what it is that go beyond experiments.

Disrupting experiments

A true controlled experiment, as the name suggests, requires control. The one way to get around some of the constraints imposed by controls are to have large numbers of participants. The problem facing the controlled experiments is that we often do not have both — either control or large numbers of participants. Further, the amount of control needed is dependent on the amount of social complexity inherent in the problem. For example, imagine comparing two options: renewing a required document like your drivers licence online or in person. This is a process that might involve a lot of effort (depending on where the nearest location of your motor vehicle registration office is), but it’s not complex. This is something that is amenable to experimentation.

However, a policy or program designed to help individuals manage chronic disease conditions involves enormous complexity given that each participants’ condition will have a combination of shared qualities with unique manifestations and that those will all be mediated by different social, biological, economic, geographic and situational variables that, depending on the chronic condition, might play a significant role in how a program is received and its impact.

This is a far more challenging task, but one that is worth doing lest we, in the words of systems scholar Russell Ackoff, do “the wrong things, righter” by imposing an experimental design on something that warrants something different. Or, put another way, perhaps we need to redesign the experiment itself to suit the conditions.

 

 

What is design thinking? A look at universities

by

Our principal, Dr. Cameron Norman, was recently interviewed for University Affairs magazine on the topic of design thinking. Speaking with journalist Tim Johnson, Dr. Norman discussed what design and designers offer those seeking to tackle complex, thorny problems.

[Dr Norman] notes that designers, especially product designers, are typically experts in conceptualizing problems and solving them– ideal skills for tackling a wide range of issues, from building a better kitchen table to mapping out the plans on a large building. “The field of design is the discipline of innovation,” he says. “[Design thinking] is about taking these methods, tools and ideas, and applying them in other areas.”

The concept of design thinking is something that’s quite new within the academic world and Johnson’s article highlights some of the academic work that is taking place in universities and beyond to understand the role that design thinking can play in tackling complex, even wicked problems.

What presents challenges and opportunities for academia is that problems and design thinking require, by necessity, collaboration and interdisciplinary contributions:

Proponents of DT posit that, with its emphasis on teamwork and its problem-based approach, design thinking is particularly well-suited to solving “wicked problems” – those big, ill-defined, complex, multi-faceted issues that don’t have a clear solution. U of T’s Dr. Norman points to climate change as an example. “There’s no climate change discipline,” he says. “We need everyone from scientists to citizens to politicians. And within universities, you have geography and sociology and biology – you name it – there’s somebody who can play a role.”

However, it’s the approach to the problems itself that also changes the perspective as noted by Greg Van Alstyne from OCAD University’s sLab:

[Design thinking] focuses on collective goals and places a premium on sustainability, community, culture and the empowerment of people, says Greg Van Alstyne, director of research and co-founder of the Strategic Innovation Lab, or sLab, at OCAD University. “It means you go about your problem-solving in a more holistic way. We can say ‘human-centered,’ but it’s actually ‘life-centered,’

By taking a systems perspective on the problem, the problem solvers and the solutions, design thinking is opening up new opportunities for academia and business alike. Indeed, the future of both institutions might be designed quite differently as design thinking moves between them, through them and from the outside to the mainstream.