Cense Ltd.

Inspiration, Innovation and Impact

Three Metrics for Design Evaluation

by

Design is about the creation of products, services, policies, and systems for use and benefit. It’s a structured, creative process that shapes what we do, how we do it, and what impact we create. Design is fundamentally about innovation: doing something different or new for benefit within a context.

Inspiration, Utility, and Values

How should we measure design and its impact on our organization? We suggest three core evaluation metrics above all to consider.

  1. Inspiration. A design has to inspire you. If you’re not excited about what you’re producing, why would it inspire your clients, customers, or users? By inspiration, we mean that it must capture or focus attention, delight, or attract energy. It does not mean a design has to be glamorous, just that it’s noticed for its purpose or appropriately invisible. For example, creating a negative opt-in to organ donation on driver’s license renewal forms vastly increases the donor pool (and reflects the desire of the population to be donors in the process) because it means people have to choose not to donate, rather than the opposite. Even with something benign like a form adjustment, Inspiration is an outcome.
  2. Utility. Use is the second key metric. Design is about being fit for purpose and if there is a poor fit then the benefits will also be poor. A great design fits the purpose for what it was designed for and is useful. It doesn’t have to be the best, just useful and better than the available alternatives. If your design is not used, then no matter how functional, attractive, or conceptually sound it is, it is a failure.
  3. Values. The alignment of your design with the values of your organization is critical to ensuring that the benefits that you accrue are the ones you want. If you value sustainability and responsibility, then your design has to reflect that. To illustrate, if you’re a company or organization that stands for human rights and ethical practices, what you produce (your designs) needs to reflect that. We see a lot of problems with organizations that say one thing and then design for something else.

From Perfection to Fit

There are no ‘perfect’ designs. A design is made within a time and context and the value and benefit of that design will change over time as the context changes. Even if we want things to stay the same and stable, the world around our designs is changing and evolving. Designs are also made with constraints posed by resources, time, and circumstances. However, design is designed — your process and procedures need to aspire and be set up to achieve the results you’re looking for.

These changes are why evaluation is such an important part of design: we need to continue to monitor and evaluate our designs in light of changes in context and circumstance.

We can’t hope for inspiration, utility, and alignment with values unless we design our designs to match that. Models like the Design Helix (below) can provide some guidance on what can be done to facilitate this.

The process of design and innovation is part art and part science. If you’re looking to design better — whether it’s to improve your products and services, make better policies, or build a culture of innovation we can help. Let’s talk.

Photo by Annie Spratt on Unsplash

Data Collection and Participation For Busy People

by

Among the greatest challenges of doing research and evaluation is ensuring you get participation from enough (and the right) people. Surveys are everywhere. It feels like everyone wants our feedback on just about everything. Yet, the more surveys out there the more we should be concerned about data quality, too.

On top of that, we are surrounded by media messages, distractions, and ‘noise’ that can make attention a very precious (and rare) commodity.

It can be daunting.

How do we get people to participate and get good quality data from that participation amongst the noise? We’re going to outline an approach to data collection that goes from methods to conversations.

From Methods to Conversations

There is a lot of research on improving existing methods like surveys or asking better questions. Tips like these can be useful, but they may also distract us from addressing other issues. For starters, consider the user case for doing research and evaluation.

Who benefits from the research? Is it those who are participating? If not, why would someone want to take time and spend energy answering your questions? We can no longer assume people will participate out of a sense of wanting to help. The deluge of research requests has made data gathering an imposition more than an opportunity for most people.

One of the ways that we deal with this is to shift the focus to creating conversations and learning opportunities. By thinking of data collection as part of a conversation we can change the way we gather data. This works for evaluation, design research, or any applied research context.

A great conversation is about creating exchange. That means some back and forth between the parties. What if you could do this with your data?

Data-Based Conversations

The concept of data-based conversations is all about using what you gather as the foundation for the exchange between people. This means gathering relevant information from people and then sharing what it is that you find. Individuals provide their thoughts, opinions, attitudes, and reflections and as researchers we provide the synthesis and opportunity to share what we’ve learned from others. It works because it creates exchange and value.

People choose to participate because they can both contribute and receive insights about their peers. Please keep in mind that this approach only works when people are interested in your topic.

We recommend that you design multiple, short engagements so you reduce response burden. Rather than use a single, large survey we suggest you break it into smaller batches of questions. In between each survey we provide rapid synthesis learning reports to share what we’ve learned from others.

Our rapid learning reports might be short summaries, infographics, or distilled tips gleaned from the data. Using visual media is particularly helpful because it’s simple and accessible. It says to our participants: “we heard you and here’s what others have said.”

Data collection can also include short interviews, social media exchanges, or panel feedback. The methods matter less than the way that we structure the engagement. This approach builds trust, familiarity and increases data quality.

Time to Talk?

People are busy and less invested in your product or service than you think. This approach to creating a conversation than just asking (and taking) from people changes the relationship. By enlisting people as partners and focusing on sharing what you learn in ways they can benefit, you serve others not use others.

As always, this must be done with transparency, ethics, respect, and commitment to delivering on your promises. We’ve found this approach works and it adds value to participants. People like to learn and know what’s going on with their peers. Gathering data this way is less intrusive, more natural, and less burdensome.

If you’ve got a big research question to ask, consider ways you can transform your data collection into a conversation. You might find that you get more participation, greater engagement, and better quality data.

Want to build this approach into your evaluations or research? We can help and share our experience using this approach to reach busy people. Contact us and let’s grab a coffee.

Image credits: Karen Lau on Unsplash, Jon Tyson on Unsplash, and Firmbee.com on Unsplash

Innovation Like An Epidemiologist

by

What if innovation was like epidemiology? What if we wanted to understand the source, scope, scale, and spread of an idea or product?

That’s one way that we think about the innovation process.

Innovation – like a virus — is similar to infectious disease epidemiology. Both of these areas look at the development of something, its effects, its spread, and how it scales over time. Both innovation and epidemiology require evaluation as well.

Viral Innovation

The parallels between infectious disease epidemiology and innovation are many. The first of these parallels is between a virus and an innovation.

A virus develops and mutates as it has more exposure to hosts. A virus ‘learns’ from what it does and adapts to fit a changing context. Innovation does the same thing only with an idea, product, or service. For innovators, the aim is for healthy development and the creation and distribution of the product or service. This is what research and development and marketing is all about.

Innovation and epidemiology are also both interested in the spread and scale of things. If an innovation works well in a context we might want it to spread to other contexts. Innovators often want their products to scale as far and as wide as possible. Sometimes innovations scale and sometimes they don’t.

The way we know this is through conducting detailed, systematic monitoring and evaluation.

Epidemiology, like innovation, is driven by evidence from the laboratory and the real world together.

Stages of Development

At Cense, we also look to another parallel between the two areas: research and development.

Innovation develops in four stages that are similar to phases of research trials.

The role of design and evaluation is different for each stage. At Stages 1 and 2 the emphasis is on working with innovators to align their intentions with their design to explore what it (the innovation) does and how it does it. This is where design thinking and strategy are most prominent.

At Stages 3 and 4, the aim is to build an evidence base and strategy to spread and scale the innovation. At these stages, the focus is on marketing, distribution, and amplification. Evaluation in these stages focuses on the larger impact and the means by which the innovation is implemented and adopted for use across contexts.

Like an epidemiologist, its important to collect data to support moving the innovation from each stage. It’s also important to explore what kind of effects — positive and harmful – are generated at each stage.

At Cense, we work with our clients to design the right evaluation and strategy for each stage of development. There are no ‘one-size fits all’ approaches to innovation. Much like with a virus, an innovator must know what their innovation does and what will change at each stage and scale of its development.

What is key is designing data collection and strategy that is fit for purpose.

Viruses aim to spread and survive just like innovations. By thinking like an epidemiologist you can help your innovation to survive and thrive like the best kind of virus.

Photo by Bill Oxford on Unsplash

Multimedia Visualizations For Strategy and Impact

by

Multimedia visualizations are powerful ways to convey a lot of information in a small space and with little time. The use of a visual — a map, a drawing, a picture, sketch, or collage — provides layers of information beyond words. When dealing with complex situations, visualizations provide clear communication pathways.

Unlike some of our other posts, this is not pointing us to a specific tool or technique, rather a general approach to thinking about and engaging with content.

Let’s look at the benefits of adding more visuals to your work to help your strategy, evaluation, and program design efforts.

1. Relationships. One visuals can help you with understanding proportions, distance, and other relative characteristics. It is useful to position things next to one another – whether that is choice alternatives, network connections, or even contrasts. Seeing things in relation to one another on a visual canvas provides our brains with new information that is difficult or impossible to gain from more abstract thinking without visuals.

2. Colour and Texture. How something looks and feels can tell us a lot about what it means. By adding colour to something we apply a simple lens that can convey meaning. Colours like green invoke environmental imagery, red is well-known to have many meanings in different cultures, while blue can be relaxing. Texture can do the same thing. To illustrate, consider New Zealand-based Icebreaker who makes wool-based clothing and uses texture-rich images of sheep’s wool to highlight how its products are natural in composition. Use of texture in the imagery gives a sense of integrity, warmth, and a feel to fabrics even when the buyer might only see them online.

3. Variety. As more people work remotely and digitally than ever before, the visual landscape for many of us has changed in most workdays. We lack contrast in our environments and also in our digital screens. By increasing the visual constraint and variety by diverging from text and boxes toward photographs, graphic images, videos, or some other form of visual media we draw attention. When so much feels the same, few things are more valuable than attention.

4. Abstraction and Narrative. Words offer us a restricted set of options due to the need for us to be linear in our the way we speak. With a visual we can better draw abstract themes from an object and use things like metaphor to describe scenarios that allow us to transcend words.

There are many ways to use visuals and software tools like online whiteboards such as Miro and Mural, visual organizing tools like Milanote, and tools such as Jamboard or Trello to showcase information in ways that extend beyond text.

Consider bringing more details into your next project and discover what visuals can do for creating richer, more vibrant ideas and creative opportunities to your work.

Using Timelines To Track Activities

by

Just as parents will use pencil notches in a wall to track the height of their children as a developmental marker, so too can innovators and evaluators use timelines to help gather and track the development of programs and projects over time.

A timeline is a simple linear visual that gathers activities together that uses time as the variable of distinction.

What makes a timeline useful is that it provides a visual display of temporal relationships between events, activities and outcomes. It makes explicit what we might have in our heads, but are also prone to confusing and forgetting over time.

When to Use Timelines

Timelines are useful in a variety of situations:

  1. Projects with activities that can be organized into a sequence (whether planned or not). They are less useful for projects where there are many activities happening simultaneously.
  2. Projects with a long time horizon.
  3. Projects operating in an environment with many different levels of activities and influences. For example, when there are external factors like policy decisions that have discrete times attached to them and can influence a project’s course, this is a good use of a timeline.
  4. Projects that have a story to tell that involves a beginning, middle, and end.

A timeline can help provide anchors between project activities and events — whether those are policy-related, tied to human (or other) resource use, environmental disruptions, seasons, or cycles. They can help provide hypotheses between causes and consequences or explain mediators.

For example, one non-profit project we worked on had a planned roll-out that was moving along well until their funder abruptly cancelled the program that they relied on. This meant that the six-months after that announcement involved finding new sources of revenue, reductions in staffing, and changes in some activities, yet also persistence in trying to adapt to the situation and still execute the original plan. By showing the data on a timeline it helped explain what happened to project activities, outputs, and outcomes within a certain time period and how that related to the overall project plan.

Examples

Below is an example of a timeline that illustrates distinctive markers along the route. These are clearly defined events that took place on specific dates. The selection of events includes those deemed to be meaningful and significant to the project.

What makes a timeline a powerful tool is that there are many different ways to illustrate events. The example above is a relatively straightforward set of data.

Below is another example that involves much more data and in different forms. This example creates a hybrid of timeline and categorization exercise.

Creating Timelines

There are many templates and tools that can be used to help develop visual graphics. The examples above are from Lucidchart, however, tools like Miro, Mural, SmartDraw, PowerPoint, Google Draw, and many others have templates that can be modified to create useful timelines. These are all simple tools that can be manipulated easily so you’re able to build them as you go.

If you are looking to develop more sophisticated models, we suggest employing a visual communicator or graphic designer to take advantage of the many ways you can represent temporal data.

The result is something that is engaging and can easily be discussed or presented to diverse stakeholders involved in a project who might be able to validate, contribute to, or constructively challenge the arrangements. We find this to be a powerful way to organize our findings, refresh our memories, and recognize all of the activities that go into a project.

Documenting Innovation Development

Lastly, if you are developing an innovation where there is no clear ‘end’ known at the beginning, timelines are useful in telling the story of the project and documenting the different pivots, changes, adaptations, and their consequences. A timeline can be a powerful asset to Developmental Evaluation and a complement to the Living History Method that we often employ in those kinds of evaluations.

A timeline can honour all the work you put into coming up with your final product and can be an engaging way to get people involved in celebrating, documenting, and tracking what you do and create.

We use these all the time and can help you track and evaluate your project. Contact us and let’s talk about timelines, innovation and impact.