Evaluating Co-production

‘Co-production’ is an increasingly popular term and approach within academic research and public service delivery. Most simply, co-production is what happens when we make things together, be those things research, museum exhibitions, archives or public services. The drive to co-produce comes from a variety of angles. For public service delivery, co-production can simply mean better service delivery – service users often have knowledge that makes them better placed to define what services should look like and how they should operate, so it make sense they should be included. For research, the drive to co-produce is similar but different, emerging from a critique of the way in which knowledge about others has often been created by experts. The claim here is not only an instrumental one that the knowledge produced is less comprehensive or detailed but also an ethical one that  speaking for about and for others without their explicit consent is unjust. Representing others is a powerful act which may not always be to their benefit. It should then be up to those to whom knowledge refers to decide how to represent themselves. It has been according to this dual (instrumental and ethical) logic that knowledge co-production became a core part of methods like Participatory Action Research (PAR) which is often explicitly oriented to the creation of knowledge for a freer and more equal world. Similarly, currents in postcolonial, anti-racist and feminist scholarship have long attempted to produce knowledge with rather than for or about and to destablise and problematise the line between researcher and research subject.

So how do we know good co-production when we see it? The New Economics Foundation (NEF) have done some excellent work developing tools for evaluating the co-production of services. Meanwhile, though significant ink has been spilled theorizing academic knowledge (co-)production, there is rather less out there in the way of practical methods for evaluating co-produced research projects. To this end, we have developed the schema below, building on the NEF’s work, but with an orientation towards co-production of research. Rather than offering a scoring system, the schema is focused on what was created, what was omitted and what if anything might be done to remedy an omission – this is applied to project ‘outputs’, relationships and organisational arrangements. This framing avoids the assumption that a perfect co-production can be defined in advance, and instead asks participants to reflect critically on why a particular ordering of the project was chosen. Thus, in a particular context, it may be the case that specialists (rather than a perfect sharing of roles) were used or that leadership (rather than a perfectly horizontal decision-making structure) was deployed. The implicit claim here is that what makes good co-production is not the presence or absence of ideal forms but rather that the particular form deployed is chosen consciously and collectively by the partners.

The schema is best used in a group context and can be printed up to size of AO to for workshop use. Have one person facilitate while the rest of the group move through the sections in turn and write down their reflections/analysis. ‘Assets created/uncovered’ is usually the best place to start.

Notes on the sections:

Assets created/uncovered

• What was it that is project created or made visible? What were the ‘outputs’? For example, journal articles, book chapters, a digital or physical archive, an exhibition etc.

Human Capacities Developed

• What are participants able to do now that could not or did not realise they could do previously? What skills, experience, training and knowledge do they have that enable this?

Degree of mutuality/reciprocity

• Think here who benefitted and how? Did partners benefit evenly? For example, individuals may have secured buy-out for their posts or career recognition, organisations may have benefitted from increased visibility, the research may have impacted on the lives of a particular group so as to be of benefit to them.

Degree to which roles are Dispersed/Shared

• Think here about who did what and why? Who did the research design, carried out the interviews, analysed the data, presented at a conference, did the writing, design work or publicity. All roles do not have to be shared, but barriers to role sharing should be minimised so that roles can be shared should participants so desire.

Degree to which decision-making is participatory

• Who is involved in making which decisions? Are some areas more open to participation than others? Why? Are decision-makers accountable to other members of the group or partnership?

Moving from periphery to centre

When moving from the outer to the inner rings of the circle, try to reflect on what aspects were missed out or and why. What was it that escaped consideration? Is action needed to rectify this lack, if so, what? Or was the focus on a particular group or area an intentional part of the project? What are the key issues or questions that emerge from this kind of reflection?

Click here to download a pdf of the evaluation wheel filled in with example text.

Click here to download a large (AO) printable version of the evaluation wheel for workshop use.