Measuring the social impact of their actions has always been crucial for NGOs and charity associations: they need to be able to assess their own efficiency in order to constantly improve and to, sometimes, prove to their stakeholders that their money is well-spent. When social businesses started to flourish after their “invention” by Nobel Prize Muhammad Yunus a few years ago, they did not address this issue right away. For a while, assessing the “business part”, using traditional financial and economic tools, seemed to be enough to satisfy both shareholders and stakeholders. However, social businesses are positioned between two worlds, involved both in social work and business creation, and yet inventing a third way: Which means that evaluating their impact also needs to be done their own way.
The necessity to measure social impact
As Deb Levy and Kathy Brennan from the Innovation Network put it,
every social entrepreneur struggles to identify when he or she has moved beyond implementing a good idea to achieve real change.
Seeing the reality of this change means going beyond mere empiric observation and accurately measuring the results, even if they are a priori unquantifiable. It is crucial that these results be assessed, in order for the shareholders and the funders to be able to “witness” the change, but not only. As Martin Hirsch, President of the Civil Service Agency, declared on this issue in the Danone Ecosystem Fund’s latest newsletter: “it is essential to measure the social impact of projects in order to justify their replication.” In the business world inventing something new means being imitated. It is the same for social work: changing the world means creating something that can be implemented on the widest possible scale, in as many places as possible. And to achieve this, everyone must be aware that your idea is good, that what you are doing works. Which is why social businesses must “prove the efficiency of new value creation models” with “academic credibility”, as recommended by the workshop which took place on this issue at the Global Communities Meeting last May.
But what must they measure other than the financial health of the business? “Social impact” can cover a wide range of aspects, which are either related to the project’s aim (i.e. empower women through sales training), or adjacent to its original goal (ex. mangrove reforestation that revives biodiversity, which is beneficial to farmers and fishermen). Governance transformation, reduction of poverty, restoration of the environment, empowerment of some categories of the population, education, training, etc. are some of these aspects. These are some of the societal changes that a successful project can bring about.
Assess societal changes
But then, how do we measure such unquantifiable effects? The main way to put a figure on these effects is to evaluate the number of people who benefitted from the project. For instance, the mangrove reforestation program led in Senegal by Oceanium has reached the population of 450 villages. This is already a clue, but it is far from being enough. And if some effects are just impossible to objectively measure with figures, this does not mean that they are impossible to assess. In fact, as Deb Levy and Kathy Brennan explain, “qualitative information can be very effective in measuring social change.” To the participants of the GCM workshop, this qualitative feedback must be obtained by discussing with the beneficiaries: they are the ones who can best evaluate the effects that the project had on them. In order to achieve this, cooperation is key, as it will help to elaborate the proper methodologywith the beneficiaries: training them to discuss these issues will help them be more accurate and relevant in the evaluation of their benefits. The approach should be participative, too, with a mix between interviews, surveys, workshops, etc. and continuous follow-up. The GCM workshop also recommends that external entities be brought into the discussion, for a complementary perspective. This is an important idea if you consider that, for social projects, some of the impacts can reach way beyond the initial “target” population. In Alnoor Ebrahim’s words, (Harvard Business School), “societal transformation involves multiple players and causal mechanisms that are still poorly understood.” The wider the scope of action, the tougher it is to correlate cause and effect, and accurately assess the weight of your own initiative, compared to other actions. Having conversations with other entities than the direct beneficiaries is crucial to gain accuracy on that matter.
Towards a scientific methodology
This qualitative data must then be processed, and this is where it becomes harder to provide a one-size-fits-all methodology. There are some clues, though. “Measuring qualitative change requires looking systematically across qualitative data and having set research questions so you know what you are looking for”, advises the Innovation Network.Which means that
you need to draw on the same creative and critical thinking skills used to launch your program. In order to measure social impact, you need to get concrete about what you hope to achieve. It’s your vision – so you have to define your own success.
Alnoor Ebrahim agrees: project managers need to “focus their attention on accountabilities that really matter for achieving their missions.” In the same range of ideas, Pascal Lamy, World Trade Organization General Manager, in the editorial of the Ecosystem newsletter, recommends that assessment methods be included right from the conception phase, so that they are part of the road map and the project leaders can incorporate them in their vision. Each of the social businesses will have then its own evaluation grid, depending on short term, intermediate and long term outcomes it wants, wishes and hopes to see as a result of its actions. This could be an attempt at defining a precise methodology; however, the diversity of social businesses and the diversity of their effects necessarily means that there will not be an agreement on “a set of hard-and-fast metrics to measure social performance”, as Julia Hanna from the Harvard Business School writes. Which points to the fact that, in Pascal Lamy’s words, “impact measurement is still an experimental science: various methodologies are available, their implementation is still in its infancy and the publication of their results remains opaque.” These are issues that must be addressed. But the goal is not to find, once and for all, the objective criteria that will allow to rapidly assess the impact of any kind of social business.
One of the raisons d’être of a social business is to invent new value creation models that are replicable in other situations but with the same purpose: reconcile business and social work, so as to build economically, environmentally and socially sustainable business models. The ambitiousness of the task may cause concern that there is no ready-made methodology to assess the efficiency of these models. But this does not worry Alnoor Ebrahim:
It turns out that highly intelligent, thoughtful people end up developing some very different approaches to measuring social performance. That suggests we might not see agreement on a common set of metrics for a while. What we could see instead is convergence on what constitutes critical reflective learning processes within the organization.
In the light of this statement, social impact measurement seems to be on the right track.
(Photo from http://blog.searchmetrics.com)