SROI Analysis: Stage 4 – Establishing impact
In the previous stage, we dealt with how indicators can be developed to collect evidence on outcomes that are occurring for various stakeholders. The process involved assessing whether an outcome has occurred and to what degree. The next step in the SROI analysis is to establish the impact. In stage 4 of the analysis, we will learn about various ways to establish that the outcomes occurring are the result of the activity or the project that has been undertaken.
Some outcomes would have occurred anyway, this stage will help you assess how and to what degree the activity has added to the outcome. This is what is termed as measuring ‘impact’.
It is better to establish the SROI ‘impact’ with a strong rationale to eliminate any room for overclaiming. This exercise is also helpful in identifying important stakeholders and activities that worked in favor of achieving the outcomes and creating the SROI impact. Those that could be termed as irrelevant or didn’t add much to the outcome could be spotted and eliminated for future projects.
- SROI Analysis: Stage 1 – Establishing Scope And Identifying Key Stakeholders
- SROI Analysis: Stage 2 – Mapping Outcomes
- SROI Analysis: Stage 3 – Evidencing Outcomes and Giving Them Value
This stage broadly comprises four steps:
- Deadweight and displacement
- Calculating SROI Impact
4.a Deadweight and displacement
In this stage, we measure how much of the outcome would have taken place in the absence of the activity. This is known as deadweight. For example, it was estimated that the economic activity in and around West Java area grew by 2 percent after the development of public spaces, however, there’s the need to measure whether or not this absolute increase could be attributed to the project to avoid overclaiming. There could be several other factors, besides the development of public spaces, which could have led to the hike in the economic activity in the region.
Remember, calculating deadweight will always be in terms of estimates or percentages. To measure deadweight, you will need to define comparison groups of people that your activity has affected and see how and to what degree the impact would have been in the absence of your project.
Tip: Make sure that your comparison group is as similar as possible for it to lead to better estimates. In case you are unable to identify an appropriate comparator group or proxy, you can take some help from a ‘best estimate’.
In case of an evaluative SROI, information on deadweight can be sourced by engaging with stakeholders. They would be able to tell you about any similar services they might or could have accessed and how did they benefit from them. This way, you can draw a comparison. Then, there are other public and private sources from where you can gather relevant information on indicators.
One of the simplest ways to measure deadweight is by looking at the trend associated with the indicator. Monitor the difference in the trend before and after the activity started. Simply put, an increase in the trend after the commencement of the activity will help you establish how much of the outcome was the result of the activity.
Your stakeholder group could be part of a wider population which could be reporting a similar trend. For example, the elderly living in and around West Java could be part of a nationwide initiative aimed at improving health and wellbeing among the elderly as a result of which a 4 percent improvement in their general wellbeing was reported. Now, the NGO needs to measure how much of a change in their health and fitness is due to the usage of the newly opened gym and park and how much of it would have happened anyway as a result of the national health initiative.
How to calculate deadweight?
Consider the previously discussed example. During an interaction between the NGO and the stakeholder group, the elderly population shared that access to the open gym and park contributed to the regular activity that they were getting as part of the national initiative. They agreed to have worked out 20 percent more than what they were used to under the national programme.
Here, the benchmark was 100 percent, which was being fulfilled by the national programme, in addition to it comes the 20 percent contributed by the ‘Redefining West Java’ project. This gives a total of 120 percent. To calculate the deadweight, we just need to measure how much the national programme contributed. Therefore, the estimate of deadweight would be 100×100/120 or around 83 percent. In this case, the deadweight would amount to 83 percent.
This way the section under ‘deadweight’ can be filled out in the impact map in the SROI analysis.
Calculating deadweight also measures social value towards outcomes for stakeholders that are hard to reach or for whom deadweight is low. An increased deadweight means the stakeholder group is being already targeted and is receiving similar benefit. A low deadweight means your contribution to the outcome is significant, exhibiting that the change achieved is due to the support received by your activity.
Deadweight is measured as a percentage which will be deducted from the total quantity of the outcome.
Now, we come to the concept of ‘displacement’ which is another part of ‘impact’ in the SROI analysis. . As the name suggests, it is an assessment of how much of the outcome displaced other outcomes. This is a concept which was discussed previously while we were learning ways to identify stakeholders.
The same example would work here to understand the concept of displacement. The development of public spaces around West Java may attract more investors and builders to the region and as a result, those in the neighbouring areas could bear the brunt of this shift. This would be an example of an outcome displacing another outcome. In such a scenario, a new stakeholder group could be included in your analysis. You can consider estimating the percentage of your outcomes that are double-counted because of any displacement and deduct it from the total. However, it may not occur in every SROI analysis.
Remember, the previous stage was about identifying whether the outcome set by your organisation would have happened anyway and to what degree. In this stage, you will factor in any other party, individual or organisation which would have contributed to the outcome. “It shows the part of deadweight for which you have better information and where you can attribute outcomes to other people or organisations,” Social Value UK states in its guide.
Assigning complete and accurate attribution would be difficult. This step of the analysis is to be aware of the fact that your activity may not be exclusive and that others could be contributing to the outcome as well. This will call for reassessing your stakeholders and include those parties who could be contributing to the outcome but are somehow not included in your scope. You may also want to take contributions made by other organisations and individuals in the past into account. It may also be useful to consider such stakeholders who had other support in the past.
It may be useful to reconsider your stakeholders and club them into groups based on their experiences before their involvement with your activity. The need to estimate attribution is crucial and here is how you can calculate it in three easy steps:
- Your estimate can be based on your experience. You may already be dealing with various other organisations or individuals to know and estimate how they contribute towards the outcome.
- As stated several times previously, it is always a good practice to engage with your stakeholders. It is best to ask them how and to what degree has your activity impacted or resulted in the outcome.
- You can also engage with other parties – organisations, individuals – involved in the outcome to assess their contribution toward it so that you can assign an accurate estimate to it and work out percentages accordingly.
Tip: Don’t spend too long on this exercise, however, ensure that you are in a position to explain how you reached your estimate.
For example, during an interaction with the elderly population living around West Java, it was found that their decision to use the newly constructed open gym and park was also primarily owing to a weekly workshop conducted by another NGO that emphasized the significance of the regular physical activity. It was estimated that nearly 15 percent of the outcome was the result of the contributions of this NGO.
This way, the attribution column in your impact map in the SROI analysis can be filled. Again, maintain a rationale behind this estimation and also relevant information about organisations or individuals relating to the attribution and the outcome.
In previous stages, we had learned about the period of outcomes – while some can be seen immediately, others take time to take shape and may last longer than others. Outcomes that last long may also be affected by several other factors in the long run and, in most cases, their amount is also likely to be less. In such a scenario, attribution to your organisation gets lower. This is where the concept of ‘drop-off’ enters and is calculated for outcomes that last over a year.
How to calculate drop-off?
It is usually calculated by deducting a fixed percentage from the remaining level of the outcome at the end of each year. You will need to study and research the amount (percentage) by which the outcome drops off annually. You can turn to some historical data or can base your estimate on thorough research from academic sources, case studies or parties who have carried out similar activities in the past.
For example, an outcome of 100 lasts for about two years but drops off by 10% annually. In such a case, the outcome will be 100 in the first year but will drop to 90 in the second (100 less 10%). In the long haul, you will need to have a more robust system to measure this more accurately.
Thus, the drop-off column in the impact map can be filled. Ensure maintaining the rationale behind this estimation and also other relevant associated data with the stage that will go into your final SROI report.
4.d Calculating SROI impact
Now we come to the final step in the stage which deals with the calculation of ‘impact’. For this, we need to go back to the estimates that we made in the previous stage. Remember, we had assigned value to the outcomes and had quantified them too in the previous stage? Now is the time to use those values.
The impact is calculated by multiplying ‘financial proxies’ and ‘quantity’ of the outcomes. The resulting figure will be used from which ‘deadweight’ will be subtracted. Now, deduct ‘attribution’ from the resulting figure.
For example, when the elderly population paid fewer visits to the doctor as a result of better health, it led to savings worth Rs $80 per person for three months on the clinic visit. The elderly visited a general practitioner a total of 6 times in three months as opposed to ten visits previously, so the change here (quantity) will be four visits. To calculate impact, we will first multiply $80 and 4 and deduct deadweight and attribution from the final value.
Total outcome: 4 x $80 = $320.00
- $320 – 83 percent
- $320×83/100 = $265
- $320-$265 = $55
Next, attribution will be deducted, or how much of the change was due to others involved.
- $55 – 15 percent
- $55×15/100 =$8
- $55-$8 = $47
Note: Refer to the examples provided in the ‘deadweight’ and ‘attribution’ sections of this blog to understand the corresponding values involved in the above calculation.
Now, you can complete the impact column on your map. This wraps up the fourth stage of the SROI analysis. The remaining two stages of the calculation will be discussed in the subsequent blogs.