2.4 Managing the Analysis Process

Performing systematic and comprehensive impact analysis can be a significant task, requiring input from both investor and investee sides. It can also, through the findings it makes, play a significant role in how investment decisions are made, and capital allocated. For both reasons, it is essential investors are transparent about their impact analysis processes, and takes steps to ensure their quality. This implies attention to:

2.4.1 Transparency

Is there transparency toward investees regarding the analysis?
Do investees know: what to expect; how and on what grounds they will be analysed; and how the process can be of use to them?

Being clear with potential investee organisations — when they are first seeking capital, and then at the more involved impact analysis and financial due diligence stage — is crucial. Investees need to understand what the process will be, firstly to make a realistic assessment if they can and want to go through with it, and secondly to ensure buy-in throughout the process. Impact analysis, as well as the subsequent reporting it implies, can be demanding upon investee organisations, and it is important they do not feel they are being subjected to a series of unexpected or arbitrary-seeming requirements.

Maintaining transparency relates to four main points:

  • timeline

    Investors must be clear as to what the timeline of the process will be: How long will the whole process take? How long will the in-depth analysis take? When will the capital become available?

  • content

    Investors must be clear as to what will be involved: What will investee organisations be expected to produce — in terms of financial information, information about the activities and impact plan, and supporting research and so on?

  • use

    Investors must be clear as to how the information will be used. The full details of any analysis system an investor is using, either for credit checks or impact purposes, may well be confidential. Transparency however need not mean absolute disclosure, or full publication on a website. It is important investees know the use to which the information they supply will be put, in part to be able to believe in the process, and in part to ensure they are indeed supplying the right information. Also if an investor has a particular emphasis or interest that will weight the analysis in a certain direction (for example, preferring specific outcomes, strategies or approaches over others), making this clear to applicants is only fair.

  • usefulness

    The purpose of impact analysis, and subsequent reporting demands, is not to create a burden, but to ensure that capital is being used effectively to generate the desired outcomes and impact. This is of interest to the investee organisation as much as to the investor. The organisation potentially has a great deal to gain from working through the impact plan and measurement system collaboratively with the investor, and ensuring appropriate indicators and reporting mechanisms are in place. Many social purpose organisations are only now starting to develop robust impact systems, and investors can play a useful and supportive role in this regard (as they often do with helping organisations develop their financial accounting and management — see investor impact). The process of undergoing impact analysis can contribute to the organisation being able to manage its own impact better, as well as being in a stronger position to bid for contracts and funding in the future.

2.4.2 Quality Control

Are there procedures in place to review and improve the consistency of results?
Is the system itself subject to regular review?

The power of analysis is the extent to which it facilitates an informed understanding and decision-making process — and for this very reason, it is essential to ensure that the information it provides, and the understanding it engenders, is actually meaningful. Having a systematic approach to impact is the first step, but in addition to creating a system, it is necessary to have a degree of quality control over that system’s implementation and use.

Quality control implies attention to consistency and system-review:

  • consistency

    As with all aspects of impact, the most important part is clarity. Consistent analysis relies firstly upon having a clear structure, but also a clear process as to how to use it. The process covers:

    • how to go about doing analysis
    • what information is needed for the different parts of the analysis
    • what to do when information is lacking

    The aim of the structure and accompanying process guidance is that the analysis can be applied to different organisations by different analysts, and produce consistent and comparable results.

    Even with a clear and well laid out system, the analysis will inevitably involve a mixture of quantitative accounts and qualitative values, which will imply a degree of qualitative judgement. The consistency and merit of these judgements can be greatly improved by a peer-review process. It may be useful to have more than one person working on the analysis, to allow for internal reviewing of judgements, or for the analysis to be reviewed once completed (e.g. by the Head of Impact). It is generally advisable to ensure at least two people have been through a piece of impact analysis before it is presented to the investment committee. When there are multiple officers performing impact analysis separately, it is advisable for these officers to meet regularly to discuss how they are using the system, what results they are getting, and to ensure there is reasonable consistency on these two points.

  • system-review

    Once a system is in active use, it is important to review it regularly. There are two key questions for a review:

    • is the system providing information that we can use in investment-decision making — i.e. are the results of analysis meaningful to what we do as investors?
    • is the system proving to be accurate? Are there any identifiable sources of bias or inconsistency? Is the analysis being borne out by what happens: i.e. is the volume of impact being generated what we expected from our initial analysis? Do the levels of impact risk we initially associated with investments correlate with their subsequent success in generating impact? If there are discrepancies, can these be explained?

It may also be appropriate at reviews to consider any recent developments in the field of impact analysis, and advances in the understanding of best impact practice.

Within an impact investment fund, it is likely there will be a number of people who need to be familiar with the impact analysis process. Among them will be the loan officers or investment team, who are involved at the primary level in sourcing deals, and the financial and impact teams (if different) who will perform the analysis and produce reports for the investment committee. Within these teams there should be a degree of impact expertise, including knowledge of the common challenges and concerns involved, as well as of key sector issues and obvious pitfalls, including ways in which data or measurements can be manipulated, and impact plans be either unrealistic or incomplete.

Likewise the members of the investment committee, who will need to read the impact reports and consider the analysis in making their investment decisions, will need to be versed in how the analysis works, and have a firm understanding of the terms and concepts involved. Also, the ongoing monitoring and evaluation of existing investments will require an ongoing attention to impact, and a knowledge of impact analysis on the part of those performing this task.

Given the reach of impact analysis throughout the operations of an impact fund, it is important to have a person who is responsible for overseeing the process as a whole, ensuring the consistency of results, and attending to the ongoing management. This person may be given the title of “Head of Impact”. The responsibility for managing impact across the fund may or may not be a full time position in itself — what is however crucial is that there is a clearly identified person with such a role, and that the duties implied are acknowledged to be a significant part of that person’s overall job (i.e. the title “Head of Impact” is not simply appended to an already full-time position).

Website built and hosted by Strangecode