Innovation in Government Initiative — General Support

Published: January 2019

Note: This page summarizes the rationale behind a GiveWell-recommended grant to the Innovation in Government Initiative (IGI). This grant is part of our exploratory work into opportunities to help improve the uptake and implementation of evidence-based policy by governments. This grant relies on more subjective and difficult to justify judgment calls than GiveWell’s typical Incubation Grants or recommended top charities.

IGI staff reviewed this page prior to publication.

Summary

As part of GiveWell’s exploratory work into opportunities to improve the uptake and implementation of evidence-based policy by governments, in December 2018, GiveWell recommended the Effective Altruism Global Health and Development Fund make a grant of $1,000,000 to J-PAL’s Innovation in Government Initiative (IGI). From 2015 until December 2018, IGI was known as the Government Partnership Initiative (GPI).

IGI is a grantmaking entity within the Abdul Latif Jameel Poverty Action Lab (J-PAL). It plans to make grants to partnerships between governments and J-PAL offices and affiliated researchers to help pilot and scale evidence-informed programs in education, health and social assistance.

The grant will be used to support IGI’s general operating costs and run two requests for proposals (RFPs).

Table of Contents

The organization

The Innovation in Government Initiative (IGI) is a grantmaking entity that sits within the Abdul Latif Jameel Poverty Action Lab (J-PAL) at MIT. IGI currently has two staff members, each spending about half their time on IGI. Until December 2018, it was known as the Government Partnership Initiative (GPI).1 Funding decisions are made by a seven-person Advisory Board based on the criteria in this footnote.2

GPI made 35 grants totaling $2.3 million.3 Grants were typically under $100,000, with no single project receiving more than $300,000 over its lifecycle. Grants can support technical assistance to governments from J-PAL offices and/or affiliated researchers.4 In the past, GPI made three different types of grants:

  • Type 1: Research partnerships between governments and J-PAL-affiliated researchers with the goal of answering a question of policy-relevance to a particular government.5 GPI made eight Type 1 grants totalling $517,000.6
  • Type 2: Technical assistance to scale up policies which have been evaluated by a randomized controlled trial (RCT).7 GPI made eight Type 2 grants to six different projects totalling $590,000.8
  • Type 3: Technical assistance to institutionalize the use of evidence in policymaking.9 GPI made 20 Type 3 grants to 17 different projects totaling $1.2 million.10

In RFPs over the course of this grant, IGI plans to fund primarily Type 2 grants, as it believes they have been the most successful in contributing to the scale-up of evidence-informed programs so far.11

How will the grant be used?

IGI intends to use this grant to partially fund its operating expenses, and to run two RFPs focused primarily on Type 2 scale ups over the course of two years.12

Why did we start investigating this grant?

We started investigating IGI (called GPI at the time) as part of our work researching opportunities to help governments improve the selection and implementation of evidence-based policies in low- and middle-income countries (more here). We have had some initial conversations with other organizations working in evidence-based policy, but do not yet have a strong understanding of the ecosystem, or the relative strengths and weaknesses of different organizations. We believed this grant was worth investigating before a broader scoping of the funding landscape because:

  1. GPI sent us a draft case study of one of their grants, which appeared to result in substantial savings to the Indian government (more on this case study below).
  2. As the policy appeared to be tested through a randomized controlled trial we found credible, the case appeared to be amenable to GiveWell’s criteria for identifying evidence-based cost-effective giving opportunities.
  3. We believed that investigating one grant in some depth would help us learn more about how we should conduct these investigations in general.

The case for the grant

This grant relies on more judgment calls than the majority of GiveWell’s Incubation Grantees or top charities.
The case for the grant is:

  1. A previous IGI-funded project appears to have had some success informing government policy in India. We have not seen strong evidence that this particular policy change had a positive effect (although our best guess is it did), but it suggests to us that IGI will be able to successfully identify opportunities to influence government policy. A rough break-even analysis suggests that, under plausible assumptions, IGI is comparably cost-effective to marginal dollars to our top charities (more below).13
  2. Our understanding is that IGI has substantial room for more funding, and has had difficulty raising funds in the past. This grant may give IGI sufficient runway to raise additional funds from other funders, or preserve option value if we decide to renew the grant (more below).
  3. Our subjective impression is that IGI is a strong organization. IGI’s staff have communicated well with us, and have been open about the limitations of the evidence for assessing their impact.

Reservations about the grant

GPI submitted two alternative funding requests; for $3.1 million, and $5.7 million.14 We decided to give less than that because:

  • The case study we reviewed was substantially less straightforward than it appeared at first (more below). This invalidated our main reason for investigating GPI ahead of a more thorough review of other groups working on influencing or informing government policy with evidence. We are cautious of overcommitting to a single organization without carefully reviewing the broader ecosystem of evidence-based policy.
  • We currently believe it is unlikely we will review similar organizations in the short term (although we would like to do so in the long term). We are cautious of overcommitting to funding an organization which falls outside our short-term priorities.

Case study of a grant GPI made

GPI initially approached GiveWell with case studies of previous historic grants it believed had an impact.15 After an initial shallow review of these studies we focused on one of them, a $70,000 project to provide technical assistance to a reform of how funds flowed through the Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGS), a large social program in India. We focused on this case study because the estimated impact appeared to be large enough to make GPI more cost-effective than our top charities.16

After reviewing the case study in more detail, we concluded that the data that is available from the Indian government on the flow of MGNREGS funds and related metrics is not sufficient to accurately measure the true impact of the reform to which GPI contributed. However, while we lack strong evidence to support this conclusion, our best guess is that GPI’s support likely played an important role in a reform which resulted in substantial savings to the Indian government.

Given IGI's limited spend to date, we think the expected benefits of this case study, when taken in conjunction with the other reasons given above, are sufficient to justify a grant to IGI.17

We discuss the case study below.

Timeline of events

GPI sent us a timeline of events leading to the reform of MGNREGS. We summarize the key points below.18

  • In 2012, J-PAL-affiliated researchers, collaborating with Mr. Santhosh Mathew, a senior Indian Administrative Service officer, conducted an RCT on a financial reform of The Mahatma Gandhi National Rural Employment Guarantee Scheme (MGNREGS) in the Indian state of Bihar (Banerjee et al. 2016).19
    • The intervention involved "delayering" the system by requiring fewer administrative levels to approve each fund request, and basing transfer amounts on documented rather than forecasted expenditure.20 The RCT found a 24% reduction in expenditure in the treatment group relative to control, statistically significant at the 1% level.21 The authors present suggestive evidence that the reduction in expenditure was likely due to a combination of reduced "leakages" (missing funds) and reduced "fund float" (funds idling, not earning interest), rather than reduced participation in the program.22
  • Between April and August 2015, a J-PAL South Asia policy manager provided research support to Mr. Mathew to help him make the case for further delayering MGNREGS.23 This support was not funded by GPI, as the initiative had been created, but had not yet launched its funding competitions.
  • In August 2015, the Indian government issued a Cabinet note approving a national reform to MGNREGS.24 The new reform meant funds would flow directly from the (central) Government of India to the beneficiary’s account, rather than being held in the State consolidated fund.25 The Cabinet note estimated potential savings of 500 crore Indian rupees ($80 million) each year from the reform due to the interest rates savings from reduction of idle funds.26
  • In August 2015, GPI provided a grant to hire a consultant to provide full-time research support to Mr. Mathew (replacing the J-PAL policy manager).27 This consultant assisted Mr. Mathew with making the case, and drafting an implementation plan for the delayering of other government programs.28
  • In May 2016, Mr. Mathew presented the implementation plan to the Prime Minister’s Office.29
  • In July 2016, the Ministry of Finance issued a note mandating that all central government schemes implement the financial management system which was needed to roll out “just in time” release of funds.30
  • In March 2017, a newspaper article claimed that almost all states had been integrated onto the new financial management system, resulting in substantial savings to the Government of India.31

Did GPI play an important role in the fund flow reforms?

We believe that J-PAL’s assistance to Mr. Mathew likely played an important role in the decision to scale up the reform because:

  • Mr. Mathew told us that his other duties meant he likely would have been unable to make the case for the reforms mandated in August 2015 or July 2016 if he had not received support from J-PAL or GPI.32
  • The August 2015 Cabinet note cited Banerjee et al. 2016, which Mr. Mathew co-authored, in support of its decision.33 This suggests to us that (a) the RCT played a role in the decision to scale up the reforms, and (b) Mr. Mathew was a key player in advocating for these reforms at an early stage.

On the other hand:

  • The August 2015 Cabinet note was issued before GPI made a formal grant to support Mr. Mathew’s efforts in making the case for adopting these reforms. Mr. Mathew was previously supported by a J-PAL policy manager. J-PAL South Asia told us that the reason they were able to assign someone to provide this support was because they knew that GPI funding would soon become available and the project fit very well with the type of projects that GPI was set up to support. We are uncertain if this was the case (due to the difficulty of knowing what would have happened otherwise).34
    While the work this policy manager did to assist Mr. Mathew appears to be reasonably representative of the grants we expect IGI to consider in the future, it suggests that J-PAL may be able to pursue the most promising opportunities without IGI, at least in the short-run.35
  • A number of other actors played an important role in causing fund flow reforms to MGNREGS. We do not feel confident in ascribing a percentage figure to the chance that these reforms would have happened without J-PAL’s assistance.36 In our breakeven analysis, we assume a 20% chance GPI's support accelerated the reform by three years. Our best guess, based on the information above, is that this is a conservative assumption.37

    Did the fund flow reforms result in savings to the Government of India?

    We believe it is more likely than not that the reforms mandated in August 2015 and July 2016 did lead to savings through reduced fund float because:

    • The August 2015 Cabinet note estimated $80 million of annual savings from the reform (before it was implemented) through the reduction of fund float.38
    • A press article on 4 March 2017 reported that the Controller General of Accounts for India claimed that reductions in parking of funds would help save $1.6 billion each year when rolled out to all central government schemes (not just MGNREGS).39
    • While we place limited weight on these figures (we do not know how they were calculated), we believe it is more likely than not that the reform led to substantial savings because there is a plausible mechanism through which we would expect these savings to be realized. Specifically, transferring funds directly from Central to the beneficiary would reduce the amount of funds "parked" in the system (not earning interest) at any time.

    However, we do not believe Banerjee et al. 2016 constitutes strong evidence that the reform outlined in the August 2015 Cabinet note would have led to savings by the Government of India because:

    • Banerjee et al. 2016 removed the need for District and Block approval of fund requests while the reform in the August 2015 Cabinet note changed how funds were transferred by removing the intermediate step of the funds being held in a State Consolidated Fund.40 These two reforms seem sufficiently different that we would not expect to be able to extrapolate from the results of Banerjee et al. 2016.
    • The August 2015 Cabinet Note notes that, by that time, the majority of villages in India had already implemented a system similar to the one evaluated in Banerjee et al. 2016.41

    We concluded that the data that is available from the Indian government on the flow of MGNREGS funds and related metrics is not sufficient to accurately measure the true impact of the reforms to which GPI contributed. We have therefore not seen strong evidence that the fund flow reforms led to savings for the Government of India.

    Room for more funding

    We believe IGI can productively absorb $1 million of additional funding because:

    • In January 2017, when we started investigating GPI, it had $600,000 remaining in funds, and expected to award all of those by the end of the year.42
    • IGI gave us plausible reasons for which they have had difficulty fundraising, which are broadly consistent with our understanding of the philanthropic ecosystem.43
    • IGI told us that J-PAL endowment funding is restricted to core staff and general operating expenses. J-PAL funds its other projects and initiatives separately.44

    As IGI is now planning to focus primarily on Type 2 grants, we believe it is likely that the most promising grant opportunities of this type will be funded by this grant over the next two years. However, due to the strategic considerations listed above, we are less likely to renew this grant than we would be for a typical GiveWell Incubation Grantee.

    Why are we recommending a grant through Effective Altruism Funds?

    We decided to recommend this grant through the Effective Altruism Global Health and Development Fund because our understanding is some donors give to that fund because they want to signal support for GiveWell making grants which are more difficult to justify and rely on more subjective judgment calls, but have the potential for greater impact than our top charities.45

    If we had not funded this grant through the Effective Altruism Global Health and Development Fund, we would have sought funding from Good Ventures through the Open Philanthropy Project, who have generally followed our recommendations in the past.

    Plan for follow up

    We plan to have conversations with IGI every few months to check in on progress, with a major check-in 10-12 months after the initial grant is received.

    Internal forecasts

    For this grant, we are recording the following forecasts:

    Confidence Prediction By Time
    35% GiveWell makes another grant to IGI January 1, 2021
    60% IGI is able to raise more than $1 million in funding from other sources January 1, 2021

    Sources

    Document Source
    Banerjee et al. 2016 Source
    Effective Altruism Global Health and Development Fund Source
    Exchange rates Source
    Financial Express press article, 4 March 2017 Source
    GiveWell conversation with Santhosh Mathew, 25 June 2018 (unpublished) Unpublished
    GiveWell's breakeven analysis of GPI Source
    GiveWell's non-verbatim summary of a conversation with Claire Walsh and Samantha Carter, February 16, 2018 Source
    GiveWell's non-verbatim summary of a conversation with Claire Walsh and Samantha Carter, January 17, 2018 Source
    GiveWell's non-verbatim summary of a conversation with Iqbal Dhaliwal, February 16, 2018 Source
    GPI Application form and instructions Source
    GPI draft funding scenarios (unpublished) Unpublished
    GPI expenditure 2015-2017 Source
    GPI learning document (unpublished) Unpublished
    GPI previous grants Source
    GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished) Unpublished
    GPI Two year funding scenario Source
    J-PAL Government Partnership Initiative website Source
    J-PAL GPI Concept Note Source
    MGNREGA data portal Source
    Ministry of Finance Memorandum, July 2016 Source
    Ministry of Rural Development Cabinet Note, June 2015 (unpublished) Unpublished
    Unpublished email from Claire Walsh on November 29, 2018 Unpublished
    Unpublished email from Claire Walsh on October 24, 2018 Unpublished
    • 1

    • 2
      • "Policy relevance: Does the study address questions crucial to the government partner?

        Evidence-informed policy: How will this project further J-PAL’s mission to promote evidence-informed policy and build long-term partnerships with governments?

        Viability of the partnership: Is the relationship with the government and other partners strong and likely to endure through the entire life of the project? Are there any logistical or political obstacles that might threaten the completion of the proposed activities, for example, government authorization or human subjects review?

        Commitment to use evidence in decision-making: Is there demonstrated demand from the government partner to use evidence from the proposed research or past research to make a key decision or institutionalize the use of evidence in a decision-making process? Is the government committing its own resources, especially finances, to this project?

        Scale-up potential: Is there potential to widely scale-up either a proven policy or the process of using evidence in policymaking? What steps will the project take to gather program costs, document implementation and scale-up processes, and disseminate them so others may also benefit?

        Institutional support: Since building partnerships with governments requires long-term commitments and on-the-ground presence, does the project have necessary institutional support of the regional J-PAL office? What other local institutional support is available? (e.g. IPA country office, strong local staff)

        Level of J-PAL affiliated researcher involvement: What is the level of J-PAL affiliated researcher involvement? Not just in research projects, but also in proposals seeking to strengthen evidence-informed policy in governments; ongoing leadership, guidance, and advice from affiliates provides significant value to the government and J-PAL staff." GPI Application form and instructions

      • "The GPI Advisory Board comprises seven members: Mr. Dhaliwal and Dr. Abhijit Banerjee, who are GPI’s co-chairs, as well as five review board members. The review board includes one affiliated researcher with extensive experience in each of J-PAL’s geographical regions, so that for every proposal there is someone on the board who understands the regional research and policy context.

        Once the proposals have been received, Claire Walsh (Senior Policy Manager, J-PAL) and Samantha Carter (Senior Policy Associate, J-PAL) create a matrix of the sectors and regions covered by each proposal, then map that matrix to the sectoral and regional expertise of the seven advisory board members, and allocate the projects accordingly.

        Simultaneously with creating the matrix, Ms. Walsh and Ms. Carter reach out to the regional offices under whose ambit the project was submitted, and ask several questions, including:

        • Is the proposed partnership or research question policy-relevant for the government and region?
        • What other organizations might be trusted to provide input on this question?
        • If this partnership and/or study succeeds, do you also think other policymakers will be interested in the results?

        The board looks at the information Ms. Walsh and Ms. Carter have collected, reviews the proposals, reads the letters submitted by implementing policy partners, and then meets for several hours to make a decision on each proposal." GiveWell's non-verbatim summary of a conversation with Iqbal Dhaliwal, February 16, 2018

    • 3

      See GPI previous grants for a list of all GPI grants.

    • 4
      • Four of GPI’s 35 grants in the past have totaled over $100,000. The project to receive the most funding over its lifecycle is a project to improve immunization rates in Haryana ($250,000 over two grants). See GPI previous grants for more details.
      • IGI has told us that it plans to grant a maximum of $300,000 to any single project in the future. Unpublished email from Claire Walsh on November 29, 2018

    • 5

      "Research grants are integral to the overall GPI enterprise, because some of J-PAL’s biggest scale-ups in the past have involved a government and a group of researchers working together to co-design an intervention and evaluation in response to either a policy window or a government partner’s urgent need. In these cases, the evaluation is designed to answer a specific question with the goal of informing a particular policy decision, which is ideal because the government is already committed to using the evidence to address a specific problem. We like to provide funding to these policy-relevant RCTs where we have confidence the results will be applied. In our experience, there are generally about two to four years between the beginning of an RCT and the implementation of a policy informed by that research." GiveWell's non-verbatim summary of a conversation with Claire Walsh and Samantha Carter, January 17, 2018

    • 6

      See GPI previous grants for a list of all GPI grants.

    • 7

      "Scale-ups have the potential to inform a particular policy on the shortest time horizon, because at the time of application the government is already at the point where it has committed to scaling something up based on evidence. Implementation of a scale-up usually happens one to two years after the grant application, barring unforeseen political challenges. Three of the six scale-ups we have funded so far have already resulted in implementation." GiveWell's non-verbatim summary of a conversation with Claire Walsh and Samantha Carter, January 17, 2018

    • 8

      See GPI previous grants for a list of all GPI grants.

    • 9

      "Type 3 grants are important because having long-term collaborative partnerships with governments on the ground is the only way that we’re able to develop a clear pipeline of Type 2 and Type 1 projects. Rather than only influencing one program decision, Type 3 grants are meant to help change the entire relationship of the government to evidence by creating the guidelines, institutions, incentives, and norms that help make evidence-informed policymaking the rule rather than the exception." GiveWell's non-verbatim summary of a conversation with Claire Walsh and Samantha Carter, January 17, 2018

    • 10

      See GPI previous grants for a list of all GPI grants.

    • 11

      Unpublished email from Claire Walsh on November 29, 2018

    • 12

      Unpublished email from Claire Walsh on October 24, 2018

    • 13

      GiveWell's breakeven analysis of GPI

    • 14

      GPI draft funding scenarios (unpublished)

    • 15

      GPI learning document (unpublished)

    • 16
      • GPI made two grants, totalling $33,092 and $36,828 to this project. See GPI previous grants for more details.
      • GPI expenditure was $2.59 million in total up to December 2017 (GPI expenditure 2015-2017).
      • We conducted an initial, rough cost-effectiveness estimate of GPI's historic work, assuming: (i) $131 million saved annually (a preliminary estimate that GPI shared with us in a draft case study they have asked to keep confidential) (ii) that, conservatively, this case study was the only GPI grant to result in positive benefits (iii) GPI gets 100% of the credit for the savings to the Indian government (iv) GPI's work accelerated the reform by three years. Under our current valuation of the benefits of saving the Indian government funds, this resulted in an estimate that GPI was ~100 times as cost-effective as GiveDirectly. After further investigation of the case study, we no longer believe this estimate is informative, but include it here to be transparent about our reasons for investigating GPI. GiveWell's breakeven analysis of GPI (Sheet: Initial BOTEC)

    • 17

      We created a rough break-even analysis to see what we would have to believe about the savings to the Indian government for IGI to be comparably cost-effective to our top charities. We conducted this analysis as a rough calibration exercise and it relies on more subjective assumptions and judgment calls than our typical cost-effectiveness analyses. We do not have a robust quantified view on the expected future cost-effectiveness of IGI.

      We concluded that, if we (fairly arbitrarily) assign 20% of the "credit" for the scale up to GPI’s support, and conservatively assume this case study was GPI’s only historical impact, we would have to believe the reform of NREGA saved the government of India in the region of $150 million (in total, rather than each year) to be comparably cost-effective to marginal dollars to our top charities.

      GiveWell's breakeven analysis of GPI

    • 18

      The timeline is consistent with conversations we had with Mr. Santhosh Mathew, the ISA officer who GPI supported. We believe the events in this timeline took place, but have only been able to corroborate or falsify parts of the timeline with other sources. GiveWell conversation with Santhosh Mathew, 25 June 2018 (unpublished)

    • 19
      • "In collaboration with the Government of Bihar, India, we conducted a large-scale experiment to evaluate whether transparency in fiscal transfer systems can increase accountability and reduce corruption in the implementation of a workfare program. The reforms introduced electronic fund flow, cut out administrative tiers, and switched the basis of transfer amounts from forecasts to documented expenditures." Banerjee et al. 2016, Abstract.
      • Mr. Mathew was a coauthor on the study. Banerjee et al. 2016, Pg 1.

    • 20

      "In the status quo system, funds flowed through four tiers of administrative hierarchy on their way from the Department of Rural Development to the village authority: state, district, block and Gram Panchayat (GP). The GP could request advance funds without specifying intended purpose, but authorities at intermediate levels of the hierarchy (the block and the district) had to approve the request before it went to the state treasury. In the reformed system, fund disbursement to a GP for labor expenses was based on incurred expenditures. Specifically, GP officials entered the names of those employed and wages owed in a central database, which automatically triggered fund release into the GP account. The GP official no longer required approval from block or district officials for the submission of the fund request (although many block officials remained involved, as the data-entry infrastructure was typically only available at the block level). All other aspects of the fund-flow process remained unchanged." Banerjee et al. 2016, Pg 3.

    • 21
      • "By April 1, 2013, MGNREGS expenditure in treatment GPs relative to control had declined by 17% and GP account balances were reduced by 30%. Panel C in Table 2 shows that the combination of lower spending and a decline in idle funds in the treatment GP accounts, reduced program expenditure by 24% in treatment GPs. This, in turn, translates into a cost saving of roughly 6 million dollars." Banerjee et al. 2016, Pg 19.
      • Estimated treatment effects and standard errors are reported in Banerjee et al. 2016, Table 2, Pg 36.

    • 22
      • "Our empirical analysis focuses on identifying this net impact. A key contribution is to provide robust evidence of declines in leakages by triangulating across multiple data sources, including administrative databases and independent surveys. First, using administrative data on daily GP finances, we show a 17 percent expenditure reduction in treatment GPs relative to control GPs. We corroborate the decline with spending data reported in the MGNREGS public access database where we also observe a corresponding decline in the reported number of hired workers in treatment GPs. Meanwhile, in an independent household survey, we find that the number of beneficiaries, the wage payments received and assets built are statistically indistinguishable across treatment and control GPs. While this seems to be evidence that the reform reduced leakage, a remaining concern is that the relatively small survey sample size may have limited our ability to identify employment declines in treatment GPs. We, therefore, bolster our analysis with two pieces of direct evidence on the reduction in corruption.

        First, we construct a measure of leakage that fully exploits the scale of our experiment and the large amount of data available. To systematically identify "ghost workers" (households who are reported to have worked but, in fact, do not exist), we take the 6,292,307 names that the public database reports as having worked on the program and match them with names from the Socio-Economic Census, which the Government of India conducted in 2012 (and itself yields a database containing 34 million names for the 12 districts of our sample). First, using a Hindi-specific Levenshtein linguistic algorithm we match the names of our approximately 18,000 sample villages across the two databases. Then we use the same method to match household names within a given village. The name of a ghost worker should fail to be matched, except in the rare case where there are two persons with the same full name and gender in the same village. This matching-based strategy systematizes and implements at scale the audit approach pioneered by Niehaus and Sukhtankar (2013) and also used by Muralidharan et al. (2014) where investigators physically track down workers reported in the public database. The matching process is imperfect (there are errors in both directions); however, the scale of our experiment allows it to serve as a statistical test of the impact of the reform on corruption. The fraction of unmatched households is significantly lower in GPs where our reform was implemented: for example, 35.5 percent of single-worker households (which make up the majority in our sample) are unmatched in the control group, compared to 33.6 percent in treatment villages – a reduction of 5 percent. This difference is absent outside the reform period.

        Our second measure of corruption traces the "missing money" by examining affidavit data on public employee assets reported just after the reform period. We find that median wealth of block and GP officials is 14 percent lower in treatment relative to control areas, and a Kolmogorov-Smirnov test rejects equality of the two distributions. If we use mean wealth as the measure, the decline is of similar magnitude, though more noisily estimated. Taken at face value, the point estimate would imply that this decline in officials’ wealth accounts for half of the savings the reformed program achieved" Banerjee et al. 2016, Pgs 4-5.

      • "Turning to other dimensions of program performance, we observe a decline in idle funds sitting in GP accounts, which represents an implementation efficiency gain from a public accounting perspective, since disbursed funds are considered a government expense. Specifically, the reform reduced fiscal transfers in treatment GPs by 24 percent, of which the decline in expenditure accounted for two-thirds. The other one-third reflects a decline in idle funds in GP accounts. It is conceivable that treatment, everything equal, increased the budget available for the program in all GPs (treatment and control), by freeing up funds that were previously idle balances. On the other hand, the reform did not directly improve program delivery for villages or beneficiaries: we do not see an increase in the number of work-days or constructed assets in treatment GPs, and we do see an initial increase in delays in payment for beneficiaries in the treatment group, though they declined over time." Banerjee et al. 2016, Pg 5.

    • 23
      • "Immediately following this meeting Santhosh Mathew requested J-PAL South Asia to provide full time research support on reforming the funds flow. From April-August 2015, J-PAL South Asia policy manager [redacted] provided research support to Mr. Mathew on his efforts to reform e-FMS for NREGA based on the RCT evidence and expand the efforts to other GoI schemes." GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished)
      • "Among other things, [Person 1] worked with Mr. Mathew to create a report and presentation slides to help him make the case for the reforms." GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished)

    • 24
      • In August 2015, a Cabinet Note approved the reform of the NREGA e-FMS. The Cabinet Note was issued in June 2015 and approved in August Paraphrased from GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished)
      • Ministry of Rural Development Cabinet Note, June 2015 (unpublished)

    • 25

      “Motivated in part by the results of this experiment, in August 2015, MGNREGS officials put in place a nationwide system that combined direct payment to beneficiary bank accounts (though not always based on a smart card) and expenditure-based transfers.” Banerjee et al. 2016, Pg 27.

    • 26
      • Ministry of Rural Development Cabinet Note, June 2015 (unpublished)
      • We use the exchange rate of 62.125481 INR to 1 USD from 1 April 2015. See Exchange rates.

    • 27

      "While [Person 1] was working on this project, J-PAL SA started recruiting for a full-time person to replace her on this project in May/June 2015 so [Person 1] could return to her other work. In August 2015, J-PAL South Asia hired [Person 2] as a Policy Consultant to work full-time providing research support to Dr. Mathew. [Person 2]’s work was to help Dr. Mathew make the case for funds-flow reform in other Central and Centrally-Sponsored Schemes based on the evidence from the Bihar RCT. [Person 2] worked with Mr. Mathew to draft concept notes and presentations to share evidence on fund-flow reforms, policy lessons from the Bihar study, and tailored scale-up advice for individual Ministries. She also conducted research on public service delivery best practices around the world, led media outreach on the fund-flow issues, worked with Dr. Mathew to create a policy implementation plan to propose to the Prime Minister’s Office (PMO), published op-eds and academic articles, and supported discussions with the Ministry of Health and the Ministry of Water and Sanitation about reforming their schemes." GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished)

    • 28

      "In January 2016, the Prime Minister invited all of his secretaries to a meeting and asked them to pitch innovative government reforms to his office. At the meeting, Santhosh pitched the idea of including the fund flow reform to a working group on innovative budgeting. The working group really liked the idea and included it in their proposal. The Prime Minister’s Office also liked the idea and requested a policy implementation plan, which Mr. Mathew helped develop with support from the GPI funded staffer. The plan recommended moving government programs to "Just in Time" budgetary releases, a fund flow system designed based on the reforms tested and found effective in the Bihar RCT." GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished)

    • 29GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished)

    • 30

      “The Public Financial Management System (PFMS) which is administered by the Controller General of Accounts in the Department of Expenditure is an end-to-end solution for processing payments, tracking, monitoring, accounting, reconciliation and reporting. It provides the scheme managers a unified platform for tracking releases and monitoring their last mile utilization.

      In order to abide by the directions to implement Just-in-time releases and monitor the end usage of funds, it has been decided to universalise the use of PFMS to cover all transactions/payments under the Central Sector Schemes. The complete monitoring of these schemes will require mandatory registration of all Implementing Agencies (IAs) on PFMS and mandatory use of Expenditure, Advance & Transfer (EAT) module of the PFMS by all IAs.” Ministry of Finance Memorandum, July 2016

    • 31

      “Despite initial opposition from a few states, the Centre has managed to integrate treasuries of almost all states into the Public Finance Management System to track fund utilisation up to the last mile as well as transfer funds “just-in-time” for central schemes. This would help the Centre save R10,000 crore annually by ending floating of funds or idling of cash at banks, Controller General of Accounts Archana Nigam told FE.

      West Bengal, which was opposed to integration of state treasuries with the PFMS, is not yet on board while Arunachal Pradesh could not join it due to connectivity issues. The Centre has set a target of integrating all state treasuries with PFMS by March 31, 2017.

      Integration of treasuries has virtually wiped out indefinite parking of central funds at the state level from about R1.25 lakh crore annually to virtually nil now, Nigam said. Floating of funds was very high in the case of government’s flagship Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA), another official said. The Centre has budgeted to spend R48,000 crore in MGNREGA scheme in FY18, nearly the same as in FY17.” Financial Express press article, 4 March 2017

    • 32

      GiveWell conversation with Santhosh Mathew, 25 June 2018 (unpublished)

    • 33

      Ministry of Rural Development Cabinet Note, June 2015 (unpublished)

    • 34

      "At the time, J-PAL SA did not have the funds or time to provide this level of support to Mr. Mathew, but they decided to provide it and jump on this policy window anyway because since February 2015 they knew GPI funding was opening soon and they could apply to hire a full-time staffer to take over from [Person 1], so [Person 1] would only have to stop her other J-PAL work temporarily." GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished)

    • 35
    • "Among other things, [Person 1] worked with Mr. Mathew to create a report and presentation slides to help him make the case for the reforms." GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished)

    • 36

      The question of how to think about ascribing counterfactual probabilities of an event depends on a number of difficult conceptual questions, which we have not resolved. For example:

      • When multiple actors are coordinating, it is likely that the marginal impact of each of those actors (assessed individually) will sum to more than the marginal impact of those actors treated as a single group. While this is not logically inconsistent, we believe the complications introduced mean that overly focusing on explicit quantification may lead us to abstract from important considerations, which in turn would lead us to make suboptimal grant decisions.
      • We believe it is unlikely that events would have played out exactly the same without J-PAL’s research support to Mr. Mathew. However, we believe it is more likely that other reforms to solve the same problem may have taken place without J-PAL’s research support to Mr. Mathew. In this case, it is unclear how broadly to define whether a "similar" reform would have taken place.

      GPI and Mr. Mathew have told us other factors which played an important role in the decision, but we do not feel confident we have a full understanding of the political context which led to the approval of the reform. These factors included that another government official was working alongside Mr. Mathew to make the case for fund flow reforms, the anti-corruption results from the RCT were important for convincing other government stakeholders, and the innovation to make MGNREGS wage payments a Central government scheme. GPI timeline of events leading to the fund flow reform of MGNREGS (unpublished)

    • 37

      GiveWell's breakeven analysis of GPI

    • 38
      • Ministry of Rural Development Cabinet Note, June 2015 (unpublished)
      • We use the exchange rate of 62.125481 INR to 1 USD from 1 April 2015. See Exchange rates.

    • 39
      • "Despite initial opposition from a few states, the Centre has managed to integrate treasuries of almost all states into the Public Finance Management System to track fund utilisation up to the last mile as well as transfer funds "just-in-time" for central schemes. This would help the Centre save R10,000 crore annually by ending floating of funds or idling of cash at banks, Controller General of Accounts Archana Nigam told FE.

        West Bengal, which was opposed to integration of state treasuries with the PFMS, is not yet on board while Arunachal Pradesh could not join it due to connectivity issues. The Centre has set a target of integrating all state treasuries with PFMS by March 31, 2017.

        Integration of treasuries has virtually wiped out indefinite parking of central funds at the state level from about R1.25 lakh crore annually to virtually nil now, Nigam said. Floating of funds was very high in the case of government’s flagship Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA), another official said. The Centre has budgeted to spend R48,000 crore in MGNREGA scheme in FY18, nearly the same as in FY17." Financial Express press article, 4 March 2017

      • We use the exchange rate of 62.125481 INR to 1 USD from 1 April 2015. See Exchange rates.

    • 40
      • "In the status quo system, funds flowed through four tiers of administrative hierarchy on their way from the Department of Rural Development to the village authority: state, district, block and Gram Panchayat (GP). The GP could request advance funds without specifying intended purpose, but authorities at intermediate levels of the hierarchy (the block and the district) had to approve the request before it went to the state treasury. In the reformed system, fund disbursement to a GP for labor expenses was based on incurred expenditures. Specifically, GP officials entered the names of those employed and wages owed in a central database, which automatically triggered fund release into the GP account. The GP official no longer required approval from block or district officials for the submission of the fund request (although many block officials remained involved, as the data-entry infrastructure was typically only available at the block level). All other aspects of the fund-flow process remained unchanged." Banerjee et al. 2016
      • “Motivated in part by the results of this experiment, in August 2015, MGNREGS officials put in place a nationwide system that combined direct payment to beneficiary bank accounts (though not always based on a smart card) and expenditure-based transfers” Banerjee et al. 2016, Pg 27.
      • Ministry of Rural Development Cabinet Note, June 2015 (unpublished)

    • 41

      Ministry of Rural Development Cabinet Note, June 2015 (unpublished)

    • 42

    • 43

      "GW: What difficulties have you encountered in fundraising?

      GPI: For the most part we’re still in the early stages of conversations with a few foundations, which is part of the reason that our funding is uncertain. However, some factors that might be contributing to a lack of interest in funding GPI are:

      1. Cause agnosticism. Most of our potential funders have strategies that are specific to a certain cause or geographic region. Since GPI is agnostic to sector and region, it is difficult to convince these people to fund it. We see this as the biggest obstacle to our funding.
      2. Uncertainty of returns. Some potential funders have concerns about how sure we can be that these projects will actually contribute to meaningful policy change because the policymaking process is complex and iterative, the time horizons are unpredictable, and there are many different actors at play.
      3. Working with governments. Some people are reluctant to work with governments in general because they assume they’re all corrupt or that they’re not committed to using evidence. In fact, we’ve seen quite the opposite – in the past five to ten years there has been a movement towards evidence-informed policy that seems to be coming mainly from governments themselves.
      4. Short track record. GPI has existed for less than two years, so some potential funders are waiting to see if we have any more successes. We expect that by the end of 2018 there will be other cases to evaluate."

      GiveWell's non-verbatim summary of a conversation with Claire Walsh and Samantha Carter, January 17, 2018

    • 44

      "GW: How much of J-PAL’s unrestricted funding is GPI eligible for?

      GPI: None – GPI is only eligible to access the restricted funds that were given to J-PAL specifically to support GPI. This includes ~$2.68 million in project awards, as well as an addition to a portion of J-PAL’s endowment earmarked to cover part of the staff time spent on the initiative."

      GiveWell's non-verbatim summary of a conversation with Claire Walsh and Samantha Carter, January 17, 2018

    • 45

      "This fund will support charities that the fund manager believes may be better in expectation than those recommended by GiveWell, a charity evaluator focused on outstandingly effective giving opportunities. For example, by pooling the funds of many individual donors, the fund could support new, but very promising global health charities in getting off the ground (e.g. Charity Science Health or No Lean Season). These organizations may not be able to meet GiveWell’s rigorous evaluation criteria at the moment, but may be able to meet the criteria in the future. If no such options are available, the fund will likely donate to GiveWell for granting. This means we think there is a strong likelihood that the fund will be at least as good as donating in accordance with GiveWell’s recommendations, but could be better in expectation." Effective Altruism Global Health and Development Fund

Source URL: https://www.givewell.org/research/incubation-grants/innovation-in-government-initiative/december-2018-grant