The most important thing to know is that we are always ready to look at charities that don't pass this heuristic, if they meet our broader criteria. If you know of such charities, please alert us using our submission form. If it appears that the information we require is available - whether or not it's available on the charity's website - we will change a charity's status to "Pending" until we have reviewed it more thoroughly.

Why do we look at what information is available on a charity's website, instead of searching more comprehensively and contacting them directly?

We have found that going back-and-forth with charities to see what they have internally is extremely difficult and time-consuming for both us and them. We are generally first connected to fundraising staff, and it takes a lot of communication and waiting just to end up talking to someone who knows what information is available. Repeating this process for all 300+ charities we have examined would not be practical, so we use a heuristic to identify the most promising candidates for further investigation.

We are explicit that our research is constrained by practical considerations. Our goal is not to be "perfect" in our assessments but rather to provide better information than donors can find anywhere else.

We do contact all rated charities to let them know about their status and how they can change it if they feel we are in error.

Is there independent evidence that "what information is on the website?" is a reasonable proxy for "what information is available at all?"

Yes. We have also used alternate research methods that involve much more back-and-forth with charities, and feel that the results give support to the “website scanning” heuristic as an imperfect but pretty good predictor of which charities actually have the information we require (particularly evidence of impact).

• Our first-year research process involved applications for grants of $25,000-$40,000. All non-confidential application materials have been publicly posted. For all five charities that earned a 2 star or better rating through this process, the primary evidence of impact we used is available on or via their website.
• We're currently conducting a grant application process for $250,000 and will be publishing the full details of what it turns up in early 2010.

Also note that our "website scanning" heuristic is similar to the method used by William Easterly and Tobias Pfutze to rate aid agencies (PDF). Our aim is similar in that we seek to reward organizations that have both good practices and the transparency to share their practices publicly.