The final scientific score is the arithmetic mean of the reviewers’ scores. If there is a wide divergence in scores with a sizable proportion (greater than 35 percent) of the GWG being in disagreement with the majority view, a minority report is forwarded to the ICOC along with the final score (CIRM, 2011g).
The next stage is the programmatic review, chaired by one of the patient advocate members of the GWG appointed to this position by the ICOC (CIRM, 2011g). The purpose of this review is to evaluate all of the applications taking into account not only their scientific scores but also the overall purpose of the RFA, with the goal of segregating the applications into three tiers—recommended, provisionally recommended, or not recommended for funding. This process has two steps. First, a histogram of the scores of all of the applications is generated. Of note, at this stage the applications are deidentified, and only the scores are revealed. The GWG examines this histogram and identifies natural breaks to divide the applications into the three tiers based on their scores. Next, the applications are identified so that the scientific score (and tier) of each is made known. GWG members (except those with conflicts, who leave the room) begin a discussion to determine whether any of the applications should be moved from one tier to another in an effort to achieve a balanced portfolio representing a spectrum of priority disease areas, scientific approaches, innovation, and so forth. For an application to be moved from one tier to another, a majority vote of the GWG is required; all members of the GWG not in conflict (scientists and patient advocates) participate in this vote. Once the GWG is satisfied with the final ranking of proposals, a final vote is taken, and the rank order is proposed to the ICOC for approval. For each application, in addition to its final ranking, the scientific score voted by the scientists on the GWG is provided to the ICOC (CIRM, 2011g; IOM, 2012e).
The ICOC makes funding decisions at a meeting scheduled and publicized in advance. As with other ICOC agenda items, deliberations on the funding of applications begin in a session that is open to the public. ICOC board members in conflict with any particular application are recused from both this public discussion and any subsequent private deliberations. Prior to the ICOC meeting, summary information about each application is available on the CIRM website, including how that particular application ranked relative to the others and its tier designation. Applications are redacted, however, to remove information that would identify applicants or institutions. Individual applicants are aware of how their proposal scored and how likely it is to be funded, and they have the opportunity to make an “extraordinary petition” to the ICOC. Any ICOC board member may request that the petition be heard. In such cases, petitioners are invited to the ICOC meeting to explain why they believe the assigned score and priority ranking are not appropriate. The ICOC takes this information into