course, particularly for custom-tailored, performance-based standards (ISO or LEED for buildings) but in general, these voluntary standards systems have emphasized changes in production practice, and their baselines and performance measures relate to this. The most straightforward way to measure success, then, has been to use existing market share or number of compliant producers as a baseline, and focus on expanding the program’s reach.
As some participants noted, most existing programs would have a difficult time backing up their claims of success (in terms of on-the-ground sustainable outcomes) because little is actually being measured, especially at scales of importance to sustainability, namely ecosystems and communities. Even if the particular standards are considered rigorous and based on sound science, certification programs rely heavily on their “theory of change” with the belief that, by changing certain practices through prescriptive guidelines, production will become more sustainable. However, the programs rarely conduct baseline assessments against which they could measure improvements in performance. Doing so would be a valuable way to demonstrate how effective, and efficient, certification programs are at enhancing positive outcomes, relative to alternative approaches. If the majority of compliant producers were top performers to begin with (as has been the case with several leading standards), this begs the question as to whether alternative approaches could have more “bang for the buck.” As some participants noted, though, a direct comparison of approaches would be difficult to carry out, akin to conducting an experiment without being able to isolate variables.
Participants pointed out that success is still not well defined in any sector or particular certification scheme. Defining sustainability for each and every sector would be a daunting task, and would seem counterproductive for individual certification programs to do. Most programs have a notion of the direction they must head, but as many participants emphasized, the bar will likely need to be raised and continually adjusted as programs mature and knowledge accrues. Perhaps more importantly, even when programs are directionally correct, they may need to expend more effort determining their starting point to help assess how far and how fast they will need to progress to achieve sustainable outcomes.
Data availability is a critical limiting factor. As was emphasized at other points throughout the workshop, outside observers would like to see data-driven approaches. Supporters of certification schemes, however, generally want to see on-the-ground changes. With resources being limited, it is difficult to apportion a large amount of up front costs to field work and data gathering before a certification program even commences. Data collection and additional bookkeeping is often cited as a significant transaction cost for producers participating in a certification program. One participant