compare the output from a land change model to the output from a corresponding naïve model that is applied to the same study site. A naïve model is one that is based on a simplistic conceptualization of the land change process and that offers a baseline that is easy to understand and implement. For example, a naïve model of deforestation could allocate the simulated deforestation on the edges of the initial forest patches. Then the output from the naïve simulation could be compared to the output from a more complex model. It is important to compare the output from a complex model to the output from a naïve model to measure whether there is any increase in predictive ability in the more complex model. A naïve model might use randomness to allocate change, but researchers frequently already know that the process of change is not random; thus, a random model is likely to produce an extremely low baseline. A naïve model that is based on one simple idea such as proximity to a single feature is likely to generate a much more challenging baseline than randomness. For this reason, it can be misleading to use metrics, such as kappa, that compare model output to a random pattern (Pontius and Millones, 2011). The literature sometimes uses the term neutral model to convey the idea of a naïve model that offers a baseline for comparison to a more complex model; however, if neutral models are based on randomness, then such neutral models are likely to produce an unchallenging baseline.

If there is no baseline for comparison, then the investigator is frequently tempted to use universal standards for model performance, such as defining good as greater than eighty-five percent agreement between the simulated map and the reference map. Universal standards for model performance are problematic, because they are by definition not specific to any particular research question or study site.

The concepts of equifinality and multifinality also need to be considered when selecting a metric for model assessment, especially when that metric measures only the pattern in the output map (Brown et al., 2006). Equifinality is the situation where two different processes produce the same result. For example, uniform versus highly variable patterns of risk aversion might, in some settings, produce identical patterns of agricultural activity. In this situation, it is possible that the model uses an incorrect process to produce the correct pattern.

In other cases, a process-based model uses the correct process to generate an incorrect pattern. Multifinality is the situation where a single process has the ability to generate many different patterns. One possible cause for this phenomenon is path dependency, whereby a few initiating events occur due to a poorly understood process, and then those events trigger numerous other processes. For example, there might be tremendous uncertainty where a corporation will build a facility, but then the facility generates urban growth near wherever it is placed. Thus, a model can simulate correctly the process of growth that follows the initial siting of the facility, but the model realizes that there is uncertainty in the placement of the initial facility. In this situation, a process-based model simulates the

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement