TABLE 6-1 Initial Evaluation of Existing Forecasting Systems

 

Tech Cast

Signtific (X2)

Delta Scan

Data sources

Partially meets. Largely from selected panel experts. Other sourcing methods and geographic sourcing not apparent. Open to invited members only. Novel data sources, collection techniques, and data audit methods not apparent. English only.

Partially meets. Open system. Multiple data sources (stewards, workshops, games) and baseline data. Geographical diversity improving (workshops, Web site) but still is English only. Participant’s profiles captured. Some applications of novel data sourcing and collection techniques. Data audit methods not apparent.

Partially meets. Open to access but not to contribute. Data gathered from more than 250 experts with a wide array of backgrounds through workshops and interviews. English only. Novel data sources, collection techniques, and data audit methods not apparent.

Forecast methods

Partially meets. Self-selected expert opinion with some predictive elements. Part qualitative and quantitative. No apparent barriers to recruiting experts.

Partially meets. Forecasts and signals derived from stewards, experts, and public via workshops and games. Signals and forecasts largely qualitative. Quantitative methods not apparent.

Partially meets. Principally expert opinion. Forecast supplied by the Institute for the Future. No apparent way for the public to contribute or collaborate.

Forecast team

Partially meets. Self-selected experts and small team. English only. No apparent barriers to recruiting experts. Methods to assess expert diversity, country or quals not published. No public participation.

Partially meets. Team consists of employees, experts, and public. Public participation strong and growing in some areas—7,000 participants in the last AGR. More limited public participation in signal generation and forecasting. Evidence of third-party community development, collaboration, and initiative becoming apparent. English only.

Partially meets. Forecasts supplied by the Institute for the Future. Appears to be English only. No apparent public participation. Methods to assess diversity of experts by country, culture, discipline, etc. not apparent.

Data output

Partially meets. Quantitative and qualitative. Each forecast quantifies estimated time of realization, confidence levels, market size, and range of dispersion. Qualitative assessments of forecast strengths and weaknesses. Visualization limited to static graphs. Unclear if data are exportable.

Partially meets. Principally qualitative. Quantitative representation is limited or not apparent. Third-party access (and export capabilities) to the data on player behavior is not apparent.

Largely meets. Qualitative with some quantitative. 1-5 scale assessment of impact, likelihood, and controversy. Qualitative assessment of geographical impact. Signals, enablers, inhibitors, Centers of excellence, data sources, analogies, word tags and links identified. Visualization and navigation could be strengthened.

Processing tools

Limited. Some enablers and inhibitors identified in forecast narrative. Processing done by the expert community. Diversity of experts unclear. Other processing tools (dashboards, data visualization, signal and link processing) are not apparent.

Limited. Some enablers and inhibitors identified by the community. Diversity of the community processing the data appears to be improving. Other processing tools (dashboards, data visualization, signal and link processing) are not apparent.

Limited. Signals, enablers, inhibitors identified but no apparent way to automatically measure progress toward thresholds. Diversity of community processing data unclear. Other processing tools (dashboards, data visualization, signal and link processing) are not apparent.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement