New Research Directions for the National Geospatial-Intelligence Agency: Workshop Report (2010)

Go to 4-D space-time maps and ability to search and analyze for events and scenarios

Use multi-sensor (cameras, sound) and IMUs on people to do internal mapping of buildings in real time (fire fighters; soldiers)

Develop situationally aware tools: need to have products and analysis tools suited to the purpose

Analytic integration:

• Using photogrammetry in the aid of social intelligence: (e.g., automated personal identification, crowd estimation, automatic generation of searchable maps)

• Use of interactive systems, including gaming, needs to be leveraged by the geo-spatial science in a whole different level to support decision science

Need to move away from four traditional NGA core areas

Blending of computer sciences, statistics, electrical and computer engineering, geodesy, geography, bioinformatics

Integration of uncertainty and error into sensor models and analysis

• Characterize multiple sources of uncertainty

• Sensor errors

• Confidence in data (subjective sources)

• Models (empirical vs. physics based)

• Utilize advanced statistical estimation, numerical methods, optimization

Adopt new strategies to address complex problems

• Interdisciplinary

• Multi-scale and multi-resolution data integration and analysis

• More effective use of human in the loop

Leverage consumer photogrammetry and merge metric and non-metric technologies

Merge traditional and non-traditional sensing methodologies (kinematic, participatory networks, social media, surveillance networks)

Exploit hyperspectral imagery

• Integrate with other data, GIS, etc.

• Add time (as described in photogrammetry)

• Exploit other information (culture, context, etc.)

Adaptive sensing (real-time) based on information value of sensor

Link the above with text information to aid classification and event and scenario recognition; link with visual analytics

Exploit atmospheric impacts as signals

Uses networks of “small satellites” to gain distributed data

Adapt products, tools to end user (first responder, soldier, analyst, etc.)

Emphasize multi-sensor fusion and information extraction

• Decrease uncertainty

• Exploit redundant capabilities

Greater utilization of state-of the art algorithms

• Estimation theory – statistics and electrical engineering

• Robust nonlinear optimization – numerical analysis

• Statistical sensor measurement models - nonlinear filtering

• Advanced software – Object oriented C++

Coordination with other government agencies

• DARPA, Air Force, Army, Navy

Exploitation of knowledge sources beyond image data mining; make relevant knowledge sources available; knowledge-based classification

Enhance change analysis – beyond the process of measurement and classification to dynamics, behavior, and prediction (issue of sensor control and tasking)

Need more than just the inanimate landscape, but also the dynamic, social environment (e.g., the flux of a living city) = GEOINT

Metadata and tagging – hey for fusion; relate to other non- GEOINT sources (semantic and tagging interoperability challenge)

Augmenting the image analyst –more tools, knowledge, visual analytics, automation, mining given a specific remote sensor

Infrastructure implications – data storage, distribution, and throughput to the analyst

• Remote sensing: We have lots of data (increased availability of commercially collected data). Can we analyze this data?

• Data collection agency, delivering tools for data analysis (multi-resolution, multi-sensor, multi-platform, multi-temporal, current and future sensor technologies – including new sensors that are not fully understood)

Data: bring processing closer to the data acquisition system (selective provision of data)

How to incorporate third party data and information into NGA processes (reliability, metadata, etc.)

Need more comprehensive metadata

Processing to support near-real time processing of constant data streams from drones and UAVs

Need to blur processing distinctions between satellite, aerial, and terrestrial data acquisition systems

Quality of information

• Reliability and integrity of automatically generated spatial information

• Scalability

• More comprehensive use of supporting information (e.g., environmental)

Quality assurance: system calibration, mission planning for different applications

Quality control: verifying the quality of the different products at different levels (sensors, data, information, and knowledge)

• Develop test sets for different products

Blending of information:

• Interface across different information types

• Information fusion (integration of open source information – quality control of information – evaluating the reliability of this information)

Information and data presentation

• How to compress petabytes of data to kilobytes of information for presentation to the end-user

• Supporting information needs to be more fully utilized

Automation:

• Is full automation possible and do we need full automation? (reliability issue)

• Provide increased human support to carry specific tasks

• For example: Tuning the learning models (more of an art that relies on the expertise of the operator ◊ reducing the level of expertise required

Modeling and data processing:

• Modeling of non-traditional and emerging sensors (e.g., DSLR, flash LiDAR, range cameras, etc.)

• Data, information, to knowledge transformation

• High resolution versus low resolution – local versus global coverage – smart sampling of the landscape

• Considering the time dimension in geo-spatial data analysis (e.g., pattern of life assessment)

Fusion

• Models for determining the optimal sensors and data needed for deriving desired information (requires data repository that have been geometrically, radiometrically, stochastically checked or pre-processed)

• Information fusion: facial reconstruction, CV

• Evaluate the results (how it relates to the end goal), understanding the data

Multi-scale to continuous scale maps

• Beyond tile-based mapping, beyond Mercator projection

Improve speed of map presentation

• Multiple scale levels, all scales

Interactive cartography driven by eye tracking, brain sensing, other body sensors

Beyond cartographic scale

• Investigate semantic aspects of scale

• Represent human activity at multiple temporal scales

Need timely access to GEOINT at differing scales based on differing user tasks

Incorporation of volunteered geographic information

Social “mapping” in space and time in addition to physical objects and terrain… research challenges? Social links and networks have geospatial characteristics, how to deal with them in a spatial sense? … have to interface with other types (agencies) of “INT”

Address challenge of how to visually present data and information quality, reliability, and confidence

Determine what information is needed by particular users and determine the appropriate evaluation methods

Integration of GPS in all aspects of geospatial technology

• Applications still in infancy

Increase proficiency in use and interpretation of GPS positioning

• Provide means of assurance that people using GPS for particular tasks know what they are doing

Ubiquitous GPS

• Integration of multiple receivers; phone, navigation

Expansion of continuously operated reference system (ground based) – CORS

Geodesy does not deal with humans directly (classical defn.), but gives information that supports societal and scientific needs… but reference frame is “invisible”

Impressive progress in geodetic accuracy… but how to “operationalize” geodesy missions and services? What should NGA do? GPS/GNSS used in many positioning apps… could we cope without it? What about difficult environments where GNSS doesn’t work?

Establish a geodetic reference frame at sub-millimeter level; research needed at observational level; drives high performance computing research, etc.

Next generation of positioning instrumentation and inertial navigation systems stable to the centimeter level over time

Geophysics: collaborative research, could be informative to NGA (in terms of data)

Application oriented datum; provide transformation

Gravimetry: UAVs; time dependent gravity; GRACE mission

Continue to pursue temporal dimension

True, comprehensive and complete space-time GIS and geospatial analysis does not exist

Expand the narrative

• Geospatial discourse constrains possible tasks

• Restricted GIS vocabulary to communicate tasks

• Production of narrative products at multiple levels of explanation

Incorporation of volunteered geo-information

• Rating system for accuracy

Need to understand how to work with the narrative framework

Need to achieve timely automated extraction; need OO software approach

Automated service and workflow discovery to enable automatic tool application

Conceptualize complex information into story line

Communication of geo-spatial issues

• Static and dynamic communication of narratives

• Visualization of narratives

Are the core NGA areas “stovepipes?” Are they the right ones?

How do people respond, perceive, and trust quality statements, especially for large amounts of data?

Need integration of geo information from unstructured sources (text), physical domain, social domain, and knowledge domain with GEOINT

Use game based analytics: explore data set in terms of games; analyze game strategy and pattern; use information for interview techniques

Cognitive effectiveness of geo-spatial technology

• Brain scans, MRI, eye/scan patterns, etc.

• logical

• physical

Broader cross training of students in geo-spatial workforce … computer science, behavior, …

• understanding … geophysics, geodesy, …

• facilitating interdisciplinary training and research

Challenges

• Predicting human behavior - relating social factors to physical factors. Geospatial elements need to become part of social network theory

Spatial data analysis methods need to be incorporated to get better predictions that put in spatial relationships and meaning

Need to tie together of spatial data and temporal forecasting

What are the validation methods? Need to develop general validation approaches. Need sensitivity analysis

Should we distinguish between prediction and forecasting? Be sure that all aspects of these areas are being covered

The forefront of modeling. More complex models that are combinations of very different models for actionable results

Technosocial predictive analytics

• Interesting interplay between social networks and physical infrastructure

• Needs systematic work on defining priors

• • black swans are a challenge since not enough data on extreme events

• Needs visual analytics as a visual tool to gain better insights

• Links possible with geocollaboration

• • collaboration over time, space, expertise

• Beginnings of applying computing to sociology and anthropology exciting!

Modeling of human behavior – more interactive and real-time forecasting tools where problem domain is constantly changing. Use of normality modeling and anomaly detection as alternative to deductive based forecasting

Computational modeling, prediction, and analysis are important research topics for the future

• Potential to guide data collection and assimilation

Uneven distribution of sensed data

Privacy issues

Crowd sourced data aggregation methods need to be developed

Understanding when crowd sourcing is useful

What about foreign countries or areas where you can’t apply your structure?

A very powerful way of collecting GIS data “unstructured collection.” Need on-the-fly planning

Use the GIS as a framework. May already have some 3D models, images, etc.

Building shared spatial knowledge bases with participatory input and sharing. Active knowledge bases

Directed planning; opportunistic planning. Situationally aware models. Need to get actionable results. Spatio-temporal models of social, political dynamics

Trust and confidence; how to account for biases and keep this information with collected data. How to do quality control in a messy data environment. This needs new ideas

Add reference data (reference models?) as points of validation with data of uncertain accuracy and provenance. This could be a general approach

Embed social networking in spatial-temporal. Insert the idea of locality and spatial structure in social network analyses

Quality control

• Need methods to aggregate measures of quality

• Timeliness is an important dimension

• Measures of trust, reliability, provenance: don’t trust; verify

• Spot-checking with high-quality, calibrated sensors to improve trust and quality

Judicious use and context of information collected

• For example, owner-defined property lines and conditions valuable in non-legal contexts

Systematic approaches to integrate information from multiple sources:

• Domain knowledge and expertise such as local context (cultural)

• Participatory data analysis – Wikipedia over GIS

• Counterpoint to the deep and intensive thinking of the analyst

How to engage all relevant sub-groups (age, gender, socio-economic) in participatory data collection?

Develop the wider model against which participatory data can be tested. Use of prior knowledge for improved registration and classification

Understanding of the quality compromises and strengths of having mixed use of authoritative and public participatory data – requires broader development of the models of use

Understanding the relationship of culture and social factors to policy and practice of collection and use of public participatory data. Research into security issues of participatory data

Participatory sensing: Integration is important!

• How to influence social media to generate data that is needed

• Research to calibrate and judge quality of sensor in participatory sensing to allow decision making

• Data fusion from this data with serious geo information?

Specific interfaces for specific users? Emphasize the generalization. What are the underlying fundamentals?

How to get from visualization to underlying methods? Need to understand the domain areas. Can general principles be extracted?

Developing a repeatable body of knowledge within visual analytics; for example, generic rules applying to the interpretation of data. Develop evaluation criteria

Integrated tools. Integrated, iterative, interactive—this is the new thing that visual

Interactive part of visual analytics is a key aspect of its contribution here analytics can bring, even using existing analysis tools (no toolkits)

New ideas derived by looking at aggregates. Individual locations to aggregations that make sense for groups. Functional and meaningful scales and multi-resolution methods. Attach meaning to aggregations. Space and time aggregations

Visual is not the only sense as you only reach a small part of the population (19%)

Interactive analytics is may be the right term

Metaphors for interaction with models and animations need to be developed

Integrated spatial and temporal analytics

Understanding the use of animation

Modeling, simulation, and high performance computing

Proper depiction of data quality and error uncertainties

Games

Social interactivity

Importance of design and art as an additional skill to be embraced

Workflow

Domain-driven integration of information from multiple sources

• Take advantage of human cognitive abilities

Need to address how techniques work across scales

• Agent-based approaches, links, etc.

Need new advances in interaction for visual analytics

Further strengthen bridges between visual analytics and other areas

Visual narratives

• Causality

Quality of the visualization

• Develop techniques to measure quality of the presentations

• Minimizing unintended artifacts, illusions, confounds, etc.

Visualizing and communicating uncertainty. Development of interactive visualization tools - dynamic feedback with analyst through eye-tracking and other sensors

Collaborative two-way participatory augmented reality

Achieving the correct balance between full automation and visual analytics assisted decision making – how to decide which to use in specific situations?

Computational modeling and/or visual analytics

• How to enable human reasoning with large amounts of heterogeneous geospatial data?

• Data fusion

• Deal with users

Science of interaction: Need to develop adaptive visual analytical methods to support geospatial users

Data fusion

• Relate to geo-space:

  • represent spatial and non-spatial dimensions

  • incorporate spatial structure: spatial variation or spatial correlation

  • couple spatial and non-spatial algorithms

Different forms of spatial data

• High-resolution, attributes cross space

• Location (point or area), boundary, space of different scales

• Models

• Best way to combine GIS data layers, coding, incorporating uncertainty

Non-spatial data fusion (as in Haesun Park’s talk): cognitive domain

• Cognitive aspects of knowledge fusion

Fusion challenges

• Scale

• Semantic interoperability

• Different resolutions

• Fusion at different levels (data, information, and knowledge)

Heterogeneous data of different fields and kinds of knowledge, disparate terms and understanding

GPS positioning

• Data on positioning and gravity are uncorrelated, nicely separated

• 2-, 3-, 4-D geodesy, not much to gain from data fusion

• Essentially, it’s about data understanding

Fusion has a lot to achieve, let alone beyond fusion

• Is it the same as merging? Conflation is part of fusion

• Need for clarification, vocabulary, a scientific language

Can disparate data, information, and knowledge be put together? Redcross, trusted feedback, outdated geospatial data together

• Would techniques presented take care of these?

• Overarching issue of uncertainty labeling for broad NGA data set needs to be addressed; what is uncertainty of high-dimensional data?

A set of techniques for understanding relations in high-dimensional data

• See also manifolds, etc.

• Applications to GI data not shown

• Loss of visibility of space and time at “preferred” scales

Powerful, but evaluation methods need to be developed

Do not stand alone—insight needs to be developed alongside

• Analyst interaction important

Also need methods to understand large disparate data bases

• Interrelationships possibly not understood

Both broader understanding and uncertainty reduction will likely require complementary, non-GI data

Comparison of fusion algorithms from visual analytics with existing fusion algorithms

Early fusion, mid fusion, late fusion

Bayesian fusion algorithms

Hard-soft fusion using hard sensor data and text, human generated, web derived information

Compare and evaluate the accuracy and applicability of these two types of fusion algorithms

Need scalable algorithms to handle large volumes of data in real-time and interactive mode

Would approximate, but faster algorithms be desirable?

Need to develop systematic approaches to matching computationally driven interfaces to user work practice

Need to investigate existing standards such as the Predictive Model Markup Language (PMML) to use the same data for different classification algorithms.

How to retrieve geospatial documents and extract geospatial information from text is still a challenge

How to use existing geospatial ontologies to inform the information extraction process

How to enable human computer interaction when complex modeling is involved

Develop methodology to create heterogeneous benchmark data sets for research

Formulation of standards for methodology and data structures

Human landscape is a better term – human condition, biophysical conditions

• Economy, sociology, transportation, anthropological, ethnic, religious, cultural, historical

Geospatial, social, cultural data integration and analysis

• More systematic approaches in collection, coding, displaying, understanding

• Categorizing trivial and non-trivial data

• Voluntary and non-voluntary contributors may not be aware of the consequence of making data available

Data uncertainty, quality, consistency, reliability, disparity, fuzzy

• Tools to filter and clean up data

• Identify what data is necessary for a given task

Collaborative tools for crowd-source data

• Interactive tools

Proper analysts with specialized knowledge

• Human intervention to double check the quality (human in the loop)

Human terrain

• Relate analytical outcome based on the significance of consequences of prediction errors; should we weight the outcomes accordingly?

• Assess possibility or level of confidence on data and analytical outcomes

• Interoperability: customized system vs. open system; closed sourced black box? scalability? Need to consider modularized system and develop API to couple with other systems

• Need a stronger geospatial component in social network analysis; dynamic relationships over space and time,

• Social networks in virtual space vs. in physical world

Cross-cutting

• Complexity of analysis: ability to interpret the results

Methods to enable analysis in native language

Human terrain-based dynamic network analysis seems to serve well as one basis for structuring a broad range of social phenomenology in space-time

• Representation and visualization in GI space an issue

Quality assertion, quantification an issue

• Highly disparate underlying data quality levels; need agreed ontology

• NGA to develop technical and ethical best practices for collection?

• Interplay between space-time accuracy and relational accuracies

• Deception possible, not easy to detect

A form of “narrative?” Perhaps useful to assess commonalities, distinctions in these methods

A larger issue lurking here? Methodological synthesis to deal with the space-time dynamic

Computation (cloud computing, mobile computing, analytical servers)

Distribution of data and data storage

Customization of products—making dynamic products for end users to dissect, modify, traceability of evidence and logic (case files FBI or doctors)

Validation, data quality, spatial uncertainty

• Populist information: privacy, uncertainty, NGA’s role? Use to validate directly gathered data

Multiple levels of uncertainty (data, model)

• Move to knowledge, wisdom, insight

• New paradigm of uncertainty (based on analytical needs at hand)

Advancement of sensors

• Sensor calibration

• Smart sensors, miniaturization, on-board computing

• Infrared, radar (better sensors)

• Don’t lose focus; don’t forget the sensors

• Don’t forget the core areas

• Scenario modeling to deploy appropriate sensor for task (weather, geography, etc.)

Temporal analytics

Can’t lose track of the 5 core areas

• Cross-cutting themes support the 5 core areas, but can’t ignore or replace them

• Cross-cutting themes need to show value to the core areas, not a substitute

• Mathematics, visual analytics can directly benefit NGA and its missions

Adding time to space

• Rich extension

• How to do this? Visual analytics, 4D GIS. Time is difficult to represent; temporal analytics?

No stove-piping in 5 core areas

• Also applies to cross-cutting areas

• These areas blend together (look for and/or promote innovation at the intersection of these areas)

Science development needs to be plugged into international science community