Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 457
457
Alexander Levis
"introduction by Session Chair"
Transcript of Presentation
Summary of Presentation
Power Point Slides
Video Presentation
Alexander Levis is chief scientist of the U.S. Air Force, Washington, D.C. He serves as chief
scientific adviser to the Chief of Staff and Secretary of the Air Force and provides assessments
on a wide range of scientific and technical issues affecting the Air Force mission.
Dr. Levis received his professional education at the Massachusetts Institute of Technology. Prior
to his current position, he was University Professor of Electrical, Computer and Systems
Engineering at George Mason University in Fairfax, Virginia, and head of the System
Architectures Laboratory of the C31 Center. For the last 20 years, Dr. Levis has conducted basic
and applied research in and taught many aspects of command and control, from organization
design for command centers, to operational and system architectures, to decision support
systems.
Dr. Levis has served as senior officer in national and international professional societies, is on the
editorial board of several professional journals, and on the Board of Directors of the AFCEA
Education Foundation. He has held two appointments to the Air Force Scientific Advisory Board,
where he participated in several summer studies and several ad hoc studies.
457
OCR for page 458
458
DR. LEVIS: Just on a serious note on that, the
term stegonography was mentioned in the first session, and
people asked what is it. As far as I know, Al-Queda has
used it already. So they are fairly sophisticated.
Good afternoon. My name is Alex Levis, and I
will be chairing this session. My current position -- I
have never worn a uniform in my life, but my current
position is with the Air Force. I am chief scientist. It
is the best job in the world. It has no specification.
What I am supposed to do is give advice to the chief of the
Air Force and the Secretary of the Air Force. The
Secretary of the Air Force did his Ph.D. on decision
analysis at Harvard. He does not need any advice from me
on matters technical.
The chief has a war to fight, he needs no advice from me.
So I am having a great time coming to workshops, and I
really appreciate the invitation.
Somewhere in my distant path actually, I do have
a degree in mathematics, but it is an undergraduate one. I
noted when David earlier -- there was a question asked
yesterday, how many people have a degree in mathematics,
and four or five people raised their hands. The question
today that David asked is, how many mathematicians, and
458
OCR for page 459
459
two-thirds of the people raised their hands. I wonder what
the significance of this is.
I would like to make some comments first. I have
learned a lot of interesting things yesterday and today.
Information security I am a little more familiar with. But
the thing that hasn't come up very much is the problem.
Those who want to contribute in mathematics, and I believe
strongly in that, and I'll try to show a couple of
examples, mathematics to the problem.
I am an engineer, and I need to understand the
problem before I apply solutions to it. The problem has
not been defined in this workshop. We all took it for
granted. From the discussion it is apparent that our
knowledge is what we all hear from the newspapers, et
cetera. I don't think even the White House has defined
homeland security, let alone -- I don't know the difference
between homeland defense and homeland security, by the way,
but different people use them in different ways.
So let me try to define in non-technical terms
the problem. We don't really know who the adversaries are.
We know some individuals, we know some organizations, but
we don't have a complete knowledge of the adversaries. We
don't know where they are. You see every night on TV that
we really don't know where they are, and we don't know
459
OCR for page 460
where they are going to attack, whether it is going to be
in the U.S. The call just went out again, after being
prepared for a year and a half.
We don't know when they will attack. We do not
know what they will attack, and we don't know how they are
going to attack. We know bits and pieces of that, but this
is the classic journalism, the five questions. So when we
talk about homeland security, we have to look at all those
aspects.
You can characterize the uncertainty. We talked
a lot about probability yesterday and a little bit today,
but there are different kinds of uncertainty that are
associated with this. You have temporal uncertainties, you
have locational uncertainties, you have all sorts of things
that somehow have to mix and match together. This session,
remember, is about integration and fusion.
Now, the strategies that you can consider fall in
three categories: reactive strategies, --
PARTICIPANT: By the way, you forgot one
question. We don't know why they are attacking us.
DR. LEVIS: No, I 'm sorry, that we know. They
don't like us. We can start a debate, but I come from that
part of the world, and I can tell you in great length why
they don't like us.
460
OCR for page 461
461
We have concentrated mostly in terms of what
people have been discussing and the approaches they are
indicating. We are looking primarily at the reactive
problem, what happens if somebody does that. Much of the
work that was presented in epidemiology would go into the
reactive part: once there is an attack, what can we do
about it.
There is also the anticipatory part. Some of the
work yesterday relates to that. We know that somebody is
crossing the broader. We don't know where, but we are
going to look for it, we are expecting an attack on the
bridges of San Francisco during that week. This is
anticipatory.
The other one is the proactive; get them before
they get a chance to get us, which is what we are doing
right now, what the Air Force is actually doing. The term
that the Air Force uses for this kind of a notion, that is,
the last one, is predictive battle space awareness. It
doesn't really mean anything. This is a bumper sticker to
put at the end of the car. It tells you it is great to
know what they have done. It is great to have all the
sensors to tell me what happened, but that really is of
very limited value. What I really want to know is what
461
OCR for page 462
462
they are going to do next, so I can go and get them before
they do it. That is the hard problem.
With the expansion of sensors, information
technology, et cetera, as many of you indicated, we have a
wealth of data, but our difficulty is to take that data and
start determining intent, which takes us away from just
projection of the data. We have to understand what is
going on, because you need to know intent, to understand
intent, to make the predictions.
The first part I have already mentioned.
Yesterday we talked a lot about searching for a needle in a
haystack. I am the last person to tell you how to do that.
I have no idea now to do that. But there are alternative
approaches, how to think about the problem.
First of all, somebody mentioned yesterday
getting a magnet or changing the properties of the needle
so that you can find it more easily, given the kind of
sensors that you have. A lot of our intelligence community
is worrying about things like that. There are ways of
trying to mark things that we eventually may want to find.
That is one way of approaching it.
The last one is, find the needle before it gets
in the haystack. As an engineer, this appeals to me very
much. I am lazy. I want to solve the easy problem, I
462
OCR for page 463
463
don't want to solve the hard one. We need to keep this in
mind. There is a whole spectrum of activities over here.
To get to this predictive integration notion,
given that there are large gaps in knowledge, because you
can get to intent -- intent is human, and you can't really
model all that stuff. You cannot go to databases to find
those things. You need to do some modeling and simulation.
This is now a case study to respond to a question
that was raised later, you have all those good solutions,
how do you go and give them for people to use. That is a
very hard problem. You cannot just say, I have a wonderful
solution. I think it was mentioned, you have to understand
the science of the problem, we have to understand the
application of the problem, because nobody is going to pay
attention. You expect somebody else to understand the
solution and then convert it and apply it. Not anymore;
nobody has the patience for that. One needs to go out.
One example over here very quickly which is
relevant to the presentations today, that is why I chose
it, has to do with influence nets. After 9/11, actually a
couple of weeks later in September, I went to the Air Force
Studies and Analysis. These are the people who do the
actual math and do modeling and so forth for the Air Force.
Originally they were the whiz kids. Some of you who are my
463
OCR for page 464
464
age will remember McNamara; this is the whiz kids group of
McNamara's. They still exist, with a fancy name.
I gave them some software from the lab, research
software, to start modeling Al-Queda. They did it, and it
has been used since then to do a number of analyses. That
software was certainly not for prime time. It was homemade
by my master's level students. Since then, we have got
people from the Air Force Research Laboratory, this is from
Rome Lab up in upstate New York, the information
directorate, and they were developing software over there
which had some relation to mathematics, but it was not very
explicit what the relationship between the software and the
mathematics was. It was very heuristic in a way, but it
did approximately the same things, but it is much more user
friendly.
Once we established from the laboratory the
validity of the approach using more rigorous tools, a
heuristic approach was being used and is used currently.
Now we need the data. Some of the data came from
databases, as described yesterday, but some of the data had
to do with judgments. We brought in the analysts from many
of the three-letter agencies that were discussed yesterday
to provide that information.
464
OCR for page 465
465
The moment the analysts got that stuff, after
they learned within two or three days how to use the
software and how to model in that way, immediately they
started asking for more. All this does is, you wiggle the
inputs to see what the outputs are.
This is good for planning, but how do I go to the
execution? This is what we showed to the generals. This
way they understood the influence nets and Bayesian
networks, except for the Secretary, who does know Bayesian
networks. Fortunately, I did my homework, and before I
discussed those things I knew that he knew Bayesian
networks, and I didn't say silly things to him.
But the idea of how to use them is a little bit
different here. You have to put yourself in the mind of
the adversary and define a set of effects that you want to
achieve. We want Milosovich to change his mind, we want
Saddam Hussein to stop bothering us and go on vacation or
something like this. You put that stuff over here, and
then you start looking, what are the things that influence
his decisions.
This blob here is where the modeling takes place.
You want to bring them to the point that you have what is
called actional events, the things that we can do -- we are
465
OCR for page 466
466
the good guys, the blue -- that will eventually have an
effect here.
This is the kind of models -- now we have made a
bunch of models of those things. Actually we have been
using this kind of stuff in DoD since 1994, in the
intelligence part of DoD. They use it that way.
But now the question came up -- and I wish I had
mathematicians working with me to solve the problem,
because I don't know -- and my students don't know the
answer, and this is not the field that we work in. Suppose
now that they start the engagement. Bombs start dropping
or things are happening in Afghanistan. I start having
observations that come over here from various intermediate
nodes; this thing occurred, that thing occurred.
Can I propagate forward? Sure, I can propagate
forward except I have to update first and update the priors
and do all those things, and do it. That is brute force,
one can do it. It takes a few weeks,
implement. But then they are a
question. If I look at this node
happened
well am
interest
sensors
you get it done, you
more sophisticated
and I see what has
and that improves my information over here, how
I doing in reaching the outcomes, can I solve the
problem? Can you tell me where I should put
to see what is happening, so I can get a better
466
OCR for page 467
467
understanding of whether I am achieving my goals or not.
That is a lot harder problem, but from what I know,
mathematicians will know how to do them correctly, as
opposed to trying to ad hoc them, the way it is occurring
right now.
But you cannot do it in two years. You need to
be ready to go and work and do it in a couple of weeks and
put it in. I hope it will be over before it is actually
needed.
This is a real transition, but it took a lot of
people. That is one of the good things when you are chief
scientist; I could call people and cash in all my IOUs.
This is how it has been established now, studies and
analysis, now has the capability to do modeling, using
Bayesian nets. We never had that capability before. The
research laboratory, Air Force Office of Scientific
Research, indeed, some of you in the audience probably know
it, that is the basic research component of the Air Force
research establishment. This is the applied research, the
Air Force Research Laboratory. They are providing the
algorithms and so forth.
Here is the alphabet soup of the intelligence
community, providing the data and the updates. We are
using it, responding to decision makers. What we need here
467
OCR for page 556
556
So the problem is very complex, it is very
difficult, it is a large country. There is no way that you
can border it totally, disallow anybody to enter it and so
forth, the way of life that has been discussed. We have a
huge intelligence establishment for which we pay a lot of
taxpayers' dollars. Part of their job is to do the
anticipatory and not even suggest, just bring the data up
front for the appropriate decision makers to decide whether
to be proactive or not.
I am speaking as an individual, but I don't think
we are doing that badly, when you consider how many real
instances of terrorism we have had in this country. It is
the most open country in the world that I know.
PARTICIPANT: I wasn't suggesting that we have
done badly. It was just a concern of mine towards this
sort of strategy around a lot of what we heard today.
There was a sense that we could not --
PARTICIPANT: You are not by the microphone.
PARTICIPANT: There was just a very defensive
kind of mindset, and I was just concerned that this feeling
of vulnerability could itself -- I was just concerned about
the strategy part.
DR. DEMPSTER: It seems to me to be the new thing
that there are events of extremely low probability that
556
OCR for page 557
557
have extremely high costs.
focusing on a bit.
DR. LEVITT: Alex mentioned something in his
opening. He said, I could tell you, but then I'd have to
kill you. I think the problem with the offensive stuff is
that it tends to be handled by the intelligence community.
I think there is an enormous amount going on, but I don't
know much of it, and therefore I wouldn't know, as far as
germane to this workshop goes, where the mathematical
challenges lay in offensive activity, which I think are
highly focused.
However, I completely agree with the scope of
homeland defense. There is the newspaper version, or what
I consider the public political version. On the other
hand, where I see the real payoff is that there is enormous
amount, and there has been for man years, of data in
automation, collection, and critical things going on, that
really can be exploited. I think that we don't know what
the limits of automation might be, we don't have to have
people, you are talking PCs and memory. If we could deal
with the inferential issues, so that we didn't generate
false alarms all the time, that there might be a tremendous
payoff.
That is what maybe we should be
557
OCR for page 558
558
DR. LASKEY: I wanted to comment on that, and say
that I agree thoroughly with the points you made about not
sitting around and just
sorry if my talk came
how it was intended.
passively observing. In fact, I'm
across that way, because it wasn't
Inference can be used to passively
observe, but you notice that level five of the five levels
of fusion that I talked about was placing your sensors.
Part of what you are doing is, you are not just passively
observing, but you are watching and anticipating what is
going to happen and preparing yourself to respond to it,
but also at the same time, if we discover that if there is
an Al-Queda cell that is planning something, I'm sure that
our military people would go in and stop them before they
had the chance to do anything about it.
But it is not just waiting until the horse is
gone to close the barn door. The technologies that I was
talking about would apply much more broadly.
DR. LEVIS: Other comments? Questions to the
speakers? The timetable for the airplanes?
I'd like to make one comment and a question, and
then we can adjourn, if you like. Earlier today there was
a question about funding. This is Washington, so I would
like to address it very briefly.
558
OCR for page 559
559
For mathematics, the Air Force has the Office of
Scientific Research. There is the Mathematics and Computer
Science Division within it. I think one person was present
here today. They are looking into those problems, and it
is basic research.
However, things have changed a little bit, and
you have to write more than one paragraph or one opening
sentence. The old joke, everything looks like a worm. You
have to write a little more and show some understanding of
how that work will have an impact.
Of course, good mathematics has a good impact,
but that is not sufficient. There is a lot of scrutiny for
relevance. So understanding the problem makes also a much
better proposal, even though you are doing the mathematics
that you want.
People mentioned DARPA. I have known DARPA for
many, many years. For DARPA you have to know the problem.
There is a very small part of the DARPA budget that is in
basic research. Most of it is in applied research and
demonstrations. In the process of using those large three
to five year demonstrations, a lot of research is being
done, and I am sure some of you have participated. But
those programs are fairly focused in trying to solve a
particular problem and make a demonstration. So in order
559
OCR for page 560
560
to play, you don't have to know every aspect of the
problem. They are usually large themes, and you spend an
awful lot of time going to meetings and coordinating on
those things.
But there is a substantial amount of funding that
allow the basic research to be done in the context of the
applied research. But you have to know -- the quick kill
three program is, if you are going to have a new algorithm
that is going to do it faster, cheaper, better and all the
other things. There is a substantial amount of funding.
The Defense Information Systems Agency together
with DARPA, they have a joint program office called JPO.
That office has kicked off a major homeland security
initiative. It is not exactly new money yet, there will be
new money in the future, but right now they are earmarking
existing money everywhere that could be pointed in that
direction.
I have been through that exercise, I write a lot
of proposals. I can see how we can take some of the work
that we are doing, dress it up appropriately, and look at
that problem. But in order to get funding for that, I will
have to be credible that at least I understand some aspects
that are relevant to homeland security, not leave it to the
reviewer to make the connection. I have seen enough from
560
OCR for page 561
561
the inside, the paperwork, there is somebody that will have
to write a paragraph that is that long on a form,
explaining why your work is relevant to the problem. So
you are passing the buck to somebody else. When you are
doing that, things don't work very well. It is much better
if you write it yourself, if I write it myself and submit
it and discuss it. It helps the proposal along.
But there is funding that is developing within
the various -- I haven't checked with the Office of Naval
Research, but I am sure they are looking at that problem in
the mathematics section, but one has to be a little more
closely aligned to the problem than was discussed in here,
at a fairly abstract and high level.
Thank you. I don't have any money, by the way.
Whoever would like to close --
DR. KELLER-McNULTY: I guess I was voted in on
this. I am the only other board member besides you two
here.
I just want to first of all thank everybody for
coming to the workshop. I think it has been pretty
exciting. I know that I have learned a lot, and I have
tried to listen carefully.
I think that there are clearly great challenges
mathematically for us to try to address that can be very
561
OCR for page 562
562
supportive to homeland security. I also think that there
is a fair amount of what we already do that can almost be
directly applied to some of the immediate problems, and it
behooves us to seek out those opportunities and try to do
that.
We heard a lot of things in the last couple of
days. Some of the things that resonated with me were the
following. First of all, we have to keep the human in the
loop. No one is talking about -- and some of our
colleagues when we talk to them, some of our scientific
colleagues, they get really nervous that we are trying to
develop methods that somehow take them out of the decision
making loop. I think it was pretty clear from everything
we heard that the human has to be in the loop, the science
has got to be incorporated into the problem, the domain
knowledge has got to be included in the solutions and in
the frameworks we build to solve these problems.
Which then of course points to one of the very
first things that Alex said when he opened the final
session, which was that you have got to know the problem.
Clearly, if we want to get collective funding to work in
this area, we had better darn well be willing to admit to
knowing that we are trying to solve a real problem, and try
to figure out how to pull the pieces together.
562
OCR for page 563
563
The other thing that came out really clear to me
is -- and something that we sometimes forget to talk about
when we are talking to people about the problems in the
work we are doing, is that quantification underlies almost
everything we are doing here. We tend to forget about it,
because it is part of our being as mathematical scientists
and as computer scientists. But everything is about
uncertain quantification.
The question that came up about the nuclear
weapons certification, it is not just about predicting a
point value, whether or not we think these systems will
work. It is all about uncertainty quantification and
knowing when we have to go back and start testing again, or
collect other types of information. That is true
throughout all the homeland security problems.
A lot of people talked about how we are in a
stage and in an era of being swamped with data, we just
have so much of it. What was it that Art said? That data
is dumb, which is true. Kathy commented that we have all
this data, data, data, we don't have time to think. That
is true on a certain level, but as soon as we ratchet these
problems up to the incredible dimensions that they are --
and Tod showed us what the dimensionality of the space is,
there actually is a lot of data sparsity.
563
OCR for page 564
564
In the last session, I think that Val's talk
tried to show how we can try to put things together in the
presence of a lot of data when you take some slices through
the problem, but not a lot of data or information when you
take other slices through that problem space. So that is
really important that we remember as well, that it is not
just about massive data mining, but it is about how to
integrate all of this information.
The final thing I want to say is that what we are
talking about is modeling complex systems. Perhaps we do
have to put hats on a bit like engineers and take a systems
approach to looking at these problems and figuring out how
we can build and develop the mathematical frameworks to
actually make progress.
I don't think the problems are impossible. I
actually disagree with one of the last comments that was
made from the floor, that a lot of what I heard at this
workshop is about prediction and forecasting, how do we
take what we have learned, what we are seeing, and project
ahead and forecast ahead, to try to understand what the
next thing is that needs to be done or where the next
vulnerability is.
These are not easy problems. We have to come
together and work on them. I hope that we do. I have been
564
OCR for page 565
565
pretty energized by this
and I look forward to continuing
to interact with many of you on these.
That's it
, .
Safe travel.
565
OCR for page 566
566
Remarks on Data Integration and Fusion
Alberio Grunbaum
Mathematics has anmazing, almost poetic, ability to look at a problem in a certain area
and then take a step back and sucicten~y realize that maybe we can use these tools for
some other problem in a different area. For example, there are a number of people who
have worked first in geophysics and then in mectica] imaging!
He cautioned, however, that as he and his colleagues movect in this poetic fashion from
one area of application to a different one, it was very important to remind themselves that
they tract to go back and start from scratch and talk to the experts in this new fieict. The
problems might look the same from the mathematical point of view, but they have all
very different features.
566
Representative terms from entire chapter:
little bit
