Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 157
Appendix E Transcript of Breakout Sessions GROUP 1 Group 1 Participants: Moderator: Derek Bothereau Richard Genik Fred Lybrand Jim O’Connor Peter Schwartz Paul Twohey Norman Winarsky Philip Wong Michael Zyda NRC Staff Member: Shannon Thomas Team Activity: Designing a Scanning System ZYDA: The big (loud crash) on the Internet was the Webcast and the fact that somebody had defined HTML. Those are two like boom! events. I remember this very clearly because I had a student come in my mine, Don Brutzman, and he goes, “Mike I got to show you Mosaic.” I said, “Go away, Don, I'm busy.” SCHWARTZ: Mosaic was the moment. ZYDA: And he goes, “I got to show you Mosaic.” I said, “Don, why?” He goes, “Oh, you'll like it.” So he showed it to me and I'm like – instantly I could – I had a use for it, which was information dissemination. Being a professor at that time I was stuffing envelopes with copies of reprints. [chuckles] ZYDA: All of a sudden I wasn’t in the envelope stuffing business anymore, I just had to build a Web page. And I will tell you within three years – three years later I went to a conference and there were 24 papers in the conference and 23 of those papers referenced papers off my Website. And so to me, the CD E-1

OCR for page 157
CD E-2 Persistent Forecasting of Disruptive Technologies – Report 2 Internet was a big boom! So the question is – but there was a lot of long- term work there. SCHWARTZ: One of the places that this diagram begins is decision-makers defining needs. Is a place that this system begins – Recognize we have to have a real client, namely the Department of Defense, Defense Intelligence and so on, do we – It seems to me that one of the important things we have to understand is what is this going to be used for. What do these customers actually want from this information? What they need to know, who needs to know it. WINARSKY: These “define needs” is right at up the top. That we surely need. ZYDA: So we should have at least one of those. BOTHEREAU Peter, you were just posing that. SCHWARTZ: Yeah. And it seems to me that we want to begin with a conversation of these -- O’CONNOR: Who’s paying for this and what do they want? SCHWARTZ: We know who’s paying for it but what do they want from it? How do they interact with it? How do they get to participate, what – [Simultaneous comments] BOTHEREAU: Should we start answering that question now? WINARSKY: This is what – this openness is what I'm constantly stressed – not stressed in a bad way, just stretching my mind in a good way. It isn't the needs of the funders that will make this a success, it’ll be the needs of people to consume and participate – SCHWARTZ: A very good point. WONG: I think that’s why I think you can almost – sort of on two streams right? You’ve got one stream which is the government side, which is their user base. And you can look at the other stream which is like the public -- SCHWARTZ: The participants. WONG: The participants, the public. And then think about that openness. Because I don't think the government cares that their interpretation to the system is open but I think the public does. WINARSKY: And then they interact with each other. WONG: Well, they interact or maybe they don't. [Simultaneous comments] Transcripts were not edited.

OCR for page 157
Appendix E CD E-3 SCHWARTZ: And what they want from it may be very different. WONG: The systems may even be different. WINARSKY: If the open system isn't any use to the government there’s no point in doing it. SCHWARTZ: That’s right. SCHWARTZ: So it’s got to meet that for sure. WONG: Yeah. SCHWARTZ: But if it doesn't meet the needs of the participants, they don't participate. UNKNOWN: Exactly. SCHWARTZ: So I think the point is that there are two different sets of needs, which may not be identical. WONG: Right. BOTHEREA: Incentives are a huge part of our criteria so I want to bring that in. WONG: I was thinking about interaction. I don't know if it’s necessary to somebody on the government side to interact directly with somebody, I guess, on the public side or the participant side, like have a direct communication line. WINASKY: No, no, no. WONG: But it could be a different type of communication. WINARSKY: That’s a benefit. In other words, this open forum about IEDs will have an impact on the government side, for sure. WONG: Right. Absolutely, yeah. WINARSKY: Right. And, yet, you know, that’s – the contribution shouldn’t be driven by the government. Contributions are driven by everybody that participates whether it’s Facebook++. O’CONNOR: We've got some people here who have done more deliverables like this to government entities. What would you think would be an acceptable deliverable from a system like this and what frequency should that be handed off? Is it access to some kind of Website, is it a quarterly report that’s written, what -- GENIK: I thought the frequency was it’s updated when it’s ready. Transcripts were not edited.

OCR for page 157
CD E-4 Persistent Forecasting of Disruptive Technologies – Report 2 SCHWARTZ: Well, yes and no. My experience in this says that actually – with bureaucratic institutions you actually have to have timing and milestones in situations: a quarterly meeting, annual meeting, semi – [Simultaneous comments] SCHWARTZ: The report is delivered at time and the people are prepared to receive it at that time and so on. And those are forcing function. They force you to update, they force people to come to the meeting, to attend and so on. WONG: That’s actually a good point, the frequency. Or does the system need to be designed so that if somebody asks a question, which is probably more likely in terms of “Oh, this has popped up on the radar as a potential threat to the country, go ask the smart guys down – the analysts – SCHWARTZ: So it’s probably both/and. [Simultaneous comments] UNKNOWN: Yeah, both/and. UNKNOWN: Yeah. WINARSKY: That’s what I wonder. My mental model has been more like a Disruptipedia, right. SCHWARTZ: Disruptipedia, I like that. That’s a good word, Disruptipedia. [chuckles] SCHWARTZ: The Disruptipedia system, I like that. ZYDA: Who’s checking out the URL? WINARSKY: Then it’s always there, people are always contributing. WONG: [chuckle] That is a good one. TWONEY: Yeah, that’s great. SCHWARTZ: Can we agree on a name? I think Norm’s actually come up with a great name, Disruptipedia. ZYDA: I like it, I'm in. TWOHEY: There’s a subtle point that we've been kind of dancing around which is that I think the assumption here is that the output was going to be the same across all times scales. And it’s not clear that we even want that. So maybe what it is – I've got a question – I've got a quick question so I can get a quick approximate answer, right, versus, “Hey, I need to have a more detailed understanding of these things.” Like these are very different use cases, right, Transcripts were not edited.

OCR for page 157
Appendix E CD E-5 they're even different ways of thinking, like kind of high-level visualizations versus starting at some point and I wander around and maybe I find something that’s interesting. It doesn't have to be the same system that does both, right. It doesn't have to the same results, it doesn't even have to be the same process but just understanding that like these are very different use cases. So maybe what we should do is come up with a bucket sets of use cases and be like “Okay, here’s this kind of thing. This kind of user probably wants this, this kind of user probably wants that,” and we should just pick them off and figure out, “Okay, well, we'll get to some of these guys in the first one and, you know, some of the other guys, well, we won't.” SCHWARTZ: A good friend is named Michael Nacht. Michael is the Assistant Secretary of Defense from everything interesting. He does cyberspace, all the cool stuff that’s not main warfare, right. He doesn't have a lot of tanks but he has programmers and all kinds of things like that, rockets, so he does all that stuff. So he asked me a question, he said, “Is there any chance in the next ten, fifteen years that somebody’s going to develop a really low-cost way of getting satellites up – fast.” ZYDA: Yes. SCHWARTZ: That’s the kind of question that might come along mid-year. A new guy comes in says, “I'm looking at this stuff,” and he says -- BOTHEREAU: And where would it come from and who’s working on it and – SCHWARTZ: Yeah, and how would I answer that question? ZYDA: And you want to also invert that question because the question maybe is “When will NASA lose its capability to launch?” SCHWARTZ: There’s also that. But the question he specifically – that is not his remit. His remit was to ask the question – Because NASA doesn't launch his satellites, Air Force does. And so the question is really more is there somebody else who could do something -- WINARSKY: Who could launch – BOTHEREAU: Pose a threat. SCHWARTZ: A new threat, that’s all. O’CONNOR: So there’s the immediate query and then there’s the – If you think about all the kind of – the list, the technology list, there’s also the function of list replacement or list aggregation. SCHWARTZ: So am I wrong? Maybe I was wrong to begin with when I said it ought to be a kind of regular cycle, maybe it ought to be question driven. Maybe this whole process ought to be simply – Michael Nacht says, “I got to know does somebody have cheap launchers,” and it feeds it into the system. Maybe it’s purely question driven. Transcripts were not edited.

OCR for page 157
CD E-6 Persistent Forecasting of Disruptive Technologies – Report 2 ZYDA: So what I envision is something like Facebook where – because if you put that as a post on Facebook then all of the relevant friends that you have could comment and say, “I've got a whole study over here, here’s the link.” Bang. UNKNOWN: [chuckle] O'CONNOR: I can go post that question on Yahoo right now and in 30 minutes – [Simultaneous comments] O'CONNOR: -- there’s no groundbreaking. ZYDA: There’s nothing to build. UNKNOWN: That specific of a question. WONG: Well, I think you can do that right now but how do you synthesize the information? How do you synthesize it? SCHWARTZ: I judge the quality and so on. WONG: Yeah, well, how do you actually – like all the stuff we're doing today, right, somebody should be sitting down and synthesizing this into a story or a recommendation, right? So I'm not too sure that piece you can really -- O'CONNOR: I mean on the existing platforms that are already there there’s one level of synthesis that goes on -- ZYDA: But then what happens is in this Facebook system we end up with ratings for people. Which is I've got a name, I can click on them, I can go get the info page about them and their expertise. SCHWARTZ: So that would – Well, now we do the details on system design. That’s a good example of what we want to include. Rating systems for participants, there’s that kind of system. ZYDA: And the value – you know, and there could be a retrospective look at the value that – the posts that person has made in the past. Which is when you see a post from someone with some knowledge you can – You’ve got like the Netflix star system, I click four stars, “This is great,” I click one star, “This guy’s wrong.” TWOHEY: You’ve got to be really, really careful about this. When you're rating a movie you're rating a distinct action or you're rating a distinct thing, right. So you're like, okay, before me like this noun had this like overall quality thing. When you start rating a person you really -- ZYDA: No. Rating the knowledge that they're putting out there in the post, not the person. Transcripts were not edited.

OCR for page 157
Appendix E CD E-7 TWOHEY: Okay. All right. ZYDA: And that will lead to overall rating of the person. TWOHEY: But you don't want to do that. As an incentive design structure you want to be very careful about rating people ZYDA: Sure. TWOHEY: Because I can know a lot about computer science and I don't know a freaking thing about combinatorial biology. ZYDA: So if you make a comment on combinatorial biology we're going to downgrade you with a low score and you deserve it. TWOHEY: Yeah. WINARSKY: But the way you're doing it is just what they're talking about not the individual. ZYDA: Right, not the person, what they're talking about. WINARSKY: But if a person has many times been rewarded with many thumbs up signs from -- ZYDA: This guy’s a sage. WINARSKY: -- in a given area, that person then becomes an influence. TWOHEY: Like Bernie Madoff with money? ZYDA: No, no, like when you go and buy something on Amazon. TWOHEY: The whole point is we're trying to find outliers, right? WINARSKY: The unknowns, yeah. TWOHEY: If you're trying to find outliers and you're saying, “Oh, great, I'm going to go find all the popular people.” WINARSKY: Yeah, you're right. TWOHEY: When do the popular kids ever like find the thing that is different, right? That’s the whole point. TWOHEY: The whole point we're trying to solve is like not what the popular kids want. So if you go after – [chuckle] WINARSKY: True. Transcripts were not edited.

OCR for page 157
CD E-8 Persistent Forecasting of Disruptive Technologies – Report 2 WINARSKY: That’s really true. This kid from Afghanistan that’s in a tribe is not going to necessarily be popular but he’s going to come up with something particularly valuable. UNKNOWN: Right. BOTHEREAU: But how should our system account for some of these kind of lesser-known information sources or kind of some of these – like how do you find them? How do you find those people? WONG: Well, I think you have to – can't you rank by relevance. I mean you have to have some system to rank relevance, I mean give certain weights to – SCHWARTZ: Yeah, but I think the point is how do you connect in with the outliers? How do we find those people out there, how do we engage them and seduce them into the system – TWOHEY: But what question are we answering? What question are we using these people to answer? Let’s stop thinking about how we're going to go rate people and do things. Let’s find like three or four different people who are going to be consuming our system and like how they want to use it. Then once we figure out what we want to give them then let’s figure out how we're going to do it, right. Not like “Oh, yeah, so we're going to go exploit these guys in a village and like make them farm for beans and like it’ll be great for us.” [chuckles] [Simultaneous comments] TWOHEY: I'm trying to say it in the most hyperbolic way possible but – SCHWARTZ: It’s good. BOTHEREAU: We appreciate that. GENIK: I think we should have at least one government customer and one consumer customer and then maybe -- BOTHEREAU: And I think your template – that’s the guy and that’s the question, right? GENIK: Yeah. GENIK: Consumer customer can be the movie. Give us the next plot of a Steven Spielberg short that’ll be the next technology to go awry and kill a family or something. UNKNOWN: Jim Cameron. SCHWARTZ: James Cameron movie. Transcripts were not edited.

OCR for page 157
Appendix E CD E-9 BOTHEREAU: As we look back at this as kind of a general flow is there anything that we feel like we haven’t talked about yet? Data feeds I feel like is – We've talked a lot about people, are there data feeds, things that are already out there that could be collected or -- ZYDA: I have a project right now funded by L&R to build an online game that does behavioral model testing. So we're actually building an entertainment game, putting it up onto the Internet, putting big servers up supporting several thousand people and we're putting various behavioral models that watch play and also interact with the people. But one of the things we're doing in that project is taking in extremes of real-time news feeds. So we're pulling RSS feeds off of CNN.com, off of the BBC, off of economic sites, and plugging that into the world that’s in the game such that it impacts the game play of the autonomous characters. SCHWARTZ: Oh, the autonomous characters, not the human characters? ZYDA: Well, and human characters by – So, for example, if we see a terrorist bombing coming across the RSS feed, the autonomous characters today are very nervous and upset and angry whereas if they don't see that they calm down over time -- LYBRAND: That’s pretty cool. ZYDA: -- and then they interact with different people. So we're building this, we're at the end of nine months right now, it’s going really well. But the whole idea of news feeds is the big deal because we're going to have this interact in interesting and experimental ways with people. SCHWARTZ: So you could answer a question for me. ZYDA: Yeah. SCHWARTZ: When I think about these data feeds – do I have to necessarily have in a sense a human being reading the data, understanding it, or are there interpretive models for taking it and further digesting it before it gets to me as a human being? BOTHEREAU: An automated model. SCHWARTZ: Yeah. ZYDA: Yeah. Yes, you can do automated stuff. TWOHEY: Everyone’s already gone over this. There’s this intelligence apparatus right now for dealing with the things that we know about, right. We already know about RSS feeds. They're already funding people to deal with this, right. Like we don't need to worry – I mean the whole point is coming with the system, it’s going to come with things that we don't know about, right. So I think that – Transcripts were not edited.

OCR for page 157
CD E-10 Persistent Forecasting of Disruptive Technologies – Report 2 SCHWARTZ: Well, no, no, no, no, no. I think that’s not quite right. In other words, the way in which we come up with things which we do not know about may not be out of anomalous data points of themselves but of the interactions of many different things together, the pattern of things which in and of themselves may not be sufficient. BOTHEREAU: The correlations not the events. [Simultaneous comments] TWOHEY: But I guess what I was – so there’s – the take that I'd like to put out there is this should be people driven because there’s going – Like for the data to be useful, for people to act on it, for them to disrupt they're going to find it. The axiom that we should us is that they're going to find it before we know about it and they're going to use it and act on it before we know about it. If you want to have an algorithmic solution for finding sets, you want to have a big Google, right, so you want to have a disruption rank or whatever it is – SCHWARTZ: But isn't it both? TWOHEY: It’s a totally different way of looking at it. Like the algorithmic things that you can do for these kinds of minings, they all assume that people have already built up and structured the data in some way, right? Like the reason page rank works is because everybody on the Internet has a local incentive to point to the thing that they care about and then Google goes and scrapes it and like puts it together. But if what you're trying to do is find the nascent things before they’ve established a pattern, right – So you might have this – you're like building some cloud of possible futures to go look at and maybe head off legislatively or politically or whatever. Then like by definition these things won't have been built, right? SCHWARTZ: But it’s a both/and. You're right in some sense but let’s just take a real example. I spend a lot of time thinking about the future of social dynamics in places like the Middle East. So not surprisingly I look at a lot of demographic data, cultural data and so on. Well, in the end, yes, I'm looking for myself, I'm the interpretive engine but before it got to me there were a whole bunch of people and a lot of data that went into massaging the demographic numbers and I'm looking at a bunch of different demographic projections and so on. So there’s an elaborate architecture underneath the human interpretation, some of which is purely mechanical, some of which is simply the integration of a whole bunch of demographic data. So it’s not an either/or, it’s a both/and. In the end it is a human interpretation, I fully agree, but along the way there can be digestion of data that is purely mechanical. That’s all I'm saying. TWOHEY: But it just sounds like what you're asking for is better tools to do your existing job. It turns out like the data analysis tools that you have, that they're not as good, they don't have as many feeds. You'd like them to be able to do better, you'd like to do all this kind of analyses but that’s a different question. That’s like “Hey, I've got a pretty good handle on what I want, I've got these six features, can you software tools people like go please Transcripts were not edited.

OCR for page 157
Appendix E CD E-11 make my data analysis things not stink? I'll pay you some money and like make it happen,” right? I don't know, I think this is a different question entirely. SCHWARTZ: What’s the different question? TWOHEY: The different question is I'm – what – where do we get the data and the leads to start looking at things that we don't even know about yet that should be the problem? WONG: Right. I wanted to make on that which is right now – I always get the sense that right now we're thinking about specking out a Website which, you know – for better or for worse, that’s the impression I get, that we're going to create a Website and we're going to have this exchange of information similar to a Facebook. But the reality of it is isn't the information already out there? People are just already using other sites and doing the things that we kind of want to observe? TWOHEY: A lot of it, probably. WONG: Right. So wouldn't a better solution [be] not to actually create – for us to interface with the end user but for other people who are already – the end user who’s already interfacing with whatever application out there, for those people to look and see what they're doing on those other applications. [Simultaneous comments] SCHWARTZ: Can you give us an example? WINARSKY: Mining that information. WONG: Yeah, mining that information. [Simultaneous comments] WONG: In an ideal world if you could create a search spider -- WONG: -- that could go out and collect information from every single Website out there that you're interested in and pull that information and then process it, wouldn't that be as useful as having your own site and being able to process that information? LYBRAND: Produce widgets for other people to put on their sites. [chuckle] ZYDA: There was a really cool search system that one of my former students has been trying to raise funding on and I liked it a whole lot. I'll describe it to you because it’s cool. I don't think he'll get upset. You're a scientist and you write a technical paper in an area and what you'd like to know is “Did I get all of the prior literature right and is there anybody else working in this area?” You take your PDF and you drop it onto the search engine and it goes out and it reads the whole document and finds people who are working in Transcripts were not edited.

OCR for page 157
Appendix E CD E-161 PAYNE: That’s assuming somebody else is doing it. If we’re ahead of the problem or we think we’re ahead of the problem, we’re looking at can we do it and how do we do it and finance [..?..]. GRAY: Well and that’s the question. And the VC guys are going to look at it and say, "Is this something that the private sector’s going to want to bid on, yes or no?" Even if we want to do it first, is the private sector going to want to bid on this or is this something that’s only going to fly if the government is there? LONG: So Harry, would you have put financial backing on GPS twenty years ago? BLOUNT: I have no idea, really have no idea. LONG: Probably – I would hazard they would not. PAYNE: Yeah, because the timeline on getting your return on investment is too long. SAFFO: Well but this is just a matter of granularity, it’s the scale problem when you’re deciding to invest. LONG: No, well this is a question of who’s going to do it, right? PAYNE: And the other thing too is that, you know – LONG: The government had a compelling need for this, right? It wanted the guys in the Army to know where the hell they are, okay? They didn't care about people driving around in their cars or me and my iPhone navigating the streets. PAYNE: That’s when Harry’s folks come in. LONG: That’s when they come in. VELOSA: No, no. But the semiconductor firms and some of the electronics firms were in there because one, they had products that were relevant and they had contracts. LONG: What is this? VELOSA: The semiconductor and electronic firms. LONG: In 1980’s? VELOSA: They were starting to work on some of the parts for it. They didn't know GPS but they had some of the parts for it. BLOUNT: I think we’re asking the wrong question. Transcripts were not edited.

OCR for page 157
CD E-162 Persistent Forecasting of Disruptive Technologies – Report 2 LONG: I don’t know which part you’re talking about. SAFFO: I wouldn't push too hard on the GPS example. It’s a nice one at a general level but it’s not – BLOUNT: I think you’re getting to the question of asking the right question at the wrong time. So if we’re designing this correctly, is GPS going to be feasible, is a disruptive? The financial guys may say, "Right now it’s not feasible but here are the signposts for feasibility." That in and of itself becomes a persistent question that continues to echo through and then when you hit, boom, a price point, the semiconductor guys start making investments, what have you. So I think these guys are here to say -- GOLDHAMMER: So who are these guys? Who’s doing that? BLOUNT: I think in the social/political realm is if it is something like the National Highway System, you know, if one of the enablers is the National Highway System, that has to take some kind of political decision so that becomes an enabler in terms of length. That’s a big enough project. PAYNE: And like I said, those folks are kind of fairly senior folks but then whatever organization’s asking the question, you know, it wasn't -- In that case it was like Department of Defense was asking that question so then you’d have some of the folks probably in DDR&E that were involved in that and just OSD, Office of Secretary of Defense. VELOSA: But you’ve also got, you know, X congressman with his pet company and his territory is going and saying why are you buying? PAYNE: Yeah, but we can’t put them in the process. VELOSA: But they are in the process though. PAYNE: They’re wildcards though, they’re wildcards, 'cause you don’t know. There’s no logic behind – UNKNOWN: They’re external. Yeah. [simultaneous comments] LONG: You wouldn't go and get X congressman and put him in your system, right? They’re external actors. VELOSA: They’re already in the system. LONG: Not in this system if we’re building a system. [Simultaneous comments] GOLDHAMMER: Yes, yes. Transcripts were not edited.

OCR for page 157
Appendix E CD E-163 BLOUNT: China’s going to need to consume massive quantities of resources over the next 20 years to support their economy. What are the key things as they go out and try to secure those resources to support their country long term, what are the social and political considerations that are going to enable the forks in the path? GOLDHAMMER: But who’s doing -- A question I’m asking, which is who’s doing that evaluation? Is that just inside the government or are these academics doing it? [Simultaneous comments] LONG: No. VONOG: Maybe it's like intelligence LONG: I think you need to reach out, you need to be able to reach out to like external academics as well, right? GOLDHAMMER: Uh-huh. Yep. LONG: So for – whatever the experts are. PAYNE: National Academies. LONG: Sometimes but, you know, you need to be able to -- look, if you’re in the government and you see something going on and you have a concern, you need to be able to reach to somebody. National Academies has a long cycle time, okay? You should have your Rolodex, call up your favorite professor of, you know, minerals politics at a university and say, “Western Australia is being sold to the Chinese. What’s the impact here?” I mean, literally, that’s the – Australian economic boom is the fact they’re digging up all of Western Australia and shipping it to China. GOLDHAMMER: Gaming and -- I’m just going to keep you guys focused here. Gaming and cloud sourcing. In this product, who’s doing this? PAYNE: That’s internal and external because as far as Department of Defense is concerned, we do our own game in a lot of things, but we also use external organizations to do that as well. GOLDHAMMER: Contractors, yeah. GOLDHAMMER: So contractors? GRAY: So you can use, yeah, so DOD, you’ve got contractors. GOLDHAMMER: Okay? So this whole thing, I mean, the principle here is you actually have people inside and outside, across all the evaluation who are doing it. Is there any entity that is sort of managing this process that’s, you know, Transcripts were not edited.

OCR for page 157
CD E-164 Persistent Forecasting of Disruptive Technologies – Report 2 sort of figuring out what questions go to what people, what answers are good, what answers are bad? VELOSA: I think it would be the team that – there has to be a team. GOLDHAMMER: Okay. GRAY: There has to be some sort of standing group that -- Because -- GOLDHAMMER: It’s an evaluation group. GRAY: Yeah, that’s an evaluation group and they don’t necessarily do the evaluation. They coordinate the evaluation. VELOSA: But it's the stewards. They [..?..] the system design stewards. GRAY: They don't necessarily do the evaluation, they coordinate the evaluation. BLOUNT: Yeah, that’s critical. GRAY: They’re not, you know -- so the nice thing there is these can be information managers but not technical people. BLOUNT: Actually I would argue this is the National Academies’ role right here. A stakeholder comes to the Academy with a hypothesis, they bring together -- VELOSA: Oh, like a TIGER committee for this? BLOUNT: Well or this committee or any other committee. They come with a hypothesis and the role of National Academies is bring a bunch of people together -- LONG: Hopefully there’s way too much traffic going through there. [Simultaneous comments] VELOSA: There has to be a standing group though. It can’t be just to come together -- GOLDHAMMER: But this seems like a small group of people to me, you know. GRAY: Yeah, it does. LONG: What I’m saying is for every agency that’s interested in doing this kind of thing there’s a large number of hypotheses flowing through here being evaluated and meeting quarterly at the TIGER meeting isn’t going to do this for you. PAYNE: And, you know, one of the fundamental flaws within DOD is that – Transcripts were not edited.

OCR for page 157
Appendix E CD E-165 LONG: I think the TIGER can oversee some big process like this. PAYNE: -- the science and technology folks, the senior scientists and technologists, they don’t have a, you know, they get together but then they have coffee and they just talk hypothetically, have really no power, almost no power to get anything done. And it’s actually the Acquisition Committee that actually has the power because they have the money. The money goes to them, you know. And even, they even control the DARPA money, you know, and so it’s, what happens is, and part of the hypothesis evaluation, the hypothesis engine is they gather – and we’re talking DOD – is they gather requirements, they gather, from the combat and command, which is like SOUTHCOM and CENTCOM, all that stuff, and they gather all the things from the intelligence community within the Department of Defense as far as, you know, concerns. And you need to have, where that comes together with science and technology has a role in it and right now it doesn't, it really doesn't. I mean, they give it lip service. They go, “Oh, this person’s the Science and Technology Advisor,” you know, and if they can get in and say, "Hi" and like, “Oh, yeah, what have you been doing?”, “Oh, we’re working on such and such, such and such, you know, and here’s some decimal points” and then, you know, that political appointee’s like, “Hey, man, nice talking to you. I’ll see you next month.” And that’s really the problem is, from our perspective, is getting those folks in with the folks who actually write the checks. Because everything that you buy today in the Department of Defense you’re going to take away from somebody else. That’s how it is. GOLDHAMMER: Yep, zero sum game. Yes. PAYNE: There’s a zero sum game issue. And I used to go to the guys who make decisions in the Department and say, “Hey, we need more money for this program.” And they’ll say, “Okay, where are you going to get it from?” I say, “I came to you. You know where it is?” GRAY: I see this, I see this, the management of hypothesis evaluation, the hypothesis engine, I see this as being an entity, whether it’s a contracting firm, whether it’s some sort of internal organization within – PAYNE: You don’t want it to be a contracting firm. GOLDHAMMER: Well no, I’m just saying but, you know, inside the, you know, whether it’s a new sub-piece of the TIGER committee or something like that. PAYNE: Well here’s the answer for today. Today this will come from the Acquisition Committee, okay? What needs to be added in here – LONG: Well the money will but the staff, you staff over there. PAYNE: The money and the decisions and some of the hypothesis generation will actually come in there. But, you know, maybe DARPA. DARPA has some say, but – as far as where they want to go. But there needs to be Transcripts were not edited.

OCR for page 157
CD E-166 Persistent Forecasting of Disruptive Technologies – Report 2 more emphasis on the science and technology, the scientists and technologists, within the Department of Defense, have a say in that. CULPEPPER: Let me say something though. I think one of the -- when I look at this I think of Hollywood and the whole market of Hollywood, right? And these things happen all the time in Hollywood and there’s an entire ecosystem that’s built up around it and if there’s a way that you could replicate that ecosystem, you know, be it DOD, be it Department of Transportation, be it the private sector, finance, it doesn't matter, where you can basically have the -- because the enablers are the active and passive data feeds, right, and then a vested set of interested bodies who are generating the hypotheses, right? GRAY: But see -- I didn't mean to cut you off earlier, Ken, but what I’m saying is we need to think of this as coming out of our report. We need to think of this as DOD, you know, and the Acquisition. That’s great. That’s one potential customer. The Federal Reserve, you know, somebody else could be another customer as well. PAYNE: But DOD is paying for this. GRAY: Well I understand that. PAYNE: And the DOD wants to know -- because the Federal Reserve is different than DOD. GRAY: Well, I understand that but the architecture of the system – VELOSA: [..?..] DOD if it has multiple partners on this. GRAY: -- the architecture of the system is -- GOLDHAMMER: I want to interject, I want to interject because I want to just kind of focus in a more practical matter. There are lots of things that we’re saying that we’re doing here and the conversation about who ultimately is going to pay for it or what that might mean is important but actually what would be very helpful right now is to figure out what exactly is the staff that does these things. LONG: Right. So I don’t think this is going to be staffed with acquisition officers, okay? It’s the wrong kind of person, okay PAYNE: I agree with you. LONG: Okay? This is – PAYNE: I’m not saying that’s how it should be. I’m saying that’s just how – LONG: It will fail. I guarantee you if you put Acquisition people, this will fail. PAYNE: But I want to put in context the way it is right now. Transcripts were not edited.

OCR for page 157
Appendix E CD E-167 VELOSA: But to me, this and this are the same people. LONG: No, absolutely not. VELOSA: They have to be 'cause they drive the -- They don’t do it at – LONG: No, absolutely not. The people that understand the, that have the scientific knowledge, the other knowledge here, are not the people generating the hypothesis. VELOSA: They have to have the same scientific knowledge. GRAY: Six, I would say six data analysts there. That’s – I’m going to put a number. Six analysts. GOLDHAMMER: Six analysts doing passive ID? GRAY: Doing. Doing passive analysis and this is going into kind of what the other table had with -- LONG: [..?..] cell biology [..?..] -- CULPEPPER: Nah. I think you could do that, I think a team of three development oriented, totally development oriented. [Simultaneous comments] GOLDHAMMER: It’s a small number of people but are there partnerships there as well? You’ve got to be doing all kinds of – or is this all coming from inside the government? GRAY: No. I think there’s partnerships. I think there’s -- LONG: See this arrow here? That’s the [..?..] and that’s the CIA and that’s the DIA. GRAY: Right. I think you’re getting input from analysts that are in other communities. I think you have another, you know, another two people that are doing the data mining of different online systems, you know. VELOSA: Right. But you then have to have the hypothesis generation and you have to have folks with an ability to understand the technical issues as well as the objectives from the questions. And to me, I’m not sure I understand your logic about why you wouldn't want them to then be involved later. LONG: I don’t want them evaluating their own hypotheses. These guys, their job is to think up scenarios which may be a – you know, in IBM we call them “wild ducks”, okay? These guys think up things and we have this whole set of here, you know, all of these guys here that take these Transcripts were not edited.

OCR for page 157
CD E-168 Persistent Forecasting of Disruptive Technologies – Report 2 hypotheses and say do these make any sense. You know, your job is to be a visionary and think of things that are possible. GOLDHAMMER: We’re not going to resolve this now so Paul, I think Paul has a comment. SAFFO: I wish Harry were here 'cause he would support me on this. The secret sauce here in the Valley for identifying technology disruptors is really simple and it’s called lunch. It is, “Hey, Darrell, let’s do lunch” and we talk. It’s lunch at the sun deck, it’s social events and the biggest problem I see that the government has is everything’s a goddamned pain in the ass to do in terms of, you know, forums to buy someone a beer. [General laughter] You know, it’s really good form. And so I just, I throw this out because what worries me as we start getting down to this, it starts getting formal. And the top-level thing I would say is this is a small team of people total. I once had a secretary who was a born-again Christian and I was complaining about the politics in my firm and she said, “Hey, look at Christ. He had 13 partners and that was one too many.” [chuckles] LONG: Ouch. SAFFO: And so I would say, I would go the hypothesis you could do this entire thing with 12 people. GOLDHAMMER: So I hear a general consensus that this is not a lot of people. GRAY: No, it’s not a lot of people. GOLDHAMMER: We can decide whether it’s the same people or not, we can solve that, but it’s not a lot. VELOSA: Okay, so we have a number here of 12 total for everything. GRAY: I could go with 12, I could go with 12. LONG: I could go with 12 too. CULPEPPER: I could go with 12. And so the question is, you know – and I think you -- SAFFO: Or you can put 12 or less, 12 or less. GRAY: And I think you divvy it up, divvy the 12 people up, based on what you’ve got going on at the time, what’s coming in that you need to have done. SAFFO: It’s a small team of people and it’s in the NAS model which, you know, let’s face it. The National Academies, the way it does its work is the same way that, you know, Blanche Dubois in Streetcar Named Desire. “I have always relied upon the kindness of strangers.” Transcripts were not edited.

OCR for page 157
Appendix E CD E-169 GOLDHAMMER: Are there any specific partnerships that you see as being critical to doing this with specific organizations? CULPEPPER: Search engine. I would go search engine big time. CULPEPPER: Search engine on this side. I think this is -- VELOSA: But to Darrell’s point, the CIA -- I mean, it already exists. GRAY: So search engines on that side. On this side I think you’ve got social networks, on this side I think you have social networks, I think you have special interest blogs, you know, so -- CULPEPPER: This is like cloud shields and like – [simultaneous comments] LONG: Yeah. GOLDHAMMER: And then what about up here? Is there anyone, is there – you know, Stewart had the like, you know --Is there anyone, is there – you know, Stewart had the like, you know – LONG: This is the big boss. GOLDHAMMER: This is the big boss. LONG: Yeah. GRAY: This is, as Ken was saying, this is, you know, this is – LONG: You know, the Customs Service, right? It’s interested in -- PAYNE: I’m staying out 'cause I got too involved earlier. GRAY: Okay, yeah, you got too involved earlier. No problem, no problem. CULPEPPER: I think there is something else over here though, potentially. I call them gravity wells. They’re big, they’re portals that people are drawn to because they’re connected to other people, right? And, but I think the long bet or the Long Now Foundation is actually, I’m not going to say it’s the right one but it has many of those characteristics. GOLDHAMMER: For up here? CULPEPPER: Yeah, for up here. It bears that fabric, right? BRAND: At Long Now we continue to be astonished by the impact. [Simultaneous comments] Transcripts were not edited.

OCR for page 157
CD E-170 Persistent Forecasting of Disruptive Technologies – Report 2 CULPEPPER: Yeah, yeah. VELOSA: And I would also argue that even the portals that connect them with the feds, the State Department and all that there, that would also have here -- GOLDHAMMER: This is an interagency, interagency? VELOSA: Yeah, interagency and sort of -- LONG: Well certainly there’s, you know, there’s communication, right? GRAY: Scientists don’t do reality. LONG: You don’t want everybody doing the same thing and coming up with different answers and not telling anybody or you don’t want to build another set of stovepipes. GRAY: How many perfectly good hypotheses were killed by a collection of data? GOLDHAMMER: Any other – just look at the table quickly. Any other resource requirements that you think are important before we wrap this up? We have a small number of people, we’ve got – LONG: The hard thing that you’re going to have dealing with this is inter-agency cooperation, okay? This one, it’s a nice green arrow but this is hard, right? You need to convince NSA, you need DIA, CIA, to give me the information that I can look at to try to find the signal, right? GOLDHAMMER: How many people are cleared in this system? You play in that world a little bit. LONG: It depends on who this is, right? If this is the Customs Service it’s different than if this is DIA. GOLDHAMMER: But no, but since -- I mean, hypothesis generation was your idea. One question I have for you is like how many of these hypotheses are classified and get stored in a vault versus – LONG: It depends on where this information comes from. If this is all open source stuff, then these guys are all unclassified. If this came from HUMINT then -- GOLDHAMMER: Then it’s highly classified. LONG: -- it’s highly classified. So it really depends on the inputs here. GRAY: I mean, any, yeah, 'cause any public data mining and stuff like that is [..?..]. [Simultaneous comments] Transcripts were not edited.

OCR for page 157
Appendix E CD E-171 LONG: If this is just going out to the Internet and reading blogs and, you know, whatever, then this is completely unclass. If this came through, you know, interesting ways then – GOLDHAMMER: Any other final comments and we can otherwise head back to the table? All right. Thank you, guys. Feedback to workshop participants in Appendix D. Transcripts were not edited.