10
Modeling of Behavior at the Unit Level

INTRODUCTION

Within seven minutes, the crew of the American Warship Vincennes, faced with a dynamic and hostile environment, had to decide whether an approaching aircraft was hostile (Cooper, 1988; Rochlin, 1991). Whether the decision made—which resulted in shooting down a nonhostile aircraft—was right or wrong is not the issue. The decision certainly was critical: failure to respond appropriately could have resulted in the Vincennes being attacked and possibly sunk. Such decisions are the result of a variety of factors at both the individual and unit levels (Roberts and Dotterway, 1995). Among the factors at the unit level are the number of opportunities for the organization to review its decision within the abbreviated decision period (Cohen, 1988), the type and appropriateness of the training provided to the decision makers (Duffy et al., 1988), the quality and accuracy of the information available (U.S. Congress, 1989), the hierarchical command structure, and inherent bias in the operating procedures involved (Watson et al., 1988). As Admiral William J. Crowe Jr., then Chairman of the Joint Chiefs of Staff, commented: ''The rules of engagement are not neutral. They're biased in favor of saving American lives."

Are there changes that could have been made to the Vincennes' command, control, and communications (C3) structure that would have altered the course of action taken? Many have argued that changes in the C3 structure could in fact have led to a different outcome. Research on organizations and teams suggests that a less hierarchical structure, more opportunities for review, and alternative training scenarios could all have altered the outcome. The Vincennes incident



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations 10 Modeling of Behavior at the Unit Level INTRODUCTION Within seven minutes, the crew of the American Warship Vincennes, faced with a dynamic and hostile environment, had to decide whether an approaching aircraft was hostile (Cooper, 1988; Rochlin, 1991). Whether the decision made—which resulted in shooting down a nonhostile aircraft—was right or wrong is not the issue. The decision certainly was critical: failure to respond appropriately could have resulted in the Vincennes being attacked and possibly sunk. Such decisions are the result of a variety of factors at both the individual and unit levels (Roberts and Dotterway, 1995). Among the factors at the unit level are the number of opportunities for the organization to review its decision within the abbreviated decision period (Cohen, 1988), the type and appropriateness of the training provided to the decision makers (Duffy et al., 1988), the quality and accuracy of the information available (U.S. Congress, 1989), the hierarchical command structure, and inherent bias in the operating procedures involved (Watson et al., 1988). As Admiral William J. Crowe Jr., then Chairman of the Joint Chiefs of Staff, commented: ''The rules of engagement are not neutral. They're biased in favor of saving American lives." Are there changes that could have been made to the Vincennes' command, control, and communications (C3) structure that would have altered the course of action taken? Many have argued that changes in the C3 structure could in fact have led to a different outcome. Research on organizations and teams suggests that a less hierarchical structure, more opportunities for review, and alternative training scenarios could all have altered the outcome. The Vincennes incident

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations illustrates how the C3 architecture of the organizational unit can influence its performance. Reasoning about the unit requires understanding not just how the individual agents perform their roles, but also how the distribution of resources and tasks across personnel and the specific C3 architecture constrain individual action, alter the flow of information, and ultimately influence the unit's performance. If appropriate computational models of organizational units and their C3 structures were available, one could begin to reason about questions regarding the design of C3 architectures such as that of the Vincennes. One could also reason about the behavior of other structures, such as coalition C3 and the overall behavior of opposing forces. The purpose of this chapter is to describe current computational approaches to modeling of C3 architectures and the tools and techniques needed to reason about C3. The focus is on unit-level models in which each of the actors (commander and subordinates) is modeled, as well as some of the factors and procedures that link these actors together. Thus, a unit-level model of a fixed air wing might model each team member, along with the lines of communication and types of messages, reporting functions, norms about contacting or not contacting others, social knowledge about how to operate as a team, intelligence about what to do when one or more team members become disabled, the tasks, the authority structure, the procedures, differences in actors due to physical position, available resources, training both individually and as a team, and military level. We might characterize such C3 models as social agent models in which there are multiple agents connected by one or more networks. Such computational models of organizational units have a large number of potential uses in military settings. For example, imagine a man-in-the-loop simulation used for training a brigade commander and staff. Unit-level models could be used to simulate the behavior of units under the commander's direction. Realistic models of the subordinate units would enable the commander to use more realistic simulations of battle engagements to explore the impact of communication losses, personnel losses, manpower reduction, misunderstanding of orders, resource attrition, and so on. Having such models would also reduce training costs as not all units would have to be present at the same time. Moreover, units could take part in joint task force operations without the complete unit being sent. As another example, imagine a computational model of information flow in a C3 architecture. Realistic unit-level models could be used by commanders to perform a series of "what if" analyses that would help determine the relative merits of putting in place various types of information and communications equipment. Models of the organizational unit could also be used for examining resource reallocation strategies, determining the fragility or flexibility of an organizational unit under stress, and exploring viable adaptation strategies. Computational models of the unit's C3 structure may also be an important part of planning modules. Finally, C3 models could be useful in determining areas of vulnerability in opposing forces.

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations Currently within the Defense Modeling and Simulation Office (DMSO), and indeed across the services, there are very few organizational unit-level models of any sort, and even fewer of the multiagent or network type that are discussed in this chapter. Those models that do exist within DMSO and the services are often inflexible and cannot be adapted to different environments. Further, the current military models of opposing forces tend to be static or to represent the unit as an aggregate, and so sidestep issues of command and control. Unit-level learning models, such as those described in this chapter, in which the collection of agents and the networks connecting them are modeled, have the potential to change this situation. Such models could be used to examine the potential impact of changes in C3 structures. They could also be used to examine the relative effectiveness of a given C3 structure in conflicts with opposing forces having a different C3 structure. Unit-level learning models may be particularly useful for examining issues of unit effectiveness and flexibility in a dynamic and volatile environment. Finally, general improvements in computational models that involve multiagent learning may have general application as optimization techniques for determining the optimal C3 structure when the relationship between performance and the various characteristics of individuals and the unit is extremely complex. In the military context, models that attempt to speak to unit-level issues typically take one of two forms: (1) the big-picture simulator and (2) expanded individual models. The big-picture simulators are simulation models in which the unit is typically characterized by a set of very general or aggregate features, such as size and number of resources. Such models have been used to calculate general levels of attrition or overall troop movement. They are typically instantiated as a series of equations reflecting average behavior. Details on individual cognition and differences in individual behavior are typically ignored. Team- and group-level issues, such as the impact of the existing knowledge network (who knows what, and who knows who knows what), the underlying communication network (who talks to whom), effects due to the specific authority structure (which is particularly crucial in joint operations), and so on are also completely ignored. The big-picture simulators that have been used for modeling blue forces, red forces, joint forces, or coalition forces often have only a minimal, if any, model of the C3 architecture. Such models are particularly useful for making high-level predictions—for example, about attrition—regarding very large groups when the statistical properties of the group are more important than individual differences. These models are not sufficient, however, to address most C3 concerns. For example, a typical way of representing C3 issues is as an error rate around some distribution of response. However, since the C3 architecture itself can affect performance in complex and nonlinear ways that vary by task, having a computational module that alters organizational unit behavior based on the C3 architecture could increase the validity of these models. The expanded individual models are simulation models of individual agents in which each agent has, as part of its knowledge base, some organizational unit

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations knowledge. An example of this approach is the Soar-intelligent forces (IFOR) model that was deployed in synthetic theater of war (STOW)-97. These models are particularly useful for modeling very small units (3 to 12 members) that are engaged in tasks requiring a high level of cognitive processing. Such knowledge might include rules about who takes over if the commander is incapacitated, rules about what communication can and should be sent to whom and in what format, and expectations about the behavior of other members of the organizational unit. A key characteristic of these models is that each agent is modeled separately, with a high level of attention being paid to the modeling of individual cognitive capabilities. Sometimes team models are built by combining a small number (fewer than 12) of these individual agent models. Organizational unit behavior is expected to arise from the ongoing interactions among the agents or to be stored as default knowledge about what to do when certain preplanned contingencies (such as attrition) occur. Details on the social organization of the unit and measures of unit-level behavior are typically not included as an integral part of these models. Norms, roles, and general social knowledge are treated as exogenous. Team models built by expanding individual models are rare. In the few existing models of this kind, the organizational unit-level knowledge of the team member agents is often based on a characterization of standard operating procedures, expert opinion, or doctrine. Thus, these models are organizationally static because the structure is fixed by these opinions or procedures. Moreover, since the mechanisms for acquiring and using social knowledge are often major determinants of organizational unit behavior, having socially realistic agents could increase the validity and use of these models. The vast majority of the work on organizational unit-level models has occurred outside of DMSO and the military context. Within this larger context, a third type of organizational unit-level model also exists—the social agent model. These are simulation models of organizational units as collections of multiple agents engaged in performing a distributed task. These models are particularly useful for modeling small to moderate-sized units (3 to 100 members) when the focus is on communication, reorganization, reengineering, or planning. A key characteristic of these models is that each agent is modeled separately, with a high level of attention being paid to modeling of the connections among these agents as determined by the C3 architecture and the mechanisms for acquiring and using social knowledge. In many of these models, the agents are adaptive. Organizational unit behavior is expected to arise from the ongoing interactions among these agents as situated in the specific C3 architecture. Details on individual cognition are often minimal. Clearly, models of unit-level behavior in the military context are currently lacking. The next section examines the rationale for developing such models. This is followed by a brief review of prior work in unit-level modeling that has been done in nonmilitary settings. The various potential applications of such models and some of the additional techniques necessary to make them useful in

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations the military context are then discussed relative to three areas of application—design and evaluation of C3 architectures, modeling of opposing forces, and unit-level learning and adaptation. Next, four overarching issues that arise during unit-level modeling are reviewed. This is followed by a synopsis of the current state of unit-level modeling languages and frameworks. The final section presents conclusions and goals in the area of unit-level modeling. WHY MODEL THE ORGANIZATIONAL UNIT? Models at different levels are useful in different contexts. In physics, for example, Newtonian mechanics is sufficient for modeling and predicting the behavior of large masses, whereas quantum mechanics is needed for modeling and predicting the behavior of specific particles. In general, one does not need to resort to modeling the behavior of each individual particle to model large masses. Indeed, it is impossible to specify the behavior of large masses by specifying the quantum mechanics of all particles because of the large number of complex nonlinear interactions involved. Similarly, models at both the unit (Newtonian) and individual (quantum) levels play an important role in the social world. Different tasks require different models. Sometimes, particularly for man-in-the-loop simulations, both individual-level models (of, for example, key actors) and unit-level models (of, for example, secondary actors), may need to be included. There are a variety of reasons for pursuing models at both the individual and unit levels. First, even if it were possible to build and run organizational unit models by modeling all the constituent agents, doing so might be too computationally intensive. Second, the behavior of organizational units appears to be more than a simple sum of the parts. Rather, the behavior of the organizational unit is often strongly determined by the ongoing pattern of interactions among the agents, the norms and procedures governing group interaction, and legal and institutional constraints beyond the control of any one agent. This point is discussed further in a later section. Third, at the present time, even if the computational resources needed to build and run organizational unit-level models composed of large numbers of cognitively accurate individual agents were available, it is unlikely that such models would generate organizational behavior. The reason is that to generate realistic social and organizational behavior, each agent would have to possess huge quantities of social knowledge, including knowledge about others, how to act in a group, when to follow a norm, and so on (Carley and Newell, 1994). There is no compendium of such social knowledge that is universally agreed upon and in a machine-readable form or easily translatable into such a form. Thus, the computer scientist looking for a list of the 300+ rules that govern social behavior will not find a set that will be acceptable to social scientists. Although there is a large amount of information, theory, and data that can be drawn upon, accessing such information will require either working with social scientists or making a substantial effort to understand a new field or fields.

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations Even if such a compendium existed, it would still not enable agents to generate all social behavior as not all behavior emerges from ongoing interactions; rather, some behavior is a result of social or institutional decisions made at earlier points in time or by agents not in the current system. Further, such a social knowledge base might well be far larger than a task knowledge base. Finally, there is a disjunction between the current state of knowledge of unit-level and individual-level behavior. A fair amount is known about how organizational units influence the information to which individual agents have access. A fair amount is also known about how individual agents in isolation learn to do tasks. However, little is known about how individual agents as members of groups perceive, interpret, and use information about the C3 architecture in deciding how to perform tasks and partition tasks across other agents. PRIOR WORK IN UNIT-LEVEL MODELING Research on organizational unit-level models has a fundamentally interdisciplinary intellectual history, drawing on work in distributed artificial intelligence, multiagent systems, adaptive agents, organizational theory, communication theory, social networks, sociology, and information diffusion. One of the earliest such models is presented by Cyert and March (1992 [1963]). Theirs is a relatively simple information processing model. They use this model to explore the impact of cognitive and task limits on organizational economic performance. This particular model, and the results garnered from it, are not directly relevant in the military context, but the newer models are relevant. Current organizational unit-level models take an information processing approach (for a review of these computational models see Carley, 1995b). The information processing approach is characterized by attention to the cognitive and social limits on the individual's ability to process and access information (Simon, 1947; March and Simon, 1958; Thompson, 1967; Galbraith, 1973; Cyert and March, 1992 [1963]). Today's models are much more realistic than the firm model of Cyert and March and the garbage can model of organizational decision making under ambiguity of Cohen et al. (1972). Today's models also draw on research in various traditions of organization science, including social information processing (Salancik and Pfeffer, 1978), resource dependency (Pfeffer and Salancik, 1978), institutionalism (Powell and DiMaggio, 1994), and population ecology (Hannan and Freeman, 1989). These models vary dramatically in the level of abstraction and level of detail with which the agent, task, C3 structure, and technology are modeled. Some are simple abstract models of generic decision making behavior; examples are the garbage can model (Cohen et al., 1972) and the Computational Organizational Performance model (CORP) (Lin and Carley, 1997a). Others are very detailed, specific models of organizational decisions or decision making processes; examples are the virtual design team (VDT) (Cohen, 1992; Levitt et al., 1994; Jin

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations and Levitt, 1996) and Hi-TOP (Majchrzak and Gasser, 1992). In general, increasing the level of detail in these models increases the precision of their predictions and decreases their generalizability. The more abstract models are often referred to as intellective models . Intellective models are relatively small models intended to show proof of concept for some theoretical construct. Such models enable the user to make general predictions about the relative benefit of changes in generic C3 structures. The more detailed the model, the more specific their predictions will be. Intellective models provide general guidance and an understanding of the basic principles of organizing. To run these models, one specifies a few parameters and then runs one or more virtual experiments. The results from these experiments show the range of behavior expected by organizational units with these types of characteristics. These models are particularly useful for demonstrating the necessity, inadequacy, or insufficiency of specific claims about how organizational units behave and the relative impact of different types of policies or technologies on that behavior. Additionally, these models are useful for contrasting the relative behavior of different organizational units. The very detailed models are often referred to as emulation models . Emulation models are large models intended to emulate a particular organization in order to identify specific features and limitations of that unit's structure. Such models enable the user to make specific predictions about a particular organization or technology. Emulation models are difficult and time-consuming to build; however, they provide policy predictions, though typically only on a case-by-case basis. These models typically have an extremely large number of parameters or rules and may require vast quantities of input data so they can be adapted to fit many different cases. To run the emulation, the user often needs to specify these parameters or rules, using data on the particular organizational unit being studied. The model is then adapted to fit the past behavior of this unit. Once this adaptation has been done, the model can be used to explore different aspects of the organization being studied and to engage in "what if" analysis. One of the main uses of emulation models is getting humans to think about their unit and to recognize what data they have and have not been collecting. These models are particularly useful for engineering the design of a specific unit. Validated emulation models can also replace humans or groups in some training and planning exercises. An example of this was the use of Soar-IFOR and Systems Analysis International Corporation (SAIC)-IFOR in STOW-97. APPLICATION AREAS FOR ORGANIZATIONAL UNIT-LEVEL MODELS As previously noted, there are numerous possible applications of organizational unit-level models in the military context. Rather than trying to describe all existing models of organizational unit-level performance and their potential ap-

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations plication in the military context, we focus here on three application areas: C3, opposing forces, and organizational unit-level learning and adaptation. For each of these application areas, the current state of modeling within the military is discussed. Where possible, relevant findings, models, and modeling issues that have arisen in other contexts are examined as well. Application Area 1: Command, Control, and Communications As previously noted, C3 issues are rarely addressed in military simulations. However, under the auspices of DMSO, many training and assessment simulations are beginning to be developed to the point where such issues need to be addressed. Some inroads have been made in the representation of communications that are either commands or required reports and in the division of knowledge in knowledge-based systems such as Soar into that which is general and that which is specific to a particular role in the command structure. Further, some of the DMSO-sponsored models have progressed in terms of command generators, parsers, and interpreters. Exploratory models have been used to look at general battlespace planning, though none of the generic planning models have moved beyond the basic research phase. Even within the existing models, a variety of details remain to be worked out, such as (1) development of standard protocols for representing and communicating commands at varying levels of detail, (2) definition of the task-based knowledge associated with different roles in the authority structure, and (3) modeling of the process whereby command relationships are restructured as various organizational units become incapacitated. The above steps are necessary for the development of a fully functioning model of C3, but they are hardly sufficient. The current approach is limited in large part because it focuses on C3 at the individual level (e.g., representation of officers or the task-based knowledge of specific officers). Current models omit and tend not to consider the full ramifications of structural and cultural factors. An example of a structural factor is the distribution of resources across organizational units or the degree of hierarchy in the command structure. An example of a cultural factor is the degree to which subordinates do or do not question the authority of their commanders. Information and processes critical to C3 are ignored in many of the models currently employed by DMSO. Such critical information and processes include allocation and reallocation of resources, task assignment and retasking, coordination of multinational forces, coordination of joint task forces, information about who knows what, the information criticality of particular positions in the C3 structure, and the management of information flow. Current models are often built assuming either that (1) structural and cultural effects will emerge out of the aggregated behavior of individual organizational units, (2) structural and cultural features are relatively simple additions once accurate models of individual soldiers or commanders have been developed,

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations or (3) the effects of structural and cultural factors are second order and so need not be attended to at this point. Much of the research in the area of teams and organizations suggests that these assumptions are false. Structural and cultural factors are often imposed rather than emergent. Because of the complexity and nonlinearity of the behavior of these factors, as well as their interaction, structural factors should not be represented by simply "adding on" unit-level "rules." Moreover, much of the work on organizations suggests that structural and cultural factors with cognitive and task factors are dominant constraint on human behavior. Rather than being emergent, organizational units are often put in place deliberately to achieve certain goals (Etzioni, 1964); for example, joint task forces are assembled, almost on the fly, to meet specific military objectives such as securing a particular port or rescuing a group of civilians. However, the organizational unit must face many uncertainties along the way. For example, the mission plan may be vague, the weather may be changing, various intelligence (such as maps) may be unavailable, personnel may become incapacitated, or communications technology may break down. Organizational units try to maintain control by structuring themselves to reduce uncertainty (Thompson, 1967), constraining individual action (Blau, 1955; Blau and Meyer, 1956), or interacting with the environment in certain preprogrammed ways (Scott, 1981). Much of this structuring is at the C3 level, and some C3 forms are much less prone to the vagaries of uncertainty than are others (March and Weissinger-Baylon, 1986; Lin and Carley, 1997a). Within organizational units, the effects of structure and culture may be so strong that effects due to individual cognition or limited rationality may not be the primary determinants of unit-level behavior. Indeed, in the organizations arena, results from many current models relevant to C3 issues demonstrate that if the command structure, the flow of communication, and task knowledge specific to certain roles or levels in the C3 structure are represented, a great deal of the variation in organizational unit-level behavior can be explained without considering individual differences or affect (Levitt et al., 1994; Carley, 1991b; Lant, 1994; Lant and Mezias, 1990, 1992). In general, there is a strong interaction among C3 structure, task, and agent cognition in affecting organizational unit-level performance. This interaction is so intricate that there may be multiple combinations of C3 structure, task decomposition, and agent cognitive ability that will generate the same level of performance. One reason for this is that the C3 structure can substitute for direct communication (Aldrich, 1979; Cyert and March, 1963), absorb uncertainty (Downs, 1967; March and Simon, 1958), and increase the need for support staff or reports (Fox, 1982). Moreover, the C3 structure is a primary determinant of what individuals can and do learn (March and Simon, 1958). Similarly, the way a task is structured may affect what individuals learn (Tajfel, 1982), the flexibility of the organizational unit (Carley, 1991b), and the need for communication

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations (Thompson, 1967). Finally, the agent's cognitive and physical capabilities affect what actions the agent can and needs to take, whereas the social or organizational context affects which of the possible actions the agent chooses to take (Carley and Newell, 1994). Much of the work in the area of computational organization theory has involved the development of computational models for examining the relative strengths and weaknesses of various C3 architectures, their susceptibility to performance degradation under different conditions (such as information overload, erroneous feedback, missing information, and missing personnel), and the impact of communications technology on information flow through the organizational unit. Those who have designed, built, and evaluated these computational models of organizational units have faced a variety of modeling and measurement issues to which partial answers are now available: (1) how to measure and monitor changes in the C3 architecture, (2) how to compare C3 architectures, and (3) how to determine whether a C3 architecture is effective. The designer of any C3 model in which all personnel, resources, tasks, and so on are modeled will face these issues. Measuring and Monitoring Changes in C3 Architectures As noted by Cushman (1985:18), "The nature of theater forces command and control systems is a web of interconnected subsystems supporting the entire spectrum of functionality of the operation from top to bottom. …" Good computational models of C3 architectures will require an effective method for representing this entire web of subsystems and the degree to which they change in response to various threats or the utilization of new telecommunications technology. For example, suppose one wants to evaluate the impact of electronic conferencing or workdesks on a unit's performance. One approach would be to examine changes in the tasks in which personnel engage. However, when new technologies are put in place, organizational units typically restructure themselves by, for example, reassigning tasks, altering the reporting structure, or changing personnel. Consequently, differences in the workload of individual operators may not capture the full impact of the new technology. What is needed are measures of the C3 architecture as a whole and how it changes. The ability to measure, and thus monitor, changes in the C3 architecture makes it possible to validate and use models to conduct "what if" analyses about the impact of new policies, procedures, and technologies. Three common approaches are used to represent the C3 architecture in models that examine the behavior and decision making of organizational units: the rule-based approach, the network approach, and the petri net approach. Each of these is discussed below. The Rule-Based ApproachOne approach to representing the C3 architecture is to use a rule-based representation for procedures. For example, communication

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations protocols could be represented as a series of rules for when to communicate what to whom and how to structure the communication. This approach is complementary to the network approach described below. Representing C3 procedures as rules facilitates the linking of organizational unit-level models with models of individual humans, since the unit-level rules can be added to the individual-level models as further constraints on behavior. This is the approach taken, for example, in the team (or multiagent) Soar work (Tambe, 1996b; Tambe, 1997; Carley et al., 1992) and the AAIS work (Masuch and LaPotin, 1989). There are several difficulties with the rule-based approach. First, much of the research on the impact of C3 structure may not be directly, or efficiently, interpretable as rules. Second, it is difficult to measure and monitor coordination, communication, and other organizational unit-level activities when they are represented as rules. Third, procedures for segregating agent- and group-level knowledge in a rule-based system need more research attention. The Network Approach The most common approach to representing the C3 architecture is to use one or more networks (or equivalently matrices) representing the organizational unit as a set of interconnected networks (hence a set of matrices). For example, a communication structure might be represented as a matrix such that each cell contains a 1 if the row person can communicate with the column person and a 0 otherwise. This approach could be expanded to the full C3 architecture by specifying the set of matrices needed to represent C3. Other networks needed for specifying the full C3 architecture include the skill matrix (the linkages between people and knowledge), the command network, the communication network, the resource access network, and the requirements matrix (the linkages between knowledge and task requirements). This network approach has many desirable features. First, the representation facilitates the measurement of many factors that influence organizational unit behavior, such as throughput, breaks in communication, structural redundancy, and workload assignment errors (see, for example, Krackhardt, 1994, and Pete et al., 1993, 1994). Second, having commanders or team members specify the structure often helps them identify specific problems in the C3 architecture. Given a set of matrices that represent the C3 architecture mismatches, it becomes possible to calculate information flow mismatches and redundancy. This basic approach of using sets of matrices of relationships to represent the organizational structure is widely used (Cohen et al., 1972; Jin and Levitt, 1993; Carley, 1992). Such a representation is valuable in part because it has face validity. Further, it provides a ready tool for acquiring information from commanders and organizational unit members and for providing the organizational unit with information that will help in altering the C3 architecture to improve performance. Reconfiguration is readily measured under this scheme as the number of changed linkages. Using this approach, the developers of VDT have been able to capture the structure of various organizations engaged in routine

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations Scalability The work on organizational unit-level modeling raises an important substantive question: What changes as the size of the organizational unit increases? It is commonly assumed that the behavior, problems, and solutions involved in unit-level behavior differ only in scale, not in kind, as the size of the organizational unit increases. Thus it is assumed that units of hundreds of individuals will make decisions in the same way and face the same types of problems as units of 3 to 10 individuals. Carried to its extreme, this line of reasoning suggests that organizational units and individuals act in the same way, and the behavior of an organizational unit is just an aggregate of individual-level actions and reactions. Numerous studies indicate, however, that such is not the case. Even in a less extreme form, the scalability assumption may not hold in all cases. For example, research in crisis management suggests that disasters are not scalable (Carley and Harrald, 1997). The lessons learned in responding to a small disaster, such as a minor hurricane that damages a few houses, are not applicable to large disasters, such as a major hurricane that completely destroys all the housing in several communities. The types of response entailed are different—say, ensuring that insurance claims are accurate versus providing shelter, food, and medicine to large numbers of people. Research in many areas other than crisis management also suggests that the scalability assumption is false under various conditions. When scalability can be assumed, the same model of organizational unit behavior can be used regardless of the size of the unit; when scalability cannot be assumed, different models are needed for units of different sizes. To date there has been little research on which types of problems, processes, behaviors, and so forth are scalable and which are not. Determining what factors are scalable is an important step in deciding when new models are or are not needed. Further, regardless of whether problems, processes, and behaviors are scalable, different metrics may be needed for measuring these factors at different levels. For example, consider the measurement of various aspects of the command structure for small six-person teams and for theater-level forces. In both cases, one might be interested in the span of control, the number of levels, and the degree of decision decentralization. There are two different problems involved. First, in both cases these factors can be measured if one knows the complete mapping of who reports to whom. However, obtaining this mapping is more time-consuming in the case of the theater-level force than in the case of the six-person team. A second and more difficult problem is that the range of variation on certain metrics, such as span of control, depends on the size of the organizational unit. In very small organizational units, this range may be so small for some metrics that those metrics have no value in predicting performance outcomes, whereas for large organizational units, these metrics may be the critical ones for predicting performance. Consequently, different measures of C3 structure may be needed at different scales.

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations Organizational Unit-Level Task Analysis and Performance At the organizational unit level, task analysis involves specifying the nature of the task and C3 structure in terms of such factors as assets, resources, knowledge, access, and timing. The basic idea is that the task and C3 structure affect organizational unit-level performance. Task analysis at the organizational unit level does not involve examining the motor actions an individual must perform or the cognitive processing in which an individual must engage. Rather, it involves specifying the set of tasks the organizational unit as a whole must perform to achieve some goal, the order in which those tasks must be accomplished, the resources needed to accomplish them, the individuals or subunits that have the necessary resources, and so on. (For example, see the task analysis done for the VDT by Levitt et al., 1994, and Cohen, 1992, or applications involving petri nets in Remy and Levis, 1988). There has been and continues to be a great deal of research in sociology, organizational theory, and management science on how to do a task analysis at the organizational unit level. For tasks, the focus has been on developing and extending project analysis techniques such as PERT charts (Levitt et al., 1994) and dependency graphs building on the dependency forms identified by Thompson (1967). For the C3 structure, early work focused on general features such as centralization, hierarchy, and span of control (see, for example, Blau, 1960). Recently, however, network techniques have been used to measure and distinguish the formal reporting structure from the communication structure (see, e.g., Krackhardt, 1994). These various approaches have led to a series of survey instruments and analysis tools. This work involves a variety of unresolved issues, including how to measure differences in the structures and how to represent change. Additionally, a great deal of research has been done on how the task and the C3 structure influence performance (see Carley, 1995b, for a review of modeling work in this area). In particular, research in the past three decades has repeatedly demonstrated that there is no one right C3 structure or design for an organizational unit (see, e.g., Lawrence and Lorsch, 1967; Pennings, 1975). Rather, the way the organizational unit should be organized depends on the specific tasks to be performed, the volatility of the environment; the extent to which personnel move into and out of various jobs in the organizational unit; the amount, quality, and accuracy of information available for making decisions; and the level and type of training or experience acquired by the participants (Malone, 1987; Roberts, 1990; Carley and Lin, forthcoming). These findings follow from empirical work in both the field and the laboratory. The theory has found expression in the form of verbal descriptions of organizing and decision making processes, as well as computational and mathematical theories. At the organizational unit level, task analysis is an operationalization of the

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations information processing approach to the study of organizational units. The following factors are often addressed: The way the structure of the organizational unit and the nature of the task limit what information is available when to which personnel in the organizational unit, and the way such limits influence organizational unit performance The way the interaction between these structures and the cognitive limits of human personnel influences organizational unit performance The way these structures interact with the complexity, quality, quantity, and accuracy of information to determine performance The set of processes the organizational unit carries out and the order in which they can occur Increasingly during the past decade, organizational unit-level theories and models with an information processing focus have found expression as computer programs (see Carley, 1995b, for a review). By calibrating these programs to actual organizational units, predictions about organizational unit behavior can be generated. The precision of these predictions varies with the level of detail used in representing the task and the C3 structure (Carley and Prietula, 1994). In fact, simply involving a team member or CEO in specifying the structural information needed for these models helps him/her see barriers to performance in the unit. Similar models are being developed by researchers in the distributed artificial intelligence community (see, for example, the work of Durfee, 1988; Tambe, 1996b, 1996c; Gasser et al., 1993; and Gasser and Majchrak, 1992). Currently, even the most detailed of these models (e.g., the VDT model) are sufficient only for small classes of very routine design tasks. However, these models are potentially useful for evaluating alternative task assignments, organizing schemes, or new technologies because they allow the researcher or CEO to engage in "what if" analyses. Ease of Modeling Organizational Units One of the main problems in rapidly building organizational unit-level models is that the available languages do not have a set of primitives geared to the organizational unit level. Thus, each researcher implementing an organizational unit-level model creates his/her own tools for modeling a variety of organizational unit-level behaviors, such as moving personnel between divisions, promoting personnel, measuring performance, combining decisions, and communicating commands. Moreover, for organizational unit-level models in which the unit comprises a set of agents (multiagent models, discussed earlier) the researcher must model both the agent (or types of agents) and the organizational unit. Consequently, organizational unit-level models can be much more time-intensive to develop than individual agent models. Another difficulty with organizational unit-level models is

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations that they are often quite computer intensive, particularly when each agent is modeled as a separate entity. Rediscovery One of the major difficulties in this rapidly developing, multidisciplinary area is that researchers are continually rediscovering well-known results from other areas. For example, researchers in distributed artificial intelligence rediscovered the well-known result from organizational theory that there is no one right organizational design for all tasks. Likewise, social scientists rediscovered the well-known result from computational analysis that performance improvements in neural net-like models require a huge number of training trials. Thus it is imperative in this area that artificial intelligence modelers and computer science system builders collaborate with organizational or social scientists. ORGANIZATIONAL UNIT-LEVEL MODELING LANGUAGES AND FRAMEWORKS Researchers have begun to develop organizational unit-level modeling languages or frameworks. None of these tools dominates, and each has some failings as a comprehensive approach for implementing organizational unit-level models. Any organizational unit-level meta-language must enable rapid modeling of agents, task, and C3 structure. The existing systems vary in their adequacy on these dimensions, the severity of the constraints they place on the types of objects that can be modeled, and their flexibility for modeling objects. For the most part, these systems are documented only minimally, and many are still under development. MACE MACE (Gasser et al., 1987a, 1987b) was one of the first general (domain-independent) testbeds for modeling multiagent systems. It introduced the idea of using agents for every aspect of model construction and development (including user interaction and experiment management). MACE is one of the few distributed object systems that is truly concurrent. It includes explicit concepts drawn from social theory, such as recursive agent composition, or the idea that a group of agents (an organizational unit) can itself be treated as an agent with distributed internal structure. Further, the notion of social worlds developed by symbolic interactionists is operationalized in MACE as knowledge-based agent boundaries. Each agent has a set of potential interaction partners, and that agent's knowledge about others, rather than explicit rules, constraints, or programming structures, defines the boundaries of communication and interaction (and so the C3 communication structure). MACE includes specific facilities for modeling a

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations number of features of other agents, such as goals, roles, and skills. It is this type of knowledge of others that is used to define the pattern of interaction over time. Today, the idea of modeling the agent's knowledge of others is commonplace within computer science and artificial intelligence. Constructuralism The constructural model (Carley, 1990, 1991a; Kaufer and Carley, 1993) focuses on agent interaction. Agents are modeled as collections of knowledge and propensities for interaction. Propensities for interaction change dynamically over time as the agents interact and exchange information. Such changes can be constrained, however, by the imposition of a formal C3 structure that imposes demands for certain interactions. Within the constructural model, a communications technology is available to agents that affects the number of agents with which they can communicate simultaneously, whether a communicated message must be the same to all receivers, how much information the agent can retain, and so on. The communications technology itself may act as an agent with particular information processing and communicative capabilities. Virtual Design Team (VDT) In the VDT model (Levitt et al., 1994; Jin and Levitt, 1996), the organizational unit is modeled as a set of agents linked by task and C3 structure. The agent is modeled as an in-box, an out-box, a set of preferences for information handling, a set of skills, and so on. The VDT uses very detailed modeling of specific organizational tasks, focusing on the dependencies among subtasks (much as one would in PERT or GANT charts), but leaves the content of what is done in each subtask unspecified. Agents have a suite of communications technologies available to them, such as telephone, face-to-face meetings, and e-mail. However, the use of technology is affected by predefined preferences for the use of particular tools for certain types of messages. Performance measures focus on task completion and rework. Strictly Declarative Modeling Language (SDML) SDML (Moss and Edmonds, 1997; Edmonds et al., 1996; Moss and Kuznetsova, 1996) is a multiagent object-oriented language for modeling organizational units. It has been used to model both team (flat) and hierarchical C3 structures. SDML does not contain a built-in model of the cognitive agent. Rather, it is sufficiently flexible to represent both simple agents (such as those in the constructural model) and more sophisticated agents (such as Soar agents). One feature of SDML that facilitates studying the impact of the agent is that it includes libraries for alternative architectures, such as genetic programming and

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations Soar. These libraries facilitate exploring the interaction between agent cognition and organizational design. The pattern of ties among agents can also be represented in SDML. The ease of representing both agents and linkages means that the C3 structure can readily be modeled as a set of predefined linkages, roles, and knowledge bases. Within SDML, the structure of the multiagent system is represented as a container hierarchy, such that agents may be contained within small organizational units that are contained within larger organizational units. Containers and their associated agents are linked by an inheritance hierarchy. Multiagent Soar There are a number of multiagent or team Soar models. In these models, the individual agents are built into the Soar architecture (Laird et al., 1987). Thus the agents are goal directed, although the goals need not be articulable. Knowledge is organized in problem spaces, and actions are directed by operators and preferences. Preferences can be used to represent shared norms or cultural choices. However, it is difficult to change preferences dynamically. In multiagent Soar, each agent has a model of other agents, and chooses interactions and dynamically forms the C3 structure by reasoning about those others (Tambe, 1996a, 1996b; Carley et al., 1992). Such knowledge may include expectations about the other agents' goals, preferences, and actions. Agents also have mental models about the social world, which can include information about the nondynamic aspects of the C3 structure. Communication involves the passing of messages among agents. Agents may have various communication-related problem spaces, such as for determining when to communicate what to whom or how to compose and parse messages. SWARM SWARM is a multiagent simulation language for modeling collections of concurrently interacting agents in a dynamic environment (Stites, 1994; Minar et al., 1996; Axelrod, 1997). It was designed for artificial-life simulations and thus is best used to explore complex systems comprising large numbers of relatively simple agents. Within SWARM, agents can dynamically restructure themselves to accommodate changes in incoming data or the objective function. Over time, systems of SWARM agents come to exhibit collective intelligence beyond the simple aggregation of agent knowledge. Task Analysis, Environment Modeling, and Simulation (TAEMS) TAEMS is a framework for modeling a computationally intensive task environment (Decker and Lesser, 1993; Decker, 1995, 1996). For the most part, it is

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations simply a way of representing tasks. The TAEMS framework is compatible with agent-centered approaches to team modeling. Within TAEMS, however, an agent is simply a locus of belief and action. Agents can thus, at least to a limited extent, communicate, gather information, and execute actions. Tasks can be described at three levels of abstraction: objective, subjective, and generative. At the objective level, tasks are described in terms of subtasks and their interrelations, such as enables, precedes, facilitates, and hinders. At the subjective level, the view of each agent is characterized. And at the generative level, the range of alternatives, the distributions used, and the generative processes needed to specify differences in specific instantiations of tasks are specified. ORGAHEAD ORGAHEAD is a framework for examining the impact of the C3 authority structure on performance for distributed choice and classification tasks (Carley and Svoboda, 1996; Carley, forthcoming). Within ORGAHEAD, organizational units have the capability of adapting their C3 structure dynamically over time in response to environmental changes. Organizational unit-level action results from actions of multiple agents, the C3 structure connecting them, and the distribution of knowledge or resources across agents. Individually, agents either learn through experience or follow standard operating procedures. The organizational unit also learns structurally by altering procedures and linkages among the agents, such as who reports to whom and who does what. This latter types of learning is implemented as a simulated annealing algorithm. Within ORGAHEAD, the user can vary the authority structure, the degree of training received by the agents, the amount of information the agents recall, the rate of structural change, and the ways in which the C3 structure can change. CONCLUSIONS AND GOALS Computational models of organizational unit-level behavior are becoming increasingly sophisticated. Work in this area holds great promise for the military context. At this point, however, none of the existing models can simply be plugged directly into a current DMSO platform as the model of an organizational unit. In all cases, important extensions or modifications are needed. In part, this is because existing organizational unit-level models have either too limited a repertoire of C3 structures or too limited a representation of tasks. Likewise, existing DMSO models are not set up to capture organizational unit-level performance measures. However, several of the models could be adapted or used in concert with other models to examine aspects of organizational unit-level behavior. Further, the network-based representation of C3 structures and the various network measures of structure could be incorporated into some current DMSO programs.

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations The current set of organizational unit-level models demonstrates the viability and necessity of using simulation to examine organizational unit-level behavior. To enhance the applicability of organizational unit-level computational models to the military context, several steps need to be taken, including data collection, development of measures, improvement and extension of existing models, development of visualization techniques, and basic research on models of unit-level adaptation. Short-Term Goals Extend existing models so they are more directly applicable to military settings. Three types of activities should be considered: (1) extending current models to new tasks; (2) creating teams of military and civilian research personnel to extend existing models to a C3 model that could be used in some specific military setting, such as synthetic theater of war-Europe (STOW-E); and (3) developing a list of contexts in which C3 models are most needed. Develop databases against which to test and/or validate existing models. To this end, network data for key C3 features for particular engagements, training exercises, or war games should be collected. Such data include, for example, the command structure for specific units, the associated communication structure, what kind of information is communicated by what technology, information on resource and platform assignment, and performance measures. Such data are often collected, particularly in war games, but are rarely made accessible in network form (who is talking to whom about what). Analysis of such network data would provide information on the range of existing structures, the strengths and weaknesses of those structures, and the types of factors that need to be included in the C3 model. Data should also be collected to examine the relationship between C3 structure and performance for a series of specific tasks. Locate and develop measures of C3 architectures that are meaningful in the military context. These measures should then be integrated into new and existing models. Data on these measures could be gathered for analyzing and validating the resulting models. Examine and address the general principles of command—such as unity of command. Is there any empirical evidence for when these principles do and do not work? These principles, along with associated empirical constraints, should be made available as a unit-level analog of conceptual model of the mission space (CMMS). These principles could also be used to inform and validate models of C3 architectures, thus increasing the realism of these models in the military context. Develop a C3 analysis tool kit. This tool kit should include procedures for determining when C3 architectures are meaningfully different; procedures for segmenting networks based on various performance criteria; and techniques for visualizing networks, particularly C3 architectures.

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations Begin research on constrained-adaptation and multiagent learning models. Current multiagent learning algorithms need to be extended to be more robust, learn more quickly, and find better solutions in complex and rapidly changing environments. There are many near-term needs in this area. First, finding algorithms that will generate better solutions in dynamic environments will involve extending and examining new optimization techniques (possibly hybrid techniques), as well as developing new techniques that will enable rapid and accurate learning under various constraints, such as little feedback or changes in available options. Second, the various organizational unit-level learning algorithms need to be compared. Third, organizational unit-level researchers, such as organizational theorists, should be brought together with computational modelers to develop valid organizational unit-level models. Validated unit-level models would be valuable decision aids for commanders trying to reason about the impact of novel strategies and new technologies on the forces under their command. Finally, work is needed on expanding existing models and validating them against data on actual organizational units. To represent organizational unit behavior accurately, models of unit-level learning should not only learn to make the correct decision, but also mislearn under certain circumstances to better represent real human behavior. Intermediate-Term Goals Continue research on constraint-based adaptation, and try to incorporate the models of adaptation developed in the short term into models of C3 structure. This work would be particularly useful in making forces behave more realistically when facing changes in orders or casualties. Develop a multiagent system that combines both a network interface for examining the impact of changes in the C3 architecture and a set of highly intelligent agents engaged in various tasks, possibly a force-level task. The use of hybrid models that combine cognitive agent models (see Chapter 3) with social network models is a critical step in being able to address C3 issues. Develop a better understanding of organizational unit-level adaptation. This can be accomplished by working in at least five areas: Develop measures of adaptation. Can adaptive behavior be recognized? At the organizational unit level, basic research is needed on what aspects of the C3 architecture facilitate adaptation, given the constraints of the military situation (e.g., command hierarchy, rules for transferring personnel among organizational units, existing resources and skills). Determine the scalability of these measures. Are the factors that inhibit or promote adaptability at the small group level the same as those that inhibit or promote adaptability at the organizational unit level? Which metrics are valuable for which sizes of forces?

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations Investigate what types of changes in C3 structure are most likely to be adaptive, and address the way C3 architectures can be set up so that they can adapt to changing task environments. This may involve developing optimization procedures for environments in which the performance surface changes over time. Begin to gather field data for evaluating dynamic C3 systems. Models for C3 adaptation at the organizational unit level often must take into account institutional, technological, and political factors that cannot be covered adequately by laboratory experiments in a limited time frame. A better understanding is needed of what kinds of data are required and how those data can be gathered. Develop organizational unit-level models with C3 architectures that adapt dynamically in response to triggering events, such as depletion of resources or alterations in the rules of engagement. Develop a framework or meta-language for describing and implementing organizational unit-level models. Progress on individual-level models was facilitated by the development of platforms, such as Soar, that integrate various aspects of human cognition and/or physiology. At the unit level, one of the most pressing issues is that development of unit-level models is extremely time-consuming, and each modeler spends part of his or her time reinventing basic procedures, such as communication protocols and algorithms for traversing the command structure. Research in this area indicates the need for a meta-language that will facilitate rapid linking of different types of agent models with models of both task and organizational structure. Such a language should have built-in default procedures for measuring performance and aspects of the task and C3 structures. No current language is sufficient for this purpose. Long-Term Goals Gather information on the conditions under which organizational unit-level behavior is or is not the simple aggregate of individual-level behaviors. Develop unit-level models in which the organizational unit is a combined force. In combined and coalition forces, additional issues such as cultural clashes, language barriers, and technological differences combine to complicate the C3 process (Maurer, 1996). Thus the development of coalition organizational unit-level models requires attention to details not relevant for forces from a single country. Examine interactions among different types of learning and the implications of such interactions for unit-level performance. Organizational unit-level learning is not simply the aggregation of individual learning. That is, it is possible for the organizational unit to learn and adapt even when all the individual agents are acting according to standard operating procedures. C3 architectures may learn or adapt by reassigning resources or tasks or by developing new procedures or information systems. The same basic standard operating procedures may apply for individual agents, but the number of agents or the time to

OCR for page 269
Modeling Human and Organizational Behavior: Application to Military Simulations complete the task may differ. At the organizational unit level, adaptation may take the form of emergent structures rather than individual learning. However, learning at the unit level may interfere with or be aided by learning at the individual level. For example, unit-level learning in corporations is frequently embedded in the connections among personnel and the roles the personnel play, but the value of such learning is often negated when personnel are replaced or the organization downsizes. There is a need to assess whether this is the case in military settings and what impact such interference would have on unit-level performance. In the course of developing models of planning, take unit-level issues into account. Explore how the output of unit-level models is turned into plans such as those that might be generated by a commander or by staff personnel for a commander. Currently, the messages passed within many of the unit-level models have minimal content (e.g., they may contain implicit decisions and belief or trust in those decisions). Research is needed to link this output to messages whose content reflects C3 issues.