5

World Café

With the wealth of available metrics sets relevant for population health, the planning committee decided that an important activity for this workshop would be to actively discuss options for using metrics to catalyze and assess efforts to improve population health. As a means of increasing the amount of meaningful dialog among all of the workshop participants, the planning committee chose to use the World Café format (Brown et al., 2005). In this format, small groups gather around a table and tackle a specific question posed to all the participants. After 20 minutes, the participants change tables and discuss the same question again. For this workshop, the planning committee chose the following two questions for the participants to consider, and this process was repeated twice, once for each question.

1. What kinds of measures are helpful to communities working to improve health?
2. What are the barriers in your community to using measures to inform action? (see Box 5-1 provides some highlights from the wide range of comments made in the World Café breakout discussions).

A host assigned to each table took notes during the discussion and reported back to the workshop after all four rounds were completed. These reports summarized a few key points from the discussions and were not intended to be all-encompassing or to suggest that there was any consensus among the discussants in these small groups. The hosts

included Alina Baciu, senior program officer at the Institute of Medicine (IOM); Amy Geller, senior program officer at the IOM; Mary Lou Goeke, executive director of the United Way of Santa Cruz County; Marthe Gold, visiting scholar at the New York Academy of Medicine; Lyla Hernandez, senior program officer at the IOM; Katherine Papa, director of Public Health Initiatives at AcademyHealth; Steven M. Smith, clinical assistant professor of pharmacotherapy and translational research at the University of Florida and the IOM Anniversary Fellow in Pharmacy; Brenda Sulick, policy outreach director at AARP Public Policy Institute; Darla Thompson, associate program officer at the IOM; Matthew Trowbridge, associate professor at the University of Virginia School of Medicine; Julie Willems Van Dijk, co-director of the County Health Rankings and Roadmaps Program, University of Wisconsin Population Health Institute; and Kelly Warden, project manager at the U.S. Green Building Council. An open discussion among the reassembled workshop participants, moderated by Steven Teutsch, followed the table reports.

HIGHLIGHTS FROM WORLD CAFÉ DISCUSSIONS: THOUGHTS ABOUT HELPFUL MEASURES¹

Alina Baciu started the reports from the four rounds of discussions at her table. The first point she shared was that some participants suggested new or novel methods to collect data to complement non-real-time data from various surveys. One idea along those lines was to involve neighborhood residents and perhaps middle and high school students in gathering information, such as on the food environment in schools. Another point raised by a few participants in the discussions was that language and communication are important factors to mind when collecting data and relaying the results to the community given that data collection tools can fail because they ask the wrong questions for a particular ethnic

___________________

¹ This section is based on the reports by Alina Baciu, Senior Program Officer at the IOM; Amy Geller, Senior Program Officer at the IOM; Mary Lou Goeke, Executive Director of the United Way of Santa Cruz County; Marthe Gold, Visiting Scholar at the New York Academy of Medicine; Lyla Hernandez, Senior Program Officer at the IOM; Katherine Papa, Director of Public Health Initiatives at AcademyHealth; Steven M. Smith, Clinical Assistant Professor of Pharmacotherapy and Translational Research at the University of Florida; Brenda Sulick, Policy Outreach Director at AARP Public Policy Institute; Darla Thompson, Associate Program Officer at the IOM; Matthew Trowbridge, Associate Professor at the University of Virginia School of Medicine; Julie Willems Van Dijk, Co-Director of the County Health Rankings and Roadmaps Program, University of Wisconsin Population Health Institute; and Kelly Warden, Project Manager at the U.S. Green Building Council. These reports were not meant to infer a consensus from the discussions, and the statements have not been endorsed or verified by the National Academies of Sciences, Engineering, and Medicine.

or racial group or they are ask them in a language that people do not understand, either literally or figuratively.

One idea that Amy Geller recounted from the discussions at her table was how useful it would be to have more granular data. Often, however, such data are proprietary and must be purchased or they must be collected through oversampling. The exchange among participants about this point included the importance of first talking to the community to select a problem for study before deciding on whether and how much granular data are needed, but even taking that step may require funding. A second set of points raised at this table was the importance of sustaining efforts and the challenges of securing stable funding to collect data over the long term and provide ongoing feedback to the community and to provide tools with which the community can take action based on that data-driven feedback. One participant suggested that hospital community benefit funds could be a source of sustainable funding if hospitals and health systems were shown meaningful measures that would enable them to take actions relevant to their goals and mission regarding public health. A participant suggested that it might be useful to have data on investment metrics that communities could use to benchmark against one another. An example given was a metric based on data on the costs of scaling up and leveraging funds.

Mary Lou Goeke reported that some participants at her table noted that the measures that have proven most useful were those that had purpose and relevance to the daily lives of the people involved, at least in part because the community had been engaged in selecting the measures and believed they could use them to make a change in areas important to them. A few participants also raised the point that measures of isolation, social support, and connectedness could be important but may be difficult to find.

Marthe Gold listed the many characteristics of helpful measures proposed by various participants at her table. For example, participants suggested one or more of the following: that helpful measures would be resonant, fit the context and pressing needs of a community, and be actionable, easy to measure, and inexpensive to measure. Helpful measures would also be understandable by members of the community so that they can be motivational, aspirational, and empowering. They would show economic potential, because health alone is not always a motivating factor for action, and they would be able to serve different kinds of narratives and be useful in different contexts to tell stories to support those narratives.

Lyla Hernandez reported that various participants at her table described the type of measures that communities would find useful: those that provide social, environmental, and demographic information at the local or neighborhood level; generate data on social capital, connec-

tivity, and engagement; and show the types of resources available to the community. Other characteristics of useful measures enumerated by participants included easy to understand, important to the community, and generating data that can be linked to education, health, criminal justice, and transportation systems in the community. Various participants noted that workforce data for health, social work, and other kinds of professions would be important for communities to have. For example, a useful measure might show how well the “helping” workforce in a community reflects the population it is serving. Also mentioned during the discussion was the desire for qualitative data to help inform quantitative data.

Reporting on what he called the robust discussions at his table, Steven Smith listed several ideas that the various discussants raised with regard to helpful measures. One idea was that measures of community capacity would be useful, including measures of political will, the extent of cross-sector interaction already existing in a community, the availability of leadership and on-the-ground providers, and the willingness of a community to engage. Other potentially useful measures included those that might break through stereotypes or assumptions made by people in a community; those for which data can be gathered quickly and used to take action quickly, as opposed to those for which data are gathered and published on an academic time-frame; and those with cross-cutting measures that link different sectors of a community. Smith also listed two measures that he characterized as out-of-the-box measures: a measure of the extent of community engagement in developing metrics and another one measuring the extent to which a community understands how its data are being used to inform change (e.g., to inform health improvement efforts).

Brenda Sulick, first noting that many of the points raised at her table had already been mentioned, reported that the discussion at her table raised the importance of putting purpose before measurement and of thinking of how measures will be meaningful, accessible, and tangible to the community and representative of the community. “Sometimes, we start with the research and do not think about the community until later,” said Sulick. Various participants in this group also pointed out the importance of producing narrative stories to which community members can relate and of linking data to the level of the family as a unit of health so that users might believe they are doing something meaningful for their children, not just the community.

Katherine Papa reported that framing was a topic of discussion at her table and that useful measures are ones that are motivational, asset driven, and able to highlight the positive aspects of a community. The discussants also noted the importance of using communication to drive public support and of community will and a common agenda that together can serve as a rallying point around the data and the actions suggested by the data.

Darla Thompson reported that some of the discussion at her table centered on how to create measures that capture the complexity of people’s lived experiences. Another topic discussed was how to measure cultural sensitivities. Along the same lines, this group discussed the importance of using language that reflects the cultural sensitivities of the community in the design and execution of a measure.

Matthew Trowbridge said one thing he concluded from listening to the discussions at his table was that the public health community would benefit from recognizing that it is at a fundamental moment of transition from simply measuring health outcomes to measuring and understanding the social and environmental determinants, and using them as project outcome measures. Some discussion at his table focused on the idea that current measures are geared toward the average (i.e., at the national or local level) and are not illustrating any particular point of view. In that case, measures that identify outliers might be useful. As an example, it was suggested that useful measures could assess attributes of a community that would “work” for both an 8-year-old and an 80-year-old. With regard to health care, one idea raised was that if the health care system was designed to serve the 5 percent of the population that uses the biggest share of health care resources, perhaps a delivery system designed to serve those 5 percent optimally would be a better system for everyone.

Julie Willems Van Dijk said that she was struck by an idea voiced at her table about the centrality of the community voice in thinking about measures and how different that is from what the conversation would have been about even 5 years ago. In terms of what to measure, ideas around her table included the importance of ensuring that measures have community relevance and both language and cultural sensitivity, and the question of how to present measures in a way that reflects the motivation and inspiration of the community and its individual members. The group discussed whether to start with measures or priorities and how to merge these two approaches. She said there was a rich conversation about whether one should look at measures first and decide what is important or vice versa, to better understand the issue.

One of the themes that Kelly Worden noted at her table was that measures should be human-centric and patient oriented. One comment that struck her addressed the difference between functional and clinical measures when interacting with patients. There was also discussion at her table about presenting data in an actionable manner and in ways that enable conversations with both scientists and community members.

HIGHLIGHTS FROM WORLD CAFÉ DISCUSSIONS ABOUT BARRIERS²

A barrier that was mentioned by several participants at Baciu’s table was the difficulty of aligning data with action when the evidence is thin. In that regard, various participants noted that more research is needed in areas such as inter-sectoral social determinants of health, though there are not enough funds available to support research that spans sectors.

Geller reported that participants at her table made the point that because funding is not always available, it may not be possible to always have the perfect measure that everyone desires, but that should not stop researchers from collecting data or communities from taking action. It was also suggested that the field develop innovative methods for collecting and using data from new sources such as social media. One comment made during the discussion was that politics can get in the way of funding streams and compelling data do not always promote change, suggesting there may be a need for alternative approaches to framing data to make a more compelling case for change. Along the same lines, a participant noted the importance of documenting and sharing examples of how measures have been used successfully to help address the sense that change is always difficult and that population health outcomes take a long time to improve.

One of the barriers discussed at Goeke’s table was the difficulty of turning metrics into convincing stories that people could use for change. Another barrier mentioned was the lack of trust that a community might have regarding the accuracy of the data and the motivation or ideology of the people presenting data for action.

Gold reported that one of the barriers cited at her table included institutional resistance to measurement that shows itself as defensiveness or the attitude that an institution wants to do what it has always done. Another barrier discussed at this table was that big measures—the

___________________

² This section is based on the reports by Alina Baciu, Senior Program Officer at the IOM; Amy Geller, Senior Program Officer at the IOM; Mary Lou Goeke, Executive Director of the United Way of Santa Cruz County; Marthe Gold, Visiting Scholar at the New York Academy of Medicine; Lyla Hernandez, Senior Program Officer at the IOM; Katherine Papa, Director of Public Health Initiatives at AcademyHealth; Steven M. Smith, Clinical Assistant Professor of Pharmacotherapy and Translational Research at the University of Florida; Brenda Sulick, Policy Outreach Director at AARP Public Policy Institute; Darla Thompson, Associate Program Officer at the IOM; Matthew Trowbridge, Associate Professor at the University of Virginia School of Medicine; Julie Willems Van Dijk, Co-Director of the County Health Rankings and Roadmaps Program, University of Wisconsin Population Health Institute; and Kelly Warden, Project Manager at the U.S. Green Building Council. These reports were not meant to infer a consensus from the discussions, and the statements have not been endorsed or verified by the National Academies of Sciences, Engineering, and Medicine.

validated metric sets that are being promulgated in the field—may not be sufficiently relevant or flexible with regard to what communities want. As a result, there may be a lack of buy-in from the community with regard to such metric sets, communities may voice concerns about the sensitivity of the questions being raised, and they may experience burnout from being asked the same questions repeatedly with little accountability or feedback. The discussion at this table also raised the issue of a lack of necessary resources.

Among the barriers enumerated at her table, Hernandez reported that the participants discussed the challenges of defining measures, having the resources to collect and analyze data, and then taking action. It was pointed out during the discussions that data can be rejected as being an accurate picture of the community when it does not fit with the ideology of the group to whom the data are being presented. Some participants voiced the concern that members of a community can have the attitude of blaming the victim and that health outcomes are inevitable, making them resistant to data that could enable change.

Smith said there was some discussion regarding barriers around the idea that excess measurement coupled with a lack of action or a lack of feedback to the community can erode trust within the community. Inadequate marketing of the importance of metrics was also noted as being a barrier, as was the challenge of getting communities to internalize data and buy into data-driven ideas for change.

Barriers listed by the discussants at Sulick’s table included the challenge of making measures meaningful for different audiences and understandable by the community; the difficulty of linking datasets; and the struggle to help stakeholders see the value of metrics and be in a position to make decisions based on the data the metrics produce. An example that was discussed was how the real estate website Zillow cuts its data by neighborhood, walkability, and schools to make its data more meaningful, appealing, and personalized for users. One point raised during the discussion of barriers was that it might be useful to create a clearinghouse of datasets and metrics to avoid duplicating what others have already developed.

Papa reported that the discussions at her table produced a list of three barriers: politics, lack of capacity, and poor data quality and data hoarding. Politics can be a barrier to action, she reported. With regard to capacity, some participants noted that there are not enough epidemiologists involved who would know what to do with the data these measures generate. Other participants suggested that the metrics community does not have a good enough understanding of what policy and systems changes these data can be used to drive.

At Thompson’s table, the challenge of getting data that are granular

enough so that people think the data apply to their community was listed as a barrier. Competing interests in a multisector environment was also noted as a barrier, as was the challenge of identifying who would take action when the time comes to translate data into purpose. The issue of trust in the data when there is no sense of collective ownership in a community was also noted as a barrier, as was the lack of good measures for social impact and the difficulty of integrating data from various measures that may be in different formats. Other barriers enumerated during the discussions included the time and resources needed to get data to the right people, connect data to stories with which community members can identify, and package data with stories that are compelling to different audiences.

One barrier that Trowbridge noted from the discussions at his table was the tension between the goal of fundamental change and the intransigence of the existing infrastructure. Another barrier noted at his table is the relatively short duration of the grant funding cycle, with even the 20-year commitment on the part of Robert Wood Johnson Foundation (RWJF) being relatively short given the types of change that are the goal of these efforts.

At Willems Van Dijk’s table, some discussions on barriers turned into thinking about action items. She reported that there were good conversations about building trust with other sectors to enable multisector collaborations, building ownership, and building common stewardship. The realities of limited resources were noted as a barrier, and the discussions at her table listed the challenges of dealing with the cost of data and the dependence on grants and resources that are not permanent. One opportunity the discussion mentioned was for the research community to do better cost/benefit analyses that can inform its work and to conduct research to identify effective strategies for change.

One barrier that Kelly Worden noted from the discussions at her table was the difficulty in collecting data across sectors and then having to involve those different sectors when acting on the data given that there is often a lack of communication among different sectors. One way in which this manifests itself is that data collected by the public health sector may not be actionable by another sector.

DISCUSSION

Steven Woolf observed that although he expected the workshop to be heavily weighted toward technical and methodological issues, datasets, and statistics, the conversation has instead been dominated by talk about the importance of community and stakeholder engagement. He noted that the same thing happened at a workshop earlier in the year

on modeling and its role in population health. Steven Teutsch agreed, adding that the common message he heard throughout the day was the importance of putting a human face on the data to make data meaningful and impactful.

George Flores from The California Endowment pointed out that little was said about using metrics to make a business case for population health or about the kind of cost data that would satisfy not only cost/benefit analysis, but industry profiting generation and economic sustainability. “The community may not care as much about those things, but economic viability is what drives a great deal of decision making and policy,” said Flores, who wondered if the roundtable should to be doing more exploration of the factors that drive economic viability. Teutsch noted the difficulty in capturing the social benefits that matter to most people in financial calculations.

Israel Nieves-Rivera said that from his perspective from the San Francisco Department of Health, which is a health delivery system, a nexus of population health initiatives, and a research organization, the problem is not that there is a lack of measures, but rather deciding what measures to use to answer a given question and how to decide what data to share with the community so that he and his colleagues can bring the right partners to the table. If a question is germane to the health care delivery system, measures on meaningful use and how the population of a specific clinic is doing are appropriate, while if the goal set is to bring partners to the table, a different set of metrics would be germane. Regardless of the question and specific measures, Nieves-Rivera believes health systems need to move away from data ownership and toward using data in the best way possible to address specific community goals. In his opinion, what is important is for all of the partners to agree on goals and vision because enough metrics are available to serve whatever purposes the community decides are important. He acknowledged that this might not be true in every jurisdiction.

A participant commented that much of what is being discussed involves looking for new ways for people with different perspectives to work together on a common goal. She noted that while challenging, this can happen if those involved are all focused on making change happen in a community. Often, this participant said, those involved in multidisciplinary efforts need to be taught new leadership skills to merge these different perspectives, pose questions differently, and look for new ways to integrate different datasets in a way that enables cross-sector approaches to analysis. She noted, too, that this type of cross-sector collaboration is not how most people working in public health or medicine, including herself, were trained to work or think.

David Kindig wondered about the tension between local purpose and

the responsiveness to local need and energies on the one hand, and some standardization and synergy on the other hand. “You lose something when you go towards more standardization, but I am not sure that it is the most efficient approach for each community to create its own wheel,” said Kindig. “I think there may be some opportunities for thinking not necessarily about a single approach but of a set of approaches that communities can learn from without having to reinvent the wheel.”

Rajiv Bhatia noted the tension that exists between top-down and ground-up approaches, and said these two approaches can exist with a healthy tension and inform each other. Centralized measures, he added, can be used in combination with localized measures that inform the central core set. He then wondered how it was going to be possible to connect the rich set of data generated within the health care world and community-level data produced outside of what he called the “HIPAA firewalls,” referring to data collected under the regulations of the Health Insurance Portability and Accountability Act (HIPAA). One possibility he suggested was to start asking questions on the HIPAA side about social determinants of health and collect data on which those outside of health care would act. In this scenario, health care would merely be the data producer. “I think we are not leveraging the power of the health care system and all of the health outcomes data in that system,” said Bhatia.

Teutsch reminded the workshop attendees about a recent IOM report on incorporating specific social metrics into the electronic health record (IOM, 2014). Bhatia replied that this was a milestone report that was, in part, about standardizing the doctor’s social history, but he did not think that the set of behavioral and social measures proposed in that report reached the scope of social determinants of health, nor that the electronic health record was the place to collect those data. In Bhatia’s opinion, social determinant–related questions, such as on food security, should be asked of every member of a health plan at enrollment. “If every member in a plan was asked about their level of food security, you would then be able to easily look prospectively at differential health outcomes and health care costs related to different levels of food security,” said Bhatia. “Then you have an economic argument for the public sector for making investments in food systems.” He suggested that the same could be done for social isolation, housing instability, difficulty paying for daily living expenses, and similar questions. In this way, he added, the attributable burden of disease to unmet social needs could be collected in the health care system and translated to those who are trying to control health care costs in the long run by making investments in other systems.

Daniel Gallagher from the San Diego Association of Governments voiced his opinion that it is important to form partnerships in the emerging areas of public health and the built environment. His organization,

for example, partners closely with the County of San Diego Health and Human Services Agency, with his group providing data on mobility and the built environment and the Health and Human Services Agency providing health data. He also noted the importance of working with partners that complement one another, and he gave an example of how public health, community design, and economic development groups worked together to implement traffic calming measures, including roundabouts, in the Bird Rock area of La Jolla, California. A study conducted after the roundabouts were installed showed that these traffic calming measures helped stitch the community together so that more people were walking and biking and were frequenting local businesses more often.

Stiefel offered the final comment that he said could be construed to be more about consternation than insight, and it had to do with perspective and bias. This workshop, sponsored by the IOM, is focused on the social determinants of health, but he imagined that there are people in other meetings talking about education who think of health as a determinant of educational outcomes or in economic development meetings who think of health as a determinant of economic vitality. “We are in this web of means and ends, and we have selected this one end that we think trumps the others,” said Stiefel. “It is just a bias we come with, and I think there is some benefit about reframing to think of health as one of the components of this complex system that produces some higher level end, whether that is individual, societal, or community well-being.” What that reframing would do, he explained, would make the discussion about public health be part of the multistakeholder collaborations that this workshop has noted are so important.