8

Exploring Advantages, Barriers, and Challenges to Global Harmonization of Methodologies for Nutrient Intake Recommendations

OVERVIEW

In the final session, moderated by Susan Fairweather-Tait and Amanda MacFarlane, panelists shared experiences from countries that have collaborated with other countries or entities to develop nutrient intake recommendations. This chapter summarizes these presentations and discussion, with major points highlighted here and in Box 8-1.

First, Christophe Matthys provided an overview of how the EURRECA (European Micronutrient Recommendations Aligned) network has addressed a range of scientific questions and policy development issues, beginning with how to define endpoints in the determination of nutrient requirements. EURECCA’s approach, he described, is based on three criteria: (1) new scientific evidence, (2) public health relevance, and (3) heterogeneity in existing recommendations (e.g., different vitamin D recommendations in neighboring countries). EURRECA’s other work revolves around:

• translating physiological requirements into recommendations, particularly in the case of uncertainty;
• questions on the added value of a scientific advisory body and issues around stakeholder involvement;
• the use of nutrient recommendations in policy making; and
• involvement of consumers in dietary guideline development, with a main challenge being how to explain uncertainty.

Unlike EURRECA, which Hildegard Przyrembel described as a project, the European Food Safety Authority (EFSA), she emphasized, is an institution. Specifically, EFSA is an independent organization that assesses and communicates risks associated with the food chain, but it is not allowed to make recommendations in nutrition. She discussed EFSA’s recent growth in response to a 2005 request from the European Commission to revise and add missing values to the 1993 dietary reference values (DRVs). In Przyrembel’s opinion, the approach for setting reference values is similar in different regions of the world. The difference, she said, lies in the amount of money, personnel, knowledge, data collection, and time available. Thus, one of the challenges to harmonization, and one that EFSA has faced, is finding experts with enough time and knowledge and the readiness to travel. Another challenge, she observed, is finding consensus in a heteroge-

neous group of experts. But there are advantages too, she continued, such as EFSA’s comprehensive databases on food consumption and composition. She encouraged between-country technical conferences and better between-institution communication when setting nutrient reference values.

James Ntambi offered insights into several challenges to harmonization across Africa:

• limited capacity to handle food and nutrition issues, including nutrient intake values (NIVs);
• diversity in foods, dietary patterns and habits, seasonal food supplies, and agricultural practices (e.g., the nutrients in plantain in one region may be very different than those in plantain in another region);
• the reality that the challenge is not only to develop a unified approach to determining NIVs, but to build the capacity needed to develop this approach both in individual countries and across Africa as a whole;
• the scattering of information among academic institutions and interested agencies; and
• questions around how to coordinate existing national and regional bodies across Africa that handle food and nutrition issues. He noted that many of these organizations have begun the process of generating NIVs, but have run out of resources.

Even though Norway is considered “one of the richest and best countries in the world,” Helle Margrete Meltzer said, it cannot afford to develop guidelines on its own. It has been collaborating with the other four Nordic countries for decades, most recently on the fifth edition of nutrition recommendations (issued in 2012). The fifth edition, according to Meltzer, focuses on nutrients of particular relevance to the Nordic countries, namely vitamin D, iodine, iron, and fatty acids and fats. Of note, it was also the first revision that involved working with systematic reviews. Because working on systematic reviews was new to most of the approximately 100 nutrition experts involved with the revision, combined with the fact most of the reviewers were unpaid and working during their free time, it was, she said, a “huge undertaking.” It was a slow process, one with many pitfalls (e.g., conducting systematic reviews requires knowledge about biases) and with large volumes of publications to examine. One of the “learning points” from this work, Meltzer concluded, is that, in her opinion, global harmonization will require a nutrition specialty in systematic reviews.

The final speaker of this session, Emorn Udomkesmalee called attention to the wide range of recommended protein intakes across Southeast Asia. For example, among 4- to 6-year-olds, the recommended protein intake

is 35 grams per day (g/day) in Indonesia, 21–22 g/day in the Philippines (females/males), 25 g/day in Vietnam, and 16 g/day in Malaysia. Such differences across very connected countries is “why we are here,” she said. In her opinion, it is not the approach to setting recommendations that is the problem, as all of these countries use similar approaches. Rather, the problem, she said, is judgment. These countries are challenged by limited expertise on local committees and major gaps in relevant data, even when setting recommendations for healthy populations. Udomkesmalee called for harmonized guiding principles that countries can follow when decisions need to be made and evidence is missing. More specifically, she called for harmonized guiding principles that address the diversity in context among countries.

EURRECA (EUROPEAN MICRONUTRIENT RECOMMENDATIONS ALIGNED)¹

Christophe Matthys provided an overview of critical scientific questions and policy development issues addressed by the EURRECA network. Funded under the European Union Sixth Framework Programme Network of Excellence, EURRECA brought together many universities that are working on setting micronutrient recommendations. Its aim was to conduct preparatory work for EFSA, although there has been some political discussion around the relationship between EFSA and EURRECA, Matthys remarked. He noted that EURRECA is coordinated by International Life Sciences Institute (ILSI) Europe, and ILSI Europe receives part of its funding from the food industry. In his opinion, source of funding is yet another challenge to harmonization, in addition to those identified during the breakout discussion, as listed in Chapter 7.

Matthys summarized EURRECA’s work as a series of five questions, beginning with how to define endpoints in the determination of requirements, taking into account public health importance, available new evidence, and priority populations. He noted that, when discussing physiological aspects, EURRECA always discusses requirements, not recommendations, as the latter are considered risk management and the former risk assessment and therefore “purely science,” he said. Matthys described the scheme by Cavelaars et al. (2010) for prioritizing micronutrients to review based on three criteria: (1) new scientific evidence, (2) public health relevance, and (3) heterogeneity of recommendations. As an example of the latter, he explained that he lives in Belgium, with the Netherlands just a 1-hour, 50-kilometer drive away from his home. Yet, based on current vitamin D

___________________

¹ This section summarizes information presented by Christophe Matthys, Ph.D., assistant professor, Human Nutrition, University of Leuven, Belgium.

recommendations, he would need to increase his vitamin D intake by a factor of five if he were to go the Netherlands. He and his EURRECA colleagues, such as Cavelaars et al. (2010), searched PubMed for studies on inadequate intake and status and their association with health outcomes, focusing on randomized controlled trials since 2003. They examined studies by population group and then built a multidimensional evidence pyramid for each group. Each pyramid has four tiers, with the top tier occupied by evidence that fulfills all three criteria (i.e., new, relevant for public health, heterogeneous), the second and third tiers occupied by evidence that fulfills only two of the criteria (either new and relevant for public health or new and heterogeneous), and the bottom tier being evidence that fulfills only the “new scientific evidence” criterion. Matthys remarked that these priority pyramids allow EURRECA to divide its resources based on evidence and to move forward.

The second question EURRECA’s work revolves around, Matthys continued, is how to translate requirements into recommendations, using the coefficient of variation (CV) in cases of uncertainty. Matthys described the creation by Dhonukshe-Rutten et al. (2013) of a conceptual framework to develop a projected “requirement distribution” and projected “optimal scenario” for each micronutrient. Although the projections are based mostly on dose–response curves for different health outcomes, Matthys explained that in certain situations, for example, for certain population groups, they had to rely on the factorial approach.

Because recommendations have to be “read,” Matthys said, this raises the third question around which EURRECA’s work revolves: What is the added value of a scientific advisory body, and how can representatives of the different stakeholders be included? He described how Timotijevic et al. (2011) looked at the way different scientific advisory committees across Europe were working and found that, in some countries, it was very clear how these committees were working, with very transparent documents and decisions and clear terms of reference about the composition of the committee and its tasks. But in other countries, it was, Matthys said, “the one who shouted the most” who decided what a reference value would be. Thus, he said, “There’s really a clear need to have a specific framework.” He added that the framework should also allow for consumers to rely on and understand what experts are doing.

Regarding stakeholder involvement, Matthys said that there is a clear north–south gradient in Europe, with a lot of involvement of stakeholders in the northern countries. In the south, in contrast, most food and nutrition policy decisions are made by experts and policy makers, not stakeholders. In Matthys’s opinion, when thinking about stakeholder involvement, another question to consider is how to include the hard-to-reach part of the population, especially immigrants, in the development of recommendations.

The fourth question around which EURRECA’s work revolves is on the use of nutrient intake recommendations in policy making. Matthys noted that EURRECA conducted an inventory of nutrient-related policy types and instruments across Europe and found a big difference among countries (Timotijevic et al., 2013).

Finally, the fifth question, involvement of consumers in dietary guideline development, with a main issue being how to explain uncertainty to a consumer, has led EURRECA to many discussions on the effect of a change in a recommendation on the population (Brown et al., 2013). For example, if the recommendation for vitamin D were to change from 2.5 to 2.6, based purely on the science, what effect will that have? How can the relevancy of that change be explained to consumers? Another issue is the selection of consumers to contribute to discussions on recommendations. In Belgium, for example, it is often the same people speaking at the food safety agency stakeholder meetings, Matthys observed.

EFSA (EUROPEAN FOOD SAFETY AUTHORITY)²

Hildegard Przyrembel began by emphasizing that EFSA is an institution, in contrast to EURRECA, which is a project. More specifically, EFSA, founded in 2002, is an independent scientific source of advice, information, and risk communication on risks associated with the food chain. “We are not allowed to make recommendations,” Przyrembel emphasized.

EFSA’s opinions and advice are formulated by the scientific committee and the scientific panels, which, in turn, are composed of external, independent experts who are recruited on the basis of an open application process. No one is admitted, Przyrembel noted, unless someone thinks that he or she has competence in some field. The balance of men and women is okay, according to Przyrembel, as is the balance among countries. She described it as a very difficult selection procedure, but one that she thinks EFSA manages well.

EFSA’s work on DRVs began with a 2005 request from the European Commission to revise the existing population reference intakes from 1993 (SCF, 1993) and to add missing values for DRVs not set in 1993. As a first step, the Panel on Nutrition, Dietary Foods, and Allergies wrote a 2010 opinion on principles for deriving and applying DRVs (EFSA NDA, 2010). Przyrembel described how, based on many papers, particularly from the Institute of Medicine (IOM), but also from member states, a decision was made to define:

___________________

² This section summarizes information presented by Hildegard Przyrembel, M.D., Ph.D., professor and director emerita, Federal Institute for Risk Assessment, Berlin, Germany.

• average requirement (AR),
• population reference intake (PRI), which she noted was equivalent to the King and Garza (2007) NIV,
• lower threshold intake (LTI),
• adequate intake (AI) (if average requirement cannot be defined), and
• reference intake range for macronutrients (RI).

Przyrembel noted that RIs have also been defined for some micronutrients as well, when no agreement could be reached on a point value for PRI.

These values are set by working groups, with additional external experts invited when they are known to have special knowledge for a particular nutrient and with the support of EFSA scientific officers, statisticians, and the data unit (evidence management), Przyrembel explained. She noted that EFSA has grown over the last 14 years from a very small unit into what she described as a “very impressive institution.”

The approach for setting reference values is similar across different regions of the world, Przyrembel remarked. She stated that this is particularly true of the AR, provided the criteria are fulfilled. In her opinion, differences in how these values are set lies in differences in the amount of money, personnel, knowledge, data collection, and time available. Based on her experience, a main challenge in this work is finding experts with enough time, knowledge, and readiness to travel. A second challenge is finding consensus in heterogeneous groups of experts with different backgrounds and varying levels of expertise in a particular field. “You learn a lot when you are on one of these scientific panels,” Przyrembel said. “There is always somebody who knows something else, and perhaps better.”

Regarding the advantages of “this setup,” Przyrembel continued, one is that, as of about 3 years ago, protocols can now be written for literature searches and appraisals of pertinent papers to permit systematic reviews of the available evidence. In the past, Przyrembel recalled, nutrition panels relied on the collecting ability of the experts in a panel. So now, it is done on a more systematic basis, because of an increased demand for transparency and replicability of panels’ assessments. The literature searches are often outsourced to external contractors.

Another advantage is the backup provided by EFSA’s databases on food consumption and food composition.³ Przyrembel described the database

___________________

³ Przyrembel was referring to the EFSA Comprehensive European Food Consumption Database (http://www.efsa.europa.eu/en/food-consumption/comprehensive-database) (accessed June 13, 2018) and the EFSA Nutrient Composition Database (http://www.efsa.europa.eu/en/data/food-composition) (accessed April 25, 2018).

as a comprehensive one that covers all member states of the European Union and is divided by category (i.e., age, food group, type of consumption [regular and high consumption]). It allows for calculations for each category of consumer and intake of any nutrient. Also worth mentioning, in Przyrembel’s opinion, is the anthropometry database for European children between the age of 0 and 18 years (van Buuren et al., 2012). EFSA’s reference body weights for children to 2 years old are those of the World Health Organization (WHO) growth standard, she said, but those for older children are based on this database. She cautioned, however, that if EFSA’s DRVs were to be used in other settings in other regions of the world, they would need to be adapted.

Although EFSA received some priorities for activities from the European Commission when it started this work, it was free to choose the sequence, Przyrembel recalled. Thus, it has relied on other mechanisms for setting priorities, as necessary, including systematic reviews, technical briefs, and toolkits. In addition to relying on the 2010 opinion paper on how to derive DRVs and the means needed for assessment (EFSA NDA, 2010), between-country technical conferences are also sometimes convened to find out which DRVs are universally applicable and which are not. In Przyrembel’s opinion, these conferences could be held on a greater scale. She urged more conferences between institutions that set up NRVs as a way to explain differences in results.

In Przyrembel’s opinion, the potential for acceptance of methodological approaches across countries exists in the European Union. Given that EFSA is a huge, well-working institution with financial and personnel resources that are so much greater than of any single member state, she wondered, “Why do member states make less use of the work which is done there?”

Moving forward, she called for mutual recognition and respect and, whenever feasible, cooperation. She suggested identifying the most urgent problems and tackling them first or selecting from available NRVs those that need population- or region-specific reassessment.

In closing, Przyrembel mentioned a recently completed DRV summary report that contains summaries of all of the opinions, but also provides information (on how the values were decided) in tabular form by age and sex (EFSA, 2017a).

AFRICA⁴

James Ntambi offered insights into the challenges to harmonization across Africa, beginning with Africa’s limited capacity to handle food and

___________________

⁴ This section summarizes information presented by James Ntambi, Ph.D., professor of nutritional sciences, University of Wisconsin–Madison.

nutrition issues, including NIVs. Compounding the challenge, Africa has diverse foods, dietary patterns and habits, seasonal supplies, and agricultural practices. Additionally, in contrast to developed countries, where most foods are processed, most foods consumed in Africa are natural foods, and because of regional differences in seasons and soils, this means that the composition of these natural foods may be very different. For example, the nutrients in plantain in one region may be very different than the nutrients in plantain in another region. Because of this diversity, Ntambi continued, harmonization will require not only building capacity in individual countries and across Africa as a whole, but also developing unified approaches and methods to determining NIVs. He expected that this will require use of existing well-developed, science-based methods.

He was aware of work that had been done in Uganda to determine nutrient content in some foods, such as iron in amaranth, but information on similar work on other nutrients was scattered across many academic institutions and interested agencies. He suspected that this was the case for other African countries as well. Additionally, he remarked that while there are existing national and regional bodies in Africa that were formed to handle food and nutrition issues, the question is, how can all of these bodies be coordinated and harmonized with a greater global system?

In his opinion, harmonization is possible. In fact, the process started in Uganda and East Africa and part of southern Africa, but they ran out of resources. “There is an opportunity to restart,” he concluded.

NORWAY⁵

Norway is considered “one of the richest and best countries in the world,” Helle Margrete Meltzer began, “but even with that starting point, we would never be able to afford to make dietary guidelines on our own.” Fortunately, she said, the country has a long history of very close, good collaboration with the other four Nordic countries. Since 1980, their collaboration on nutrition recommendations has been supported by the Nordic Council of Ministers, with the fifth edition published in 2014. According to Meltzer, because resource constraints make it out of reach to fully reassess every nutrient when revising the recommendations, they put their main emphasis on those nutrients with specific issues or challenges of relevance to the Nordic countries. For instance, vitamin D, iodine, iron, and fats and fatty acids have almost always been on the agenda.

For this last edition, for the first time, they worked with systematic reviews. In the past, they relied on what Meltzer described as “knowledgeable

___________________

⁵ This section summarizes information presented by Helle Margrete Meltzer, Ph.D., research director, Norwegian Institute of Public Health, Oslo, Norway.

professors.” But that was back when individual professors could have their own overviews of what was going on in the field. “That’s almost impossible today,” she said, “with some 350 papers coming out daily now related to nutrition issues.” Conducting systematic reviews were new to most of the approximately 100 nutrition experts who were involved in this last edition. Thus, it was a huge undertaking. In the process, they realized how challenging it was, with many pitfalls. “We probably went into every ditch that it is possible to go into on the road,” she said. But they managed, in the end, to produce about 20 systematic reviews, which, after peer review, were published in the journal Food and Nutrition Research. The reviews were fed into the central working group and used as a basis for the recommendations. For Meltzer, this was a “sunshine story” of five countries who have been working together for many years.

Additionally, the two most recent Nordic nutrition recommendation revisions (i.e., the 2004 and 2012 editions) have included not only “go-throughs” of nutrients, but also dietary guidelines. Both also have included physical activity recommendations. Plus, the last edition contains chapters on what Meltzer called “hot topics” that do not necessarily apply to specific nutrients, but are pressing public issues (e.g., diet and sustainability).

Regarding challenges to global harmonization, Meltzer commented on the many pitfalls, the knowledge of biases required, and what she described as the humongous volumes of publications that need to be searched when conducting a systematic review. She suggested that systematic reviews be established as a specific specialty for students at the Ph.D. level. She described how most of the review committee members conducted the systematic review work on their own free time, and many worked unpaid. So the process was quite slow. “If this is to be feasible,” she said, “I really think it should be a specialty in its own right. That’s one of the learning points from our work.”

In closing, she referred to the many challenges faced by EFSA, as described by Przyrembel. “Every one of those points would apply to the Nordic countries as well,” she said.

SOUTHEAST ASIA⁶

In the past 4 years, several countries in Southeast Asia have revised their recommendations on nutrient intake. Emorn Udomkesmalee discussed these recommendations, emphasizing differences among countries, For example, in the case of recommended protein intake (grams/day), for the age group of 6–9 years, the levels range from 49 (Indonesia 2014) to 29–30

___________________

⁶ This section summarizes information presented by Emorn Udomkesmalee, Ph.D., senior advisor, Mahidol University Institute of Nutrition, Thailand.

(Philippines 2015) to 32–33 (Vietnam 2016) to 12 (Malaysia 2017) (Personal communication, Professor Poh Bee Koon, Universiti Kebangsaan Malaysia). In Udomkesmalee’s opinion, considering that these countries are in the same Association of Southeast Asian Nations (ASEAN) region, the average requirements for similar age ranges should not differ this much for any particular nutrient, including protein. She remarked that the wide variation seems to reflect different “judgments.”

While the stepwise process to derive recommended nutrient intake is similar across countries, different countries follow different sets of existing international recommendations (e.g., the IOM, United Nations’ Food and Agriculture Organization [FAO]/WHO, EFSA, the United Kingdom) depending on the preference of the country committee. Udomkesmalee explained that most of the national committees are assembled with expertise from broad and overarching areas in nutrition, food, and health-related disciplines and that very few members have expertise in in-depth, nutrient-specific areas of knowledge.

Another challenge is the limitation of quality data to help with decision making. Udomkesmalee speculated that a systematic review of locally available data to screen for the strength of evidence, based upon the previously mentioned criteria,⁷ would reveal an urgent need to strengthen the quality of data. As an example, she cited controversy around the vitamin A bioconversion factor in the ongoing revision of Thailand Dietary Reference Intakes (DRIs). In 2003, the Thailand DRI committee adopted the IOM recommendation for a retinol activity equivalent bioconversion factor of 12:1 for beta-carotene (i.e., 12 µg of beta-carotene = 1 µg retinol [active form of vitamin A]), based on the evidence at that time from plant-based diets in developing countries. However, since then, as new data have become available, an EFSA panel has decided to maintain the older recommendation of a 6:1 bioconversion factor. According to Udomkesmalee, the Thailand DRI committee is not willing to adjust their current bioconversion factor because, as they see it, the new evidence is controversial and there is no international consensus yet regarding which factor suits what context. In addition, changing the bioconversion factor from 12:1 to 6:1 would require substantial efforts to correct all the raw data in the current national database.

A final challenge, Udomkesmalee continued, is that the DRIs are set for healthy populations, yet, for example, a significant proportion of the Thai population carries a thalassemia trait, which should be considered when setting the DRI for iron. In conclusion, Udomkesmalee emphasized the

___________________

⁷ In session 2, “Exploring Approaches to Evaluating the Evidence,” as summarized in Chapter 5.

critical need for guiding principles that the DRI committees can use when there is no clear “yes or no” decision.

DISCUSSION

Following their presentations, the panelists participated in an open discussion with the workshop audience, as summarized here.

Africa: Lack of Resources

Anura Kurpad asked “what went wrong” with the effort initiated to harmonize efforts in Africa and, if a similar effort is to be resumed, which institution or institutions would be plausible leaders. Ntambi responded that there are a number of organizations in African countries whose core function is nutrition, such as the National Food and Nutritional Council in Uganda. These organizations began collecting data some time back, he said, but a lack of resources led them to “getting stuck.” If resources were to become available, that effort could be revived.

Additionally, Ntambi emphasized that most countries in Africa have national academies of sciences, and the academies have a lot of expertise in computer science, mathematics, and nutrition. If these experts were given the charge or task to conduct these evaluations, they should be able to do it, in his opinion. He suggested encouraging the African academies to hold workshops like this one, but on the African continent, and to begin discussing harmonization. “The expertise could be there,” he said, “but the resources are lacking.”

While on the topic of resources, Matthys added that many resources are donor driven, not necessarily needs driven, and that this needs to be taken into account. Additionally, he mentioned the issue of research waste and the need to reduce the “double-work” that many people do.

Nutrient Recommendation Values in the Nordic Countries

It appears that there is agreement around the vitamin D NRV among the Nordic countries, Peter Clifton observed, but disagreement around how to apply these values in practice, that is, whether via fortification or supplementation. He asked, has there been any discussion around the public health application of the vitamin D NRV in the different Nordic countries? Meltzer responded that the decision on how to implement recommended NRVs is left to the separate countries. Regarding the actual recommendations, she noted that there are only 2–3 nutrients with slight differences among countries. Most are exactly the same.

The Nordic Countries: Barriers and Lessons Learned

When asked by Laura Martino what barriers the Nordic countries have come up against and lessons learned, Meltzer mentioned the same list that Przyrembel had described for EFSA. She commented that, early on, they agreed that they should use systematic literature reviews. But because none of them were experienced with those, they enlisted the help of professional librarians. One of the biggest challenges within the review itself, she said, was that they started with two large questions that covered “everything” and had to narrow the questions quite substantially to make them “doable.” She described the new literature that is emerging all the time as overwhelming, at least it was for the issues they decided to address, like vitamin D and fatty acids. Quality assessment was another challenge, she added, that is, “being able to really grasp the aspects of quality.” They ended up using an A, B, C, or D quality grading system, one used by the World Cancer Research Fund International. Hardly any study received full marks, she noted.

Meltzer emphasized again, as she had during her talk, the need to have experienced people conducting systematic reviews. “It should become a specialty in its own right within the nutrition society,” she urged, the kind of specialty that Ph.D. students or postdoctoral students work on, given its demanding nature. Yet, she said, “All of us should know a bit about how to do it, to appreciate the work behind it.”

Systematic Reviews

Most of the discussion during this session revolved around systematic reviews, beginning with Lindsay Allen asking, “How many systematic reviews do we need?” She questioned how many systematic reviews are really needed within each country. In her opinion, different systematic reviews do not yield very different numbers, perhaps with the exception of a few nutrients with criteria that one could “quibble about.” She expressed concern that the systematic review process cannot be applied everywhere and that the expenditure of resources by some groups is perhaps not necessary. For example, is it necessary in Thailand, or could that committee use results from EFSA’s work, for instance?

Matthys responded that one of the challenges is that the defined health outcome could be different from country to country. That is, a health outcome of relevance in one country may not be relevant in another. This is especially true with chronic disease outcomes, he added. Chizuru Nishida agreed with Matthys that systematic reviews address multiple health outcomes and that the reviews change based on the scope of the work and the

endpoints being addressed. Depending on a country’s priorities, one could focus on those reviews that include specific evidence of relevance.

Susan Fairweather-Tait pointed out that an interesting scientific experiment is under way with the National Academies conducting a review of sodium in parallel with EFSA conducting a separate review. As far as she was aware, they were examining virtually the same endpoints. But the National Academies may be using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system, while EFSA would be using a risk-of-bias tool. She expressed curiosity about how these two reviews will turn out and whether the results will conflict. She added that the Australian review of sodium was also available for comparison. “We don’t want endless systematic reviews,” she said. “We want updates, of course, but it shouldn’t be a business. It should be a means to an end.”

Regarding the systematic reviews themselves, Nishida emphasized the importance of transparency regarding how the quality of evidence is evaluated. She noted that, in the past, FAO and WHO used the same World Cancer Research Fund International grading system used by the Nordic countries, adapting it for both the 2002 update of dietary goals and the 2008 update of fats and fatty acids. Since 2010, however, WHO no longer uses that system. Instead, it uses the GRADE system, with the quality of evidence being ranked as high, moderate, low, or very low. Her understanding was that EFSA uses yet another classification system. Use of these different systems can cause problems. As an example, she mentioned Codex’s coining of the term generally accepted, which she found very confusing. To her and her colleagues, “generally accepted” evidence did not seem like convincing evidence. She encouraged harmonizing the use of different terms to express the quality of evidence.

Following Nishida’s comments, however, Przyrembel pointed out that, in EFSA, the generally accepted terminology is old language used by experts in a group who, according to their conscience and knowledge, reached a conclusion that was generally acceptable. EFSA’s current assessment process involving systematic reviews does not use that language, Przyrembel said. Additionally, in its assessments, in addition to conducting its own systematic reviews, EFSA also makes use of existing systematic reviews.

Regarding the dissemination of existing systematic reviews, Nishida mentioned that, in fact, there is a database, the WHO e-Library of Evidence in Nutrition. In the guidelines section, she said, not only are the full systematic reviews that guided the guidelines available, but so too are related existing systematic reviews from Cochrane and other sources. She explained that WHO has started collaborating with Cochrane to provide not just a link to their reviews, but also summaries of existing Cochrane reviews so people who are interested, but do not have the time to read a 400-page Cochrane review, can still access that information.

With respect to relying on systematic reviews, in Clifton’s opinion, the problem is with chronic disease. He mentioned the controversy around saturated and polyunsaturated fats and how the six or seven available systematic reviews (of cohort and intervention studies) yield very different answers despite ostensibly doing exactly the same thing with the same kind of question. The “real bias,” he said, is in the selection or omission of studies. While Cochrane reviews justify very fully why they omit certain studies, a major problem with many systematic reviews in the literature is that they do not. Thus, different systematic reviews may reveal different answers, depending on the omission or selection of studies. Yet, all of these reviews are published in peer-reviewed journals, some quite high ranking, according to Clifton. Although saturated and polyunsaturated fats are a good example, there are going to be many more micro- and macronutrients with the same problem, he predicted.

Matthys referred to a recent article in JAMA on the misuse of meta-analysis in nutrition (Barnard et al., 2017). He agreed that the problem is not just with saturated fatty acids but also red meats. Some reviews, depending on what they have omitted, recommend eating more red meat. “Quality assurance,” he agreed with others, “is one of the key issues to tackle.”

Laura Martino suggested further research on this topic, that is, how to combine different detailed assessments and come up with an overall judgment. She urged building on the GRADE approach in a way that adjusts for biases, accounting for both the direction and magnitude of biases.

One of the challenges that Thailand and other countries are going to face, Udomkesmalee observed, is the need for a review of systematic reviews. With different reference agencies having different ways of grading quality of evidence, she asked, on what basis are you asking countries to choose? She described the 2003 dietary guidelines in Thailand as a “cocktail mix” of FAO, WHO, and IOM recommendations and predicted that the 2017 guidelines would be even more mixed. They will be the result of what she described as a “judgment decision” on the part of the eight working groups in Thailand, each group coming up with its own “favorite.” She encouraged harmonization of all the “background noise” created by the multiple approaches being used to develop reference values and suggested that the reference agencies themselves talk around the usefulness and application of their work.

The Value of Stored Raw Data

Matthys remarked that, despite repeatedly insisting that EURRECA’s data would be open access, in fact, if he were to look into one of his external hard drives, he suspected he would find a lot of stuff. He suggested

that perhaps EFSA too has data on its shelves that sit, with nobody able to access them. In other words, he said, EURRECA’s data are not just a collation of papers, but they also include stored raw data. Making raw data from different institutions publicly available would be very beneficial, in his opinion. Committees from different countries could examine those raw data first and determine whether they are relevant. It may simplify things, he suggested.

Moving Forward: The Idea of a Global Consultative Group

“I feel like we’ve opened Pandora’s box,” Janet King said. She expressed being depressed after hearing about the many barriers to harmonization and did not want to leave the meeting without thoughts about how to come back together and go forward. She wondered whether there was a way to set up a global consultative group on nutrient recommendations to serve as a resource for all countries and regions. Such a resource would allow countries to build on the experiences of others and to modify others’ approaches, as necessary, to accommodate population differences and also to provide feedback so others can benefit.

Rather than feeling depressed, Joseph Lau said that he was heartened by this discussion. He shared with the workshop that this same issue around systematic reviews was addressed in the context of health care and medicine 20 years ago. It is new only to nutrition, he observed. He mentioned a 2017 JAMA commentary (Barnard et al., 2017) addressing the same problem that he himself addressed in a Lancet commentary 20 years ago. The message, he said, was the same: one answer does not tell the whole story. In his opinion, the challenge for nutrition is that the quality and information available from the primary studies are lacking. He interpreted this discussion as a call for better, more comprehensive reporting of information in the primary studies. “People are beginning to recognize the challenges and problems,” he said, “which is the first step in addressing an issue.”

Matthys expressed optimism as well. There are positive steps being taken, he said, such as the African Evidence Network, which aims to increase the use of evidence-based decision making in nutrition and health and involves trainings at many universities. Another example is the Joint Programming Initiative in Europe to make primary data available. While there is still a lot of work to do, he said, “I would like to look at the bright side.”