As a broad generalization, the HRED work is being done at a high level of competence. As noted above, there might be questions about why a specific study was worth doing, but as a general rule the mechanical details of research are well done. Questions have been raised in the area of statistical analysis and interpretation. In some cases, the analysis did not seem to be the correct analysis. In other cases, the distinction between statistical significance and scientific or practical significance was lost. These issues tend to arise in presentations by junior investigators and point to the ongoing need for mentorship, “brown-bag presentations” of research to internal groups, and the vetting of research by the processes of peer review provided by participation in national and international meetings and the submission of work to rigorous, peer-reviewed journals.
In terms of its scientific output, HRED has increased its emphasis on publishing work in the open, peer-reviewed literature, but for a group the size of HRED, that output remains modest. The bulk of publications are book chapters, conference proceedings, and technical reports. Although useful, this output is not the same as publication in journals that are routinely indexed in databases (e.g., PubMed) and whose citations would be tracked by ISI Web of Knowledge or other citation indexes. Publication in these outlets is, in a sense, the currency of academic science and the most direct way for work in one laboratory to influence work in other laboratories. Promotion cases in university departments are heavily influenced by the quality and quantity of such publications. The current output at HRED would not support academic promotion in many cases.
The Human Research and Engineering Directorate should give stronger consideration to the publication of path-breaking HRED research in high-impact scientific journals.