This type of simulation cannot determine whether the new DOT&E protocols will decrease the government’s risk of buying substandard body armor. What it can do is to assess, from the manufacturers’ perspective, whether the new DOT&E protocol will result in higher FAT/LAT failure rates for body armor that probably would have passed under the previous Army protocol.
From a manufacturers’ economic risk perspective, this is a very reasonable comparison since, from that perspective, the issue is not how the armor performs in the field but whether the new DOT&E protocol increases test failure rates (and thus costs) for existing products and processes.
Note that this is essentially a nonparametric approach to evaluating manufacturer risk. That is, by using actual historical data drawn from passed tests, one does not have to make any parametric assumptions about the distribution of backface deformation (BFD), nor estimate the probability of penetration, nor try to model whether and how the two measures are jointly distributed.
However, this approach does depend on having sufficient historical data from which to resample. If insufficient data are available, then a parametric approach may be taken in which distributions are fit to the BFD and penetration data, and those distributions are used to simulate future data.
Another way to assess the impact of the new DOT&E protocols also uses historical test data for body armor. This approach can be used to compare two protocols. Instead of considering only manufacturer and design combinations that passed the historic protocols, consider a representative range of manufacturer and design results. For illustration, the committee compares the new DOT&E protocol and a historic Army FAT protocol.