The Department of Defense states that Sunday’s test was the 65th successful hit-to-kill intercept out of 81 tries since 2001 for the Ballistic Missile Defense System. This statistic includes not only the Ground Based Midcourse (GMD) missile defense system tested this weekend, but all ranges of interceptors, including the Patriot system which targets short range missiles.
Only 17 of those tests have been of the GMD system. The Missile Defense Agency (MDA) will state the GMD record as 9 successes out of 17 tests, which includes tests of interceptors that were prototypes and those with both the CE-I and CE-II kill vehicles. The Pentagon also claims “four intercepts using the operationally configured interceptor since 2006.”
GMD Test Record
We score the GMD’s intercept test record as 8/17, and three intercepts since 2006. This differs from the Pentagon’s count by one success. The Pentagon’s record includes FTG-02 as a successful intercept, and we disagree with that. (Click here for a table of GMD intercept tests.)
In that test, the interceptor struck a “glancing blow” off the target but didn’t destroy it. Dr. Michael Gilmore, Director of Operational Test & Evaluation said “I score the FTG-02 flight test a hit, but not a kill.” George Lewis has a good discussion of this on his Mostly Missile Defense blog. We count this as a failed intercept, since the aim of hit-to-kill technology is to demonstrate that a hit can cause a kill. Although the Pentagon’s fact sheet says that (strangely) an intercept wasn’t a primary test objective—which might argue that the test should be left off the intercept test list altogether—General Obering, the MDA director at the time, stated in a Senate hearing that an “intercept” was indeed a secondary objective.
In contrast, Sunday’s test was described as “…using only the force of the direct collision between the interceptor and the target to destroy the target warhead.” That seems to be pretty explicit. The MDA’s video seems to show the dummy warhead being destroyed.
So the record for operationally configured interceptors would more accurately be described as three successful kills, three failures, and one “hit, but not a kill.”
Common-mode Failures
Firing more than one interceptor at a target only increases the probability of interception if the kill probability of each interceptor is independent of the others. But if there is a problem common to all the kill vehicles—a “common-mode failure”—then if one fails due to that problem the others are likely to as well. In that case firing more interceptors doesn’t necessarily increase the kill probability. So far at least two of the failure modes of the operationally configured interceptors appear to have been common: the problem with the CE-II EKVs inertial guidance system in FTG-06 and the problems with the batteries uncovered in last summer’s FTG-07 CE-I test.
Common mode failures can be masked by other problems, as it appears the CE-II inertial guidance system problem was. The CE-IIs first test, FTG-06 in January 2010 failed because, according to the MDA, a “lockwire was not inserted during the EKV manufacturing process,” which would be a quality control issue. The near-repeat test in January failed because of a common-mode failure—an issue with the design of the kill vehicle. Vibrations from the kill vehicle’s onboard thrusters caused the inertial guidance system to fail to work properly. The January 2010 test presumably would have experienced this problem if it had gotten that far.
Common mode failures may also not be apparent without testing under a variety of conditions. For example, last summer’s FTG-07 test of the CE-I-equipped interceptor reportedly failed because of an issue with the battery, an issue that is apparently common to all CE-I and CE-II interceptors, since the MDA has stated that all interceptors will undergo a fix for it. This failure mode appeared under the more stressing test conditions used in FTG-07 and had not yet appeared in the other CE-I or CE-II tests.
Testing and Kill Probability
However, since the GMD, while in development, is apparently also being associated with real operational capability, it is instructive to look at how system effectiveness is determined.
A missile defense system’s effectiveness is often expressed as a “kill probability”—the probability the system will intercept and destroy a warhead. This kill probability can only be determined by testing, and the more testing the better we can trust that we know the kill probability accurately.
The GMD system is in the developmental stage, and its tests are necessarily scripted in order to generate needed engineering data. Since the tests are not operationally realistic, the kill probability determined in this phase of testing does not tell you what it would be in a real attack.
This is why we say that the GMD system does not have any demonstrated capability in a real-world situation. This is in contrast to others’ statements that, for example, the kill probability is well-known and high.
Moreover, it’s important to recognize that the interceptor is not characterized by a single value of the kill probability since the probability of interception will depend on the situation. The kill probability will depend on a number of factors, including the geometry of the interception, the time of day (which can impact the infrared signal of the target), the presence of launch debris and the presence of countermeasures—both intentional and unintentional (like a tumbling warhead). The addition of realistic conditions would challenge the system in myriad ways.
A test series determines the kill probability with some level of confidence. The kill probability is an intrinsic property of the system and the confidence is how well the user knows that effectiveness, with, for example, 66% or 95% confidence. The confidence level is a number determined by the statistics of the test program—it is not a subjective feeling someone has about the system. More on that later.