Scenario Modelling
Reliability prediction is a precise but inexact science. Software tools implementing the formulae defined in MIL-HDBK-217, for example, are capable of producing very precise results whose accuracy must always be treated with informed scepticism. There are undoubtedly strengths and weaknesses associated with this type of prediction.
Strengths
The strengths of this type of prediction are:
◦ It can be used throughout the design process, in its parts count form initially, followed by parts stress, to enable the technical risk of design decisions to be minimised at the earliest opportunity.
◦ It lends itself well to comparing options and performing trade-off analyses.
◦ It provides consistency and repeatability.
◦ It is a well-established methodology, supported by proven proprietary software tools available from diverse vendors.
◦ Despite its weaknesses, which are listed below, this type of prediction provides a mechanism for the fair comparison of alternatives and/or competing equipment suppliers.
Weaknesses
The contrasting weaknesses of this type of prediction are:
◦ Its lack of absolute accuracy.
◦ Its assumption that failure rate is constant with time.
◦ Its ability to consider only series configurations.
◦ Its inability to address many factors, including:
▪ Inadequate design.
▪ – Manufacturing defects.
▪ – Software.
▪ – Power on/off cycling.
▪ – Environmental flux.
▪ – Physical disruption.
▪ – Human interference.
 |
More recent reliability prediction calculation models may address some of the above factors.
|
It has been said that “Reliability prediction is about as accurate as weather forecasting; the only thing you can be absolutely sure of is that it's wrong.” At first glance this may seem a pretty damning statement; but, in the context of the proper use of reliability prediction, absolute accuracy is largely irrelevant.
It is essential to realise that the result of a reliability prediction is just a guess; an educated one maybe, but a guess nonetheless. The important factor is that the results are repeatable, and the inaccuracy is consistent across alternative proposed design solutions such that informed decisions may be made with regard to choice of options. It is worth digesting the opening paragraphs of the Reliability Prediction Manual for Guided Weapon Systems (Rex, Thompson and Partners on behalf of the MoD(PE); 1980) at this point:
“Reliability prediction is a forecasting technique by which the potential reliability achievement of a ‘mature’ system can be estimated during its design and development phases, and criteria established to aid decision making during those phases. The particular methods, which can be used at various stages during a project, depend upon the details of system design which are available, and the data that are relevant from previous experience. The true benefits of reliability prediction lie in the disciplines imposed by systematic and detailed analysis of the proposed design and its specified requirements, and the engineering interpretation of the predicted figures rather than in the absolute values themselves. In general, failure rate or MTBF predictions are likely to be optimistic and a prediction lying within a factor of two of the eventual achievement can be considered as good agreement. Despite the limitations which may be associated with any type of forecast, the prediction process provides the means to compare alternative design solutions against a common base-line, to identify reliability shortcomings which can be improved or corrected and highlight areas where trade-off studies or decisions may be required.”
So, although reliability prediction tools are capable of generating very precise results (to several decimal places), we must not allow ourselves to be tricked into believing that this must mean that they are very accurate results. Precision does not equate to accuracy.
If the above is true for predictions of inherent reliability, then the problems associated with the translation from predicted inherent to expected operational are far more complex and severe. Once deployed in the field, the subject equipment is exposed to many reliability threatening factors, although in this context we are actually referring to observed or perceived reliability rather than the inherent reliability of the original predictions.
These prediction techniques were derived from the assumptions that failures occur in a random manner with respect to the time domain and that the failure rate of individual components is constant. This concept provides a framework for collection and analysis of component failure rate data and for feedback of this data into the reliability prediction models. The effects of some operating stresses and environmental conditions on component failure rates were recognised early in the development of prediction techniques and have been incorporated into the currently accepted failure rate models where possible.
Most of this data was collected at the component level during life tests with no power on/off cycling and very little cyclic electrical, mechanical or thermal stress. As a result, cyclic effects that are significant in many equipment applications have not been adequately reflected in the data and thus are not explicitly represented in component failure rate models. This omission is the main reason that many reliability predictions for complex electronic equipment differ markedly from the values subsequently observed or perceived during service use.