Prognostics is an emerging concept in condition based maintenance (CBM) of critical systems. Along with developing the fundamentals of being able to confidently predict Remaining Useful Life (RUL), the technology calls for fielded applications as it inches towards maturation. This requires a stringent performance evaluation so that the significance of the concept can be fully exploited. Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few issues. Instead, the research community has used a variety of metrics based largely on convenience with respect to their respective requirements. Very little attention has been focused on establishing a common ground to compare different efforts. This paper surveys the metrics that are already used for prognostics in a variety of domains including medicine, nuclear, automotive, aerospace, and electronics. It also considers other domains that involve prediction-related tasks, such as weather and finance. Differences and similarities between these domains and health maintenance have been analyzed to help understand what performance evaluation methods may or may not be borrowed. Further, these metrics have been categorized in several ways that may be useful in deciding upon a suitable subset for a specific application. Some important prognostic concepts have been defined using a notational framework that enables interpretation of different metrics coherently. Last, but not the least, a list of metrics has been suggested to assess critical aspects of RUL predictions before they are fielded in real applications.

Prognostics is an emerging concept in condition basedmaintenance(CBM)ofcriticalsystems.Alongwith developing the fundamentals of being able to confidently predict Remaining Useful Life (RUL), the technology calls for fielded applications as it inches towards maturation. This requires a stringent performance evaluation so that the significance of the concept can be fully exploited. Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few issues. Instead, the research community has used a variety of metrics based largely on convenience with respect to their respective requirements. Very little attention has been focused on establishing a common ground to compare different efforts. This paper surveys the metrics that are already used for prognostics in a variety of domains including medicine, nuclear, automotive, aerospace, and electronics. It also considers other domains that involve prediction-related tasks, such as weather and finance. Differences and similarities between these domains and health maintenancehave been analyzed to help understand what performance evaluation methods may or may not be borrowed. Further, these metrics have been categorized in several ways that may be useful in deciding upon a suitable subset for a specific application. Some important prognostic concepts have been defined using a notational framework that enables interpretation of different metrics coherently. Last, but not the least, a list of metrics has been suggested to assess critical aspects of RUL predictions before they are fielded in real applications.

Prognostics is an emerging concept in condition basedmaintenance(CBM)ofcriticalsystems.Alongwith developing the fundamentals of being able to confidently predict Remaining Useful Life (RUL), the technology calls for fielded applications as it inches towards maturation. This requires a stringent performance evaluation so that the significance of the concept can be fully exploited. Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few issues. Instead, the research community has used a variety of metrics based largely on convenience with respect to their respective requirements. Very little attention has been focused on establishing a common ground to compare different efforts. This paper surveys the metrics that are already used for prognostics in a variety of domains including medicine, nuclear, automotive, aerospace, and electronics. It also considers other domains that involve prediction-related tasks, such as weather and finance. Differences and similarities between these domains and health maintenancehave been analyzed to help understand what performance evaluation methods may or may not be borrowed. Further, these metrics have been categorized in several ways that may be useful in deciding upon a suitable subset for a specific application. Some important prognostic concepts have been defined using a notational framework that enables interpretation of different metrics coherently. Last, but not the least, a list of metrics has been suggested to assess critical aspects of RUL predictions before they are fielded in real applications.

Prognostics performance evaluation has gained significant attention in the past few years. *As prognostics technology matures and more sophisticated methods for prognostic uncertainty management are developed, a standardized methodology for performance evaluation becomes extremely important to guide improvement efforts in a constructive manner. This paper is in continuation of previous efforts where several new evaluation metrics tailored for prognostics were introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. Several shortcomings identified, while applying these metrics to a variety of real applications, are also summarized along with discussions that attempt to alleviate these problems. Further, these metrics have been enhanced to include the capability of incorporating probability distribution information from prognostic algorithms as opposed to evaluation based on point estimates only. Several methods have been suggested and guidelines have been provided to help choose one method over another based on probability distribution characteristics.

**Abstract** Prognostics solutions for mission critical systems require a comprehensive methodology for proactively detecting and isolating failures, recommending and guiding condition-based maintenance actions, and estimating in real time the remaining useful life of critical components and associated subsystems. A major challenge has been to extend the benefits of prognostics to include computer servers and other electronic components. The key enabler for prognostics capabilities is monitoring time series signals relating to the health of executing components and subsystems. Time series signals are processed in real time using pattern recognition for proactive anomaly detection and for remaining useful life estimation. Examples will be presented of the use of pattern recognition techniques for early detection of a number of mechanisms that are known to cause failures in electronic systems, including: environmental issues; software aging; degraded or failed sensors; degradation of hardware components; degradation of mechanical, electronic, and optical interconnects. Prognostics pattern classification is helping to substantially increase component reliability margins and system availability goals while reducing costly sources of "no trouble found" events that have become a significant warranty-cost issue. **Bios** Aleksey Urmanov is a research scientist at Sun Microsystems. He earned his doctoral degree in Nuclear Engineering at the University of Tennessee in 2002. Dr. Urmanov's research activities are centered around his interest in pattern recognition, statistical learning theory and ill-posed problems in engineering. His most recent activities at Sun focus on developing health monitoring and prognostics methods for EP-enabled computer servers. He is a founder and an Editor of the Journal of Pattern Recognition Research. Anton Bougaev holds a M.S. and a Ph.D. degrees in Nuclear Engineering from Purdue University. Before joining Sun Microsystems Inc. in 2007, he was a lecturer in Nuclear Engineering Department and a member of Applied Intelligent Systems Laboratory (AISL), of Purdue University, West Lafayette, USA. Dr. Bougaev is a founder and the Editor-in-Chief of the Journal of Pattern Recognition Research. His current focus is in reliability physics with emphasis on complex system analysis and the physics of failures which are based on the data driven pattern recognition techniques.

The estimation of remaining useful life (RUL) of a faulty component is at the center of system prognostics and health management. It gives operators a potent tool in decision making by quantifying how much time is left until functionality is lost. This is especially true for aerospace systems, where unanticipated subsystem downtime may lead to catastrophic failures. RUL prediction needs to contend with multiple sources of error like modeling inconsistencies, system noise and degraded sensor fidelity. Bayesian theory of uncertainty management provides a way to contain these problems by integrating out the nuisance variables. We use the Relevance Vector Machine (RVM), for model development. RVM is a Bayesian treatment of the well known Support Vector Machine (SVM), a kernel-based regression/classification technique. This model is next used in a Particle Filter (PF) framework. Statistical estimates of the noise in the system and anticipated operational conditions are processed to provide estimates of RUL in the form of a probability density function (PDF). Validation of this approach on experimental data collected from Li-ion batteries is presented.

Prognostics is crucial to providing reliable condition-based maintenance decisions. To obtain accurate predictions of component life, a variety of sensors are often needed. However, it is typically difficult to add enough sensors for reliable prognosis, due to system constraints such as cost and weight. Model-based prognostics helps to offset this problem by exploiting domain knowledge about the system, its components, and how they fail by casting the underlying physical phenomena in a physics-based model that is derived from first principles. We develop a model-based prognostics methodology using particle filters, and investigate the benefits of a model-based approach when sensor sets are diminished. We apply our approach to a detailed physics- based model of a pneumatic valve, and perform comprehensive simulation experiments to demonstrate the robustness of model-based approaches under limited sensing scenarios using prognostics performance metrics.

It is an inescapable truth that no matter how well a system is designed it will degrade, and if degrading parts are not repaired or replaced the system will fail. Avoiding the expense and safety risks associated with system failures is certainly a top priority in many systems; however, there is also a strong motivation not to be overly cautious in the design and maintenance of systems, due to the expense of maintenance and the undesirable sacrifices in performance and cost effectiveness incurred when systems are over designed for safety. This paper describes an analytical process that starts with the derivation of an expression to evaluate the desirability of future control outcomes, and eventually produces control routines that use uncertain prognostic information to optimize derived risk metrics. A case study on the design of fault-adaptive control for a skid-steered robot will illustrate some of the fundamental challenges of prognostics-based control design.

With recent advancements in prognostics methodologies there has been a significant interest in maturing Prognostics and Health Management (PHM) to increase its technology readiness level for onboard deployments. Active research is underway both in industry and academia to address shortcomings in availability of run-to-failure data, accelerated aging environments, real-time prognostics algorithms, uncertainty representation and management (URM) techniques, prognostics performance evaluation, etc., to name a few. At this juncture it is highly desirable to close the loop by connecting the high level customer requirements for mission planning and execution to performance specifications for prognostics methodologies at the lower technical level. This calls for integrating the pragmatics of safety, reliability, cost, and real-time viability into the prognostics methodologies to establish a connection between top-down and bottom-up approaches currently pursued in the PHM community. In this paper we identify key areas that must be addressed to bridge these gaps and provide an overview of how these areas have been addressed in part at various levels. We also discuss on how these issues can be further developed into a comprehensive and more coherent portfolio of technologies that will ultimately lead to specifying guidelines for prognostics performance.

**PHM08 Challenge Dataset is now publicly available at the NASA Prognostics Respository + [Download](http://ti.arc.nasa.gov/tech/dash/pcoe/prognostic-data-repository/)** **INTRODUCTION - WHY SIMULATE DEGRADATION DATA?** Of various challenges encountered in prognostics algorithm development, the non-availability of suitable validation data is most often the bottleneck in the technology certification process. Prognostics imposes several requirements on the training data in addition to what is commonly available from various applications. It not only requires data containing fault signatures but also that contains fault evolution trends with corresponding time indexes (in number of hours or number of operational cycles). In general there are three sources from which data is usually available, namely: Fielded applications, experimental test-beds, and computer simulations (see [Figure 1](https://c3.ndc.nasa.gov/dashlink/static/media/other/3_modes.bmp)). From prognostics point of view, data collection paradoxically suffers from the situation that the systems that do run to failure often did not have warning instrumentation installed, hence no or little record of what went wrong. In the other situation, those that are continuously monitored are prevented from running to failure or are subject to maintenance that eliminates the signatures of fault evolution. Conducting experiments that replicate real world situations is extremely expensive in terms of time required for a healthy system to run to failure and is often dangerous. Accelerated ageing may be useful to some extent but may not emulate normal wear patterns. Furthermore, to manage uncertainty multiple datasets must be collected to quantify variations resulting from multiple sources, which makes it all the way more unattainable. Simulations can be fast, inexpensive, and provide a number of options to design experiments, but their usefulness is contingent on the availability of high fidelity models that represent the real systems fairly well. However, once such a model is available, simulations offer the flexibility to rerun various experiments with added knowledge from the system as it becomes available. Where, availability of real fault evolution data from the fielded systems would be more desirable, generating data using a high fidelity model and integrating it with the knowledge gathered from the partial data obtained from the real systems is by far the most practical approach for prognostics algorithm development, validation, and verification. In this presentation we discuss some key elements that must be kept in mind while generating datasets suitable for prognostics. Furthermore, with the help of an example it has been shown how a dynamical system model can be supported with suitable degradation models available from respective domain knowledge to create suitable data. The example is discussed next. **APPLICATION DOMAIN** Tracking and Predicting the progressionof damage in aircraft turbo machinery has been an active area of study within the Condition Based Maintenance (CBM) community. A general approach has been to correlate flow and effciency losses to degradation signtures in various components of the engine. Once such mapping is available, the next task is to estimate this loss of flow and eficiency inferring information from measurable sensor outputs, which ultimtely is used to assess the level of degradation in the system. **SYSTEM MODEL: C-MAPSS** The C-MAPSS (Commercial Modular Aero Propulsion System Simulation) is a tool, recently released, for simulating a realistic large commercial turbofan engine. C-MAPSS (Commercial Modular Aero-Propulsion System Simulation) that simulates a realistic large (~90,000lb) commercial turbofan engine. It allows the user to choose and design operational profiles, controllers, environmental conditions, thrust levels, etc. to simualte a scenario of interest. An extensive list of output va