Reliable systems health management is an important research area of NASA. A health management system that can accurately and quickly diagnose faults in various on-board systems of a vehicle will play a key role in the success of current and future NASA missions. We introduce in this paper the ProDiagnose algorithm, a diagnostic algorithm that uses a probabilistic approach, accomplished with Bayesian Network models compiled to Arithmetic Circuits, to diagnose these systems. We describe the ProDiagnose algorithm, how it works, and the probabilistic models involved. We show by experimentation on two Electrical Power Systems based on the ADAPT testbed, used in the Diagnostic Challenge Competition (DX-09), that ProDiagnose can produce results with over 96% accuracy and < 1 second mean diagnostic time. **Reference:** B. W. Ricks, and O. J. Mengshoel. "The Diagnostic Challenge Competition: Probabilistic Techniques for Fault Diagnosis in Electrical Power Systems." Proc. of the 20th International workshop on Principles of Diagnosis (DX-09) Stockholm, Sweden, 2009 **BibTex Reference:** @inproceedings{ricks09diagnostic, author = {Ricks, B. W. and Mengshoel, O. J.}, title = {The Diagnostic Challenge Competition: Probabilistic Techniques for Fault Diagnosis in Electrical Power Systems}, booktitle = {Proc. of the 20th International Workshop on Principles of Diagnosis (DX-09)}, address = {Stockholm, Sweden}, year = {2009} }

Systems health monitoring is essential in guar- anteeing the safe, efficient, and correct opera- tion of complex engineered systems. Diagnosis, which consists of detection, isolation and identi- fication of faults; and prognosis, which consists of prediction of the remaining useful life of com- ponents, subsystems, or systems; constitute sys- tems health monitoring. This paper presents an integrated model-based diagnostic and prognos- tic framework, where we make use of a com- mon modeling paradigm to model both the nom- inal and faulty behavior in all aspects of systems health monitoring. We illustrate our approach us- ing a simulated propellant loading system that in- cludes tanks, valves, and pumps.

The goal of prognostics and health management (PHM) systems is to ensure system safety, and reduce downtime and maintenance costs. It is important that a PHM system is verified and validated before it can be successfully deployed. Prognostics algorithms are integral parts of PHM systems. This paper investigates a systematic process of verification of such prognostics algorithms. To this end, first, this paper distinguishes between technology maturation and product development. Then, the paper describes the verification process for a prognostics algorithm as it moves up to higher maturity levels. This process is shown to be an iterative process where verification activities are interleaved with validation activities at each maturation level. In this work, we adopt the concept of technology readiness levels (TRLs) to represent the different maturity levels of a prognostics algorithm. It is shown that at each TRL, the verification of a prognostics algorithm depends on verifying the different components of the algorithm according to the requirements laid out by the PHM system that adopts this prognostics algorithm. Finally, using simplified examples, the systematic process for verifying a prognostics algorithm is demonstrated as the prognostics algorithm moves up TRLs.

Prognostics is an emerging concept in condition based maintenance (CBM) of critical systems. Along with developing the fundamentals of being able to confidently predict Remaining Useful Life (RUL), the technology calls for fielded applications as it inches towards maturation. This requires a stringent performance evaluation so that the significance of the concept can be fully exploited. Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few issues. Instead, the research community has used a variety of metrics based largely on convenience with respect to their respective requirements. Very little attention has been focused on establishing a common ground to compare different efforts. This paper surveys the metrics that are already used for prognostics in a variety of domains including medicine, nuclear, automotive, aerospace, and electronics. It also considers other domains that involve prediction-related tasks, such as weather and finance. Differences and similarities between these domains and health maintenance have been analyzed to help understand what performance evaluation methods may or may not be borrowed. Further, these metrics have been categorized in several ways that may be useful in deciding upon a suitable subset for a specific application. Some important prognostic concepts have been defined using a notational framework that enables interpretation of different metrics coherently. Last, but not the least, a list of metrics has been suggested to assess critical aspects of RUL predictions before they are fielded in real applications.

Prognostics is an emerging concept in condition basedmaintenance(CBM)ofcriticalsystems.Alongwith developing the fundamentals of being able to confidently predict Remaining Useful Life (RUL), the technology calls for fielded applications as it inches towards maturation. This requires a stringent performance evaluation so that the significance of the concept can be fully exploited. Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few issues. Instead, the research community has used a variety of metrics based largely on convenience with respect to their respective requirements. Very little attention has been focused on establishing a common ground to compare different efforts. This paper surveys the metrics that are already used for prognostics in a variety of domains including medicine, nuclear, automotive, aerospace, and electronics. It also considers other domains that involve prediction-related tasks, such as weather and finance. Differences and similarities between these domains and health maintenancehave been analyzed to help understand what performance evaluation methods may or may not be borrowed. Further, these metrics have been categorized in several ways that may be useful in deciding upon a suitable subset for a specific application. Some important prognostic concepts have been defined using a notational framework that enables interpretation of different metrics coherently. Last, but not the least, a list of metrics has been suggested to assess critical aspects of RUL predictions before they are fielded in real applications.

Presentation given by SRI NRA winners on Verification and Validation for Complex Systems.

# Summary There is a major thrust worldwide on developing affordable Integrated Vehicle Health Management (IVHM) technologies in aerospace, automotive and other areas. The fundamental objective is to enhance availability, increase safety and reduce unscheduled maintenance costs. IVHM enables fault detection, isolation, root cause analysis and potential diagnostics. In addition, it aims at developing robust algorithms to predict the onset of a fault, minimizing false alarms and estimating the remaining useful life of the mission despite the adversity. IVHM optimally integrates technologies in sensors, vehicle systems, prognostics and diagnostics. NASA and CSIR-NAL have received an award to jointly organize the first Indo-US Workshop on IVHM and Aviation Safety (WIAS), sponsored by Indo-US Science and Technology Forum (IUSSTF) from Jan. 5th to 7th, 2012 in Bangalore, which is the aerospace hub of the country. # Purpose The purpose of the workshop is to deliberate, discuss and evolve the state of the art aerospace systems’ health management strategies, and identify opportunities for collaboration between US & Indian Institutions. This will help orchestrate preparation of IVHM roadmap into the future. This is an attempt, initiated at NAL, in the direction of creating an ecosystem among R&D, Academics and Industry on the subject matter as a part of the IVHM Mission for aerospace industry. We thus see unprecedented opportunities for discussions and knowledge networking in the areas of IVHM. # Participation An active participation from following organizations / agencies is expected: Leading US academic institutions including Stanford, Berkley, Georgia Tech and Auburn University US Industry including GM, GE, Honeywell, Boeing Research, Lockheed Martin, Rockwell Collins European organizations such as the Center of Excellence, Cranfield, UK; LMS International, Belgium; Airbus & Dassault systems, France Top Indian Academia like IITs & IISc, R&D (like DRDO, CSIR, DST, DOS), Industry, Regulatory (DGCA) and Armed Forces. The IVHM program envisages close cooperation of R&D, Academia and Industry (national and international) and has immediate applications to legacy, current and future generation aircraft and other programs. # Expected outcome,

We believe our approach to gathering and organizing prognostics V&V information from relevant literature, and then applying it to our specific prognostics application, provides a novel methodological way to approach V&V. Conventional literature surveys have a different purpose – they attempt to distill general themes, offer comparisons and contrasts, etc. Our intention was to provide an aid that allows the application of knowledge gleaned from the literature. It also allows the user to break down the V&V process into smaller segments and to identify potential bottlenecks which then allow focusing the attention. The specific approach we took to organizing the information – into categories of “Barriers” and “Solutions”, where each is accompanied by explanatory text from the original sources, coupled with references to the sources themselves – seems to have worked reasonably well overall. There are aspects that could be improved; the descriptive text recorded proved on occasion to be insufficient to serve as a standalone explanation of the item in question – fairly often we found the need to trace back to the original source and read more of the explanatory context; the reference to the source helped, but still meant a somewhat cumbersome process; we also feel it would have been better to have taken the time to record a reference back from an item (Barrier or Solution) to all the sources where that item, or its equivalent, were discussed; the overall hierarchical organization of these items (see Appendix for the listing of their titles) could probably be improved upon. What we found to be most useful was to use the Barriers as a series of talking points, taking notes as we went along as to our prognostics experts' understanding of whether, and if so how, that Barrier applied to our application. This process both serves as a means to capture the rationale that justifies faith in the prognostics application's approach, and to stimulate (and again capture) thoughts on areas of concern and possible approaches to addressing them.

Prognostics performance evaluation has gained significant attention in the past few years. *As prognostics technology matures and more sophisticated methods for prognostic uncertainty management are developed, a standardized methodology for performance evaluation becomes extremely important to guide improvement efforts in a constructive manner. This paper is in continuation of previous efforts where several new evaluation metrics tailored for prognostics were introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. Several shortcomings identified, while applying these metrics to a variety of real applications, are also summarized along with discussions that attempt to alleviate these problems. Further, these metrics have been enhanced to include the capability of incorporating probability distribution information from prognostic algorithms as opposed to evaluation based on point estimates only. Several methods have been suggested and guidelines have been provided to help choose one method over another based on probability distribution characteristics.