Improvement of Operational Risk Measurement Accuracy on Power Plant Using Qualitative and Quantitative Method

Implementation of maintenance programs in order to improve the performance of power plants required considerable cost, while existing funding sources are limited. Prioritization of maintenance program implementation based on risk level is needed to optimize the performance of power plant unit and maintenance cost required. The process of determining the level of risk can be done with qualitative and quantitative methods, each has advantages and disadvantages. This research compared accuracy of risk number on main equipment of power plant determined using qualitative method, quantitative methods and combined method which is the combination of weighted qualitative and quantitative risk number. Weight of qualitative risk is determined by the engineers in a focus group discussion (FGD) based on technology and capability criteria of equipment such as condition monitoring technology, warning system technology, ability to activate protection system, and load pattern of equipment, to improve the accuracy of risk measurement. Result from the calculations shows that risk number generated from the calculation using combined method has the closest average value and standard deviation to the actual loss. Keywords―Risk Priority Number, Quantitative Risk Assessment Method, FMEA, Risk Based Maintenance.

I. INTRODUCTION 1 Risk measurement and prioritization of risk mitigation programs is one of the important activities in the business process of Power Plant Operation and Maintenance Services. Risk identification and assessment process serves to determine the mitigation measures to be incorporated into the operation and maintenance plan. The limitation of operational and maintenance budget, making the selection of priority maintenance program to be implemented as a form of risk mitigation becomes important.
Inaccuracy in prioritizing risk mitigation programs might have an impact on the decline in equipment reliability leading to an increasing risk of potential loss of production opportunities. Inaccurate selection of risk mitigation programs will potentially lead to inefficiencies in operating costs, and increased opportunities for loss of operating income of the plant. Therefore, the measurement and prioritization of the risk mitigation program becomes 1 Hani Wibieanto is with Department of Technology Management, Institut Teknologi Sepuluh Nopember, Surabaya, 60111, Indonesia. Email: wibieanto@gmail.com. 2 Adithya Sudiarno is with Department of Industrial Enginering, Institut Teknologi Sepuluh Nopember, Surabaya, 60111, Indonesia. E-mail: adithya_sudiarno@ie.its.ac.id. important in supporting the achievement of the operation of the generating unit.
Failed to anticipate potential risk occurred in a coal fired steam power plant could make the unit not being able to achieve optimal performance and loss of maximum production opportunity for 810 hours. Failure on one of the generating equipment cannot be anticipated because the equipment is rated as having a low risk of failure by the engineers, so the maintenance program on the equipment is not a top priority for execution.
Risk is the potential to gain or lose a value such as physical health, social status, financial well-being and others. Risk can also be defined as an intense interaction with uncertainty [1]. Uncertainty itself is a potential, unpredicTable and uncontrolled outcome. Risk is a consequence of actions taken against an uncertainty [2]. International definitions of ISO 3100 (2009) Guide 72: 2002 define risk as the effect of uncertainty on purpose, in this definition of uncertainty including events and uncertainties caused by ambiguity and lack of information [3].
There are many methods used to measure risk, one that is often used is to calculate the probability of occurrence of negative events using the frequency of similar events that have occurred in the past. This makes risk assessment difficult for industries with high hazard levels such as the nuclear energy industry, due to very rare failure frequency whereas the consequences of failure are considerable. Risk is always measured as the value of expectations against an undesirable outcome. This combines the probability of various events that may occur. Ratnayake and Antosz (2017) in the study of risk matrix development and risk based maintenance in the manufacturing system using the determination of the frequency and impact of failure with a qualitative approach. The impact classification and the number of failures are measured based on personal safety (PS) criteria, percentage of nonconforming product (PoNCP), time of failure eliminations (ToFE), and number of breakdown (NOB) [4].
While Pui et al (2017) used a quantitative approach to calculating failure as the basis for determining equipment inspection on offshore oil drilling rigs, it was found that the estimated time between failure events on a device. With a more accurate estimation of failure estimates, maintenance programs can be planned more timely just before failure occurs according to estimates so that equipment reliability can be improved, while maintenance costs can be reduced to more optimal and efficient levels [5].
Risk-Based Maintenance. The purpose of the maintenance process is to improve the profitability of operations and optimize the total life cycle cost without overriding safety and environmental issues. Selection of maintenance policy is a decision-making process with multiple criteria. The criteria that are always considered are cost and reliability. Risk-based maintenance planning minimizes the chances of system failure and its consequences.
Maintenance management techniques have evolved through several stages of metamorphosis, from techniques that focus on periodic overhauls to condition monitoring, reliability-centered maintenance (RCM), and expert systems. Risk-based maintenance (RBM) and risk-based inspection (RBI) are two emerging methods. Among the several methods mentioned above RCM is designed to minimize maintenance costs by balancing high corrective maintenance costs with the cost of other maintenance strategies [6]. While RBM is the method used to define maintenance priorities using risk criteria that integrate safety and failure. The main purpose of this method is to reduce the overall risk that may arise as a consequence of the unexpected failure of an operating facility [7].
Research conducted by Aller, Horowitz, Reynolds, and Webber [8] became the basis for RBI development on equipment owned by Brunei Shell Petroleum [9]. The riskbased approach has also been successfully applied to pipeline maintenance [10] discussing simple risk-based modes for maintaining cross-country pipelines.
The goal of implementing a risk-based maintenance strategy is to reduce the overall risk of failure at an operating facility. In high-and medium-risk areas, focused maintenance efforts are essential. While in low risk areas, maintenance efforts are minimized work scope and maintenance cost through a structured effort. Implementation of a risk-based maintenance strategy will reduce the likelihood of unexpected damage.

II. METHOD
In this study the methodology used to determine the value of risk is a combination of qualitative and quantitative methods. Determining the possibility and impact of failure qualitatively is done by engineers using agreed criteria. While the determination of the possibility and impact of failure qualitatively is done based on historical data of equipment failure ever happened and recovery time required, obtained from database history of operation and maintenance of equipment.
Risk assessment using the combined method is a risk assessment using the weighted value of risk number the resulted by qualitative calculation and summed with the weighted value of risk number resulted by the quantitative calculation. The amount of weight of the qualitative and quantitative risk values used in the calculation of risk using the combined method is determined by the engineers in a focused group discussion (FGD).
The research stages include, collecting data on the frequency and impact of the power plant main equipment failure. Furthermore, to calculate the value of risk by using qualitative methods, quantitative methods and combined methods. Then compare the results of risk calculation using these three methods with the actual loss value to find out how close the deviation value generated.
Qualitative Risk Method. Qualitative risk value is obtained by multiplying the value of the failure frequency with the impact of the failure both obtained by qualitative calculations by the engineers The value of qualitative risk is obtained through the calculation process using the following equation = * (1) Where Rl is qualitative risk number, Fl is Equipment Failure Frequency Assessed by Qualitative Approach, and Dl is Impact of Equipment Failure Assessed by Qualitative Approach. The criteria used to assess the failure frequency are qualitatively agreed upon by the engineers in an FGD forum, as well as the criteria used to determine the impact of equipment failure qualitatively. The criteria for assessing the frequency of failure qualitatively are as follows. While the criteria used to assess the impact of failure qualitatively are as follows Where Rl is qualitative risk number, Fl is Equipment Failure Frequency Assessed by Qualitative Approach, and Dl is Impact of Equipment Failure Assessed by Qualitative Approach. The criteria used to assess the failure frequency are qualitatively agreed upon by the engineers in an FGD forum, as well as the criteria used to determine the impact of equipment failure qualitatively. The criteria for assessing the frequency of failure qualitatively are as follows.
Quantitative Risk Method. In quantitative method, probability and the impact of failure on each main equipment of the plant can be obtained by withdrawing the history of the ever-present failure. Mean time between failure on main equipment can be calculated from the date of failure. Furthermore, by using formula (3) failure rate of the equipment can be calculated. Probability for equipment failure can be calculated using the formula (2) after the failure rate is obtained. To calculate the possibility of failure can be used the following.
= 1 − μ μ (2) and Where is Opportunity occurrence of failure at a certain time t, is failure rate and MTBF is mean time between failure.
Impact of failure from main equipment is calculated based on the loss of production opportunities of plant during the equipment recovery process. The magnitude of the impact of failure relies heavily on the length of the recovery process of the equipment, the longer time need to recover the equipment will make the impact of a equipment failure more significant. Cost required to restore the equipment is not taken into account in the value of the impact because the restrore cost number is less than 1% of the value of lost production opportunity, so it is considered insignificant. The value of the failure impact is calculated based on the number of lost production opportunity values which can be calculated by the following equation.
= * * (4) Where Dn is impact of equipment failure assessed by quantitative approach, Tr is the time needed to repair equipment, Pl is decrease in performance arising from equipment failure and Pr is selling price.
Number of risk are obtained by multiplying the probability of failure by the impact of failure that it may ocure in an equipment. The greater propability of failure and impact resulting by failure of an equipment will increase the risk of equipment. Risk number can be obtained using the following equation, where Rn is risk number obtanied by quantitative method.
= * (5) Risk Assessment by Combined Method. Risk assessment by combined method is a combination of risk assessment results with qualitative and quantitative approaches done by summing the quantitative and qualitative value of an equipment after first being weighted at each value by a panel of experts consisting of engineers responsible for monitoring the condition of the equipment. The calculation of the risk value by the combined method is performed using the following equation.
= ( * ) + ( * ) (6) Rg is risk number obtained by combine method, Rl is risk number obtained by qualitative method, Wl is weight of qualitative risk number, Rn is is risk number obtained by quantitative method, and Wn is weight of quantitative risk number.
From the results of FGDs it is agreed that there are four criteria used in determining the amount of weight given for the value of qualitative risk on the calculation of the combined risk value. 1. Operation mode, showing how the equipment is operated. The base load operation pattern is an operating pattern with a relatively sTable load on the equipment, while the cyclic load operating pattern is an operating pattern with variable / fluctuating loads. Powerplant units operated at sTable loads will have a lower uncertainty factor that may causing equipment failure than units operating with cyclic loads 2. Technology Condition Monitoring, is a technology used to monitor equipment condition. The engineers classify the equipment monitoring equipment condition of the plant to be there are three classifications based on its monitoring capability which is divided into six technological attributes. The classification of monitoring technology based on the agreement of the engineer during the FGD forum is as follows. More advance condition monitoring techology will lead to lower weight of qualitative risk. 3. Warning System, is a technology used to alert operators when the operating parameters of an equipment are close to the operating limits that may cause equipment failure. Based on the capabilities of the warning system on generating equipment can be grouped into three. More advance warning system techology will lead to lower weight of qualitative risk.

4.
Protection system, is a system that serves to protect the equipment from severe damage. This system usually works when the equipment operation parameters show a number that exceeds the operating threshold. An equipment with protection system will have a faster recovery time, resulting in lower the impact of failure as well as risk number. Based on the four criteria above, the experts from engineering department give weighting percentage of qualitative risk value that will be used in the calculation of risk number by the combined method. The greater uncertainty on probability of failure in main power plant equipment caused by the lack of a monitoring system and warning system and adequate protection on the equipment will make the weight of the qualitative risk assessment greater. The criteria to weighted qualitative risk assessment of main equipment in hydroelectric power plant, coal fired steam power plant, and combine cycle power plant agreed by the engineers in a FGD is listed in the following Table.

III. RESULT AND DISCUSSION
After calculation using the formulation that has been delivered in the previous chapter obtained the risk value of each of the main equipment of the plant. the following results of risk values that have been calculated by qualitative, quantitative and combined methods.
Qualitative Risk Number. After assessing the probability and impact of equipment failure by the engineers based on the criteria in Table 1 and Table 2, the main equipment risk values for each plant are as follows. Risk number on main equipment of hydroelectric powerplant is obtained by multiplying the equipment failure probability with the impact caused by the failure of the equipment of main are shown ini Table 6. Risk number on main equipment of the coal fired steam power plant calculated using the qualitative method are are shown in Table 7.
The data are processed using equation (1) while probability and impact of failure are determinde based on criteria as shown in Table 1 and Table 2 above. Risk number on the main equipment of combined cycle power plant calculated using the qualitative method are processed the same way as the main equipment risk calculation of hydroelectric power plant and coal fired steam power plant. The risk number are shown in Table 8. Risk number obtained from the qualitative calculations in Table 6, Table 7, and Table 8 show the difference in chance probability of failure given by engineers on the same equipment for different types of plants. This difference in value is due to differences in environmental conditions around equipment, workload and equipment age. The older the age of the equipment, as well as the heavier workload and extreme environmental conditions will increase the chance of equipment failure.
While the difference in the value of the impact of the equipment is more influenced by the installed capacity of the plant that fails or decreases in performance. The greater the generating capacity that decreases the performance, the higher the impact of equipment failure.
Quantitative Risk Number. Probability of failure on powerplant equipment are obtained using equation (2) and (3), for example, the last failure of the generator at "W" hydroelectric power plant (HEPP) occurred on January 31, 2016 while the previous failure occurred on October 9, 2014, the time interval between failures of generator at the "W" HEPP was 27,021 hours. The mean time between failure is the average failure time in the four generator at "W" HEPP samples. Mean time between failure (MTBF) of generator at "W" HEPP is equal to 27,127 hours or 1130 days. This means that the failure of the hydropower ganerator on average occurs within 3.1 years. With the formula (1), the value of the reliability of the hydropower generator can be calculated in the span of one year to reach 72.4%, or in other words the probability of failure of the hydropower generator is 27.6% in one year.
The impact of the failure of a device is measured by the length of time the equipment fails and the amount of the opportunity loses the opportunity to operate as long as the equipment fails. in this calculation we use the average duration of recovery of equipment from the generating unit that is the sample. Failure impact obtained using equation (4), For example, based on a database of equipment failure history, the average time needed to recover from a generator that was damaged was 9.475 hours or about 13 months. The impact caused is calculated based on the loss of production opportunities for 13 months multiplied by the power of the generating plant multiplied by the selling price of electricity per kWh. If the power is capable of 6 MW or 6000 kW with a selling price of 810 Rupiah per kWh, then the loss of opportunity for production reaches 265 billion Rupiah.
Risk value is obtained by multiplying the value of the possibility of equipment failure with the value of the impact arising from the failure of the equipment. Risk number of powerplant main equipment calculated using equation (6). Risk number of hydroelectric main equipment are shown on Table 9. Risk number on the main equipment of coal fired steam power plant calculated using quantitative method obtained bay processing data through the same way as the main equipment of hydroelectric power plant risk number calculation. The risk number of coal fired steam power plant main equipment are shown in Table 10. Risk number on the main equipment of combined cycle power plant are calculated using the same method as the main equipment risk number calculation of coal fired steam power plant adn hydroelectric power plant. The risk number of combined cycle power plant main equipment are shown in Table 11. Risk value with quantitative calculation is obtained from history data of equipment failure ever happened. Differences in the probable value and impact of equipment failures reflect the actual conditions of the equipment. The lower probability of failure on an equipment means that it has higher reliability and less frequent failures.
Risk Number Obtained by Combine Method. After the engineers assess the technology of the condition monitoring device, the warning system, the protection system, and the pattern of loading on the main equipment of the power plant in accordance with the agreed criteria listed in Tables  3, 4  Quantitative risk weight = 1 -Qualitative risk weight (7) Weighted value of qualitative risk number are shown in Table 12.
Qualitative risk number of main powerplant equipment in Table 6, 7, and 8 are weighted with number in Table 12 and sum with quantitative risk number in Table 9, 10, and 11 weighted using a number obtained by calculation using equation 7. After calculating the data by using equation 6 risk number of combained method are obtained as shown in Table 13. Risk Number Obtained by Combine Method. The actual loss value represents the lost value of production opportunities occurred due to malfunction in the main equipment of the plant resulting in a decrease in performance or the non-operation of the generating unit. Using the data of risk value obtained by calculations with qualitative, quantitative and combine method, we performed data comparison against actual loss value. Tabel 14 is a comparison table of risk values that is combined with qualitative, quantitative, combined approach and actual loss value.
Furthermore, data are processed using one-way ANOVA multiple comparison procedure of Dunnett method with 95% confidence level. From the data processing obtained results as shown in Figure 2 as follows. Based on the graph above shows that the value of risk calculated by quantitative and combine method has a value that does not differ significantly to the actual loss value. However, the value of risk calculated by qualitative method has a significant difference from the actual loss value. The difference of mean value data and standard deviation of risk value as measured by qualitative, quantitative and combined method can be seen in more detail in  The data in Table 15 show that the risk value generated by the combined method has the average value and standard deviation closest to the actual loss value. It can be interpreted that the risk number generated from the calculation by the combined method has a more accurate and precise value.
Risk order of the main power plant equipment is arranged based on the risk value obtained from the calculation with qualitative, quantitative and combined methods. Risk number of the main power plant equipment is sorted from the largest value to the smallest value in order to prioritize maintenance plan to mitigate risk of main equipment failure. Comparison of risk order of main powerplant equipment with actual loss. Risk number obtained by calculation using three methods above will be the input data to determine maintenance priority are ordered based on risk number of each equipment. Equipment with the highest risk number have the highest priority to maintain. The following table will shown priority order of maintenance of main power plant equipment based on risk number calculated using three different methods compared to actual loss. Maintenance priority for main equiipment of HEPP are shown in table 16.  While the comparison of priority maintenance of the main equipment of CFPP and CCPP based on risk number and actual loss are presented in tables 17 and 18. From the three tables above, it can be seen that the order of main equipment maintenance priority based on the risk number obtained by the combine method consistently has the same equation with the priority of equipment maintenance which is based on the actual loss. Calculation of risk by the combined method carried out through the sum of qualitative and quantitative risk number that have been weighted can be a method to improve accuracy of priority maintenance program planning that becomes the risk mitigation of each main power plant equipment. A more accurate maintenance plan obtained from the results of a risk assessment that accurately can reduce the risk of failure of the main power plant equipment, so as to reduce the risk of losing production opportunities. An accurate maintenance plan can also increase the effectiveness of maintenance costs so that maintenance costs that excessive or even unnecessary can be further suppressed. This led to an increase in the efficiency of electricity production cost.

IV. CONCLUSION
From the Dunnet diagram the calculation of risk number by qualitative methods provide a value wich has significant deviation from actual loss. Factors affecting the value of deviation include significant and recent major equipment failure events that affect the engineer's assessment as well as the engineer's tendency to be cautious in dealing with risks by providing an impact value on the worst conditions in the calculation of risk values.
While the risk number obtained by quantitative methods have a lower deviation from actual loss when compared with the risk number resulting from qualitative calculations. But the average value of risk generated by qualitative calculation are lower than the actual loss. Some factors that cause the occurrence of condition are the insufficiency of historical data equipment failure that has occurred in the past used as a reference in the calculation of the value of risk, especially on some equipment that has a fairly high level of reliability.
Calculating risk using combined method, will give more improvement in result accuracy because the inadequacy of the reference data used in the process of calculating risk number using quantitative methods can be improved by adding the risk number obtained from the qualitative calculations after being weighted by the engineers. The amount of weight is determined by the engineers based on how much historical data on past events are used as the basis for quantitative calculations. The more and complete history of failure data that has ever happened to a piece of equipment, the greater the calculation of risk value with quantitative methods will be higher. Conversely, the less data used in quantitative calculations, the greater the weight of the risk value generated from qualitative calculations will be greater. Qualitative risk assessment provided by the engineers based on their experience and expertise can improve the accuracy of risk number generated by quantitative methods to obtain risk number that is closer to the actual loss.
More accurate risk number provide more accurate data input for maintenance planners in determining maintenance priorities on main power plant equipment, thereby enhancing equipment reliability and optimizing maintenance costs to a more efficient level. Further research can be undertaken to evaluate how long the qualitative risk weight of each main equipment can provide accurate risk value acceptability without revision.