Evaluating the Success of E-Invoice Implementation at a State-Owned Enterprise Using DeLone & McLean IS Success Model

― E-Invoice considered as one of the most implemented financial information systems these recent years. E-Invoice had grown from a system that brings convenience, into a part of the strategy to fulfill corporate expectation. However, it’s uncertain whether E-Invoice implementation indeed successfully brings proven benefits into the company business. Therefore, the aim of this study is to identify the indicators that affect the success of E-Invoice implementation in a particular state-owned company, based upon DeLone & McLean IS Success Model. In terms of evaluation, previous research regarding E-Invoice mostly discussed about the key factors that influence the adoption of E-Invoice. Lesser literatures purposed at studying the success of the implementation, related to the system performance. According to DeLone & McLean IS Success Model, there are six variables used as basic measurement, namely information quality, system quality, service quality, user satisfaction, intention to use, and net benefits. Following that, a set of structured questionnaire arranged and distributed to 34 users of an E-Invoice system provided by a state-owned enterprise headquartered in Rembang, Indonesia, including vendors and employees. The data then will be processed using Partial Least Square (PLS) method. The result showed that Information Quality and System Quality gave significant impact to User Satisfaction and User Satisfaction gave significant impact to Net Benefits. Meanwhile, Service Quality is not proven empirically to gave significant impact to both User Satisfaction and Intention to Use.


INTRODUCTION
N RECENT YEARS, organization and companies are heavily implementing information system to boost their operational in daily basis. Information system become a vital part for the business process applied by the company to perform various tasks and works. The main purpose of using the information system by the organization is to increase the operations efficiency [1]. In the middle-upper scale company, information system is not only considered as an object to support daily activities, but involved a lot deeper as a strategy to help the company to achieve its goals and purposes.
As one of the most crucial part in business, companies pay a particular concern towards financial issues and adopt financial information system as an important part of the financial management. Financial information system is widely implemented to support any activities related to financial management. The awareness to transform the financial operation from manual works into information system aroused by the company needs of automatic financial information, electronic document exchange, and digital financial transaction process. One of the most implemented financial information system in corporate level is Electronic Invoicing, or widely known as E-Invoice. E-invoice aspires to become the most common electronic document in the world [2]. The main function of E-Invoice is to facilitate the debt and bill payment process. It plays a critical role in maintaining business information throughout the supply chain [3]. It may become a significant part of financial supply chain as it shortened the business process in a better way, hasten the document transmission, and reduce the probability of administrative mistakes potentially happened during the process. Compared to the manual invoicing, E-invoicing reduces the total costs simply by eliminating the millions of paper invoices exchanged every year [4]. Companies might consider those adventages to decide the implementation of E-Invoice.
This research took place at state-owned enterprise named PT Z, an operation-company belongs to the largest cement company in Indonesia. As the heat of the business competition constantly increasing, the company growth their competitive advantages. As time goes by, the numbers of vendors keep multiplied and the company steadfastly widen their business, the problems faced are getting more complex. The employees realize the need of strengthening their IT sector, not only as supporting system, but also as the part of the strategic plan. E-Invoice is the embodiment of that plan, since it bridges the company with the vendors. This company began to implement E-Invoice since mid-2017 with the purpose to optimize the billing payment process, related to external partner such as vendors and third-parties. On top of that, the management established the regulation regarding the utilization of E-Invoice to accomplish any task related to vendor and third party billing. The user of this system divided into two big roles, the internal side consists of the employee of financial division, and the external side or the vendors. The vendors make use of this system to submit invoice or billing info into the company, meanwhile the financial division employee check the documents submitted and use the system as a repository. Both side could track the progress of invoice approval until it is ready to be paid by the treasurer. The 389 system is highly expected to pull off a good performance and fulfill the user needs.
Nowadays, the performance of a company depends on the performance of their information systems [5]. However, the good IT implementation closely related to how the user utilize the system. If the system is all right while the users are incompetent, the result appears will be lacked in quality [6]. The more sophisticated the system, more proficiency required to operate the system. This is an important factor to yield a good-quality information. The quality of the system and the proficiency of the user are the key of the successful implementation of an information system, including E-Invoice. Behind the advantages company may obtain by implementing E-Invoice, there are still a lot of problem surfaced. The transition from manual to system might be a tough job to some users, yet they keep continue to use the system in order to obey the company's regulation. This could lead into misused of the system and the optimization desired by the company is somewhat hard to achieve.
By managing an evaluation, it could be observed whether the system is already successful enough from the view of the users or do the system need to be improved so it could performs better, to meet the benefits desired by the implementation of the system. When the system become a strategic role of the company, the successful of the system is such a critical thing. Evaluation is one of the options to uncover how success is the implementation of E-Invoice based on the user impressions on using the system at a certain period of time.
There are many method on evaluation of system information. One of the most prevalent method used by numbers of researchers is DeLone & McLean IS Success Method. This method, developed by DeLone & McLean [7], quickly become one of the dominant evaluation frameworks in IS research, in part due to its understandability and simplicity [8]. The Updated DeLone and McLean IS Success Model defined six distinct dimensions of IS success: system quality, information quality, service quality, intention to use, user satisfaction, and net benefits (see Figure 1). The DeLone and McLean IS success model was approved as a valid comprehensive model for measuring success in the IS realm [9].

A. Previous Research 1) E-Invoice
Since E-Invoice considered to be one of the most popular information system implemented this recent years [10], there are several numbers of research held to observe the subtlety of this system, either from the view of exploratory study or empirical study.
E-Invoice has been recognized as one of the most important source of profitability in Europe [11]. Penttinen & Hyytiainen [12] once conducted an exploratory study about the factors affecting the adoption of E-Invoice in Finland. This study examine the success factors related to the implementation of E-Invoice in the afformentioned country. From this exploratory study, which observed four companies and two public organization as the case study, it is inferred that E-Invoice adopted at an increasingly fast rate, thanks to the communication factors and technological readiness, as the most significant success factors. E-Invoice likely become a part of E-Government or E-Procurement. Based on the empirical study conducted by Lian [13], in Taiwan E-Invoice is a good, representative example of a cloud based e-government application. The study aimed to understand the critical factors to the adoption of cloud-based E-Government, including E-Invoice. Through online survey involving 251 respondents, result conveys that effort expectation (on using the system), social influence, trust feeling, and perceived risk are the success factor of E-Government adoption in Taiwan.     However, despite its advantages, not every company desired to implement E-Invoice to support their business. This was the highlight of the study conducted by Hernandez-Ortega [10], to know the key factors that influence the adoption and subsequent use of E-Invoice. Upon the survey involving 1,193 respondents, from two kind of company in Spain, the first is companies which already implement E-Invoice and the other is companies that have not adopted E-Invoice (non-user). The conclusion reveal that perceived usefulness and compatibility are the two most important factors, both for non-user and adopter.

2) DeLone & McLean IS Success Model
DeLone McLean IS Success Model had been implemented in many case study time by time. From the original model introduced in 1992, until the updated version in 2003, numerous researchers have attempted to extend or respecify the original model [8]. Various kind of information system fit this model, since this model is renowned by its comprehensiveness.
Prevalently, the researchers who evaluate E-Learning applied DeLone & McLean IS Success Model as the evaluation method. Numbers of researches in this topic are easily found among the literatures. Seta, et. al [14] conducted evaluation of E-Learning implemented in UPN Veteran, a private university in Jakarta. The research shows that system quality and technical quality are the main factors that encourage user satisfaction towards the E-Learning system, while intention to use the system influenced by information quality.
Sometimes, demographic factors could describe the result of evaluation. According to the research on E-Learning in higher education held by Wagimin, et.al [15], more women are using e-learning compared with men. The study took place in UNS and UMS, both are university located in Surakarta. The research also inferred that successful implementation of e-learning is strongly influenced by the perception of user satisfaction and intention to use the system.
In industrial sector, Roky & Al-Meriouh [16] once evaluated the industrial information system implemented in automotive industry company. They found out that information quality plays a key role in increasing user satisfaction and intention to use the system. Another research in industrial sector conducted by Adroni & Sitorus [17]. They evaluate the success of Decision Supporting System implemented in largest Indonesian state-owned telecommunication company. The result shows that system quality and information quality gave significant influence on user satisfaction, while user satisfaction gave significant influence on net benefits, meanwhile service quality gave no impact to the user satisfaction.
However, not every evaluation yield good result about the succesful of an information system implementation. Yu & Qian [18] assessed the success of organizational Electronic Health Record implemented in 10 residential aged care homes in Australia. The data collected from 243 nursing staff through questionnaire. The result find out that system quality, information quality, and service quality had no significant impact on intention to use. Also, it is revealed that intention to use had no significant impact to user satisfaction and there's a weak, yet significant relationship between intention to use and net benefit. This research could be a prove that an unexpected result is still a result, depends on the system itself. and the way the user treat the system. This could be happened to any system in any circumstances.

A. Research Methodology
There are four stages in conducting this research : A) Preparation Stage, B) Data Collection Stage, C) Data

1) Preparation Stage
This very first stage is preparation to shape this research objective, thresholds, problem identifications, and all the preliminary information needed. Field study is useful to analyze the current condition of E-Invoice implementation, the expected results in implementation of the system from the views of management, the obstacles and challenges. Later on, the sample is determined, with the help of the user. Literature review is concurrently done with the field study. From the information gathered in this stage, the research model is determined and the questionnaire is ready to be designed The questionnaire based on the constructs used in the DeLone & McLean IS Success Method. Each constructs explained by indicators which lead the questionnaire design. Table 1 presents the research constructs and related survey items used for measurement of each of these constructs.

2) Data Collection Stage
In this stage, the questionnaire designed is being distributed to the respondents. The respondents of this research are the users of E-Invoice from the vendor and internal employee. After the answered questionnaire collected, the result are getting tested to find out the validity and reliability.

3) Data Analysis Stage
The analysis begin in this stage, after the testing of validity and reliability done. The data will be analyzed using Partial Least Square method. The outline of the analysis are testing the measurement model and testing the structural model. All done by the help of Partial Least Square analysis software named SmartPLS. a. Assessing the measurement model Measurement model, as known as outer model, is the evaluation of constructs validity and reliability, or could be defined as the correlation between indicators and the constructs possessed that indicators [19]. There are three criteria to assess the measurement model : 1. Convergent Validity 2. Discriminant Validity 3. Composite Reliability b. Assessing the structural model Structural model is assessed to check the relationship between one construct and other construct. The assessment done by observing the R square or by analyzing the structural path coefficient. The stability of this estimation evaluated by using T-testing through the procedure of bootstrapping [20].

4) Result Discussion Stage
Lastly, after the analysis successfully done, the results of the analysis are getting interpreted and discussed to some fruitful conclusions. These conclusions will lead into recommendations for the future development of E-Invoice.

B. Sample and Population
This research took place at financial division of PT Z. The whole population are all the users of E-Invoice, both from the external side and internal side. The external side are vendors and third-parties who established cooperation with the company, meanwhile internal side consists of the employee of PT Z financial division. This research aimed around 40 respondents from the population, sorted by the experience they have on using E-Invoice. The basic requirement is the sample already have the experience of using E-Invoice for minimum six months. From more than 40 questionnaires distributed, only 34 questionnaires responded and are decent to be included on the analysis.

C. Research Model and Hypothesis
The research model is adapted from DeLone & McLean IS Success Method, where there are six constructs divided into three independent variables and three dependent variables. The research model could be seen on Figure 3.

There are some minor differences on the research model compared to the Updated DeLone & McLean IS Success
Model. There is no reciprocal relationship between User Satisfaction and Intention to Use like mentioned in the original model. User Satisfaction is affecting Intention to Use, while Intention to Use does not have that kind of relationship with User Satisfaction. The choosing of this model based on the assumption that the more satisfied the user towards the system, the more likely they have higher intention to use the system. According to the model, this study hypothesized the following nine hypothesis (see Table 2).

A. Data Analysis: Assessing Outer Model or Measurement
Model First, the outer model or measurement model need to be assessed by quantify the correlation between indicators and its constructs. By quantifying the correlation, the validity and reliability of the model would revealed as well. Ghozali [21] mentioned there are several methods used to measure the validity and reliability of the models, by observing the convergent validity, discriminant validity, and construct reliability.

1) Convergent Validity
Convergent validity means observing the validity of relationship between each indicators and its construct. By using PLS, the validity could be found in the Outer Loading section. The acceptable measure of validity is ≥ 0.7 in loading factors [21]. Table 3 presents the outer loading for each indicators possessed by constructs in the research model:

N.b : * = Not Significant
From the result seen in Table 3, most of the indicator loading values are greater than 0.70. Except for indicator SQ1 with loading factor value 0.559, which is less than 0.70. This value indicated that indicator SQ1 (Reliability) is low in validity since it's less than the standard requirement of validity.
Before moving to the next step, the indicator SQ1 need to be dropped and then do re-estimation excluding the indicator SQ1. The result of re-estimation could be seen in Table 4.
From the re-estimation, all of the indicators have loading factor greater than 0.70 and fulfill the standard requirement of convergent validity.

2) Discriminant Validity
Abdillah & Jogiyanto [22] explained that discriminant validity related to the concept that indicators should not correlate higher with construct other than the construct which possessed that indicators. There are several ways to calculate discriminant validity: a. Cross Loading between constructs and its indicators. The cross loading value between the construct and its indicator is expected to be higher than the indicators belong to other construct [21]. b. Square root of Average Variance Extracted (Fornell-Larcker Criterion). Hair et.al [23] described Average Variance Extracted (AVE) as square root average from indicator loading factors, related to its construct. The model is significant if the square root of AVE is greater than the correlation value between each constructs. Table 5 shows the discriminant validity of the research model based on the Cross Loading value. From the result, could be inferred that the correlation value between construct and its indicators are greater than the correlation with the other constructs. It is safe to conclude that all of the constructs explain its own indicators better than indicators from any other constructs.
Alternate way to describe the discriminant validity is by comparing the square root of AVE from each construct with the correlation between one constructs and other constructs. Before doing the comparison, find the AVE first. A good AVE has value larger than 0.5 [21]. Table 6 show the AVE value along with AVE square root.
The result shows that all the AVEs are larger than 0.50. After the square roots AVE from all constructs obtained, the comparison could be done from here. From Table 7, it is revealed that all constructs are fulfill the good discriminant validity standard.

3) Composite Reliability
Another examination to evaluate outer model is to find out composite reliability. Construct considered reliable if the composite reliability is greater than 0.70.
From Table 8 it could be indicated that all of the construct are reliable based on the composite reliability.
Data Analysis : Assessing Inner Model or Structural Model Inner model assessment aimed to test the hypothesis listed in the previous section. Two measures were used to assess the structural model: the ability of the model to explain the variance in the dependent variables R square (R²) and the statistical significance (t-tests) of the estimated path coefficients [24].

4) R-Square
R Square or R 2 attempts to measure the explained variance of the dependent variable relative to its total variance. Values of approximately 0.75 is considered strong, values of approximately 0.5 is considered moderate, and values of approximately 0.25 is considered weak [25].
As seen on Table 9, latent dependent variable Intention to Use (IU) has R 2 equal to 0.592 which means the constructs Information Quality (IQ), System Quality (SQ), Service Quality (SV), and User Satisfaction (US) give influence as much as 59% to Intention to Use. Later on, the latent dependent variable User Satisfaction (US) has R 2 equal to 0.691, which indicated that the Information Quality (IQ), System Quality (SQ) and Service Quality (SV) affect as much as 69% to the User Satisfaction. In addition, the latent dependent variable Net Benefits (NB) has R 2 equal to 0.809, which could be concluded that Intention to Use (IU) and User Satisfaction (US) give influence as much as 81% to the Net Benefits.

5) T-Test for Hypothesis Testing
Hypothesis testing in the research will be done by performing Bootstrapping technique to get standard deviation values, path coefficient, and value from T-Statistics. The bootstrap done by resampling all the empirical data or the original data [26]. The sample bootstrap recommended by Hair et, al [25] and Henseler et. al [26] is 5000. In this research, the sample bootstrap is sets at the number 5000 with 5% significancy level. Table 10 shows the result:

N.b : * = not significant
The T-Statistics defined above is useful for hypothesis testing that have been proposed in the previous section. The hypothesis will be explained one by one and the T-Statistic will be compared to the T- Table. The T-table value used for this research is 2.048. Moreover, for significancy level 5%, path coefficient significant if the T-Statistic is greater than 1.96 [25]. Hypothesis testing is wrapped in Table 11.

H1
H1 tests the correlation between Information Quality (IQ) and Intention to Use (IU). T-Statistic between IQ and IU is 0.494, lesser than the T- Table 2.048. The p-value is also greater than the maximum standard 0.05. From that result, it could be inferred that the quality of the content, format, information accuracy, and timeliness are not affecting the user intention to use E-Invoice.

H2
H2 tests the correlation between Information Quality (IQ) and User Satisfaction (US). T-Statistic between IQ and US is 2.200, greater than the T- Table 2.048. The p-value is also lesser than the maximum standard 0.05. From that result, it could be inferred that the quality of the content, format, information accuracy, and timeliness are affecting the user satisfaction on using E-Invoice.

H3
H3 tests the correlation between System Quality (SQ) and Intention to Use (IU). T-Statistic between SQ and IU is 0.108, lesser than the T- Table 2.048. The p-value is also greater than the maximum standard 0.05. From that result, it could be inferred that system reliability, ease of use, ease of learning, system features, and response time are not affecting the user intention to use E-Invoice.

H4
H4 tests the correlation between System Quality (SQ) and User Satisfaction (US). T-Statistic between SQ and US is 2.601, greater than the T- Table 2.048. The p-value is also lesser than the maximum standard 0.05. From that result, it could be inferred that system reliability, ease of use, ease of learning, system features, and response time are affecting the user satisfaction on using E-Invoice.

H5
H5 tests the correlation between Service Quality (SV) and Intention to Use (IU). T-Statistic between SV and IU is 1.285, lesser than the T- Table 2.048. The p-value is also greater than the maximum standard 0.05. From that result, it could be inferred that service reliability, service responsiveness, service assurance, and empathy are not affecting the user intention to use E-Invoice.

H6
H6 tests the correlation between Service Quality (SV) and User Satisfaction (US). T-Statistic between SV and US is 1.834, lesser than the T- Table 2.048. The p-value is also greater than the maximum standard 0.05. From that result, it could be inferred that service reliability, service responsiveness, service assurance, and empathy are not affecting the user satisfaction on using E-Invoice. H7 H7 tests the correlation between User Satisfaction (US) and Intention to Use (IU). T-Statistic between US and IU is 3.247, greater than the T- Table 2.048. The p-value is also lesser than the maximum standard 0.05. From that result, it could be inferred that overall satisfaction from using E-Invoice, the satisfaction gained from the information served by E-Invoice, and the satisfaction gained by the quality of E-Invoice are affecting the user intention to use E-Invoice.

H8
H8 tests the correlation between User Satisfaction (US) and Net Benefits (NB). T-Statistic between US and NB is 4.215, greater than the T- Table 2.048. The p-value is also lesser than the maximum standard 0.05. From that result, it could be inferred that overall satisfaction on using E-Invoice, the satisfaction gained from the information served by E-Invoice, and the satisfaction gained by the quality of E-Invoice are affecting net benefit resulted by using E-Invoice.

H9
H9 tests the correlation between Intention to Use (IU) and Net Benefits (NB). T-Statistic between IU and NB is 1.751, lesser than the T- Table 2.048. The p-value is also greater than the maximum standard 0.05. From that result, it could be inferred that actual use of the system and number of transaction daily are not affecting the net benefit resulted by using E-Invoice.

B. Implications
This research has addressed the concern for measuring the success of E-Invoice in PT Z. For this purpose, an E-Invoice success measurement model was developed based on the DeLone and McLean [7] updated IS success model, which captures the multidimensional nature of E-Invoice success. There are implications concluded from this research. 1) Information quality, system quality, and service quality do not have significant impact on user's intention to use the system. Assumed that the utilization of this system is mandatory. Users operate this system because the regulation obliged it. As in previous study, Budiyanto [19] mentioned that as for mandatory systems, intention to use does not makes a proper variable to measure actual use and user's intention to use the system. 2) Information quality and system quality do have significant impact on user satisfaction. Though this system is mandatory, user still find satisfaction on the result and quality provided by the system. As in previous study, Rai et.al [27], Wu & Wang [28], and Livari [29] empirically proved that information quality and system quality give significant impact on user satisfaction. The better information quality provides by E-Invoice, system quality performs by E-Invoice, the more satisfaction gained by the users. 3) Service quality does not have significant impact on user satisfaction. Wang & Liao [30] once found the same conclusion on their research, saying that there's no significant influence of service quality on user satisfaction. The responsibility of service quality of E-Invoice is lie in the hand of the developer of the system, which affiliated with PT Z. Whenever problem occurs, user directly contacts the developer and tells them the detail of the error. The developer use ticketing system and  [7] where user satisfaction significantly impact intention to use 5) User Satisfaction do have significant impact on net benefits. The users impressed by the benefit after they gain the satisfaction after using E-Invoice. As in previous study, Gelderman [31] mentioned that as level of satisfaction increase, so do the net benefit of the system. 6) Intention to use does not have significant impact on net benefits. No matter how many times user use E-Invoice a day, will not affect the benefits felt by the user. As in previous study, Gelderman [31] assumed that the association between use and net benefits was not statistically significant.
IV. CONCLUSION Table 9 shows that latent variable dependent Intention to Use has R2 value 0.592 which means the variable Information Quality, System Quality, Service Quality, and User Satisfaction give influence of 59% on Intention to Use. Besides, the latent variable User Satisfaction has R2 value 0.691 which means the variable Information Quality, System Quality, and Service Quality give influence of 69% on User Satisfaction. Lastly, latent variable Net Benefits has R2 value 0.809 which means the variable Intention to Use and User Satisfaction give influence of 81% on Net Benefit. According to Hair et. al [25], the R2 values show that the model in this study is in the range of moderate.
Based on hypothesis testing, Information Quality and System Quality give significant impact on User Satisfaction, while User Satisfaction gives significant impact on Intention to Use and Net Benefits. However, Intention to Use does not have any impact from the independent variables, also does not give any impact to the Net Benefits. Assumed the reason of being a mandatory system, the intention to use E-Invoice is remain intact, since the company obliged so.
Service Quality is not empirically proven to give a significant influence on both User Satisfaction and Intention to Use. Considered the cause is there is no technical support specially appointed to handle E-Invoice, so the service is somewhat out of expectation.

A. Recommendation
Upon the research result, suggestions can be given as follow :

1) For the company
User Satisfaction and Intention to Use from the result of the research proved give a huge influence on Net Benefit, around 81%. Financial Division of PT Z is suggested to maintain and propose a better development for E-Invoice for better target achievement. Service Quality also need some attention, since there is no technical support or helpdesk in charge for E-Invoice. Financial Division is suggested to discuss about this point with the one who responsible with the ICT for better service management.

2) For further research
For further research in the future, it's recommended to choose a system with voluntary use instead of mandatory use. Other method could be applied to evaluate the success from another point of view, such as behavioral intention (using UTAUT or UTAUT 2) or any other method suitable with the research purpose.