Analysis of Project Performance Criteria for Design and Build Contract

The design and build (DB) project delivery system recently begins to be widely used for road projects in Indonesia. It has some advantages such as shorter completion time, lower cost growth, lower schedule growth, and better performance in terms of quality. The complexity of DB projects and the lack of complete rules and regulations in this field have caused the implementation of DB projects to face various challenges and problems. Thus, DB projects need to be evaluated to improve their performance. The purpose of this study is to provide the concept of DB project performance, performance criteria, and performance indicators that used to carry out a performance assessment. The analysis was done based on a literature review and interviews. There are fifteen indicators generated from this analysis. The indicators divided into three parts: design phase performance, construction phase performance, and overall project performance and three criteria: results quality, resource usage, and objectives achieved.


INTRODUCTION
The most commonly used project delivery system for road works in Indonesia is Design Bid Build (DBB). It is the traditional project delivery where the planning, implementation, and supervision stages are carried out separately and with different work contracts to produce a separate product (Yunianto et al., 2014). Design and Build (DB) is one of the alternative procurement systems where the client makes a contract with a single entity to perform both design and construction under a single DB contract (Ratnasabapathy et al., 2006). It begins to be widely used for road projects in Indonesia due to its advantages. DB system is considered the fastest procurement system because it encourages overlapping between the design and construction process (Chen et al., 2016). Construction works can begin while the design is still in progress so that initial defects can be corrected immediately without a significant impact on the project schedule (Nawi et al., 2014;Ojo et al., 2011). The DB system also allows effective communication between the designer and the builder so that it can integrate the contractor's knowledge and construction experience into the design. The resulting design is more constructable that it minimizes the possibility of design changes. Several studies have shown that the performance of projects that use DB systems is better than projects that use Traditional Systems in terms of cost, time, and quality. DB projects require shorter completion times, have lower cost growth and schedule growth, and show better performance in terms of performance specifications (Hale et al., 2009;Ojo et al., 2011;Ratnasabapathy et al., 2006).
In Indonesia, the DB system applied to acquire faster road infrastructure development and a more efficient procurement process. Projects delivered using the DB system are projects that fell under the category of a complex project or urgent project. According to the Regulation of Minister of Public Works and the Housing Republic of Indonesia Number 1 the Year 2020, complex projects are those that have high risk, require high technology, use specially designed equipment, have difficulties to be defined technically, and have a high probability of unforeseen condition. Meanwhile, urgent projects are those that provide more benefits to the community both economically and socially, must be utilized immediately, and contain difficulties in carrying out design and construction work separately due to time constraints. These criteria limit the number of projects delivered using the DB system. Below are the road projects conducted using DB system during 2016-2018.
 Although some studies have shown that the performance of DB projects is better than projects that use Traditional Systems, some DB projects have not shown good performance. The complexity of DB projects and the lack of comprehensive rules and regulations in this field have caused various challenges and problems during the project implementation, which lead to ineffective and inefficient results. Therefore, it is necessary to evaluate the project through performance assessment. If the project is performing well, it will also deliver reliable infrastructure. Thus, construction performance will also increase. Infrastructure construction is part of the asset life cycle that is important in infrastructure asset management, which aims to ensure that infrastructure can function sustainably, efficiently, and following sustainable principles (Soemitro & Suprayitno, 2018).
Researches on DB project performance generally focused on project performance in forms of cost, time, and quality. However, those measures are not sufficient to assess their actual performance (Ratnasabapathy et al., 2006). Project performance is not only about financial success or early completion but also about the entire processes that occur during project implementation. Besides, a project is unique in that each project differs from others regarding its goals, activities (tasks), resources, and deliverables (Lauras et al., 2010). Different project definitions might lead to different ways of measuring project performance. Consequently, an appropriate performance indicator is needed as a benchmark or expected performance standard so that it can accurately describe DB project performance. The purpose of this study is to provide the concept of DB project performance, performance criteria, and performance indicators that used to carry out a performance assessment.

Design and Build Project Delivery System
Design and Build (DB) is an integrated procurement system that delivers design and construction services under one contract. In this system, a single entity performs both design and construction and takes responsibility for all aspects of the project (Chen et al., 2016;Ojo et al., 2011;Ratnasabaphaty et al., 2006). In the DB system, there is only one tender step to select one entity to complete the project (Hale et al., 2009). Thus, it has significant time savings compared to the traditional system that requires two tender steps. Design and construction, either in part or in whole, can be carried out by a single design/ builder or can be subcontracted to another contractor. In DB systems, designers work under a contractor as a team (Ratnasabaphaty et al., 2006).
The process in the DB procurement system is as seen in Figure1. It shows that an overlap between the design and construction phases is common during DB projects, which can speed up the project significantly.

Project Performance Measurement
Project performance is how the project works in terms of comparing the project results with the objectives specified in the contract (Hafidy, 2010). Successful project performance is when stakeholders get the expected requirements, both individually and collectively. The stakeholders have to consider and distinguish three criteria that exist in the project life cycle: procurement, process, and results to obtain the requirements (Takim et al., 2002). Generally, Project success or failure determines project performance. The iron triangle (time, cost, quality) is no longer sufficient to represent project success.
Performance measurement is a systematic way to evaluate the inputs and outputs in manufacturing operations or construction activities and is a tool for continuous improvement (Takim et al., 2002). According to Takim (2002), the element of success in a project deals with efficiency and effectiveness measures. Efficiency measures refer to schedule, budget, specification, and safety and would only be achieved through having standards, systems, and methodology. Effectiveness measures refer to user satisfaction, the use of the project, fitness for purpose, and free from defects. The level of project objectives achievement can also show project effectiveness. The main priorities of the project objectives achievement are project tasks and criteria, such as meeting project time, budget, and technical specifications. In line with Takim, Ikediashi et al. (2012) mention that performance evaluation compares the actual and expected performance in terms of effectiveness, efficiency, and quality regarding skill and product. Lauras et al. (2010) use project tasks, performance indicators (time, cost, quality, and risk), and performance triptych (effectiveness, efficiency, relevance) to analyze project performance. Effectiveness describes whether the results of the project meet the objectives. Efficiency indicates whether the resources used well to attain the results. Meanwhile, relevance measures the sufficiency of the methods to meet project objectives.
Performance measurement requires an appropriate indicator, which is used as a benchmark or expected performance standard and can describe project performance accurately. Performance indicators must be able to determine the measurable evidence that proves that the planned effort has achieved the expected results (Takim et al., 2002).

Compliance to the Contract
The expected results of the project must be known to be able to measure project performance precisely. Expected project results such as design, work methods, and building/ construction, stated in the contract document. According to Gransberg & Molenaar (2004), the contract is a combination of the RFP and the winning proposal to become the technical definition of the design and construction contract, as shown in Figure2. It means that DB project performance must be seen as a whole process, starting from the pre-tender stage where the project was defined.

Figure 2. Design and Build Contract
Model. (Gransberg & Molenaar 2001, cited in Gransberg & Molenaar, 2004 In DBB, the constructor must deliver the quality specified in the design. While in the DB, the design/ builder must provide not only the construction but also the design. In the DB, a set of design criteria and basic design stated in the RFP (Request for Proposal) describe the project scope. Both design criteria and basic design used as the basis for the design/ builder to prepare a design proposal to be submitted. It is necessary for the owner and the design/ builder to perform surveys and investigations before establishing the design so that it will reflect the owner's objectives and needs.
There are two phases in the DB contract: the design phase and the construction phase. Consequently, the DB project performance consists of design phase performance and construction phase performance. At the design phase, the design/ builder detailed the preliminary design into DED (detailed engineering design). The problem that often occurs in this process is that it does not match field conditions causing design change. It indicates that the survey and investigation conducted, whether by the owner when establishing basic design or by the design/ builder when setting up preliminary design, were not proper. The design change will affect both the design and construction performance. Therefore, the quality of basic design and preliminary design becomes an important variable that affects project performance.
At the construction phase, activities, and resources required to make the design a physical reality are identified. During this phase, some changes are still possible to happen. The main cause of these changes is methods, equipment, and materials chosen that are not suitable for the field condition. This condition can lead to design change. The changes that occurred in the design and construction phase show that ineffectiveness and inefficiency in the project. The more changes occurred, the more ineffective and inefficient the project.
According to Lauras et al. (2010), project performance must take into account the uniqueness of each project. It must consider several universal dimensions of project management, for example, measuring performance in the area of knowledge determined by the PMI (Project Management Institute). In this study, variables cost, time, quality, safety, and environmental performance analyzed to measure DB project performance (Chan&Chan, 2004;Ikediashi et al., 2012;Lauras et al., 2010;Takim, 2002). These variables were reviewed not only at the end of the project but also during the project. Through their progress, the performance of the project can be assessed.

RESEARCH METHOD
The research methodology of this study consists of a literature review, data collection, data analysis, and discussion. The literature review carried out is including books, journals, previous researches, and regulations to provide the source of data in the field of DB contract and DB project performance. The literature on the topic of performance criteria and indicators for the DB project was also analyzed by comparing ideas, opinions, and understandings, drawing a conclusion, and stimulating new ideas. The literature review is used as a basis to build the concept of DB project performance.
Simultaneously, data collected by interviewing experts is including the requirements, performance, and problems that often occurred in the DB projects. The interviews also carried out to identify experts' opinions about the DB project performance concept, factors that affect performance, and the technique/ method to measure the performance. An unstructured interview method adopted for this research. It is the most suitable method to bring some preliminary issues to the surface to determine which variables need further investigation (Lim, 2009). Experts are personnel who were known to have knowledge and experience related to DB project implementation, particularly at the Directorate General of Highways, the Ministry of Public Works, and Public Housing (PUPR). Directorate General of Highways is an agency responsible for planning, building, and managing national road networks in Indonesia. Interviews were conducted through face-to-face conversations and also by telephone due to geographical limitations. It was about 30 minutes each, and a note was taken. A total of eight experts were interviewed during the research.

Design and Build Project Performance Indicators
Based on data analysis and discussion above, the most appropriate way to measure DB project performance is by using the effectiveness, efficiency, and relevance measures. The effectiveness, efficiency, and relevance show the real performance of the project: not only from the final results but also from the whole processes that happen during the project. Thus, the three aspects become performance criteria with the following elaboration : 1. Effectiveness, reflected by the quality of project results 2. Efficiency, assessed by resource usage and 3. Relevance, measured by the objectives/ goals achieved in the project.
Therefore, the indicators divided into three parts: design phase performance, construction phase performance, and overall project performance. Performance indicators are then developed based on performance criteria, variables mentioned in the discussion above, and also expert opinion. Expert opinion is required to decide which process in the project can reflect project performance and how to measure it. There are fifteen indicators generated in this study. DB project performance indicators development is as seen in Figure 3.

Design Phase Performance
The design phase is a phase where the design/ builder detailed the preliminary design into the final design. Indicators must be able to assess the quality of design phase results. The results of the design phase are design drawings (DED) and the method of construction. The quality of the results affected by the quality of the survey and investigation, basic design, preliminary design, resources used (designers). Design phase performance indicators are as follows : 1. Conformity of survey and investigation results conducted by the design/ builder and initial survey results conducted by the owner (to establish basic design).
The basic design is important as it used as the basis for the design/ builder to prepare the preliminary and the final design. The basic design arranged according to field conditions based on survey and investigation. Accurate data about field conditions are needed to get a good basic design. Otherwise, it will cause many changes at the implementation stage. Therefore, surveys and investigation must be carried out well to obtain accurate data. Surveys and investigations not conducted properly will cause many changes in basic design. This indicator is to find out how well the survey and investigation conducted. It also shows whether the owner has prepared the basic design precisely. The more changes and the more significant the changes are, the worse the performance. 2. Conformity of final design and preliminary design submitted by the design/ builder.
The final design may differ from the preliminary design due to field conditions. Too many changes from preliminary design to the final design led to an ineffective project. This indicator will show whether the design/ builder has prepared the preliminary design properly. The more changes that occur, the worse the design performance is. 3. Compatibility of methods specified in the final design and those of the preliminary design.
A method is a crucial element in realizing all planned designs into physical buildings. It compiled at the time the design/ builder was preparing the preliminary design. However, there may be an incompatibility with the field conditions during the design stage so that changes are necessary. This indicator is to find out how many work items experiencing method changes and the effect of these changes on the design. The more the changes in the methods, mainly if the changes cause a design change, the worse the performance is. 4. Conformity of DED and Basic Design.
DED is the most important output of the design phase used as a reference for work execution at the construction stage. This indicator is to find out how many changes occurred from the basic to DED. The more changes that occur, the worse the design performance is. 5. Suitability of the quality of personnel (designers) and the qualifications and performance requirements of the designer. This indicator is to find out the designer's performance by the number of errors in the design documents and how quickly repairs made. It will show not only the performance of the designer but also the relation between the qualifications of the personnel and their performance in design making. The more error and the longer the repairs are made, the worse the design performance is.

Construction Phase Performance
The construction phase is a phase where the design/ builder implements the final design into construction. One of the reasons for using a DB procurement system is to reduce design changes during the construction phase. Therefore, the number of design changes (drawings and methods) that occur during the project represents the project performance. The performance also assessed from the fulfillment of resources such as personnel, equipment, and materials. Construction phase performance indicators are as follows : 1. Compatibility of methods used during construction and those of in the final design. The design/ builder detailed the method at the design phase. However, method changes are still possible at the construction phase regarding field conditions. Method changes will cause many changes in project implementation because it affects the work sequence, work schedule, and completion. Method changes can also cause design changes. If a method change causes a design change, there will be more changes in the project. This indicator is to find out whether the method used is following the method determined at the design stage. It measures the number of method changes that occur and the effect of these changes on the design. The more the changes, the worse the performance, particularly when causing design change. 2. Conformity of As-Built Drawing and DED (dimensions and elevations).
Changes in design are also possible during the construction phase. The differences between As-Built Drawing and DED shows the changes made. The more the changes, the worse the performance at the construction phase. The high numbers of changes that occur also indicate poor design performance. It shows that the design/ builder is not accurate in design making. 3. Suitability of the quality of personnel (builders) and the qualifications and performance requirements of the builders. This indicator is to find out builder performance by the number of errors and defects in the construction and how quickly the repairs made. It will show not only builder performance but also the relationship between builder qualifications and their performance in the construction phase. The more error/ defect and the longer the repairs made, the worse the construction performance. 4. Conformity of materials used during construction and those of the final design.
Material requirements include quality, quantity, and availability. This indicator is to determine whether, during the project, the material meets the specifications, quantity, and material schedule specified in the contract. If one of the three aspects is not met, it will affect project performance. The more aspects that are not fulfilled, the worse the construction performance. 5. Conformity of equipment used during construction and those of in the final design.
The selection of equipment based on the method, field condition, and equipment schedule. Equipment requirements include equipment type, number, capacity, and condition. This indicator is to find out whether the equipment used during construction is suitable for the methods, always on schedule and in good condition. The more aspects are not fulfilled, the worse the construction performance.

Overall Performance
Overall performance is seen from the suitability with the project objectives in terms of quality, cost, and time. Occupational health and safety and environmental impact management are also taken into consideration in determining overall project performance. Below are indicators to assess the overall performance of the DB project: 1. Conformity of project results with the specification. The project considered a success if the project results quality complies with the specifications stated in the contract. This indicator is to determine whether the project meets the quality from the beginning or it needs some improvements before meet the expected quality. The high number of results that do not meet the specifications indicates that there are still poor job performances, both in the design and the construction phase. 2. Accuracy of project execution time to that of the contract.
Completing time is one of the objectives of the project. Therefore, time is one indicator of project performance. This indicator will show the accuracy of project execution time through the progress based on the project schedule (s-curve). The more delays occur, the worse the project performance. 3. Conformity of total project costs (including addendum) with initial costs specified in the contract.
The innovation of the design/ builder is one of the advantages of the DB project. This innovation makes the project more effective and efficient. This indicator is to know whether there is innovation in the project. It shows whether the same project costs or even lower project costs provide better results using the DB delivery system. 4. Compliance of environmental impact management during construction with environmental documents specified in the contract. The environment has become an important issue in construction projects. This indicator determines whether the management of the environmental impacts during construction is following the environmental documents specified in the contract. If environmental documents are implemented well in the project, the project will not harm the environment and the society around the project. 5. Compliance of health and safety management during construction with occupational health and safety documents specified in the contract. Construction works must meet occupational health and safety (OSH) requirements. The application of OSH is an important factor in project implementation because it affects the productivity and effectiveness of the project. This indicator measures health and safety performance during construction through the presence or absence of work accidents and the severity level of accidents that occurred.

CONCLUSION
Based on the research analysis above, several main conclusions can be written as follows.  The DB project performance indicators divided into three parts: design phase performance, construction phase performance, and overall performance, and three criteria: results quality, resource usage, and objectives achieved. Conformity of total project costs (including addendum) with initial costs specified in the contract; [4] Compliance of environmental impact management during construction with environmental documents specified in the contract; [5] Compliance of health and safety management during construction with occupational health and safety documents specified in the contract.

FURTHER RESEARCH
As a follow up to this research, further research is required. Recommendations for future research are as follows.
 Assess DB projects using these indicators to determine the performance of each project.  Analyze the factors that cause a lack of performance in the DB project.  Determine DB project performance improvements needed based on the assessment and the analysis.