Dean Kashiwagi, PhD, John Savicky, Kenneth Sullivan, PhD, Jacob Kovel, David Greenwood, Charles Egbu

Abstract
There are two ways to analyze the problem of construction nonperformance (not on time, not meeting the quality expectations of the owner, and not having cost increase change orders). The first is a project-specific approach, which assumes that the problems are being caused by the uniqueness of each project. This approach stresses: finding solutions in better trained personnel and craftspeople, more standards, construction management, and inspection. The other is a process approach, which assumes that the problems are being caused by the process. Performance-based procurement uses the process approach. It hypothesizes that the current price-based design-bid-build procurement process is inefficient, supports an adversarial environment, is devoid of performance information, is highly inefficient, maximizes management and inspection instead of quality control, and treats highly differential construction products and services as commodities. The Performance Information Procurement System (PIPS) was created to resolve these issues causing construction nonperformance. The results of 380 tests of construction procurement will be assessed in terms of performance. Using case studies from a variety of large clients, the results of performance will be analyzed in terms of on-time, on budget, and meeting customer expectations. Lessons will be drawn on the cost of performance, the minimization of client management and inspection, and the creation of a "win-win" relationship of best value for the owner while concurrently maximizing the contractor profit.

KEYWORDS: performance results, performance based procurement, construction delivery process, Six Sigma process application

Introduction - Construction Industry Performance
For the past twenty years, the construction industry has attempted to improve its construction performance (finishing on-time, minimizing change orders, and meeting customer’s expectations). In both the United States and in the United Kingdom, overall performance has hovered between the 60% - 70% range for owner satisfaction (Egan 1998, Vickers 2000, State of the Construction Industry Report 2000, Post 1998). The performance issues can be summarized by the following numbers:

1. 33% of projects in the US end over budget.
2. 53% of clients in the US do not want to have a relationship with the contractor at the end of construction.
3. Only 68% of clients in the UK would give a 8/10 rating or better on satisfaction.
4. Only 45% of clients in the UK stated that the costs were on target
5. Only 62% of clients in the UK stated that the projects were completed on time.

The construction industry has tried various solutions to improve construction performance. These have included continuous improvement, partnering, business process re-engineering, just-in-time construction, lean construction, prefabricated systems, and long-term partnerships. Although each solution may have improved construction operations, the general problems of nonperformance have persisted.

Influence of the Worldwide Competitive Price Pressures
The construction industry is guided by two major factors: competition and performance (Figure 1) (Kashiwagi, 2004.) In Figure 1, Quadrant III represents the construction industry structure before the advent of the worldwide, highly competitive marketplace. Clients or building owners selected performing designers and contractors. Terms were negotiated, and the construction was completed. Hiring was based on performance before price. These designers and contractors had highly skilled personnel and craftspeople that did their own quality control.

With the worldwide competitive marketplace, clients sought to procure a better value. The intention was to keep high quality, but increase the competition (moving from Quadrant III to II). However, the inability to identify and measure the difference of performance resulted in the awards being price based. Instead of moving from Quadrant III to II, the majority of clients moved to Quadrant I.

Quadrant I is a price-based, commodity environment. A price-based environment is only optimal when the products and services are true commodities. Commodities are described using minimum standards and requirements. The best value is the lowest price. Procuring construction as a commodity forces the contractors to provide the given acceptable performance at the lowest price. The client’s representative (architect/engineer) uses minimum standards to reduce the risk of receiving a lower quality product. Figure 2 shows an example of four contractors, each with different levels of performance ability for a particular project. The specifications (input based specs (USA), not output based specifications as are common in the UK) put forward in the contract documents dictate a specific level of performance quality. To reduce costs and the chance of a successful bid, the contractors with a greater level of ability for the given project (quality, speed, expertise) lower their performance to the level of the specification. Thus, the contractors (and manufacturers providing the construction products/materials) use the minimum expectation as a maximum level (in order to maximize their profits and likelihood of a successful bid). By awarding to the lowest bidder, performance is guaranteed at only the lowest possible level, which maximizes the client’s risk of nonperformance by a contractor. The resulting difference commonly causes an adversarial relationship (Figure 3) where the owners see the stated specifications as a minimum level of quality while contractors (and suppliers, manufacturers, etc.) see it as a maximum level of quality.

Figure 1. Construction Industry Stability

Figure 2. Impact of Minimum Standards on Performance

Figure 3. Owners vs. Contractors: Difference in Objectives

Project Specific Approach: Construction Management and Expertise
The industry has tried to solve the adversarial, high risk environment by hiring experts who uses their expertise to differentiate every type of construction, set technical standards in each specialty; create means, methods, and material specifications; prequalify contractors; and manage, control, and inspect the contractor using technical expertise, thereby theoretically eliminating as must risk as possible. Ironically, their chief weapon, the minimum standards are a major source of risk (Kashiwagi 2004). Offering a higher performance (in a price based, Quadrant I environment) is a noncompetitive practice. This concept helps explain some of the bankruptcies of experienced construction businesses that have been in the industry for over ten years (Construction Chart Book 2002).

When standards are used, it forces the client to inspect in terms of means, methods, and materials (MMM). Minimum standards have no correlation to performance (Lam 2001, Graham 2002, Cash et al. 1993, Stenman et al. 1994, Lounis et al. 1998). Standards and specifications also allow contractors who do not have experience to bid the project. The award to the low bidding, inexperienced low bidder may actually result in a higher project cost. This environment, with it poor performance results, threatens the sustainability of low-bid construction. The risk can also be identified by the high costs to sureties in both the bonding and insurance payouts (SIO 2003, Morgan Insurance 2003, Construction Chart Book 2002, Why do Contractors Fail 2003). Due to these factors, the authors hypothesize that performance has no direct correlation with awarded price. Therefore, if price does not affect specified performance, high performing contractors do not require external management and inspection. High performance contractors will quality control their own work. They minimize the performance risk with expertise and quality control.

The authors propose that by moving to a Quadrant II, performance based environment, the efficiency of the construction process will increase, minimizing performance issues. By hiring experienced personnel and contractors (which need less control, less management, and less inspection), the contractors will maximize their profit, and the owner will get best value. The authors are proposing that the effort to minimize construction performance issues using a project specific approach (Quadrant I) has proven to be inefficient and ineffective. A process based solution (Quadrant II) is required.

Process Based Solution: PIPS
To investigate process based performance procurement in comparison to current low-bid practices, the Performance Information Procurement System (PIPS) was designed in 1991 and has since been tested, modified, and retested 380 times over the past ten years. It is a Six Sigma based process that defines the process, measures the critical elements and the level of risk, analyzes the process using fuzzy logic, forces improvement, and controls deviation through process control (fundamentally Six Sigma’s DMAIC (Define, Measure, Analyze, Improve, Control)).

The ten year, $4.2M research effort has involved the procurement of $230M of construction, and has resulted in over 80-refereed conference and journal papers. The fundamental hypothesis (construction nonperformance is a process based problem) has not been altered over the ten years, even though steps in the process has been improved through trial and error. The hypothesis includes:

Research Hypothesis:
1. Construction performance is mainly a process issue.
2. The critical element is identifying and competing performing contractors.
3. Management/control by the owner should be minimized.
4. An efficient environment will lower cost, deliver best value for the owner, and maximize the contractor’s profit.
5. Risk should be minimized by contractors rather than clients.
6. Prequalification is only used when the process is price based.

Under the above hypothesis, PIPS was formed and is composed of five major filters of procurement (contractor selection) that seek to test the hypothesis. These major components or filters of performance based procurement are:

1. Identification of past performance. Past performance includes frequency of on time completion, minimal change orders, and high customer satisfaction of critical project performance elements (general contractor, site superintendent, project manager, and mechanical, electrical, waterproofing and other critical subcontractors).

2. Project specific capability. This is defined as the capability to identify, prioritize, and minimize the risk of the project in the non-technical terms of cost, time, and quality expectation.

3. Competition based on performance (past performance and ability to minimize risk) and price. The prioritization is done using a multi-criteria decision making model, which minimizes risk of nonperformance by giving credit to the identified critical past performance elements (recorded values of filters one and two). This model does not penalize values which are near the mode, but penalizes values that are below the mode. The processing of values forces contractors to provide their best value, and compete with every other best value, resulting in a two step best value process.

4. Pre-award phase. The best value contractor (as identified by the multi-criteria decision making model) must minimize the risks identified by all competitors. They must coordinate the requirements between critical elements, clarify or seek clarification on the project. The contractor will then sign a contract that includes their risk minimization plan, the intent of the owner, and all clarifications.

5. Construction. The contractor is forced to manage the project in terms of risk. The contractor passes risk information (affecting cost, time, and quality expectation) to the client’s representative.

6. Measurement of performance. The project will be rated after completion. All critical elements of the general contractor’s team will receive the same performance rating. The rating becomes up to 50% of the critical element’s future performance rating.

Testing – Application of PIPS
Testing of PIPS was accomplished via its application on real projects for contractor procurement. Testing has been conducted over a ten year period in the public and private sectors for the following clients: Intel, Motorola, Boeing, Burr-Brown, International Rectifier, Honeywell, State of Wyoming, US Army Medical Command, Federal Aviation Administration, State of Utah, United Airlines, State of Hawaii, University of Hawaii, State of Georgia, Wyoming National Guard, Dallas Independent School District, Denver Hospital Group, Harvard University, and the US Coast Guard.

Over 380 tests and $230M of construction projects were procured using the PIPS system. The overall performance results of PIPS tests were:

1. No evidence that the first cost of the performance-based awards was more expensive than the costs of the low bid award. The Civil Engineering Unit of Oakland, CA of the US Coast Guard (USCG) concluded, via a cost analysis of PIPS awarded projects and non-PIPS awarded projects for the USCG, that PIPS represents a savings as large as 19% for a project’s life cycle costs compared to low-bid or non-PIPS procurement (Rumsey et al. 2004).

2. PIPS showed 98% performance, where performance is given as projects that were delivered on time, with no contractor generated cost change orders after the preaward phase, and high customer satisfaction. This is in comparison to the documented performance of 60 – 70 percent performance of construction in both the United States and the United Kingdom.
3. Increased performance of contractors over time or in comparison with their performance in the low bid environment.

4. Contractors performing to a higher level in the PIPS environment than in the low bid environment. This includes perceived higher performance of the same contractors in the PIPS environment than in the low bid environment.

5. Construction management minimized up to 80%.

6. Minimized means, methods, and material details in design specifications.

7. Risk of designers was minimized due to the two levels of constructability review (business level review in the risk identification filter and detailed constructability review in the preaward phase by the best value contractor).

The first repeat user of PIPS was the FAA Western Region (50 storm damage repair projects ($4M)) and the FAA provided no technical specification to the contractor. PIPS allowed the FAA engineering requirements group to increase the amount of work procured by 300 percent. Projects included building repairs, road repairs, and electrical and mechanical systems repairs. All the projects were finished on time, without change orders, while satisfying the clients (Farnoush, 2002.)

United Airlines (UAL) was the next repeat user of PIPS (results shown in Table 1, where the owner rated certain items on a scale of 1-10 with 10 being the highest (most favorable)). As in the FAA projects, technical specifications were minimized. The process was tested on roofing, painting, waterproofing, flooring, abatement, and renovation/remodeling projects. The speed, efficiency, and minimized effort of PIPS decreased the overhead of construction delivery allowing more of the funding to go into construction.

Table 1: United Airlines Performance Based Results

The State of Utah projects were the first large multi-million dollar projects (6 projects, $80M budget, the largest being $53M Olympic Village, Phase II). Due to the State’s requirements, the projects had to be run without the most critical component of PIPS, the pre-award phase. Even though the capability of the process was limited, PBSRG ran the modified process to determine if PIPS could be used successfully on large projects with multiple subcontractors. The results were the best results (Table 2) at the State of Utah in ten years (Byfield, 2001). Without the pre-award phase, the contractors were not forced to find mistakes in the design documents before construction award. In the only project that was not completed on time or without change orders, the user stated that architect missed too many items for the contractor to cover (Jacobs, 2001). The results reinforced the importance of the pre-award phase. The largest project, the $53M 2002 Olympic Village Housing project was awarded to the low bidder.

Table 2: State of Utah Project Results

The State of Hawaii ran the most projects (over 150) for the longest period of time (4 years) of the ten year test cycle. The test results were captured in the State’s internal audit. The process eventually ended due to a change in political party and the appointment of a new comptroller who wanted to return to the traditional, technical based project approach. It is interesting to note that the State has been unsuccessful in finding a process that duplicates the results of PIPS. Their current inability to identify or use performance information, and the inability to document the performance of construction projects supports the author’s hypothesis that the owner does not know the value of construction in the priced based environment. The analysis of performance in the Hawaii tests were done in several ways:

1. Of the 55 roofing clients, 100% stated that the PIPS contractors’ performance was excellent, 100% stated that they preferred PIPS over low-bid award, and 96% were satisfied with the quality of work (State of Hawaii PIPS Advisory Committee 2002).

2. Out of 20 inspectors, 100% were satisfied with the PIPS work, 94% stated that the PIPS contractors were more willing to perform, and 95% stated that PIPS required less work for their staff (Kenny, 2001).

3. A transaction cost analysis was performed on the roofing PIPS projects and low-bid projects. The analysis concluded that PIPS resulted in over 13% savings in the first cost in comparison to low bid (State of Hawaii PIPS Advisory Committee 2002). This did not take into account the increase in quality.

4. A comparison of 96 PIPS roofing projects documented that (State of Hawaii PIPS Advisory Committee 2002): 98% of the roofs were completed on time, the contractors produced approximately twice as much work per day, and stopped the practice of the State repairing its roofs during the warranty period.

A project run at the State of Georgia was very significant since it allowed a clear comparison of the first cost of running PIPS versus the cost of low-bid on the exact same project. The procurement of the $45M construction of an environmental wet laboratory was bid twice using both processes (Kashiwagi 2003a). The first round of bids was done using the PIPS process. The bid was rejected due to the perceived high cost. It was later identified that the project was over-designed. The project was redesigned cutting $4.5M from the project. It was re-bid and awarded using the low-bid process. The project was still over budget ($46.6M) and completed at $48.8M and late by 50% of the initial construction time. The project was finally completed with over $2.2 Million in change orders and approximately 1 year behind schedule. The State of Georgia ran a second, similar project using PIPS. After awarding to the best performer, they proceeded to manage and control the contractor as though it was a low bid contract. The client was impressed with the contractor but discouraged by the bureaucratic control by the State.

The Dallas Independent School District (DISD) implemented PIPS on nine roofing projects. The implementation illustrated several key concepts, including:

1. Contractors that DISD thought were very low performing (due to past low-bid work), were capable of performing very highly under PIPS.

2. Contractors and manufacturers did not know the performance of their roofs.

3. Contractors and manufacturers immediately began responding to problems that DISD had been requesting to be fixed for over two years.

Table 3 summarizes the results of the PIPS implementation at DISD (Kashiwagi 2003c). The projects were completed on time, and 13 percent under budget. Once again, the first costs were lower for higher performance than the low bid prices.

Table 3: Dallas Independent School District Project Results

Conclusion and Recommendation
The process based approach of PIPS, based upon the nearly 400 projects run using the performance based system, seems to be far more effective in minimizing construction performance issues than the project specific, low-bid approach. The success of the PIPS system shows that the Quadrant I, technically oriented, price based construction delivery process may be the primary cause of construction nonperformance. PIPS is fundamentally a Six Sigma application that defines the process, measures performance at the right time by the right party, automated the analysis process using fuzzy logic, forced continuous improvement without management and control, and minimized deviation through the process control (DMAIC.)

Performance Based Studies Research Group, Arizona State University, P.O. Box 870204, Tempe, AZ 85287-0204 USA
 Central Connecticut State University
Northumbria University, UK
Glasgow Caledonian University, UK

References
Byfield, R.E. (2001) Letter to Gordon Matsuoka of State of Hawaii, Division of Public Works from DFCM Director, 12 February.
Cash, C. and Kan, F. (1999) On the Study of Crack-Initiation Fracture Toughness of
Fiber Glass Asphalt Shingles. Roofing Research and Standards Development Fourth
Volume, ASTM STP1349, 132-151.
Construction Chart Book. (2002) The Construction Chart Book: The U.S. Construction Industry and Its Workers, 3rd edition.
Deming, E. W. (1982) Out of the Crisis. Cambridge: Massachusetts Institute of Technology.
Egan, J. (1998) Rethinking Construction: The Report of the Construction Task Force to the Deputy Prime Minister, John Prescott, on the scope for improving the quality and efficiency of UK construction. The Department of Trade and Industry. 16 July, URL: http://www.dti.gov.uk/construction/rethink/report/index.htm
Graham, M. (2002) NRCA's Performance standards initiative. Professional Roofing, 32 [9], 80.
Jacobs, T. (2001) Evaluation of the Gunnison Correction Facility Project. Letter to John Savicky, Research at PBSRG, 11 June.
Kashiwagi, D., and Savicky J. (2003a) State of Georgia Test with Performance Information Procurement System (PIPS). 31st Annual Conference of the Canadian Society for Civil Engineering (CSCE) 2003; Moncton New Brunswick, CD: GCC-311.
Kashiwagi, D., Darshit, P. and Savicky, J. (2003b) The Impact of Minimizing Specifications and Management at the University of Hawaii. Journal of Facilities Management, 2 [2], pgs. 131-141.
Kashiwagi, D. and Savicky, J. (2003c) Case Study of the Implementation of the Performance Information Procurement System at the Dallas Independent School District. ASC International Proceedings of the 39th Annual Conference, Clemson, SC, 315-323.
Kashiwagi, D. and Savicky, J. (2004) Transforming Minimum Standards & Requirements into Performance Based Requirements. ASHE PDC 2004 Conference, Tampa, FL, March 15.
Kenny, R. (2001) Implementation of the PIPS Within the State of Hawaii for Roofing Projects. Master of Science Thesis. Arizona State University, Tempe, AZ, USA.
Lam, P., Kumaraswamy, M. and Hg, S. (2001) The Multiple Roles of Specifications in Lean Construction. Lean Construction.
Lounis, Z., Lacasse, M., Vanier, D. and Kyle, B. (1999) Towards Standardization of Service Life Prediction of Roofing Membranes. Roofing Research and Standards Development 4th Volume, ASTM STP1349, 3-18.
Moran Insurance (2003) Surety Bonding: The Importance of Surety Bonds in Construction. MORGAN Insurance. 15 October,
URL http://www.moraninsurance.com/surety.html
Post, N.M. (1998) Building Teams Get High Marks. Engineering News Record, 240 [19], pg 32-39.
Rumsey, N., Dempsey, J., Westling, J., Lewis, P., and Riggs, W. (2004). CEU Oakland Quick Study, ISC Alameda Roofing. United States Coast Guard – Civil Engineering Unit. August 2004.
SIO (2003) Why do Contractors Fail? Surety Bonds Provide Prevention & Protection. SIO Surety Information Office.
State of Hawaii PIPS Advisory Committee (2002) Report for Senate Concurrent Resolution No. 39 Requesting a Review of the Performance Information Procurement System (PIPS).
State of the Construction Industry Report (2000) Construction Industry Board. [11], May, URL:http://www.dti.gov.uk/construction/stats/soi/soi11.htm.
Stenman, H., Mech, M., Paroli, R., Lei, W. (1994) Maximum Tensile Load and Elongation Properties of Heated Built-Up Roofing Membranes at Selected Cold Tempatures.” Roofing Research and Standard Development, 3rd Volume, ASTM STP 1224, 78-86.
Vickers, C. (2000) Report for the 2000 Annual Consultative Conference. Construction Industry Board, May edition.

Published In:
11th Joint Symposium: Combining Forces –Advancing Facilities Management and Construction through Innovation, Helsinki, Finland, 172-182 (June 13, 2005).