A Private Organization Utilizes the Best Value Approach on an Enter- prise Resource Planning System

Article history Received: 03 April 2019 Accepted: 04 December 2019 Published: 30 January 2020 The Best Value Approach (BVA) is a new project delivery method that has been developed at Arizona State University. It has been documented to increase performance and value on projects by the identification and utilization of expertise instead of management, direction, and control (MDC). It utilizes performance information that is simple, observable, and countable. It allows the expert vendor to know what the client project requires, why they can achieve success and what they will do before they do it. The tracking of the project cost and time deviation requires an initial plan and method to track it. Preliminary results of the BVA have shown a 90% decrease in effort by client organizations, 98% customer satisfaction and has led to 1% vendor cost and time deviation rate. It applies to construction, services/IT projects, and any long-term service. In 2014, a large private organization having difficulty delivering information technology (IT) and construction/facility services identified the BVA as a potential solution. This paper will summarize a major IT Enterprise Resource Planning case study that the large private organization used the BVA on and identify the full results.


Introduction
I n 2014, a Large Private Organization (LPO) was having difficulty delivering two types of projects: information technology (IT) and construction/facility services. The organization had recently tried to deliver an enterprise resource planning (ERP) software platform upgrade for the entire organization but was not successful. The organization ended up spending a year and $3M+, trying to work with a vendor to reach an agreeable plan and specifications, only to find out their expectations could not be met. The project was stopped, and the purchasing of the service was postponed.
The LPO was using a traditional process to deliver its IT services. This model required them to create technical specifications to relay the requirement of the service to the vendors. Since most of the time the LPO did not have expertise in the service, the process required them to use time and resources to hire an IT consultant to help them create the specifications to deliver the service.
The traditional approach to delivering services has not had a good past performance history. The documented performance of the service industry has had low performance (in terms of on budget, on time, with high customer satisfaction) (Deming, 1982;Egan, 1998;Kashiwagi, 2009 [12] Organizations are continually trying to find different methods that ensure they will receive high performing services. A recent literature search was performed, as part of a Ph.D. student's dissertation (Rivera, 2017), [14] to verify the poor performance of services. The study reviewed over 208 publications from 6 major research databases. Thirty-six of the publications had documentation of performance in terms of cost and schedule overrun, customer satisfaction and quality. Table 1 identifies six major industries' performance. The literature verified the low performance of services and identified that despite the differences in technical difficulty of each industry, the performance levels were still similar. Like many other organizations, the LPO began looking for a way that minimizes their issues and failures in delivering IT services. In 2015, the Director of construction/facility services reached out to the Performance Based Studies Research Group (PBSRG) to train the organization on the Best Value Approach (BVA). The Director had learned about the BVA in conferences and identified it as a potential solution to the organization's issues. After the organization was educated on the approach and identified its performance results, the organization was interested in using the process to try to re-deliver its ERP software upgrade.

Best Value Approach (BVA)
The Best Value Approach (BVA) was derived from the industry structure model (IS) (see Figure 1). The IS model splits the industry up into two main quadrants, the Value-Based quadrant that has high competition and performance and the Price Based quadrant that has low competition and performance. The model identifies that low performance is caused due to buyers trying to manage, direct, and control (MDC) vendors. The only way to move to the Value-Based quadrant is to utilize the expertise of the vendor, by moving the management and control of the project to the expert vendor.  Figure 1. Industry Structure Chart The IS model identifies the following buyer traditional activities that are used to MDC vendors (Kashiwagi, 2018; PBSRG, 2018): [11] [12] • Creating technical requirements and specifications.
• Partnering and developing relationships with vendors to enable the client to be involved with the management and development of the service.
• Using the contract as leverage over the vendor.
• Using a project manager to manage a vendor after they were awarded a contract.
The IS model also identifies that the following activities will enable buyers to utilize the expertise of vendors: • Minimize involvement in technical details of services.
• Move buyer activities to that of quality assurance (ensuring the vendor has created a plan and is measuring their performance through non-technical metrics) instead of quality control (ensuring the vendor is performing all of their technical work correctly).
• Require vendors to tell the client what the technical specifications and requirements should be.
• Utilize internal buyer personnel to help and protect the vendor.
The BVA was developed to help buyers to understand and move to the Value-Based quadrant and perform the activities that enable them to utilize the expertise of vendors. The BVA splits a project up into three major phases (selection, clarification, and execution) (see Figure 2): Selection Phase: Vendors will compete based on their level of expertise instead of their technical scope of work. During this phase, the vendors are not given technical requirements or specifications, but a list of expectations and explanation of "what the client thinks they want". They are selected upon their past performance metrics, ability to identify risk, and capability of their key personnel. The vendor that is highest ranked moves into clarification.
Distributed under creative commons license 4.0 Clarification Phase: This is the most important phase, as the vendor with the highest level of expertise now is required to create their scope of work and technical requirements. They are also required to explain how they will accomplish the work efficiently and with high customer satisfaction. They are required to identify their plan for beginning to end, all risks that they do not control, all major milestones, how they will measure their performance, and justify their costs. During clarification, the client will express their concerns and feedback about the contractor's plan and the contractor has to address those concerns in their plan. Regardless, if the concerns from the client are technical or non-technical, the vendor is required to resolve the concern using non-technical language. The contract is only signed when the client is totally comfortable with the contractor's plan, otherwise, the contractor will be eliminated from clarification and the next in line vendor will be notified for clarification.
Execution Phase: Upon signing the contract, the contractor can proceed to work according to their plan. Since the vendor was the entity that developed the plan and the metrics, it has now put them in full control of the project. Performance will be tracked and posted online for each contractor through Weekly Risk Reports (WRR) the contractor will turn in on every Friday. If ever another stakeholder tries to control the expert that is also reported on the WRR and the vendor identifies what the impact that control will have on the project's performance.  Many of these ideas are different from traditional delivery models. However, the LPO was convinced that the concepts were accurate due to the performance of the BVA system which includes the following (Rivera, 2017; PBSRG.com, 2018): [14] [13] • 2000+ projects and services delivered (construction and non-construction).
• $6.6B of projects and services delivered with a 98% customer satisfaction and 9.0/10 client rating of process.
• Services delivered: construction, facility maintenance, IT, professional [design], redesign of systems and organizations and supply chain applications.
• $17.6M in research funding generated, due to the effectiveness of decreasing buyer cost of services on average by 31% [57% of the time, the highest performing expert was selected and was the lowest cost].
• Contractors/experts could offer the client/owner 38% more value and decreased client efforts by up to 79%.
• 90% of all project cost and schedule deviation is caused by the owner's non-expert stakeholders.
• Change order rates were reduced to as low as -0.6% (Rivera, 2017). [14] • CIB W117 has worked with over 123 unique clients [both government and private sector] and received 12 National/International Awards.
• 5 to 30 percent of cost savings are achieved on the projects.
• The BVA is the most licensed technology to come out of Arizona State University licenses [54] .

Problem and Proposal
The Best Value Approach (BVA) proposes that the reason the Large Private Organization (LPO) was having difficulty in delivering high performing services, was due to their use of a traditional process that required them to manage, direct, and control their vendors, instead of utilizing their expertise. Performance-Based Studies Research Group (PBSRG) proposed that in using the BVA, the LPO would no longer have to perform MDC activities and would be able to begin utilizing the expertise of the vendors. Using the BVA the LPO would not only begin to see the performance of services go up but also would see that the cost and time to implement services would decrease.
A study (Kashiwagi, 2013) [10] was performed comparing the BVA with traditional delivery systems and it identified that not only did performance increase, but the cost went down and the value the buyer received went up (see Figure 3).

Methodology
The Large Private Organization (LPO) agreed to the implementation of the BVA and agreed to the following steps: (1) Educate their personnel on the BVA.
(2) Use the BVA on implementing the ERP software upgrade service.
(3) Document and analyze the project and its results.

Large Private Organization Enterprise Resource Planning Software Upgrade Service
The enterprise resource planning (ERP) service was led by the Large Private Organization's (LPO) procurement group in the Human Resources Department. The ERP would affect every area of the organization, as it would be changing the way all 4,000 employees would track their work hours, receive payment, and work with the LPO's electronic business processes. The ERP would replace its current legacy IT software platform. The first step in doing this was to educate all the LPO's upper management personnel that was included on the core team. Many of the personnel had their disagreements with minimizing the management, direction, and control of the vendors, but in the end, all agreed to follow the process.
The first step was developing the scope of work without using technical requirements or specifications.
The following was what the LPO finally agreed to publish as the SOW: The intent of the overall project is to provide Large Private Organization (LPO) with a Human Capital Management, Payroll, and Time/Attendance system. The system will replace the current systems, which are either out or soon to be out of support and compliance and will need to integrate with applications that LPO will maintain related to HR, Payroll and Time/Attendance.
The Scope of Work to be considered in your proposal includes both: Product Solutions (software, hardware, ongoing support, maintenance, and upgrades), and Consulting and Project Support (business process design, system integration design, development, testing, and implementation, technical support, technical and end-user training).
A. A cost-effective integrated Hardware/Software solution for delivery of core HR including benefits enrollment and integration with third-party providers, Payroll, and Timekeeping activities. Desired solution will provide data integrity, positive user experience, data analytics, compliance, risk mitigation, and efficiencies. It will also enable LPO to meet its complex business requirements (ie. multiple jobs, mixed FLSA types, multiple payment types and pay rates, multiple managers and approvers, labor allocation, reporting, labor laws, teacher contract pay, etc.) B. Demonstrated ease of integration of related HCM content and activities (Value Adds) including compensation, benefits, talent management, recruitment, and learning management.
C. Evaluation and estimate of "cost of ownership" for your proposed solution, including hardware/software purchase and licensing, ongoing costs for maintenance and support, and estimated support needs (LPO staff resource and non-payroll cost needs) from LPO and Partners. Costs for updates, upgrades, maintenance, security, and customizations. Provide a 5-year cost of ownership projection based on LPO employee levels (approx. 4200 employees with annual turnover approx. 13%).
D. Evaluation of existing related best practice business processes and technical support to update and redesign these processes as necessary to ensure data integrity, positive user experience, integration, compliance, and efficiencies aligned with best practices.
E. Technical support, coordination, and evaluation of system implementation and testing including SIT and UAT testing of all processes and interfaces. Development and availability of test environments.
F. Consultation, advice, and collateral material related to change management and adoption of new systems/ processes including communications plans, templates, and evaluation, development, and design of training for LPO technical users and end-users.
G. Maintenance: Provide a recommended plan which outlines ongoing maintenance requirements, including updates and upgrades for the system going forward. This was extremely different than what both the buyer and the vendors were used to seeing. Many of them questioned why more information and explanation was not provided. The response given to them was, "you tell us what should be required and what would be best to receive".
This enabled a request for proposal (RFP) to be created within 30 days compared with the previous attempt that took 1 year. The rest of this section will review all the major phases of the ERP software upgrade BVA project.

Selection Phase
On 8/17/2016, the LPO released the request for proposal and received six responses. Table 2 shows the evaluation scores. Contractor A was the lowest price ($2.9M from the most expensive and $53K from the second-lowest), and highest prioritized vendor. In the selection phase, no technical details were discussed, but the vendors were required to show their documented past performance, identify the major risks that the project could encounter, submit options for anything they thought could add more value to the buyer that no one else could offer, and price. The top three submittals' teams were brought in for interviews. The interviews only asked high-level questions and did not go into the details of the vendor's offers. A selection committee of three persons provided the ratings. Table 2 shows the total scores (out of 100). The highest-ranked vendor (Vendor A) was also the lowest cost. The selection was simple and took no decision making from the team.

Clarification Phase
Vendor A was allowed to advance into the clarification phase. The main purpose of the clarification phase are as follows: • Ensure the vendor is an expert by requiring them to create the technical requirements, create a simple plan that resolves any concerns from the buyer and shows the buyer how they will be able to know the vendor is delivering quality service throughout the entire project.
• Resolve any inaccurate buyer expectations.
• Ensure all parties are informed and accountable for their part in the implementation of the service.
They were expected to develop a complete technical scope of work and pre-plan the entire project before they could receive a signed contract with the Large Private Organization (LPO). Their deliverable for the Clarification Phase was called clarification documents (full plan). It included the following: • Scope of work.
• Assumptions and Resource Breakout.
• Risk management plan. After Vendor A created the first draft of their clarification documents, a meeting was held with the client and the following issues were identified: • Plan identified multiple testing strategies, which would identify if the enterprise resource planning (ERP) system is working, with no explanation of how it will be conducted upfront prior to award.
• Plan did not identify all resources and expectations from the LPO in order to bring the project to completion.
• Client was confused and did not know how to proceed.
Contractor A initially had a difficult time laying out the entire plan for the client. They were used to the traditional process of the client telling them what the schedule should be, what meetings and communication were required, and to figure out who was responsible for what after a contract was signed throughout the entire project. The following are examples of how by requiring the vendor to lay out a plan, it resolved many issues before the contract was signed, and ensured the project would be successful by allowing the expert vendor to determine what should be done: (1) First, the vendor had to clarify the scope of work with the client. Figure 3 shows the original scope of work submitted. After review, the client did not understand at a high level what was being delivered, the cost and time requirement, and which stakeholder would be responsible for all the major parts of the project. The vendor eventually clarified this information to the client and helped the client to understand the major deliverables (see Figure 4 and Table 3) steps the vendor would make to finishing the service.
(2) Second, the vendor had to clarify how many resources and how much time they would need to spend on the project to enable the vendor to deliver the service correctly. Figure 5 is what the vendor initially submitted. It was an 855-line detailed schedule of activities. After the detailed task line items were simplified, the vendor was able to break the project down into major phases (see Figure 6), general assumptions (see Table 4), major resources associated with hours and a schedule of when the resources are expected (see Figure 7 and 7a). When completed, it helped to ensure the client and the vendor had the right expectations and assumptions of what would happen during the contract to minimize any surprises.
(3) Third, the vendor had to clarify when and how much they would bill the client. Table 5 shows the original price schedule submitted. It was unclear to the client what was being billed, when and for how much. Table 6 shows the adjusted price schedule. The client was able to identify how much the vendor was charging for each deliverable and ensure they were comfortable with when the vendor would expect payment.
(4) Fourth, the vendor had to clarify their schedule. Table 7 shows the vendor's milestone schedule. It did not help the client understand what major activities were to be conducted and major client and stakeholder action items. Table 8 shows the adjusted milestone schedule. It helped the client to see the major phases of the project, major activities and client and stakeholder action items all associated with dates.
(5) Finally, the vendor had to clarify their risk mitigation and management plan (RMP). Table 9 shows the initial RMP submitted. It did not have any metrics to identify the cost and schedule impact if one of the risks occurred. The client was unable to prioritize which risks were more likely and critical. Table 10 shows the adjusted RMP submitted. The adjusted RMP identifies the risk, vendor's plan of actions to mitigate or manage the risk, their client assumptions and cost and schedule impact to the project.
(6) The vendor clearly identified how they would measure the performance of the project before the contract was signed which enabled the client to know exactly what information the vendor would collect, report on throughout the end of the project, and how they would determine when the project was successfully completed.
All these steps completed by the vendors, helped the client to resolve all of their concerns and issues with the vendor's proposal, which led to a contract being signed.     Table 4. General Assumptions

General Assumptions Client Questions/Concerns Vendor Response
Professional Services in this SOW will be performed at ~70% offsite and 30% onsite at a client location.
-Vendor's breakout of presence by major activities?
-How do we know this is the right approach?
-What other off-site tools [besides emails and phone] will be used to communicate?
Vendor will provide presence by major activities for each stage/role.
Typical approach for commercial side is 80% offsite and 20% onsite.
Additional offsite tools: WebEx, Skype, internal collaboration tool.       1. Vendor will document impact in WRR.
2. Vendor will provide dominant information to client.

Execution Phase
After the contract was signed. The vendor then carried out the plan that they had created in the clarification phase. Each week the vendor reported on their performance and sent a simple report out to all the key stakeholders to ensure everyone understood where the project was at. The report was in the form of an excel spreadsheet, which measures the final schedule and cost from its baseline, to identify the differential and who was responsible for it.
The following tables show the key sections of the report. Table 11 shows the first major section of the report, which records the baseline cost and schedule the report will measure from.  Table 12 shows the milestone schedule. The vendor was required to layout their plan from beginning to end using major activities with assigned dates. To assist in making the schedule simple, the milestone schedule includes all stakeholder activities. If a milestone deviates from its baseline, a deviation number (Dev #) is assigned to it. The Dev # correlates to the line item that the deviation is explained in the deviations section (see Table 13).  Table 13 shows the deviations section. Each time a milestone did not meet the baseline schedule and caused a deviation beyond the end completion date, or an additional cost was incurred, it is recorded in the deviations report. In addition to the cost and schedule impact recorded, the entity responsible is identified. At the completion of the execution phase, the project was able to be completed on time and on budget. The customer was extremely satisfied, and the internal project team could not believe how well the project went. There were no major issues that occurred on the project. Table  14 shows the final reported performance.  The LPO also found that the following characteristics are required in order to enable the utilization of expertise: (1) Transparency -The only way to utilize the expertise of the vendor and for the buyer to allow the vendor to take control of the project is if both sides are completely transparent and provide all the information and supporting documentation for all of the work they do.
(2) Simplicity / Non-Technical communication -The only way the buyer will feel comfortable enough to enable the vendor to take control of a project is if they can understand exactly what will happen and why the vendor is doing what they are doing. In order for a process to be efficient all participants, must also have the right expectations and know what their responsibilities are. This can only happen if everything communicated is clear and simple.
(3) Measure -The buyer and vendor will have no way of knowing if the service was successful and the value it produced unless clear metrics are in place ahead of time that all parties agree upon. In the BVA, a vendor is not hired to complete a set of technical requirements, they are hired to accomplish a certain level of performance.

Implications and Limitations
The LPO identified that the BVA decreased the cost and time of delivering services, while increasing the performance of the services. The implications of this study are as follows: (1) Utilizing the expertise of the vendor can create transparency and develop a contract that minimizes risk and measures performance.
(2) Transparency holds all project stakeholders accountable for the decisions they make and work that they perform.
(3) Performance information is more effective at differentiating and selecting vendors than technical information on the service.
(4) Requiring a vendor to have a plan that can be explained in non-technical terms and identify how they will measure their performance before their contract is signed will mitigate risk and increase the quality of the service.
(5) The client is the main cause of project cost and schedule deviations.
(6) The Best Value Approach is not limited to delivering IT projects. It can be used on many types of services.

The limitations of this study are as follows:
(1) This research was not able to document the satisfaction of the delivered services for the entire length of the service. Further analysis of the client's satisfaction may provide more insight on the actual performance of the delivery of services.
(2) Greater documentation of the performance of the LPO's traditional approach for delivering projects can give a more accurate view on the differences with the Best Value Approach.