Analysis - Vendor Performance: A Shared Responsibility

Ottawa
January 2014

Promoting Fairness, Openness and Transparency in Public Procurement

Introduction

  1. In March 2013, the Office of the Procurement Ombudsman (OPO) analyzed some of the measuresFootnote i used by certain jurisdictions—Canadian and international—to manage the performance of suppliers. The analysis focused on certain measures implemented by the Government of the United Kingdom, the Government of the United States and the Government of Ontario.
  2. The purpose of this analysis is to inform ourselves and share information on some of the different measures used to address vendor performance, as well as their areas of convergence.

Background

  1. A number of OPO's stakeholders have raised concerns with the office about vendor performance. In his 2011–2012 Annual Report, the Procurement Ombudsman recognized the frustration of suppliers and public servants who saw firms obtain contracts "despite being known within the community for providing inferior goods and services."Footnote ii
  2. In 2010, OPO published a study on measures used to manage vendor performance. This study, conducted in fiscal year 2009–2010, examined programs set up by federal government organizations and other jurisdictions (e.g. the United States, Ontario, Newfoundland and Labrador); and identified best practices. The study noted the implementation of vendor performance programs is key to the proper management of the procurement process: these programs support the accountability of organizations by promoting an effective management of the contracting process and risks associated with contract administration.Footnote iii
  3. Since the study was published, OPO became aware of new initiatives for managing vendor performance. Specifically, in 2012 the Government of the United Kingdom began documenting the performance of certain suppliers and applying corrective measures on an as-needed basis. In Canada, similar activities have been carried out by Infrastructure Ontario since June 2010. These activities are in line with initiatives in the United States, which were highlighted in the 2010 OPO study. In addition, in January 2013 the Government of the United States published a series of standards and procedures for making "determinations of responsibility" which has an impact on the management of vendor performance.
  4. It is in this context that OPO conducted this analysis. This report is intended to provide the opportunity to stay informed on how other jurisdictions manage vendor performance, a subject of interest to OPO and its stakeholders.

Methodology

  1. OPO analyzed examples of measures put in place by the governments of the United States, the United Kingdom, and the Province of Ontario for managing vendor performance. For each measure analyzed, OPO sought to determine:
    1. What is the strategic objective behind the management of vendor performance (risk mitigation, accountability, etc.)?
    2. On what basis is the supplier's performance evaluated?
    3. How is a supplier's (un)satisfactory performance managed (sharing of information, corrective measures, feedback, etc.)?  

Limitation

  1. The vendor performance management measures were analyzed using publicly available information,Footnote iv and this information was sourced from public websites. The analysis does not include all vendor performance management measure of all jurisdictions nor does it contain a detailed description of each measures implemented by the governments in question. In addition, OPO did not look at best practices in vendor performance management, as this was done in the study published in 2010. 

Summary of Findings

  1. The measures examined in the analysis differ in their scope and application. Nonetheless, the analysis revealed three areas of convergence that, when considered together, offer a potential lens through which to view various vendor performance management measures. Each area of convergence corresponds to one of the questions in the Methodology section.

Question 1: What is the strategic objective behind the management of vendor performance?

  1. In the first area of convergence, all of the vendor performance management measures analyzed fall within a larger frameworkFootnote v that seeks to:
    • Hold the participants (suppliers and public servants) in the procurement process accountable; and
    • Identify the minimum standards of reliability when choosing a supplier.
    Governments seem to have implemented these frameworks to mitigate certain risks associated with spending public funds through contracting.

Question 2: On what basis is the supplier's performance evaluated?

  1. In the second area of convergence, the frameworks—which include vendor performance management measures—typically use two approaches to help determine a supplier's reliability:
    • Most governments take the supplier's integrity into consideration. This characteristic is usually evaluated against extra contractual factors, such as whether the supplier has a criminal record or a history of fraud; etc.
    • Governments consider contract performance history. With this approach, reliability is based on the likelihood of the supplier fulfilling its contractual obligations.
  2. Although these two approaches are often used simultaneously (for example, before the awarding of a contract), it is contract performance history that relates directly to the management of vendor performance. This distinction illustrates the complexity of the frameworks aimed at mitigating some risks associated with management of government procurement.
Question 3: How is a supplier's (un)satisfactory performance managed?
  1. Finally, in the third area of convergence, the analysis revealed three components that seemed representative of the measures analyzed:
    • In all cases, a record was created for the purpose of making supplier performance history available to procurement officials. In general, procurement officials must consult the record prior to awarding a contract.
    • In all cases, a history of unsatisfactory performance may result in corrective measures. Although there are a number of such measures, each government considered the temporary denial of contracts as a legitimate corrective measure.
    • The temporary denial of contacts was generally not considered the ultimate purpose of the vendor performance measures analyzed. Most of the documentary sources suggested the main goal was to improve the communication/relationship between purchaser (i.e. government organizations) and supplier. Accordingly, vendor performance management measures were used mainly to provide feedback, rather than simply to penalize underperforming suppliers.

Analysis

United Kingdom

  1. On November 8, 2012, the Government of the United Kingdom published a Procurement Policy Note (PPN) entitled "Taking account of bidders' past performance."Footnote vi
  2. The scope of the PPN is limited to contracts for goods and services relating to 1) information and communications technology, 2) facilities management or 3) business processing outsourcing with a total value of ₤20 million or greater.Footnote vii
  3. The purpose of the PPN is to set out minimum standards for reliability when selecting a supplier.Footnote viii Reliability is assessed in accordance with past performance. Once a contract is complete, bidders who have delivered satisfactory performance receive a certificate of performance. A copy of the certificate is sent to the Cabinet Office,Footnote 1 which has a central repository of certificates and information relating to suppliers. This repository is available to all government organizations in the United Kingdom and can be used to verify the information provided by bidders to show their reliability.Footnote ix Agencies from the Government of the United Kingdom can ask suppliers for information on their performance in the previous three years.Footnote x
  4. The PPN states that the application of the minimum standards for reliability is only one aspect of the overall assessment of the suitability of bidders in any procurement. The PPN also recognizes the importance of considering the technical or professional ability of potential bidders.Footnote xi Though the PPN states that identifying bidders' technical and professional abilities ensures programs are delivered cost-effectively,Footnote xii it also notes that assessing the reliability of bidders would ensure better contract performance as well as increased protection for taxpayers and recipients of public services.Footnote xiii In this context, the measures set out in the PPN appear to be an attempt to assign accountability not only to bidders, but also to government authorities.

United States

  1. The measures taken by the United States to manage vendor performance appear to have been implemented primarily to hold public servants accountable for the spending of public funds. In March 2009, the President of the United States published a memorandum addressed to the heads of executive departments and agencies stating:
    • The Federal Government has an overriding obligation to American taxpayers. It should perform its functions efficiently and effectively while ensuring that its actions result in the best value for the taxpayers.Footnote xiv
    The memorandum asked the heads of executive departments and agencies to identify public funds that were being wasted as a result of inefficient contracts, in order to improve overall procurement practices.
  2. In July 2009, the United States Office of Federal Procurement Policy published a second memorandum in which it suggested that greater and more effective use of contractor performance evaluations is essential to meeting the goals set out in the March 2009 memorandum: "Holding contractors accountable for the past performance is an important tool, for making sure the Federal government [of the United States] receives good value from its contracts."Footnote xv This second memorandum states that the management of vendor performance in the United States was fragmented into different measures specific to different government agencies.
  3. A series of requirements was put in place to address this situation. Primarily, further to a change made to the Federal Acquisition Regulation (FAR) in July 2009, agencies are required to submit an electronic record of contractor performance in the Past Performance Information Retrieval System, a single government-wide repository for contractor performance information.Footnote xvi The FAR also requires agencies to establish internal procedures for collecting and reporting past performance information and to provide reports on these procedures.Footnote 2
  4. Finally, under the FAR, no contract may be awarded before the contracting officer has made a "determination of responsibility".Footnote xvii This determination is similar to the evaluation of supplier reliability that takes place under the PPN in the United Kingdom. In January 2013, the United States Government published a reportFootnote 3 identifying the legal procedures and standards required to make a determination of responsibility.
  5. A contractor's responsibility is determined by seven criteria, which must be applied even if they are not included in the solicitation.Footnote xvi For example, this determination is used to evaluate the contractor's integrity and ethics (criterion 4) as well as certain collateral requirements (criterion 7). The collateral requirements ensure that the successful contractors meet certain socioeconomic targets provided for in legislation (e.g. employment equity).Footnote xix Footnote 4
  6. Performance history (criterion 3) is therefore only one of the components to be taken into account when selecting a contractor. Furthermore, "poor performance or default on one or several prior contracts is not, per se, sufficient ground for disqualification."Footnote xx Therefore, in practice, a negative evaluation for one of the seven criteria, including past contractual performance, does not automatically disqualify a contractor.
  7. Ultimately, authorities in the United States recognize that the application of these seven criteria—past contractor performance included—addresses only one of the parameters used to select a supplier: the determination of [the supplier's] responsibility. As stated in the July 2009 memorandum, adding this step to the bid evaluation phase was meant to increase the productivity of the United States government procurement. Therefore, the existence of a responsibility determination does not absolve United States agencies from considering the value of the bids:
    • As a general rule, government agencies contract with the lowest-priced (or best-value) qualified responsible bidder or offeror. Responsibility is an attribute of the contractor, while price and qualification are attributes of the bid or offer.Footnote xxi

Province of Ontario

  1. The Government of Ontario does not have a government-wide vendor performance management program. The ministries responsible for acquisitions in the Government of Ontario appear to be individually responsible for managing the performance of their suppliers. Because of this, the Government of Ontario has a number of targeted supplier performance management programs.
  2. For example, the Ontario Ministry of Finance, through its Broader Public Sector Supply Chain Secretariat, has published a document containing procurement standards for Ontario hospitals. One of the standards in this document concerns the process for managing supplier performance.Footnote xxii Similarly, Infrastructure Ontario (IO), which is responsible for real estate management for the Government of Ontario, has developed the Vendor Performance Program (VPP), which imposes the minimum rules for managing the performance of vendors hired by IO.Footnote xxiii For the purposes of this analysis, OPO has focused on the VPP.
  3. The VPP has a limited scope. It applies to real estate management contracts where the price exceeds $100,000Footnote 5 and is intended for use only by IO procurement officers and project management service providers hired by IO.Footnote 6
  4. The purpose of the VPP is "to ensure the responsible management of IO contracts and enhancement of the value of public sector real estate and services."Footnote xxiv To achieve this, the VPP has seven objectives:
    • Establish a standardized framework for performance evaluation;
    • Ensure proper documentation of vendor performance;
    • Evaluate vendors' performance against the contractual requirements based on the principles of "quality," "partnership" and "value for money";
    • Establish clear expectations for vendors;
    • Recognize and reward top-performing vendors;
    • Encourage continuous vendor improvement through appropriate corrective action; and
    • Apply appropriate sanctions to address shortfalls.Footnote xvi
    In short, IO appears to have used its VPP to develop a formal feedback procedure to encourage continuous improvement of vendor performance.Footnote xvi The analysis of the terms and conditions of the VPP presented below confirms this. 
  5. Vendor performance is evaluated at the end of the contract by means of a previously created scorecard. The scorecard allows the evaluation of a vendor's performance on a scale of 1 to 5 according to three parameters: 1) the "quality" of the work, 2) the "partnership" between the vendor and the client; and 3) "value for money."Footnote xxvii All evaluation scores for a given vendor are combined to create a performance rating; this rating is the vendor's average score for the previous three years.Footnote xxviii When a call-up is issued, the rating is used to evaluate the vendor submissions that have met the mandatory and technical criteria.Footnote xxix The minimum weight assigned to a performance rating in a submission is 10% and cannot exceed 25%.Footnote 7
  6. In addition to the scorecard, the VPP requires vendor performance to be evaluated throughout the term of the contract. In the event of minor performance deficiencies (e.g. failure to provide milestone schedules in a timely manner), the vendor is issued a notice and the opportunity to address and improve the situation.Footnote xxx If the situation is not corrected, an infraction report is completed. This report contains all the information pertaining to the breach of contractual obligations, the corrective action taken and the consequence if the situation is not corrected.Footnote xxxi In the event of major performance deficiencies (e.g. failure to comply with the terms and conditions of a contract or non-compliance with specific legislation during the execution of the contractFootnote 8 ), an infraction report is issued immediately.Footnote xxx All infraction reports affect the vendor's performance rating. For instance, if the situation is corrected, the vendor receives a maximum of three points on its scorecard for that contract; a maximum of one point is awarded if the situation is not corrected.
  7. The scorecards and the infraction reports are kept by the IO Procurement Department. The information is made available to procurement officers or project management service providers who award contracts on behalf of IO.Footnote xvi
  8. Finally, the VPP allows for the suspension of a vendor in specific situations. Such an action would be taken if, for example, a vendor obtained one point on its scorecard or if its performance rating was two points or lower within a 3-year period. In addition, any criminal conduct, including fraud, collusion or bid-rigging, can result in an indefinite suspension.Footnote xxxiv

Conclusion

  1. This analysis has presented examples of measures put in place by three governments (the Government of the United Kingdom, the Government of the United States, and the Government of Ontario) to address the issue of vendor performance. Despite the different ways these measures were applied, the analysis revealed three areas of convergence:
    1. First, vendor performance management is generally part of a framework for mitigating risks associated with the administration of government procurement. The two goals of the frameworks are 1) to hold both suppliers and public servants accountable for the procurement process and 2) to determine minimum standards of reliability when selecting a supplier.
    2. Second, contract performance history is but one of the components used to determine a supplier's reliability. In most cases, government authorities also take into account extra contractual factors such as whether there is a criminal record, a history of fraud, etc.
    3. Third, it appears vendor performance management measures invariably include the following components: 1) the creation of a repository of supplier performance history – made available to procurement officials, 2) the option to temporarily deny contracts to a supplier, and 3) the importance of continuously improving supplier performance and the relationship between suppliers and government purchasing organizations.
  2. In conclusion, these areas of convergence provide information on the scope of the vendor performance management measures. Supplier performance is the responsibility of all actors in the procurement process, and the measures examined by OPO appear to make both public servants and suppliers accountable for the effectiveness of government procurement.

Annex A: List of acronyms

  • FAR: Federal Acquisition Regulation
  • IO: Infrastructure Ontario
  • OPO: Office of the Procurement Ombudsman
  • PPN: Procurement Policy Notice
  • VPP: Vendor Performance Program

Footnotes

Footnote 1

The word "measure" is used in this analysis to designate the different actions taken by governmental organi-zations to manage vendor performance. 

Return to footnote i

Footnote 2

  Office of the Procurement Ombudsman, 2011–2012 Annual Report, page 3.

Return to footnote ii

Footnote 3

Office of the Procurement Ombudsman, 2009–2010 Procurement Practice Reviews, "Chapter 4: Review of Procurement Practices Related to Management," page 12.

Return to footnote iii

Footnote 4

OPO had a short telephone conversation with Infrastructure Ontario (March 25, 2013) to obtain details on the information in the document entitled Infrastructure Ontario Vendor Performance Program, published June 2, 2010

Return to footnote iv

Footnote 5

The word "framework" is used to designate a group of policies and procedures that has a strategic objective.

Return to footnote v

Footnote 6

Government of the United Kingdom of Great Britain and Northern Ireland, Procurement Policy Note 09/12: Taking Account of Bidders' Past Performance

Return to footnote vi

Footnote 7

IBID, paragraph 2

Return to footnote vii

Footnote 8

IBID, paragraph 4

Return to footnote viii

Footnote 9

The Cabinet Office is the organization responsible for supporting the Prime Minister of the United Kingdom and Cabinet of the Government of the United Kingdom. It is roughly the United Kingdom equivalent to the Privy Council Office in Canada.

Return to footnote 1

Footnote 10

IBID, paragraph 17

Return to footnote ix

Footnote 11

IBID, paragraphs 13 and 23

Return to footnote x

Footnote 12

IBID, paragraph 5 and "Annex 2—OJEU Notice Wording and Guidance"

Return to footnote xi

Footnote 13

IBID, paragraph 12

Return to footnote xii

Footnote 14

IBID, paragraph 13

Return to footnote xiii

Footnote 15

United States—Office of the Press Secretary, Memorandum for the Heads of Executive Departments and Agencies, March 4, 2009

Return to footnote xiv

Footnote 16

United States—Office of the Press Secretary, Memorandum for the Chief Acquisition Officers Senior Procurement Executives, July 29, 2009

Return to footnote xv

Footnote 17

IBID

Return to footnote xvi

Footnote 18

In January 2011, an evaluation of the application of the new requirements was conducted on the 10 agencies that carry out the majority of acquisitions in the United States. It was determined that only a small percentage of these agencies evaluated the performance of their suppliers and that most of the evaluations did not contain enough informationA.

Return to footnote 2

Footnote 19

Congressional Research Service, Responsibility Determination Under the Federal Acquisition Regulation: Legal Standards and Procedures, January 4, 2013, page i 

Return to footnote xvii

Footnote 20

Responsibility Determinations Under the Federal Acquisition Regulation: Legal Standards and Procedures, published on January 4, 2013.

Return to footnote 3

Footnote 21

IBID, pages 8-9

Return to footnote xix

Footnote 22

Other governments also consider specific socioeconomic objectives. For example, the Canadian federal government has procurement programs relating to Aboriginal businesses and ensuring employment equity; these programs are not used to manage supplier performance. 

Return to footnote 4

Footnote 23

IBID, page 7

Return to footnote xx

Footnote 24

IBID, page 1

Return to footnote xxi

Footnote 25

Ontario Ministry of Finance, "Standard 5.1: Supplier Performance Management Process," Performance Measurement Phase II—A Framework for Action, pages 46 and 47,

Return to footnote xxii

Footnote 26

Infrastructure Ontario, "Preamble," Infrastructure Ontario Vendor Performance Program, published June 2, 2010, page 3.

Return to footnote xxiii

Footnote 27

The VPP may be applied to contracts valued at $100,000 or less if the business unit makes a request to do so. In this case, the application of the VPP does not appear to be mandatoryB.

Return to footnote 5

Footnote 28

Project management service providers are providers who have obtained a real estate management contract from Infrastructure Ontario. The VPP requires that these providers evaluate the performance of their subcontractors.

Return to footnote 6

Footnote 29

Infrastructure Ontario, Infrastructure Ontario Vendor Performance Program, published June 2, 2010, page 4.

Return to footnote xxiv

Footnote 30

Infrastructure Ontario, "Appendix A : Vendor Performance Scorecard," Infrastructure Ontario Vendor Performance Program, published June 2, 2010, page 15

Return to footnote xxvii

Footnote 31

Infrastructure Ontario, Infrastructure Ontario Vendor Performance Program, published June 2, 2010, page 6..

Return to footnote xxviii

Footnote 32

IBID, page 7

Return to footnote xxix

Footnote 33

An IO unit or a PMSP wishing to assign a weighting of more than 10% to the performance rating must first obtain authorization from the Vice-President of IO's Procurement DepartmentC.

Return to footnote 7

Footnote 34

IBID, page 8

Return to footnote xxx

Footnote 35

IBID, page 9

Return to footnote xxxi

Footnote 36

Occupational Health and Safety Act, Environmental Protection Act and Construction Lien ActD.

Return to footnote 8

Footnote 37

IBID, pages 10-11

Return to footnote xxxiv