Dispute Resolution Mechanisms for Vendor Performance Management

Dispute Resolution Mechanisms for Vendor Performance Management (PDF, 1.0MB)

1.0 Introduction

1.1 The Office of the Procurement Ombudsman

The Office of the Procurement Ombudsman (OPO) is a neutral and independent organization of the Government of Canada that works collaboratively with federal organizations and Canadian businesses to promote fairness, openness and transparency in federal procurement. OPO delivers on this mandate by connecting stakeholders, investigating complaints, resolving problems, making recommendations and sharing best practices.

OPO employees:^{Footnote 1}

Review the practices of federal departments for acquiring materiel and services to assess their fairness, openness and transparency and make any appropriate recommendations to the relevant department.
Review any complaint respecting the award of a contract for the acquisition of goods below the value of $25,300 and services below the value of $101,100 where the criteria of the Canadian Free Trade Agreement would otherwise apply.
Review any complaint respecting the administration of a contract for the acquisition of materiel or services by a department, regardless of dollar value.
Ensure that an alternative dispute resolution process is provided, if the parties to the contract agree to participate.

In 2018, OPO launched a knowledge deepening and sharing (KDS) initiative to better understand the root causes of recurring issues in federal procurement. Through the publication of studies, OPO intends to share its knowledge of federal procurement practices and provide meaningful guidance for federal procurement stakeholders.

1.2 Purpose

The purpose of this study was to examine the need for formal dispute resolution mechanisms in vendor performance management frameworks. The study also sought to identify different approaches for dispute resolution currently in use across multiple levels of government within Canada. Results of the study were intended to support or refute the premise that vendor performance management frameworks should include dispute resolution mechanisms.

1.3 Scope of Work

The study examined existing vendor performance management frameworks at the federal, provincial, territorial and municipal government levels. Specifically, the study focused on mechanisms within these frameworks for resolving disputes between vendors and government organizations regarding vendor performance scores or ratings.

1.4 Audience

This study may be of interest to procurement stakeholders at federal, provincial, territorial and municipal government levels who are interested in establishing or revising their approach to dispute resolution within the context of a vendor performance management framework.

1.5 Development Methodology

Literature Review

A review of the regulatory and policy framework governing vendor performance management was undertaken via an Internet search to gain an understanding of the guidance developed by federal, provincial, territorial and municipal organizations.

Questionnaire on Vendor Performance Management

Provincial and municipal organizations were asked to complete a questionnaire (see Appendix) on how dispute resolution mechanisms within vendor performance management regimes are designed and operate.

1.6 Enquiries

Enquiries should be directed to:

Office of the Procurement Ombudsman
410 Laurier Ave. W., Suite 400
Ottawa, Ontario K1R 1B7
Canada

Telephone:: 1-866-734-5169
Fax:: 613-947-6211
Toll-free for hearing-impaired:: 1-800-926-9105
Email:: ombudsman@opo-boa.gc.ca

1.7 Acknowledgements

OPO would like to thank the participating federal, provincial and municipal organizations for their input in developing this study.

2.0 Overview of Vendor Performance Management

2.1 Vendor Performance Management

Vendor performance is the action taken by a vendor^{Footnote *} to meet their contractual obligations in delivering goods and/or providing services. Vendor performance management is the monitoring, evaluating and reporting on a vendor’s performance against the requirements of a particular contract. Vendor performance management can be used to both encourage good vendor performance and hold vendors accountable for poor performance.

A vendor performance management framework typically includes an evaluation process whereby a rating system is used to evaluate a vendor on a set of pre-defined performance indicators. The rating system may be quantitative or qualitative in nature. Results from vendor performance evaluations may be taken in to consideration in the selection process for future contracts. Results may also be used to support a decision to suspend or debar a vendor from bidding on future contracts due to poor performance.

2.2 Dispute Resolution Mechanisms

In the context of vendor performance management, a dispute resolution mechanism refers to a formal method for resolving disputes between vendors and government organizations regarding the score or rating received by a vendor.

Dispute resolution mechanisms offer an avenue for vendors to appeal the outcome of a performance evaluation and resulting score or rating. They can serve as an alternative to litigation, saving time and money for both vendors and government organizations. Having a process in place that can avoid potential litigation can also help to preserve the business relationship between vendors and government organizations. Dispute resolution mechanisms also promote fairness in the vendor performance management process as they provide vendors an opportunity to express their concerns and appeal decisions they believe to be incorrect.

2.3 Previous Office of the Procurement Ombudsman Studies

Vendor performance management has been a topic of interest for OPO since the Office opened in 2008, and has been the subject of two previous OPO studies.

In May 2010, OPO undertook a study entitled “A Management Approach to Vendor Performance,”^{Footnote 2} which examined how government organizations were approaching vendor performance and identified best practices to improve fairness, openness and transparency of the procurement process. Best practices outlined in this report included the use of performance clauses in contracts and the use of tools and automated systems to monitor, evaluate and report performance results.

In January 2014, OPO released a second vendor performance management report entitled “Vendor Performance: A Shared Responsibility.”^{Footnote 3} In this study, OPO examined vendor performance measures in place in the United States, United Kingdom and Province of Ontario. The report highlighted areas of convergence among the vendor performance regimes in these three jurisdictions, including:

Vendor performance management is generally part of a framework for mitigating risks associated with the administration of government procurement;
Contract performance history is but one of the components used to determine a supplier’s reliability; and
Vendor performance management measures include: the creation of a repository of supplier performance history, the option to deny contracts to a supplier for a specified period of time, and the importance of good supplier relationship management.

2.4 Current Federal Government Focus

The Government of Canada does not currently have a common policy or approach to vendor performance management for federal procurement. Senior government officials and policy makers have recognized this and identified it as an area in need of change.

In October 2017, the Prime Minister of Canada issued a mandate letter to the Minister of Public Services and Procurement and Accessibility, then referred to as the Minister of Public Services and Procurement.^{Footnote 4} Among the priorities given to the Minister was to modernize procurement practices, which included “developing better vendor management tools to ensure the Government is able to hold contractors accountable for poor performance or unacceptable behavior.”

Also in October 2017, the House of Commons Standing Committee on Government Operations and Estimates undertook a study on how to improve access for small and medium enterprises (SMEs) and Indigenous businesses in federal procurement. In the final report, released in June 2018,^{Footnote 5} the Committee added its voice to those calling for change in the way the federal government manages vendor performance. The report recommended “[t]he Government of Canada implement a supplier performance rating system to evaluate and track vendor performance, which includes a component that recognizes new suppliers.”

Currently, Public Services and Procurement Canada (PSPC), the department that serves as a common service provider for federal procurement, is working on a “Procurement Modernization Initiative.” Under this initiative, as mentioned in its 2019-2020 Departmental Plan, “PSPC is developing better tools to manage the performance of vendors, allowing the government to hold them accountable for poor performance while incentivizing good performance. These tools will allow the department to support its clients in ensuring that they are obtaining value for money in their procurements.”^{Footnote 6}

3.0 Vendor Performance Management in Canada

3.1 Federal Government

While the Government of Canada has not developed a common approach to vendor performance management, PSPC, previously known as Public Works and Government Services Canada (PWGSC), has had vendor performance policies in effect for over 20 years. In 1996, PSPC implemented a Vendor Performance Policy^{Footnote 7} to address the Department’s responsibilities for vendor performance when it acted as the contracting authority in government procurements. This policy was subsequently replaced in 2011 when PSPC implemented its Vendor Performance Corrective Measure Policy.^{Footnote 8}

Under the existing Vendor Performance Corrective Measure Policy (2011), there are two forms of corrective measures that could be taken by PSPC—debarment or application of conditions. Debarment is the refusal to do business with a vendor for a specified period of time. A debarment renders a vendor ineligible to bid on contracts, standing offers and supply arrangements. Alternatively, conditions, such as a requirement for added financial security, may be applied against vendors in place of debarment, if considered more appropriate by PSPC. Where a vendor is subject to conditions and has not met them, the vendor can be declared ineligible to bid on or receive contracts, standing offers and supply arrangements.

Prior to taking formal corrective measures under the Policy, PSPC is required to conduct an assessment to determine if there is evidence that continued contracting with a vendor may pose a greater than acceptable risk to Canada. A senior PSPC official is responsible for notifying the vendor of assessment results. The vendor then has 15 days to respond in writing and their response could include a request to present their case verbally to the senior official. The senior official can revise the assessment results after reviewing the vendor’s response. In such cases, written notice of revised assessment results is provided to the vendor.

The final decision to impose corrective measures is made by a PSPC Assistant Deputy Minister. There is no mechanism built into the Policy for a vendor to appeal the final decision of the Assistant Deputy Minister.

Despite these vendor performance policies being in place for many years, they have not met expectations. A 2014 PSPC internal audit entitled Vendor Performance and Corrective Measures observed that the “successful application of the Policy requires participation by contracting officers and client [departments]. However, objectives and priorities may differ between contracting officers and client [departments]. Since [PSPC] does not have the authority to require client [departments] to provide vendor performance information, the communication and sharing of information between these stakeholders has not been easily established.”^{Footnote 9}

PSPC is in the process of revising its vendor performance management policy, with a vision of one-day having a common policy and approach across the federal government.^{Footnote 10} At the time of writing, the 2011 PSPC Vendor Performance Corrective Measure Policy remains in effect.

3.2 Provincial and Territorial Jurisdictions

In Canada, each province and territory has authority over its procurement processes and systems. As such, not all provinces and territories follow the same approach to manage vendor performance. OPO sought to gain a better understanding of the current state of vendor performance management frameworks at the provincial and territorial levels, with a specific focus on dispute resolution mechanisms within those jurisdictions. This involved a review of provincial and territorial government websites, followed by a questionnaire sent directly to procurement officials.

OPO examined vendor performance frameworks in five Canadian provinces: Newfoundland and Labrador, Quebec, Ontario, Saskatchewan and British Columbia. Similar to the federal government context, most provinces do not have a common, government-wide vendor performance management framework. Newfoundland and Labrador was identified as the only province with a single government-wide dispute resolution approach. Other provinces with existing vendor performance management frameworks tend to be more decentralized. These provinces typically feature some form of common vendor performance direction or guidance, with individual departments or ministries retaining final decision-making authority.

3.2.1 Newfoundland and Labrador

The Province of Newfoundland and Labrador has embedded vendor performance management, which it refers to as “supplier performance,” in its provincial Public Procurement Act and Public Procurement Regulations.^{Footnote 11} Provisions of the Public Procurement Act pertaining to supplier performance came into effect in September 2018.^{Footnote 12}

Newfoundland and Labrador’s approach to managing supplier performance is further described in the province’s Public Procurement Policy.^{Footnote 13} As identified in the Policy, the purpose and objectives of supplier performance are to:

Promote effective supplier performance management procedures;
Encourage effective communications between public bodies and suppliers;
Outline criteria for evaluating supplier performance;
Establish procedures to address concerns with supplier performance;
Improve supplier performance; and
Establish the process and parameters for supplier suspension.

The Public Procurement Policy does not specify a minimum dollar threshold or type of contract for which supplier performance monitoring would apply. Rather, “monitoring would be dependent on the type of commodity being acquired. Additionally, when determining the extent of monitoring required, a public body should consider the value of the acquisition and the degree of risk to the public body associated with the procurement.”^{Footnote 14} The public body produces a Supplier Performance Report upon contract completion. According to the Policy, “[t]he report will evaluate a supplier’s performance based on a points rating system considering major components relative to supplier performance. The points assigned will be determined by the type of performance ranging from unsatisfactory, marginal, satisfactory, very good and excellent.”^{Footnote 15} A Supplier Performance Report will include one of three recommendations: continue to be eligible to bid on procurement opportunities; issue a warning letter; or suspension based on non-compliance or substandard performance.

As described in the Public Procurement Policy, if the decision is to suspend a supplier, written notice is provided that includes the reasons for the suspension and its duration. Further, the Policy specifies:

A supplier may appeal a suspension by a public body to the Chief Procurement Officer^{Footnote **} within 15 business days of receiving notification of the suspension. If a supplier appeals a suspension, the Chief Procurement Officer shall notify the head of the public body and the head of the public body shall provide the Chief Procurement Officer with all pertinent information and records regarding the suspension within the time frame identified by the Chief Procurement Officer. The Chief Procurement Officer shall review the suspension and decide upon it within 30 business days of receiving the appeal and may confirm, vary or cancel the suspension. A suspension of bidding privileges remains in effect until such time that the [Chief Procurement Officer] renders a final decision.^{Footnote 16}

3.2.2 Québec

The Province of Québec has included vendor performance in chapter C-65.1, r. 2 Regulation respecting certain supply contracts of public bodies (the Regulation), which stipulates that “[a] public body must record in a report the evaluation of any supplier whose performance is considered to be unsatisfactory.”^{Footnote 17} The Regulation also prescribes timelines for a public body to complete a performance evaluation and render its decision on vendor appeals. Under the Regulation, a vendor has the right to respond in writing to their evaluation report. The chief executive officer of the public body makes the final decision to uphold or cancel the evaluation.

At the ministry level, Transports Québec has implemented a Vendor Performance Evaluation policy “in order to measure the quality of contractors’ work and services provided by service providers” [translated].^{Footnote 18} Evaluations are performed on all contracts valued over $25,000 in road and bridge engineering, and over $100,000 for construction work on complex structures or in information technology. Additionally, evaluations are performed on any contract for which the supplier’s performance is deemed unsatisfactory.

Since 2013 Transports Québec has used past vendor evaluation reports to produce a vendor performance rating. This rating is one of the criteria used to evaluate bids for certain contracts. Additionally, when a vendor receives an unsatisfactory performance evaluation, Transports Québec can refuse any tender submitted by the vendor for a contract of the same nature for a period of two years.

Evaluation reports must be provided to the vendor within 60 days of the acceptance of the final deliverable under a contract. If Transports Québec does not meet this deadline, the vendor may request the report be withdrawn. Following receipt of the evaluation report, the vendor has 30 days to submit comments, and Transports Québec has 30 days to review or maintain the evaluation report.

3.2.3 Ontario

As was noted by OPO in 2014, “[t]he Government of Ontario does not have a government-wide vendor performance management program. The ministries responsible for acquisitions in the Government of Ontario appear to be individually responsible for managing the performance of their suppliers.”^{Footnote 19}

Infrastructure Ontario (IO) is a Crown agency established by the province that provides a wide range of services supporting government initiatives to modernize and maximize the value of public infrastructure and real estate.^{Footnote 20} IO has a vendor performance program that includes two different methods of identifying, measuring, and acting on poor performance—one for private-public partnership (PPP or P3) projects and another for traditional delivery projects.

An IO Vendor Performance Program document states that “Vendor Performance Scorecards [are] used to assist Project Managers in considering past performance in the sourcing stages of new contracts, giving high performing Vendors a more competitive position when pursuing future work with IO and/or a Service Provider.”^{Footnote 21}

Under the Vendor Performance Program, a vendor disagreeing with a scorecard may request a review by the project manager within 10 business days of receiving the Scorecard. The request must be made in writing and must include substantiated facts and supporting documentation. The project manager may, at their discretion, meet with the vendor to discuss the matter. The project manager will issue a written response to confirm or modify the scorecard. The scorecard made by the project manager in response to a vendor’s request for review will be final.

The review process described above does not apply to disagreements related to a formal Notice of Infraction’s impact on a scorecard. IO will issue a Notice of Infraction to a vendor upon committing any infraction meeting pre-defined criteria. There is no mechanism to dispute an infraction’s impact on a scorecard. OPO was advised by IO that contract clauses provide assurances that infractions will be well-supported with documentation and are based on conditions agreed to by the vendor.

3.2.4 Saskatchewan

The Province of Saskatchewan’s approach to vendor performance management is articulated in a Vendor Performance Manual. The Manual depicts vendor performance management as a process that supplements ongoing contract management to ensure:

Vendors are receiving formal evaluation and feedback on their performance; and
Past vendor performance is considered in procurement decisions.^{Footnote 22}

Vendor performance evaluations are mandatory for all contracts valued at $1,000,000 or over, and may be applied to contracts of any value.

The Manual recommends that organizations use a four-point rating scale for individual performance evaluations, ranging from 1—Unacceptable to 4—Exceptional. Regardless of the outcome, performance feedback is to be provided to the vendor. As stated in the Manual, “[a]fter an evaluation has been completed, feedback must be provided to the vendor. Regardless of how the performance was rated, the vendor should know the results.”^{Footnote 23} Following receipt of their performance feedback, a vendor has 30 days to request a consultation meeting to discuss the results.

The Manual describes the process to resolve disputes over performance evaluations as follows:

It is the responsibility of each ministry and Crown to develop and implement a process for vendors to discuss issues they may have with their evaluation score, contract score or vendor score. The process may include:

An attempt at informal resolution with the person who assigned the score;
Submission of a written complaint to the business unit that assigned the score;
Opportunity for review by a third party within the organization; and
For future evaluations, the opportunity for the supplier to move to a neutral rating if enough evidence is available to verify the supplier has implemented processes to improve.^{Footnote 24}

3.2.5 British Columbia

The Province of British Columbia’s Core Policy and Procedures Manual, states that “a post-completion evaluation is required on every contract over $50,000 to provide a record of the contractor's performance and to assist in future contracting activity.”^{Footnote 25} According to a description of the procurement process on province’s website, “[p]ost contract evaluations are used to determine how the ministry benefited from the contract, if all the deliverables were met, how well the contractor performed during the contract, and how well each member of the Province’s team performed his/her role. These evaluations are documented to serve as a reference on the project and to identify any lessons learned.”^{Footnote 26}

Additionally, a description of the province’s post contract evaluation process states it should include a mechanism that “(i) documents the subject’s… agreement with the evaluation results, and (ii) allows them to object if they don’t agree.”^{Footnote 27} It appears that the form of such a mechanism is the decision of individual ministries.

3.3 Municipalities

Like provinces and territories, municipalities have their own procurement processes and systems, some including formal vendor performance management processes. Below are profiles of vendor performance management processes identified in five major Canadian municipalities that include formal dispute resolution mechanisms: Ottawa, Edmonton, Toronto, Saskatoon and Peel Region.

3.3.1 Ottawa

The City of Ottawa launched a vendor performance management program for construction and construction-related professional services in January 2015. It covers construction contracts valued over $100,000 and construction-related professional services contracts over $15,000.

The goals of Ottawa’s Vendor Performance Management program, as described in a Business Process Guide posted to the City’s website, are to:

Improve overall vendor (consultant and contractors) performance on City contracts;
Improve communication between Project Managers and vendors;
Improve the overall performance of vendors over time; and
Build a history of vendor performance over time, allowing future vendor selection decisions to include a historical performance perspective.^{Footnote 28}

For contracts within the scope of the vendor performance management program, a City of Ottawa project manager conducts both an interim and final evaluation of a vendor’s performance. Projects less than six months in duration only require a final evaluation. Interim evaluations use the same evaluation categories as the final evaluation, but do not count towards a final vendor score and cannot be appealed.

Final evaluations are completed at contract close-out. The final evaluations must be completed and approved within a specific timeline, depending on project type. Once the final evaluation has been completed, further discussion may take place between the vendor and the Project Manager for clarification of the evaluation. Any change proposed by the vendor and agreed to by the Project Manager requires the original evaluation to be withdrawn and modified to reflect the revised score.

As explained in the Business Process Guide, after the final evaluation has been discussed with the vendor, vendors may choose to appeal. Appeals must be submitted electronically within 20 days following the final evaluation, and must include a rationale specific to what category the appeal is pertaining to. The standard response time to respond to an appeal is 15 business days; however, this may be longer for appeals that escalate to a higher management level or are more complex.

An advisory committee or appeal board consisting of three members from the operating department will hear and adjudicate an appeal. The intention of the hearing is to give the appeal board an opportunity to ask questions of the vendor and/or for the vendor to clarify information. It is not an opportunity for the vendor to give a presentation or to provide new information. Any new information is to be provided in writing as part of the appeal submission.

A City of Ottawa official advised OPO that, at time of this study, the appeal process has only been used in approximately 1% of contracts meeting the program’s criteria.

3.3.2 Edmonton

The City of Edmonton launched its supplier performance program in January 2016. The Program applies to the City’s design and construction contracts and related agreements with a value of $75,000 or greater.

The purpose of the program is to:

Promote positive supplier performance;
Provide a standardized approach to formally measure supplier performance;
Ensure good value is derived from the contracted good, services and construction; and
Establish the process to address concerns.^{Footnote 29}

For contracts within the scope of the supplier performance program, a project manager completes interim and final evaluations of a vendor’s performance based on pre-established performance evaluation categories and criteria. To assist project managers at evaluation time, scoring guidelines have been developed for each of the evaluation criteria. Final performance evaluation scores are made available for use in future pre-qualification, tender and request for proposal submission evaluations.

A vendor can appeal the results of an interim or final evaluation to the City’s Corporate Procurement & Supply Services Branch. An appeal must be filed in writing within seven calendar days of a vendor receiving their evaluation results and requires payment of a $2,500 non-refundable filing fee. If the appeal is deemed valid, an appeal meeting will be scheduled within 45 calendar days. The appeal meeting is conducted by a referee who is given the authority to dismiss the appeal, change evaluation scores or modify any evaluation comments. The decision of the referee is final and binding. According to program documentation, there is no further right of appeal within the City or the courts.

A City of Edmonton official advised OPO, at the time of the study, that there have been no requests from vendors to appeal performance evaluation results.

3.3.3 Toronto

The City of Toronto has implemented a standardized tool, known as the “Contract Performance Evaluation” (CPE), to evaluate “a general contractor’s performance on any given construction project that has been awarded to a general contractor.”^{Footnote 30} The CPE provides an evaluative record of results, both positive and negative, and is used to encourage vendors to improve their performance. Evaluations are conducted by City project management staff with oversight from senior division management and purchasing and material management.

If a vendor does not agree with their CPE score they have the opportunity to submit their objection in writing with evidence within five business days. The submission is sent to a division manager for interim evaluations or a division director for final CPE. The City has the sole discretion to decide whether the score should be amended and a replacement CPE form issued. If a report is submitted recommending suspension, the Chief Purchasing Officer will inform the contractor of their right to make a statement to a standing committee dealing with the suspension and provide further information on the process. The standing committee ultimately makes a recommendation to Council with respect to the suspension.

3.3.4 Saskatoon

The City of Saskatoon’s Purchasing Policy,^{Footnote 31} which took effect in 2018, specifies a process for vendor performance management including subsequent appeals. The Policy states that supplier performance must be monitored and tracked in accordance with the City’s Supplier Performance Evaluation Protocol and that suppliers can be suspended in accordance with its Supplier Suspension Protocol. These protocols form part of Saskatoon’s Policy and Protocols Manual.

The purpose of the Supplier Performance Evaluation Protocol is to “establish a performance evaluation process in order to evaluate supplier performance in a transparent and consistent manner.”^{Footnote 32} The performance evaluation process includes both interim and final evaluations. If a supplier receives a “Below Standard” rating for their interim or final evaluation, the Contract Administrator will meet with the supplier to discuss the performance issues that led to the rating. Final performance evaluations will include a recommendation to either allow the supplier to continue participating in future procurements or to suspend the supplier. A suspension could be for a period up to five years depending on reasons for the suspension.

If a final evaluation results in a recommendation for suspension, the Supplier Performance Evaluation Protocol requires the recommendation be reviewed by the Contract Administrator’s director. The supplier must be notified of the final rating and recommendation and will be provided with a copy of the final report. The supplier has 10 business days to provide a written statement of dispute.

All decisions to suspend a supplier must be approved by the City’s Procurement Review Committee. The Committee will consider the supplier’s statement of dispute, if any, in rendering its decision. A decision made under the Supplier Suspension Protocol is final.

3.3.5 Peel Region

The Peel Region has procedures that provide guidance on vendor performance evaluations conducted by the municipality. The Peel Region website indicates its vendor performance evaluation procedures are currently being used solely for public works and real property asset management projects.

Information collected in performance evaluations serves the following purposes:

To provide feedback to vendors for performance improvements/or acknowledge satisfactory performance;
To determine vendors’ eligibility or ineligibility to bid on future contracts; and
To provide justification for the award or non-award of contracts.^{Footnote 33}

If a vendor is unsatisfied with their evaluation they can submit an appeal in writing to the Director of Procurement within 21 calendar days of the receipt of their final evaluation. A committee comprised of the Director of Procurement, the project lead’s director or any other individuals deemed necessary will review the appeal. The committee will decide to either confirm or vary the final evaluation and rating and this decision will be final.

4.0 Forms of Dispute Resolution Mechanisms

As demonstrated in the preceding discussion, there are various potential approaches for resolving disputes within vendor performance management frameworks. These range from a simple “no appeal” process, to a more formal one-level internal appeal process, to a two-tiered mechanism including independent third party review at the second level of appeal.

At one end of the spectrum is a vendor performance management framework with no dispute resolution mechanism and no recourse available to a supplier who is unsatisfied with the results of their performance evaluation. This approach was not observed in any of the jurisdictions reviewed in this report, which tended to, at a minimum, offer a process where a vendor can appeal directly to the individual who completed the performance evaluation. For example, a project manager could consider a vendor’s argument and decide whether or not to amend the performance evaluation. Discussions may be held between the project manager and vendor to reach an agreement, but ultimately there is no escalation or appeal to a higher authority.

Proponents of this approach could argue that a formal dispute resolution mechanism is not necessary because clear contract clauses and good contract management practices ensure that issues are well documented and vendors agree to deliver projects in accordance with those conditions. However, given the potential negative consequences to the vendor that may result from the performance evaluation, including suspension or debarment, vendors should have the opportunity to appeal performance evaluations.

More common among the provinces and municipalities reviewed in this study is a one-level dispute resolution mechanism. Under this approach, there may be a senior official identified or committee established from within the contracting organization to hear appeals about vendor performance evaluation results or debarment/suspension decisions. In comparison to the “no appeal” model, the one-level internal appeal process adds a degree of formality. Internal appeal processes also include a timeline within which an appeal must be made, and key dates to finalize the appeal decision. This may include negotiation prior to a decision being reached.

The benefits of this approach include maintaining much of the administrative efficiency of the simple model, while involving an internal third-party that was not directly involved in the dispute. On the other hand, the “in house” nature of the process can be a drawback as the neutrality of an internal appeal authority might be impaired, in perception or reality, because they also represent the contracting organization. While preferable to the “no appeal” model, this process lacks the neutrality and impartiality offered by an independent third-party.

At the other end of the spectrum is a dispute resolution mechanism with a two-level appeal process that includes an independent third-party for the second and final level of appeal. This process can include similar characteristics to the one-level internal appeal process, with an added second level of appeal involving an independent third-party that is not part of the organization involved in the dispute. A potential disadvantage of the two-level appeal process is the amount of time it could take to ultimately resolve the dispute. When using this process, clear procedures and timelines for requesting and conducting the appeal must be in place and adhered to.

5.0 Considerations for implementation

The following factors should be taken into consideration when developing dispute resolution processes within vendor performance management frameworks:

5.1 Interim and final performance ratings

Performance ratings are a tangible output of the vendor performance management process. Depending on the design of the framework, evaluations may be conducted during the performance of a contract (e.g. every 6 months) and at the end of the contact, resulting in both interim and final performance ratings. Alternatively, evaluations may only occur at the completion of the contract period, resulting in a single, final performance rating.

An important consideration in the design of a vendor performance management framework with both interim and final performance ratings is whether the interim ratings are subject to appeal. In instances where interim ratings are taken in to account in determining the final rating, consideration should be given to making them subject to an appeal process. The same principle would apply when interim ratings are used in the award of future contracts, e.g. during a multi-year contract where the final rating will not be determined for several years. Conversely, where an interim rating serves more as a “check-in” point and is not used to determine a final rating or considered in the award of future contracts, it may not be necessary to make it subject to appeal. Hybrid approaches are also a possibility for vendor performance management frameworks with two-level appeal processes. In a hybrid model, interim ratings would be limited to a first-level appeal, while final ratings could be subject to a first and second level appeal process.

5.2 Application fees

Another important consideration is whether or not a fee should be charged to vendors, and what would constitute a reasonable fee to appeal a performance rating. Whereas a high fee may create a barrier for some vendors to appeal their rating, having no fee might open the door to potentially frivolous appeals. Consideration should also be given to whether application fees should be refunded under certain conditions. In instances where the vendor’s appeal is determined to have merit, should the application fee be refunded to the vendor? From a fairness perspective, it would seem appropriate to refund fees when an erroneous score is changed in the vendor’s favour.

5.3 Deference

Deference has been defined as yielding to the judgment or opinion of another.^{Footnote 34} For the purpose of this study, it refers to the scope of authority given to the second level of appeal in the dispute resolution process. More specifically under what conditions would the second and final level of appeal change the decision made by the initial appeal authority.

In the court system, appeal courts are generally not “fact finders”, as this process has already taken place in detail in the lower court. Instead, appeal courts focus on whether or not the initial court erred in applying the law to the established facts. Deference is also seen in the context of certain ombudsman offices, for example: the Procurement Ombudsman of Canada may not substitute his/her opinion for the judgment of the persons involved in the acquisition process, unless there is insufficient written evidence to support that assessment or the assessment is unreasonable.

When determining how much deference should be provided to the initial appeal authority in the context of vendor performance dispute resolution, consideration should be given to whether the second level of appeal will take place in-person or via written submissions. Time and cost considerations may make it impractical to re-hear arguments in-person and in their entirety at the second level of appeal, thereby requiring some level of deference toward the initial decision.

5.4 Processing Time

A key drawback of the two-level appeal process is that it can lengthen the amount of time required to ultimately resolve the dispute. Both vendors and government organizations benefit from an efficient dispute resolution mechanism that provides finality. For this reason, binding arbitration should be considered as a viable option at the second and final level of appeal, as it provides the desired finality. For the same reason, procedures and timelines for every step of the process, including submitting appeals, the format of the appeal (e.g. in person vs. in writing), and the time within which a decision will be rendered, need to be clearly established and respected.

6.0 Conclusions

The following factors should be taken in to consideration when implementing a dispute resolution process in the context of a vendor performance management framework:

Clear and comprehensive guidance material is required for all involved parties. This guidance should include key information such as the purpose of the framework, the evaluation criteria to be used in determining ratings, the evaluation process, the dispute resolution and appeal process, deadlines, timeframes and the requirement to document decisions in writing.
In Canada, at the time of this writing, there is no common, horizontal approach to vendor performance management and related dispute resolution at the federal level. Examples of vendor performance management regimes at various levels of government are outlined in this report, and can be taken in to consideration when developing a vendor performance management framework.
Due to the potential negative impacts on a vendor that could result from a poor rating, including suspension or debarment, vendors should have the opportunity to appeal performance evaluations and ratings.
One level of appeal is preferable to no appeal at all, even if the appeal authority is part of the organization that provided the performance evaluation and rating. However, consideration should be given to including a second level of appeal to an independent and impartial third-party.
Where the parties are unable to reach an agreement at the first level of appeal, consideration should be given to arbitration at the second and final level of appeal, as the binding nature of arbitration provides the parties with finality.
Consideration should be given to making interim ratings subject to appeal, especially in instances where: (a) the interim rating is used in compiling the final rating; and (b) the interim rating is taken in to consideration in the award of future contracts.
Consideration should be given as to what, if anything, constitutes a reasonable appeal application fee, and whether that fee should be refunded in cases where the appeal is deemed to have merit and the vendor’s rating is adjusted upward.
Consideration should be given to the level of deference accorded to the initial appeal authority, especially in instances where it is impractical to re-hear all of the facts in their entirety at the final level of appeal.

For the federal government, the design and implementation of a clear and fair dispute resolution mechanism in the context of vendor performance management should be of utmost importance. Such a mechanism, if properly implemented, will support the objectives of promoting positive supplier performance and ensuring that good value is obtained for the expenditure of taxpayer dollars.

Appendix – Vendor Performance Management Questionnaire

Questions

Do you have a process to evaluate vendor performance?
Could you describe your vendor performance management process? For example:
1. When was it established?
2. Is there a dollar threshold for contracts?
3. Does it cover all types of contracts? (E.g. goods/services/construction)?
4. Do vendors/suppliers receive a score?
5. How are scores tabulated?
6. How are results used?
Could you provide:
1. Volumetric data on the number of vendors/contracts that have gone through your vendor performance management process?
2. An overview of the results, e.g. the number or percent of vendors/contracts that scored in a various performance range?
Does your vendor performance management process include an appeals mechanism where by vendors can dispute their results?

If yes, continue to the next question.

If no, skip next three questions.

If yes, could you explain how the appeals mechanism/process works?
1. How do you file an appeal?
2. Who adjudicates the appeal?
3. Paper based appeal? Hearings?
4. Time limits to appeal?
5. What can be appealed?
6. How long does it take (start to finish)?
7. Is there a fee?
How many requests have you received from vendors to appeal performance rating/score?
1. Of those requested, how many have proceeded to the formal appeals process?
How often (number or percentage) have vendors been successful at having their performance evaluation results changed in their favor?
What (other) avenues are available to vendors that disagree with their performance evaluation score/rating?
How many times has litigation occurred over the vendor performance management results?
Could you offer any success factors or area of improvement for your vendor performance management appeals process?

Date modified:: 2019-11-19