Research on goods versus services contracts

Preamble

An analysis of supplier contacts to the Office of the Procurement Ombudsman (OPO) since its establishment in 2008, revealed a disproportionate number of complaints related to service contracts (as oppose to contracts for goods). This despite the fact that, while there have been increases in spending on service procurement by the federal Government over the last 15 years, the number of goods contracts awarded remain higher than the number of service contracts. Since there are greater number of goods contracts awarded each year, once could presume that a higher number of goods complaints should ensue, but this is not the case.

To better understand this situation and why it is happening, OPO engaged University of Ottawa researchers to answer the following questions:

  • What inherent differences in contracting for services (as opposed to goods) may be contributing to the disproportionate number of complaints about service contracts compared to contracts for goods?
  • What issues, factors or reasons may be contributing to the disproportionate number of complaints related to services?
  • What conclusions may be drawn from the research?
  • Are comparable jurisdictions experiencing similar problems?

The University of Ottawa's report confirmed there are higher numbers of service contract complaints compared to goods contract complaints. Based on data collected for this report, the majority of comparable United States (U.S.) jurisdictions have also experienced higher levels of service contract related complaints, meaning that OPO is not unique in this area.

While the research was inconclusive with regards to causality, it nonetheless provides a potential reason for the disproportionality: the report's key observation is that a 'structural incompatibility' may exist between the actual procurement system (designed for goods) and the increasing number of service contracts, which are often viewed as more complex. This 'structural incompatibility' may be a result of the evolution of Canada's procurement system.

The report notes this area of public procurement (i.e. the difference between procuring goods compared to services) is largely understudied and suggests that more research is required.

OPO is pleased to make this report publicly available to encourage additional research in this area of public procurement.

Acknowledgements

The authors wish to acknowledge the following public procurement experts for their insightful comments throughout the development of this report:

  • Dermot Cahill, Professor, Bangor University Institute for Competition and Procurement Studies, United Kingdom.
  • Paul Emanuelli, General Counsel and Managing Director, the Procurement Office, Toronto, Canada.
  • Krista Ferrel (for National Association of State Procurement Officials (NASPO)), Director of Strategic Programs, NASPO, United States.
  • Steven Graves, Professor, Massachusetts Institute of Technology (MIT), United States.
  • Stephen B. Gordon, Professor of Practice, Public Administration, Old Dominion University, United States.
  • Khai Thai, Professor, School of Public Administration, Florida Atlantic University, United States.

Any errors or omissions in this report are the sole responsibility of the authors.

Introduction

Public procurement is undertaken in a complex environment where political interests (e.g. regionalism, electioneering) intersect with spending and the market economy. As Worthington (2013) explains, "[…] the public procurement of goods, services and construction are highly regulated processes" (p. 508). Furthermore, public procurement "involves a wider set of choices about what to buy, from whom and how" (Hartley 1991, p. 45). The vast sets of rules and regulations in place to govern procurement are meant to sanitize the process and remove as many potential conflicts of interest as possible.

Public procurement is inherently different from private procurement with repercussions not for shareholders, but for taxpayers, citizens as well as the political livelihood of a government. Paul Emanuelli conveys the interplay of public procurement, politics and public trust well:

As the Pacific ScandalFootnote 1 illustrates, government procurement controversies can bring down the best of them. Even the Fathers of Confederation were not immune. The paramount importance of good government is firmly entrenched in our political fabric. Reactions to spending improprieties run as deep in our political psyche as the memories run long. Public officials who run on the wrong side of these rules do so at their peril. Today, computer and advertising contracts may have replaced railway construction deals as the subject of controversy, but accountability for proper government spending remains a matter of national importance (Emanuelli 2005, p. 1).

Given its political repercussions, it was little surprise that procurement was targeted when political scandal surrounding accountability surfaced in the mid-2000s in Canada. Part of the reaction was to enact legislation for a Procurement Ombudsman to receive procurement contract complaints and to review the procurement processes of federal organizations. Pursuant to the Federal Accountability Act that received Royal Assent in December 2006, the Procurement Ombudsman's mandate and enabling legislation were made by amendments to the Department of Public Works and Government Services Act (2006) (section 23.1).Footnote 2 Establishing the Procurement Ombudsman was one of "several elements of the government's comprehensive plan to strengthen the fairness, openness and transparency of federal procurement" (OPO Annual Report 2008-09, p. 10). While the Act establishes the role and responsibilities of the Procurement Ombudsman, the Procurement Ombudsman Regulations that came into effect in May 2008 operationalize the mandate.Footnote 3

The Procurement Ombudsman's mandate has four basic tenetsFootnote 4 (as defined in the Department of Public Works and Government Services Act (2006))Footnote 5:

  1. Ensure government departments procure goods and services in a fair, open and transparent manner, through a review of their practices, making recommendations for improvement when appropriate.
  2. Review complaints for the contract award for goods below $25,000 and services below $100,000.
  3. Regardless of dollar value, review complaints regarding the administration of a contract.
  4. Ensure alternative dispute resolution services are available (both parties to the contract must agree to participate).

A plain read of the Procurement Ombudsman's mandate defined in the Department of Public Works and Government Services Act and the operationalization of their mandate through the Procurement Ombudsman's Regulations, appears to suggest that the Procurement Ombudsman can receive complaints once a contract has been awarded for matters related to the evaluation of bids, the contract award, as well as contract administration. There may be a risk of bias in the nature of complaints filed with the Procurement Ombudsman in relation to their mandate. However, given the perceived scope of the Procurement Ombudsman's mandate, there appear to be linkages between the legislation and regulations and the pre-contractual, contracting, and contract administration phases of the four phases of the contracting process (see Table 1). Only phase four, the post-contract phase, may be contestably part of the Procurement Ombudsman's mandate. For a discussion of how the Procurement Ombudsman operationalizes its mandate through the filing and evaluation of complaints, please see Annex 1.

The Government of Canada spends roughly 15-20 billion dollars in the procurement of goods and services every year through hundreds of thousands of contracts (Treasury Board of Canada Secretariat (TBS) 2015). Procurement practices at an aggregate level matter for politics and public finance because of the amount of money spent and the ways in which it is spent.

When contracts are assessed based on monetary value, service contracts cost consistently more than goods contracts since 1996 (Data tables 1 and Data table 2). In terms of volume, the number of goods contracts since 1996 is greater than the number of service contracts, with exception to 1997 and 1998 (see Data table 3 and Data table 4). Most contracts (90%) are worth less than $25,000 (i.e. low value) and represent less than 10% of the total value of contracts. While contracts worth $25,000 and above represent the largest portion of spending, they are far fewer in number (OPO Presentation 2013). Prior to the establishment of the OPO in 2008, suppliers for low-value contracts had no formal complaint process (vendors with higher value contracts have recourse with the Canadian International Trade Tribunal).

Table 1: Contract phases and descriptions
(Office of the Procurement Ombudsman Annual Report, 2008-09, p. 32)
Phase Phase description
(OPO Annual Report 2008-09, p. 32)
Description of complaints
1. Pre-contractual
  • Activities related to requirement definition and procurement planning.
  • Requirement understated.
  • Statement of Work unclear or biased.
  • Unfair or onerous requirements, e.g. security clearance level, liability insurance.
2. Contracting
  • All activities from bid solicitation to contract award.
  • Request for proposal (RFP) development, e.g. onerous terms and conditions.
  • How the bidders are evaluated and whether it's fair.
  • Evaluation criteria: mandatory or rated; complaints that they're too restrictive, e.g. 12 years at OPO instead of 12 years in Public Service (PS) generally.
  • Mandatory criteria that is pass/fail (they either have it or not) is treated as rated where partial points may be awarded, allowing some vendors to move to the next level even if they should not have been allowed.
  • Procurement strategy, e.g. contracting agencies not using the procurement tool accurately; perception of contract splitting offer (SO) for long-term/recurring procurement).
  • Complaint that the contracting agency did not evaluate the bid the way they said they would, e.g. inaccurate application of the point system by the contracting agency (i.e. they give partial or half points), problematic for vendors who did not receive points.
  • Wrong contractor selected.

Note: Many evaluation issues are related to planning, i.e. accurately defining (and applying) the terms of assessment clearly and regularly from the outset.

3. Contract administration
  • Activities such as issuing contract amendments, monitoring progress, following up on delivery, payment action.
  • Contract amendments, e.g. change in scope, insufficient funding.
  • Payment delays.
4. Post contract
  • Final action (e.g. client satisfaction, contractor agreement to final claim, final contract amendment, completion of financial audits, proof of delivery, return of performance bonds) and close-out (e.g. completeness and accuracy of file documentation and adherence to file presentation standards).
  • Performance unsatisfactory.
  • Key documents not on file.
Data table 1: Contract values by type, 1996-2004
Year Total value of yearly contracts Total value of construction contracts (approx.) Total value of services contracts (approx.) Total value of goods contracts (approx.)
1996 8,703,172 M 3.5 M 4.5 M
1997 9,093,408 0.8 M 0 4.2 M
1998 13,732,457 0.6 M 0 5.1 M
1999 9,858,154 0.6 M 4.9 M 4.5 M
2000 9,360,172 0.7 M 5.1 M 3.6 M
2001 13,146,679 0.9 M 6.5 M 6.2 M
2002 12,822,851 M 7.7 M 4.4 M
2003 13,393,801 M 7.9 M 5.2 M
2004 19,063,453 M 11.9 M M
Data table 2: Values of goods, services and construction contracts (yearly, 2005-13)
Year Total value of contracts (Yearly) Total value of construction contracts (approx.) Total value of services contracts (approx.) Total value of goods contracts (approx.)
2005 15,944,061 0.9 M 10.2 M 4.9 M
2006 12,153,012 0.9 M M 4.1 M
2007 14,257,457 0.8 M 6.2 M 7.3 M
2008 16,439,171 0.9 M 6.8 M 8.7 M
2009 20,351,495 1.3 M 8.8 M 10.1 M
2010 16,203,482 1.3 M 9.2 M 5.4 M
2011 16,033,758 1.9 M 7.0 M 7.2 M
2012 15,149,203 1.3 M 7.1 M 7.1 M
2013 14,552,157 1.3 M 7.6 M 6.1 M
Data table 3: Percentage of goods, services and construction contracts based on total number of contracts (1996-2004)

Source: Treasury Board of Canada Secretariat, 2015

Year % of Total number of contracts (Goods) % of Total number of contracts (Services) % of Total number of contracts (Construction)
1996 52 38 9.5
1997 47.5 48 9
1998 38 59 5
1999 71.5 38 1.4
2000 62 35 2
2001 61.5 35 2
2002 63.5 34 2
2003 63.5 33.5 1.4
2004 58 36 9
Data table 4: Percentage of goods, services and construction contracts based on total number of contracts (2005-13)

Source: Treasury Board of Canada Secretariat, 2015

Year % of Total number of contracts (Goods) % of Total number of contracts (Services) % of Total number of contracts (Construction)
2005 55 39 6
2006 55 40 5
2007 57 40 2
2008 54.5 40.5 5
2009 52 45 3
2010 51 44 5
2011 58.5 38.5 3
2012 56 38.5 5.5
2013 55 39 6

Since the establishment of the office in 2008, the OPO observed inordinate numbers of service contract related complaints. While it may be suggested that the number of service contract complaints would naturally increase as more services were procured, the argument does not appear to hold based on Treasury Board of Canada Secretariat (TBS) procurement data. Despite increases in spending on service procurement over the last 10-15 years, the volume of goods procurement contracts remains higher than the volume of service contracts. Since there are a greater number of goods contracts, one could presume that a higher number of complaints should ensue, but this is not the case.

One could also suggest that the subjective nature of service makes it more difficult to assess or quantify its outcomes, thereby incurring higher than average numbers of complaints. However, analysis of OPO data challenged this reasoning since the majority of complaints filed were related to the contracting phase of the procurement process (i.e. the period in which the request for proposals is developed, bidders' submissions are evaluated) and the contract awarded, and not the contract or post-contract phases where the contracts are amended and performance are evaluated, respectively. At the inception of this research project, the subjectivity of service was hypothesized to have led to increased instances of complaints. However, definitions from half of the participating U.S. jurisdictions provide some indication that service is not so subjective that it cannot be defined.

If the volume of contracts cannot explain high numbers of complaints and if complaints are not related to the evaluation of service contract outcomes, then how can the inordinate number of service contract complaints be explained? What variables may be contributing to this result? Is the public procurement of services inherently different than that of goods? Is the OPO the only jurisdiction experiencing this tendency or is service procurement a challenge across jurisdictions?

The researchers were engaged by the OPO to attempt to answer these questions. To undertake this inquiry, a series of research questions was developed and agreed upon by the researchers and the OPO. Two working hypotheses were proposed and refined based on consultations with international public procurement experts. The consulted experts also verified the research design and methodology. This report's fundamental observation is that a structural incompatibility exists between the actual procurement system (designed for goods) and the increasing public procurement of services (still outnumbered by goods). In our consultations, there was consensus among experts around the term 'incompatible' to explain the current goods procurement system and its mismatch with service procurement.

From the experts' perspective, the fundamental incompatibility is one of human capacity (i.e. procurement officers do not have the education/training to execute service procurement in a complex environment). Building on this perspective, this report suggests that there are various sources of incompatibility including: strategy, people, processes and infrastructure that must be considered to evaluate the existing model for service procurement. While this finding is not definitive, it is a helpful advancement in the largely understudied area of public procurement, notably through the lens of contract complaints.

This report proceeds by establishing the project's research design, followed by its methodology and discussion and analysis of the findings and the comparative data. To conclude, the report suggests future research directions to continue to investigate this subject.

Research design

Context

Canada's modern procurement system was developed during wartime, as many of the historic foundations of government procurement tend to lie at critical moments, e.g. war, a major national undertaking such as space exploration, etc. (Keeney 2007). During World War I (WWI), military procurement in Canada was ad-hoc with the army, air force and navy purchasing independently. To correct deficiencies and limit profiteering by suppliers, the federal government began to coordinate procurement efforts (Plamondon 2010, p. 3). The historic procurement processes established in the mid-20th century to acquire materials tended to emphasize goods (for a short overview of key moments in Canada's procurement history, see).

Summary of Canada's procurement history

To coordinate procurement efforts the federal government established:

  • Defence Purchasing Board (DPB) (operational July 1939).
  • With increased demands of WWII, DPB replaced with War Supply Board (est. November 1939), which became the Department of Munitions and Supply.
  • Non-military supplies at the time were procured through the Canadian Export Board (within the Department of Trade and Commerce).
  • Post-WWII, military procurement was transferred to the Ministry of Trade and Commerce (February 1947), then managed through the Canadian Commercial Corporation.
  • Government reorganization (1963, post-Glassco Commission) saw the establishment of the Canadian Government Purchasing, Supply and Repair Services.
  • In 1993, Public Works and Government Services Canada (PWGSC) was created. PWGSC managed procurement for the entire public service including the military, no longer differentiating between military and non-military procurement.
  • Departments are able to undertake their own procurement for contracts valued under $25,000.
Box 1: Summary of Canada's procurement history (Plamondon 2010, p. 3-7).

The path dependent tendencies of any policy are inclined to lock-in and perpetuate over time (Pierson 2004; Peters 2012; Hall and Taylor 1996). With these precedents, change while not impossible can be difficult. Following the concept of path dependency, the procurement system (made up of strategies, people, processes and infrastructure) has a historical precedent designed to acquire goods that persists, that may impact public procurement today.

Key terms and definitions

For the purposes of this report, goods are defined as articles, commodities, equipment, materials or supplies (following the Government of Canada's Supply Manual Glossary). Service is defined as what is not a good. Only the distinction between goods and service contracts are of interest in this report. As such, all OPO data related to procurement contracts was coded as related to either goods, services, good and service (where the file included both aspects of goods and service) or not applicable (where the file was undefined or not relevant to the study). Construction was not considered an independent category and was coded based on the most closely related of the four categories.Footnote 6

Neither the Treasury Board Contracting Policy (Chapter 16)Footnote 7 nor the Department of Public Works and Government Services' Supply Manual Glossary provide an explicit definition of what constitutes a service, yet definitions for goods and construction exist. In spite of not having a definition for service, the TBS distinguishes between goods, service and construction procurement in its procurement reporting data.Footnote 8

Among public procurement officials, there is no single agreed upon definition of what constitutes a service. The National Association of State Procurement Officials (NASPO), an umbrella organization that represents the state chief procurement officers in the U.S., distinguishes between goods and service procurement. Definitions of service range from what is not a good to clear lists of what constitutes a service, as is the case in various U.S. states (see Table 2).

Goods procurement tends to be input focused where the request for proposals seeks to acquire an article, commodity, equipment, materials or supplies. By contrast, when procuring a service (which, for the purpose of this report is anything but a good), one is more likely than not seeking a solution to a problem or a particular outcome. When procuring a service, "[…] the solicitation should include a program narrative, answering the question, […]" What problem is the agency seeking to solve?"" (NASPO 2015 p. 163). As NASPO's handbook for public procurers explains,

In the purchase of services, the specifications or scopes of work ought to emphasize achieving a specific result, and not the rigid details of the process for getting to it. The goal of a competition is to ensure that the service provider's bid or proposal responds with a solution to a defined need (NASPO 2015, p. 162-163).

The crucial difference in goods versus service procurement comes down to a distinction in purpose. A good is acquired because an article that takes the form of equipment, materials or supplies is needed. When procuring a good, be it to fight a war, to create workspaces by buying desks and chairs, to purchase fencing for a federally sponsored event in the National Capital Region, etc., the item is acquired because it is needed as a tool or input to fulfill a function. By contrast, the procurement of services tends to solve a problem instead of solely acquiring a physical item. The way that service should be bought is different than buying a good. Rather than going to market to buy a specific item (as in goods procurement), one goes to tender seeking solutions to solve a problem.

Table 2: Jurisdictional definitions of service contractsFootnote 9

Arizona

The furnishing of labor, time or effort by a contractor or subcontractor that does not involve the delivery of a specific end product other than required reports and performance. Does not include employment agreements or collective bargaining agreements.

Government of Canada

No definition exists.

Nebraska

That which directly engages in the time or effort of an independent contractor whose purpose is to perform an identifiable task, study, or report rather than to furnish an end item of supply, goods, equipment, or material (Statute 73-502).

New Jersey

A service contract primarily requires labor, time and/or effort to provide intangible general or professional assistance to accomplish an objective (construction and architecture are overseen by a separate division).

North Dakota

The furnishing of labor, time, or effort by a contractor, not involving the delivery of a specific end product other than reports that are merely incidental to their required performance. The term includes professional services.

Pennsylvania

The furnishing of labor, time or effort by a contractor not involving the delivery of a specific end product other than drawings, specifications, or reports which are merely incidental to the required performance. The term includes the routine operation or maintenance of existing structures, buildings, or real property. The term does not include employment agreements or collective bargaining agreements. The term includes utility services and those services formerly provided by public utilities such as electrical, telephone, water, and sewage service.

Given the inordinate numbers of service contract complaints OPO receives, there are likely explanatory variables that can account for the tendency. The researchers have found however, that the data collected by the OPO over its history is of insufficient depth to validate the observations presented in this report. However, leveraging the limited data and the expertise of a panel of international public procurement experts, the researchers have attempted to draw certain inferences as to the root causes of the growth in service complaints. Yet, in spite of consultations with several dozen U.S. jurisdictions and the Organisation for Economic Co-operation and Development (OECD) on behalf of its member countries, a dataset that would substantiate the professional opinions of the expert panel could not be identified. The observations in this report require further study to determine whether they in fact represent root causes.

Research directions in public procurement

While the practice of public procurement has been undertaken in many forms for over a century (see Kenney 2007), its study remains a relatively niche undertaking. In the literature on public procurement, the distinctions between public and private procurement (as well as their interplay) are discussed (Emanuelli 2005; Worthington 2013), the complexities of public procurement are studied (Hartley 1991; Emanuelli 2005; Worthington 2004 and 2013; Thai 2004; Thai et al. 2005; Thai et al. 2007), the roles of human resources and management capacity are assessed (McCue 2001; Snider 2006), as are new trends, e.g. green procurement (OECD 2003; OECD 2013), e-procurement (McCue and Alexandru 2012; Gardenal 2013), alternative service delivery (NASPO 2015), etc. The differences in the complexity of procuring services versus goods are recognized (California 2013; NASPO 2015), with special emphasis on the procurement of Information technology (IT) services (Sieverding 2008; Ancarani, Di Mauro and Giammanco 2009; California 2013). However, the implications of procuring services in systems that tend to be goods oriented, remains understudied, leaving a gap in the literature on public procurement. As governments around the world procure increasing numbers of services, the ways in which they are acquired merits closer attention.

Complaints can be used as a lens through which to study the existing procurement process. The number and nature of procurement related complaints serve as an entry point for testing whether the goods-oriented procurement process is contributing to inordinate numbers of service contract complaints. Based on data collected for this report, the majority of comparable jurisdictions have also experienced higher levels of service contract related complaints. However, no published research appears to have been undertaken to suggest explanations for this state of affairs. Literature reviews and expert consultationsFootnote 10 confirmed that using complaints as a lens to evaluate the procurement process is a unique research angle.

This report confirms that there are higher numbers of service contract complaints versus goods contract complaints (despite the greater volume of goods procurement in the federal government). Through research and expert consultations, potential explanations (that require further research to confirm) include incompatibilities between the goods-oriented procurement system and the procurement of services as well as human capacity in the system to undertake service procurement. Without changing the underlying structure and without training staff to think differently about the task of service procurement, the Procurement Ombudsman may inevitably continue to receive inordinate numbers of service contract versus goods contract related complaints.

The content of this report is based on the OPO's internal data, data shared by comparable jurisdictions, expert consultations, Government of Canada data, as well as secondary sources (including scientific articles and expert reports). This report serves not only to fulfill the OPO's research request, but will also contribute to the void in the literature on the incompatibility between the use of a goods-oriented procurement process for the procurement of services.

Research questions

In consultation with the OPO, the following research questions were defined to guide the inquiry:

  1. What inherent differences in contracting for services (as opposed to goods) may be contributing to the disproportionate number of complaints about service contracts compared to contracts for goods?
  2. What issues, factors and/or reasons may be contributing to the disproportionate number of complaints related to services?
  3. What conclusions may be drawn from the research?
  4. Are comparable jurisdictions experiencing similar problems?

Hypotheses

Based on initial research, two hypotheses were developed to confirm the higher number of service contract complaints relative to goods contract complaints and to suggest potential explanations. Both hypotheses were confirmed through data collection and verified by expert consultations.Footnote 11

This report's hypotheses are:

H1:

There are an inordinate number of service contract related complaints v. goods contract related complaints.

To validate H1, there should be higher numbers of service contract complaints than goods contract complaints. The data confirmed H1 with 79% of relevant complaints related to service contracts compared to 21% for goods contracts based on OPO's data since 2008.

H2:

Goods procurement is input focused, whereas service procurement is outcome focused. Therefore, there is a structural incompatibility of the existing procurement model (developed to procure goods) for service procurement.

If H2 holds, there should be higher numbers of complaints in the pre-contractual and contracting phases, suggesting that the procurement process may be oriented toward input-based goods instead of outcome oriented service procurement. The data confirmed H2 with 55% of filed complaints since 2008 related to the contracting phase. The researchers did not assess the complaints in the context of the Procurement Ombudsman's mandate but on which contract phase the complaint falls.

Confirmation of the two hypotheses point to possible root causes of difficulties with service procurement, rather than a definitive "theory."  To ensure that the results of this study are not overstated, it should be noted that there is broad agreement among experts that there is currently insufficient data and analysis in the literature to fully explain the endemic problems associated with service procurement—it is a work in progress.

To better establish the linkages made in this report, it may be useful to draw a statistically valid sample of the OPO files for further in-depth assessment. Identifying clearly the reason for the complaint and even interviewing the complainant can help to obtain more explicit conclusions about the variables contributing to higher numbers of service contract complaints. This supplementary focused researched can verify the report's observation of an incompatibility between the procurement process and the services being procured.

Methodology

For this project, a scoping exercise of OPO's complaints files was undertaken to assess the numbers of and differences between goods and service contract complaints. Given the confidential nature of OPO's files, approximately eight members of the office's staff reviewed all procurement complaint files since the office's opening in 2008 (the researchers were not allowed to handle the confidential files). While there may be inconsistencies in the subjective nature of data collection and recording given the involvement of eight staff, standardization was encouraged through the use of a pre-established template (see Annex 2 for the template and Table 1 for coding (i.e. OPO's contract phase descriptions)).

OPO staff reviewed complaint files and recorded relevant data based on the nature of the contract (i.e. good v. service), the type of complaint (i.e. related contract phase), the related department, as well as the actions taken or resolution obtained. Only complaints related to procurement contracts were reviewed and counted in this assessment. All other files (e.g. general inquiry calls to the office) were not counted. Data collected by OPO staff was anonymized with all indicators removed so that it could be made public. The researchers recorded and coded the anonymized data.

In total 668 complaint files were received from the OPO and coded by the researchers. Of the 668 entries, 480 were related to service contracts, 125 related to goods, 7 were classified as part goods contract and part service contract, and 56 were not applicable (N/A) as they were either undefined or irrelevant to the data set (e.g. question to the OPO) (see Table 3 and Data tables 5 and Data table 6).

Data table 5: Complaint types by number 2008-14, n=668
Complaint type Number
Goods 125
Service 480
Goods and services 7
N/A 56

For the purpose of this final report, as well as for comparability with other jurisdictions, only files clearly defined as goods or service related were analyzed. This reduces the number of applicable files from 668 to 605.

Data table 6: Complaint types by percentage 2008-14, n=605
Complaint type Percentage (%)
Goods contract 19
Service contract 72
Goods and services 1
N/A 8

This methodology was crucial for two reasons. First, for the researchers to test H1, all complaints and the split between goods and service had to be tabulated. Second, to test H2, it was necessary to link the complaint to a specific contract phase. OPO's pre-defined contract phases were adopted in this report and were used to code the "time" (i.e. related contract phase) of the complaint linking it to a set of steps and practices in the procurement process. See Table 1 for the phases and their descriptions.

When supported by a critical mass of data, identifying the related contract phase can suggest reasons for the higher number of service versus goods contract related complaints. For instance, complaints related to phases #1 and #2 (see Table 1 for definitions), may suggest pre-contract issues related to the formulation of the request for proposals as well as issues related to assessment and interpretation of terms in the evaluation process.

To test research findings, and to determine if similar jurisdictions were experiencing analogous complaint splits, a comparative assessment with data from comparable jurisdictions was undertaken. The 50 states of the United States were targeted for comparison, as many share geographical similarities with Canada and work with similar vendors. While not all states have the budgets of the Canadian federal government or similar populations to that of Canada in its entirety, many states have procurement-specific complaints processes embedded in their state system.

As a professional association, NASPO produces research, organizes education and training and convenes its members to address the challenges and practices of public procurement. In 2014, NASPO undertook a survey of "State Procurement Practices". Of its 50 members, 42 have legislation, rules or regulations related to procurement complaints. The researchers contacted these 42 states requesting data on the number of complaints their office received, the split between goods and service complaints, as well as their definitions (if they exist) for good and service (see Annex 3 for a sample data request chart and Annex 4 for a list of the contacted jurisdictions).

There were 16 participating jurisdictions that provided full or partial data (including a definition of "service") for this report. All of the jurisdictions were assured that the collected data was anonymized to ensure it could not be attributed to any specific jurisdiction, although some officials noted that their data is in fact publicly available.Footnote 12 There was significant interest among state procurement officials in the results of this study. To promote collaboration and interjurisdictional comparisons, anonymized data collected for this report will be shared in aggregate form with the participating jurisdictions.

The methodology for this report is rooted in inductive analysis. In conducting their enquiry, the researchers moved from a specific observation that OPO was experiencing inordinate numbers of service contract complaints to a discovery that most of the complaints link to the initial phases of the procurement process. While this report's dataset cannot explain why this pattern exists, inductive reasoning confirms that a pattern does exist.

Data from U.S. jurisdictions confirmed that OPO's experience is not unique. To suggest potential explanations and to establish future research directions, public procurement experts were consulted. The experts came to the conclusion that the current system is incompatible for the procurement of services and is exacerbated by a lack of human capacity in procurement, leading to higher numbers of service contract complaints. These potential explanations merit further research.

Findings

This section will discuss and analyze the report's results through the lenses of the two hypotheses and the results of their testing.

It was crucial to verify that there are consistently higher numbers of service contract complaints than goods contract complaints as the foundational premise of this report. The data confirmed H1 with 79% of relevant complaints related to service contracts compared to 21% for goods contracts based on data since 2008 (see Data table 7).

Data table 7: Goods v. service complaints (based on relevant complaints only)
Type of complaint Percentage (%)
Service contract 79
Goods contract 21

With service complaints data significantly outnumbering those on goods, a time series of complaints since 2008 was developed to ensure that the discrepancy could not be attributed to anomalies or complaint "spikes" in particular years. As Data table 8 demonstrates, since 2008, service contract complaints have consistently outnumbered goods contract complaints, suggesting a relatively stable ratio (service: goods) over the study period.

The consistently higher numbers of service contract versus goods contract related complaints may suggest that a constant variable or set of variables is contributing to this trend.

The significantly higher numbers of service contract complaints were unsurprising to the panel of experts. For instance, NASPO indicated that service procurement, if not done properly, can be challenging. The organization is in the process of developing an educational platform for public procurement support for entry to senior-level state procurement personnel, which includes access to education and training on topics such as service level contracting methods.

Having established that service contract complaint numbers are much higher than goods contract complaints, the next section focuses on identifying potential variables to better understand the state of affairs.

Underlying H2 are assumptions about the procurement of goods and services. Working from the assumption that the procurement of goods and services should reflect their different goals, the second hypothesis seeks to test whether the way in which goods and services are procured can be a variable in the higher number of service contract complaints because of their different purposes. At an operational level, it is hypothesized that the distinctions manifest themselves as a structural incompatibility between the existing procurement model developed to procure input-oriented goods and the use of the model to procure outcome-oriented services. Can a model built to procure a good be applied to procure (what are often more complex) services?

Data table 8: Number of complaints received (per annum) by complaint type
Year Number of goods contract complaints Number of service contract complaints
2008-09 29 63
2009-10 8 26
2010-11 14 53
2011-12 22 85
2012-13 14 80
2013-14 20 78
2014-15 18 95

The majority of the experts consulted pointed to a problem in human capacity when it comes to service procurement (see Box 2). From the experts' perspective, since the procurement of services is undertaken for an outcome and not an input distinguishing it from goods, it requires different processes (e.g. evaluation) and a different organizational culture to manage these files.

Experts and the importance of human capacity

In order to investigate potential root causes of service complaints in the procurement system, an international panel of experts was retained based on:

  1. Their global public sector perspective and experiences.
  2. Their knowledge of research, academic literature and their own contributions to the public procurement field.
  3. Their own experiences consulting and assessing public procurement problems to identify and understand possible avenues for future study to assess the identified problems/weaknesses in the system.

The experts generally agreed that beyond the structural incompatibilities, procurement is an inherent human problem.

Box 2: The functions of the international panel of public procurement experts and their principal conclusions.

As Professor Stephen Gordon of Old Dominion University indicated, public procurement is more than a process; it is part of a complex system of government that must respond to different pressures. Given the complex and dynamic environment of public procurement generally, the higher level of difficulty associated with the procurement of services as compared to the procurement of goods, the capability of individual procurement and contracting officials to execute service procurement as it should be executed is critical to the strategic success of public organizations.

The idea of focusing on individual procurers for change was reiterated by Professor Dermot Cahill, Bangor University Institute for Competition and Procurement Studies as well as NASPO that is providing procurement support and training. There is broad agreement among experts that change has to come from individual procurers who would adjust their analysis for the procurement of services on a case-by-case basis. The experts were clear that this type of change requires training and education programs as well as changes in organizational culture.

On the issue of human capacity, Professor Khi Thai of Florida Atlantic University specifically emphasized the importance of refining evaluation procedures to support individual procurers in assessing outcomes. Clear criteria for services do not always exist, thus procuring outcomes can be challenging. Often times, Dr. Thai indicated, procurement committees will default to price, treating lowest cost (i.e. best value for money) as the best option. This however, can be problematic because it may not fulfill the required outcome. Further investigation is required to address the question of "best value v. lowest cost" in public procurement, since how to benchmark quality against price differences remains to be studied.

Based on the researchers' consultations, there was consensus among experts around the term 'incompatible' to explain the actual goods procurement system and its mismatch with service procurement.

The data confirmed H2 with the highest numbers of complaints (55%) filed for the contracting phase (phase #2) since 2008. There were 668 complaint files received from the OPO and coded by the researchers. The related contract phases are depicted in Table 3 and their relative weights illustrated in Data table 9.

Table 3: Complaints and related contract phases
Phase Total related complaints
1. Pre-contractual
106
2. Contracting
369
3. Contract administration
109
4. Post-contract
12
1&2
5
2&3
1
2&4
1
3&4
1
N/A
64
Total
668
Data table 9: Percentage of complaints by contract phase
Contract phase Percentage (%)
Phase one (Pre-contractual) 16
Phase two (Contracting) 55
Phase three (Contract administration) 16
Phase 4 (Post-contract) 2
N/A (Undefined or unapplicable) 10
Combined phases 1

For the purpose of this report, only files clearly related to phases #1-4 were analyzed. As previously noted, combined classifications as well as files that were not applicable (N/A) were eliminated. This reduced the number of applicable files from 668 to 596. The number of applicable files was further reduced to 563 by only analyzing files that were clearly defined as being either goods or service related (eliminating, as was previously done the N/A and goods and service tags). The distribution of complaints by contract phase after reducing the sample is shown in Data table 10.

Data table 10: Complaints by contract phase using reduced sample, n=563 (2008-14)
Contract phase Number of complaints
Phase 1 102
Phase 2 350
Phase 3 99
Phase 4 12
Data table 11: Service complaints only by contract phase (2008-14)
Contract phase Number of complaints
Phase 1 77
Phase 2 281
Phase 3 79
Phase 4 8
Data table 12: Goods complaints only by contract phase (2008-14)
Contract phase Number of complaints
Phase 1 25
Phase 2 69
Phase 3 20
Phase 4 4

When sorted based on contract phase, most complaints are related to phases #1, #2 and #3 (see Data table 10), with the vast majority related specifically to phase #2. This trend holds for both goods and service complaints when they are isolated, with both demonstrating higher instances of contracting phase #2 connections (see Data tables 11 and 12).

That the problems originate principally from contracting phase #2 suggests that it is not the subjective nature of evaluating a service that is at issue. Rather, since Phase #2 contains the steps from evaluation to contract award, it can be suggested that there is an incompatibility with the strategy, people, processes and infrastructure that inform the actual procurement system, potentially for both goods and service procurement. For instance, if a government agency is tendering for software, it may release a request for proposals that seeks labour at an hourly rate. The agent appears to be procuring a good (labour) when in fact, they should be looking for a solution to a problem and should formulate their request accordingly.

Since phase #2 complaints are highest for goods and services, further research is required to determine precisely which variables are informing the result. Are unsuccessful bidders filing complaints, thereby driving up numbers? Is the evaluation of bids a source of contention for vendors? Are the underlying requests for proposals unclear and inviting unsuitable vendors to bid? Can specific elements in the procurement system be isolated to explain the complaints discrepancy? Is it a problem of human capacity? Or is it a combination of the two?

While the results of this report are not definitive, the evidence in consultation with the panel of public procurement experts appears to suggest that there are problems with the formulation and tendering of requests for proposals and the evaluation of bids that lead to higher numbers of service contract complaints. As in the example of software acquisition above, public procurers may be at an inherent disadvantage by treating procurement as a linear process (i.e. checking boxes) that can be applied in full no matter the type of procurement at hand.

Comparative data

To verify the findings from OPO's data, the researchers collected data from comparative jurisdictions from Canadian provinces and U.S. states. All Canadian provinces and territories (with exception to the Northwest Territories, Nunavut and Prince Edward Island) have provincial ombudsmen (see Table 4). These offices tend to be general ombudsmen managing complaints related to provincial services e.g. health, and report directly to the legislature (unlike the federal OPO). There is no domestic provincial equivalent of a procurement ombudsman in Canada. The majority of provincial ombudsmen do receive procurement contract related complaints (Quebec only accepts complaints related to provincial access to services). Due to low frequency of complaints, most provincial offices do not collect data on procurement complaints. Anecdotally however, the participating provincial offices indicated higher numbers of service contract related complaints when procurement related complaints are received.

Table 4: Canadian provincial ombudsmen offices' anecdotal evidence on procurement contract related complaints

Jurisdiction and comments

  • Alberta
    • Jurisdiction over procurement contract complaints; few complaints received.
    • Of few received, mainly dealt with services.
  • British Columbia
    • Jurisdiction over procurement contract complaints.
    • 289 complaint files since 2008 (unable to review goods v. service split).
  • Manitoba
    • Unresponsive
  • New Brunswick
    • Unresponsive
  • Newfoundland and Labrador
    • Jurisdiction over procurement contract complaints.
    • Approximately 16 complaints since 2002.
    • Majority related to service contracts.
  • Nova Scotia
    • Unresponsive
  • Ontario
    • Jurisdiction over procurement contract complaints.
    • Mostly related to services.
  • Quebec
    • Not applicable. Only accepts complaints related to provincial access to services.
  • Saskatchewan
    • Jurisdiction over procurement contract complaints.
    • Rarely receives such complaints.
  • Yukon
    • Jurisdiction over procurement contract complaints. No data on complaints.

Note: Canadian provinces and territories (with exception to the Northwest Territories, Nunavut and Prince Edward Island) have provincial ombudsmen.

Although the Canadian data could not further substantiate the report's initial findings, U.S. states provided a fruitful ground for comparison. The 42 comparative jurisdictions derived from NASPO's 2014 survey were contacted to request procurement complaints data.

Data table 13: Number and relevant sample of participating U.S. jurisdictions
Type of respondents Amount
Total sample population 42
Responsive jurisdictions 16
Participating jurisdictions 11
Applicable sample 5

As depicted in Data table 13, of the 42 contacted jurisdictions, 16 jurisdictions were responsive and provided some form of data and/or information including complaints data and/or definitions of goods and service. From the 16 jurisdictions, 11 specifically shared complaints data of which five tracked the difference between goods contract and service contract related complaints on an annual basis. These five jurisdictions serve as the comparative sample population for this report. Two other jurisdictions produced complaints data differentiating between goods and service but only in total/aggregate form and not on an annual basis. To ensure comparability with OPO's data and to verify complaint levels across the time series, only jurisdictions that tracked goods versus service contract complaint data annually were used.

Data tables 14-18 depict the complaint numbers based on complaint type for Jurisdictions 1-5 on an annual basis.

Data table 14: Jurisdiction 1, complaint type, annual basis
Year Number of goods complaints Number of service complaints
2008 6 9
2009 5 12
2010 11 11
2011 7 4
2012 5 11
2013 2 3
2014 1 5

Note: As of 2011, procurement of Information technology (IT) services for Jurisdiction 1 was moved to a separate administrative branch.

Data table 15: Jurisdiction 2, complaint type, annual basis
Year Number of goods complaints Number of service complaints
2008 25 16
2009 36 14
2010 24 17
2011 33 29
2012 30 18
2013 12 18
2014 10 15
Data table 16: Jurisdiction 3, complaint type, annual basis
Year Number of goods complaints Number of service complaints
2008 0 0
2009 0 2
2010 0 2
2011 3 2
2012 1 1
2013 11 11
2014 1 5

Note: Data not collected in Jurisdiction 3 prior to 2009.

Data table 17: Jurisdiction 4, complaint type, annual basis
Year Number of goods complaints Number of service complaints
2008 3 16
2009 7 12
2010 2 24
2011 0 16
2012 1 18
2013 4 9
2014 2 10
Data table 18: Jurisdiction 5, complaint type, annual basis

Note: Data not collected in Jurisdiction 5 prior to 2012.

Year Goods complaints Service complaints
2012 15 10
2013 14 9
2014 15 10

Three out of five jurisdictions indicate higher numbers of service contract versus goods contract complaints. In aggregate form (i.e. total numbers of goods and service complaint data since 2008) and with two additional jurisdictions that provided data on complaints but not on an annual basis, the collected data indicates a trend similar to OPO's where the U.S. jurisdictions experience higher numbers of service contract versus goods contract complaints since 2008.

Data table 19: Comparing total complaints by type and number of Jurisdictions 1-7 and OPO
Jurisdiction Goods complaints Service complaints
1 37 55
2 170 127
3 16 23
4 19 105
5 15 10
6 25 229
7 33 2
OPO 125 480

Given the trends in data from U.S. jurisdictions, OPO's experience of higher numbers of service contract complaints does not appear to be unique. The comparative research also highlighted that some jurisdictions make the distinction between of goods, services, IT and even construction materials. These distinctions have been recognized administratively by separating their procurement functions into different agencies.

In Mississippi for instance, there are four oversight agencies for procurement that have been established in recognition of their differences: one for Commodities (non-IT-related); one for Services; one for Construction and one for IT services. This differentiation underscores the distinct processes and expertise required for acquiring goods versus services. Pennsylvania has underscored the complexity of IT services procurement by delegating the responsibility to the Office of Administration in the Governor's Office, since 2011.

Having experienced difficulties in procuring major IT projects (e.g. canceling or suspending IT projects after years of development, projects exceeding cost and scheduling assessments), California struck a taskforce to study the situation and provide solutions named, the California Task Force on Reengineering IT Procurement for Success. Reporting in August 2013, the task force made various recommendations to improve IT procurement through a more in-depth pre-contract phase, human resources with procurement expertise, changes to bid evaluation, etc. The recommendations from California's taskforce appear to align with this report's suggestion of an incompatibility between the procurement of service and the existing procurement system, including strategy, people, processes and infrastructure.

Observations and next steps

The Procurement Ombudsman's request to assess inordinate numbers of service contract complaints through the lens of complaints data appears to be novel. The findings of this report may, in-fact, reflect the limits of potential analysis to test the two hypotheses, based on available data, a constructed comparative data set and expert consultations.

The following observations are derived from the data collected for this report:

  1. OPO is not unique in experiencing higher instances of service contract complaints.
  2. There is an incompatibility between the procurement of outcome-oriented services and the underlying goods-oriented procurement system.
  3. Further research is required to isolate explanations to definitively conclude why OPO is experiencing these results and their causes.

While the numbers of service complaints in Canada have increased beyond their relative increase in volume, causal linkages cannot be established to explain the inordinate number of service contract related complaints filed with the OPO. TBS data indicates that the quantity of goods procurement still outnumber the number of service procurement contracts, yet service contracts continue to register higher numbers of complaints.

Without more information from OPO files or vendor interviews, OPO data is limited to showing higher instances of complaints for contract phase #2. When these results were discussed with the expert panel, there was virtual unison in their interpretation of the findings. Expert commentary coalesced around the idea that human capacity is the most significant driver of service contract based complaints. The procurement process underlying the staff is unsuitable for complex outcome-based service procurement. Political and bureaucratic incentives muddy the environment, leaving accountability for procurement nebulous.

The researchers were asked to better understand why service complaints outnumbered goods. Beyond the analysis undertaken in this report, the data cannot be extrapolated further to explain or prove why there are higher numbers of service complaints. Therefore, suggestions for future study are made based on expert consultations to frame the current findings and propose directions for next studies.

Data to explain why the numbers of service contract complaints are higher can be acquired in different ways, such as internal analysis and assessment because OPO staff has most direct access to files. Future research directions may include: comparative analysis of jurisdictions with high levels of service contract procurement complaints as well as jurisdictions that treat service procurement as a separate issue (e.g. by placing it in a separate department); interviewing current public procurers to assess whether service and goods procurement are viewed differently; and interviewing complainants to better understand the nature of their complaint.

Public procurement is a multifaceted undertaking in a complex environment of political incentives and rules. To procure today is doing more than acquiring an item. With the increasing procurement of various services and the different requirements for outcome or solution oriented service procurement versus input oriented goods procurement, a rethinking of the strategies, processes and human resources required to effectively procure a service may be necessary.

Annex 1: Operationalization of the Procurement Ombudsman's mandate

The Procurement Ombudsman operationalizes their mandate by evaluating the procurement practices of departments and receiving complaints from Canadian suppliers. Departmental procurement practices are reviewed for instance, to ensure that they align with Cabinet policies, that they respect the Financial Administration Act, that the practices adhere to previous reviews of the department's procurement practices, etc.

Procurement contract complaints, that are the focus of this report, tend to fall in two categories: first, to contest a contract award; and the second, is to contest the administration of a contract.

For both types of complaints, Canadian suppliers can file a complaint in writing with the Procurement Ombudsman (PWGSC Act, 22.2 (1)) within 30 days after a contract award has been announced (and up to 90 days after the announcement if an exception is granted by the OPO) (Procurement Ombudsman Regulations 7(1)). A complaint is considered filed when the Procurement Ombudsman receives the required information that includes documentation to establish the complaint, the facts and grounds of the complaint, the potential net profit the complainant could have made and the complainant's cost of submitting the bid (Procurement Ombudsman Regulations, 7(4)). Once the complaint is filed, the Procurement Ombudsman has 10 working days to notify the complainant and the relevant department whether the complaint will be reviewed (Procurement Ombudsman Regulations, 8).

Beyond meeting the minimum information requirements established in section 7 of the regulations, the facts and grounds for the complaint can never have been subject to court proceedings or have been a matter before the Canadian International Trade Tribunal (CITT) and cannot be exempted by any legislation (Procurement Ombudsman Regulations, 9(1)). At any time, a review by the Procurement Ombudsman can be ended for not meeting requirements, if the complainant withdraws the complaint, if the contract award is cancelled or if the Procurement Ombudsman "determines that the complaint is frivolous or vexatious" (Procurement Ombudsman Regulations, 10).

The Procurement Ombudsman typically reviews complaints "under a simple review process", (Procurement Ombudsman Regulations, 11(1)). Relevant departments are given 15 working days to provide comments to the complainant on the file. An expanded review process at the request of the complainant or the department or at the initiative of the Procurement Ombudsman is also possible. Involved parties must be notified immediately if the Procurement Ombudsman is undertaking an expanded review.

For an expanded review, the nature of the complaint, the value of the contract, the complexity of the bid requirements or procurement process, potential impacts that findings and any recommendations could have on government operations and resources must be considered (Procurement Ombudsman Regulations, 11(2)). During an expanded review, the Procurement Ombudsman reviews the complaint file and mediates as the department concerned and the complainant provide comments giving the other party an opportunity to respond in 10 working days (beginning with the department) (Procurement Ombudsman Regulations, 11(2)).

Regardless of whether a simple or expanded review is undertaken by the Procurement Ombudsman, they are required to report on their findings and recommendations no more than 120 working days after the complaint was filed (Procurement Ombudsman Regulations, 14) to the complainant, the minister of the department concerned and the Minister of PWGSC. The Procurement Ombudsman is restricted to recommending compensation if found to be reasonableFootnote 13, since only the Governor in Council can impose fines (Public Works and Government Services Act, 23(2)).

When the Procurement Ombudsman reviews contract administration complaints, the same validation process as with contract award complaints is followed. The review process however, differs slightly. To review a contract administration complaint, the Procurement Ombudsman considers whether the administration was reasonable given the circumstances and whether the parties acted in bad faith (Procurement Ombudsman Regulations, 20). While the Procurement Ombudsman cannot make recommendations to alter the terms and conditions of the contract or to suggest a remedy other than those in the contract (Procurement Ombudsman Regulations, 21), there are alternate means for resolution, namely through alternative dispute resolution process (ADR).

ADR is an option to settle disputes if requested by a party to the contract related "to a dispute on the interpretation or application of contract's terms and conditions" (Procurement Ombudsman Regulations, 23(1)). In order to proceed with ADR, the Procurement Ombudsman must obtain agreement of all parties, with participants agreeing to bear the costs. ADR only proceeds if parties agree to the terms.

Annex 2: Data collection template

File/Vendor # Nature of contract
(G or S)
Industry Vendor specs Related department Nature of complaint Related phase (1-4) Resolution? (Y/N) Date of complaint (MM/YY) Date of resolution (MM/YY)
2012/13-115 (S) N/A N/A National Research Council Canada Security clearance 1- Pre-
contractual
Out of mandate; considered for future Procurement Practice Reviews (PPR) 12-Aug Not indicated
2012/13-301 (S) N/A N/A Canadian Nuclear Safety Commission (CNSC) Kickbacks 1- Pre-
contractual
Out of mandate; Privy Council Office (PCO) is conducting investigation centrally 13-Feb Not indicated
2012/13-267 (S) N/A N/A Government wide Employer-
employee relationship
1- Pre-
contractual
Out of mandate; systemic issue, considered for future PPR 13-Jan Not indicated
2014/15-159 (S) N/A N/A PWGSC obo Health Canada Security 1- Pre-
contractual
Out of mandate; not contract holder 14-Aug Not indicated
2014/15-511 (S) N/A N/A N/A Security 1- Pre-
contractual
Provided Criminal Intelligence Service Canada (CISC) information 15-Mar Not indicated
2014/15-002 (S) N/A N/A Canada Revenue Agency (CRA) Statement of work (SOW) 1- Pre-
contractual
Did not provide permission to contact department 14-Apr Not indicated
2014/15-491 (S) N/A N/A PWGSC Library and Archives Not getting work 1- Pre-
contractual
Out of mandate; Supply Arrangement (SA) 15-Feb Not indicated
2014/15-489 (S) N/A N/A N/A Evaluation and selection plan 1- Pre-
contractual
Provided options to consider 15-Feb Not indicated
2013/14-336 (S) N/A N/A Canadian Human Rights Tribunal Constructing directly with company's resource 1- Pre-
contractual
Complainant did not come forward in writing 13-Dec Not indicated
2012/13-303 (S) N/A N/A Industry Canada Restrictive mandatory criteria 1- Pre-
contractual
Out of mandate; OPO was only cc'd 13-Feb Not indicated
2011/12-221 (S) N/A N/A Not indicated Restrictive requirements 1- Pre-
contractual
Withdrawn 11-Nov Not indicated
2010/11-061 (S) N/A N/A PWGSC Selection criteria favoured one bidder 1- Pre-
contractual
Out of mandate 10-Jun Closed
2014/15-301 (S) N/A N/A Unknown SOW and evaluation and selection plan 1- Pre-
contractual
Would be considered for future PPR 14-Nov Not indicated

Annex 3: Data request chart

Jurisdiction name:

Total annual complaints
This table is only an example
and contains no data.
Year Total number of complaints
2008  
2009  
2010  
2011  
2012  
2013  
2014  
Goods v. service complaints
This table is only an example
and contains no data.
Year Good complaints Service complaints
2008    
2009    
2010    
2011    
2012    
2013    
2014    
Definitions
This table is only an example
and contains no data.
Good Service
   

Annex 4: Contacted jurisdictions

  1. Alabama
  2. Alaska
  3. Arizona
  4. Arkansas
  5. California
  6. District of Columbia
  7. Delaware
  8. Florida
  9. Georgia
  10. Hawaii
  11. Idaho
  12. Illinois
  13. Indiana
  14. Iowa
  15. Kansas
  16. Louisiana
  17. Maine
  18. Maryland
  19. Michigan
  20. Mississippi
  21. Missouri
  1. Montana
  2. Nebraska
  3. Nevada
  4. New Hampshire
  5. New Jersey
  6. New Mexico
  7. New York
  8. North Carolina
  9. North Dakota
  10. Oklahoma
  11. Oregon
  12. Pennsylvanie
  13. Rhode Island
  14. South Carolina
  15. Tennessee
  16. Utah
  17. Vermont
  18. Virginia
  19. Washington
  20. West Virginia
  21. Wisconsin

Works cited

  • Ancarani, Alessandra, Mauro, Carmela, M. Daniela Di Giammanco. (2009). "The Purchase of Technology in Health Organisations: An Analysis of its Impact on Performance." Journal of Public Procurement 9(1): 109-131.
  • Emanuelli, Paul. (2005). Government Procurement. Markham, ON: LexisNexis Butterworths.
  • Gardenal, Francesco. (2013). "A Model to Measure e-Procurement Impacts on Organizational Performance." Journal of Public Procurement 13(2): 215-242.
  • Government of Canada. (2008). Available at: Procurement Ombudsman Regulations. Ottawa: Minister of Justice.
  • -------. (2006). Available at: Department of Public Works and Government Services Act. Ottawa: Minister of Justice.
  • Hall, Peter A. and Rosemary Taylor. (1996). "Political Science and the Three New Institutionalisms." Political Studies XLIV: 936-957.
  • Hartley, Keith. (1991). "Public Purchasing." Public Money and Management 11(1): 45-49.
  • Keeney, Sandy. (2007). "The Foundations of Government Contracting." Journal of Contract Management (Summer 2007): 7-19.
  • Library and Archives Canada. (2001). "ARCHIVED - Canadian Confederation" (accessed June 12, 2015).
  • McCue, Clifford and Roman, Alexandru V. "E-Procurement: Myth or Reality?" Journal of Public Procurement 12(2): 221-248.
  • McCue, Clifford P., Gianakis, Gerasimos A. (2001). "Public Purchasing: Who's Minding the Store?" Journal of Public Procurement 1(1): 71-95.
  • National Association of State Procurement Officers (NASPO). NASPO State and Local Procurement: A Practical Guide, Second Edition. Washington, D.C.
  • -------. (2014). 2014 Survey of State Procurement Practices. Washington, D.C.
  • Website: Office of the Procurement Ombudsman. (2015).
  • -------. (2013). Presentation to the Delegation from the Ministry of Supervision People's Republic of China.
  • Organisation for Economic Co-operation and Development (OECD) (2013). Mapping out good practices for promoting green public procurement. Paris: OECD Publishing.
  • -------. (2003). The Environmental Performance of Public Procurement: Issues of Policy Coherence. Paris: OECD Publishing.
  • Pierson, Paul. (2004). Politics in Time: History, Institutions and Social Analysis. Princeton, NJ: Princeton University Press.
  • Peters, Guy B. (2012). Institutionalist Theory in Political Science, 3rd Edition. New York: Continuum.
  • Plamondon, Aaron. (2010). The Politics of Procurement. University of British Columbia (UBC) Press: Vancouver.
  • Public Works and Government Services Canada. (2015). Available at: "1. Supply Manual Glossary".
  • Sieverding, McLean. (2008). "Choice in Government Software Procurement: A Winning Strategy." Journal of Public Procurement 8(1): 70-97.
  • Snider, Kieth F. (2006). "Procurement Leadership: From Means to Ends." Journal of Public Procurement 6(3): 274-294.
  • State of California. (2013). Available at: "Recommendations to improve large information technology procurements: A road map for success in California" (PDF version 960Kb) (Help with alternative formats and plug-ins). Taskforce on Reengineering IT Procurement for Success.
  • Thai, Khi. (2004). "Challenges in Public Procurement: An International Perspective—Symposium Introduction." Journal of Public Procurement 4(3): 312-318.
  • Thai, Khi, C.M. Hartland, J. Telgen, L.A. Knight and G. Callendar. (2007). "Challenges Facing Public Procurement." In L.A. Knight et al. (Eds). Public Procurement: International Cases and Commentary. London, UK: Routledge.
  • Thai, Khi et al. (2005). "Challenges in Public Procurement." K. Thai et al. (Eds). Challenges in Public Procurement: An International Perspective. Boca Raton, FL: PrAcademics Press.
  • Treasury Board of Canada Secretariat (2015). "Publications - Contracting Data" (accessed June 16, 2015).
  • -------. (2013), "Publications - Contracting Data" (accessed April 27, 2015).
  • Worthington, Robert C. (2013). Desktop guide to procurement law. Markham, ON: LexisNexis.
  • -------. (2004). The public purchasing law handbook. Markham, ON: LexisNexis.

Experts consulted

  • Dermot Cahill, Professor, Bangor University Institute for Competition and Procurement Studies, United Kingdom.
  • Paul Emanuelli, General Counsel and Managing Director, the Procurement Office, Toronto, Canada.
  • Krista Ferrel (for NASPO), Director of Strategic Programs, National Association of State Procurement Officials, United States.
  • Steven Graves, Professor, Massachusetts Institute of Technology (MIT), United States.
  • Stephen B. Gordon, Professor of Practice, Public Administration, Old Dominion University, United States.
  • Khai Thai, School of Public Administration, Florida Atlantic University, United States.

Footnotes

Footnote 1

Return to footnote 1 referrer

In April 1873, it was discovered that then Prime Minister Sir John A. Macdonald was accepting large sums of money from Sir Hugh Allan. In exchange, Sir Allan was assured that he would be awarded the lucrative contract to build the transcontinental Canadian Pacific Railway. A Royal Commission was struck in August 1873 to investigate what came to be known as the Pacific Scandal. With evidence confirming the affair, Sir John A. Macdonald's government resigned in November 1873 and a general election ensued. In 1878, Sir John A. Macdonald's government was reelected and returned to power (Library and Archives Canada (2001), "ARCHIVED - Canadian Confederation", (accessed June 12, 2015).

Footnote 2

Return to footnote 2 referrer

See Government of Canada, (2015), Department of Public Works and Government Services Act.

Footnote 3

Return to footnote 3 referrer

See Government of Canada, (2015), Procurement Ombudsman Regulations.

Footnote 4

The Governor in Council or the Minister of Public Works and Government Services can assign other duties or functions to the Procurement Ombudsman related to procurement practices of goods and services.

Return to footnote 4 referrer

Footnote 5

Return to footnote 5 referrer

See also the Office of the Procurement Ombudsman's Website.

Footnote 6

Given the complexity of service procurement, it is advisable that the Government of Canada develops a definition of service (or define what it is not).

Return to footnote 6 referrer

Footnote 7

Return to footnote 7 referrer

Public Works and Government Services Canada (2015), "1. Supply Manual Glossary", (accessed April 27, 2015).

Footnote 8

Return to footnote 8 referrer

Treasury Board of Canada Secretariat (2013), "Publications – Contracting Data", (accessed April 27, 2015).

Footnote 9

For jurisdictions other than Canada, definitions were obtained through email exchanges for data collection with state procurement offices.

Return to footnote 9 referrer

Footnote 10

Six consultations were undertaken with academics and practitioners in the field of procurement. For a list of the consulted experts, see the "Experts Consulted" section of the Works Cited page.

Return to footnote 10 referrer

Footnote 11

Adjustments were made to the original hypotheses being tested based on the availability of data and consultations with experts.

Return to footnote 11 referrer

Footnote 12

Jurisdictions that made their procurement complaints data public cited transparency and accountability as mechanisms to improve their quality of work. If the public can track complaints and their related categories, the system may be more inclined to undertake more efficient procurement practices.

Return to footnote 12 referrer

Footnote 13

As per the Procurement Ombudsman Regulations, compensation may be recommended by the Procurement Ombudsman in the following ways:

13. (1) Subject to subsection (2), the Procurement Ombudsman may recommend payment of:

  1. compensation in respect of the profit that the complainant would have realized if, but for the actions of the contracting department, it had been awarded the contract, which compensation may not exceed the lesser of:
  • the net profit that the complainant would have realized, had it been awarded the contract, at the lower of the amount that it bid, if any, and the amount at which the contract was awarded, without including the value of any options or any extensions of the contract, less the cost of submitting the bid, if any; and
  • 10% of the value of the contract awarded without including the value of any options or any extensions of the contract, less the cost of submitting the bid, if any; or
  1. compensation equal to the costs of submitting the bid but that compensation may not exceed 10% of the value of the contract awarded, without including the value of any options or any extensions of the contract.

Return to footnote 13 referrer