Procurement practice review of Employment and Social Development Canada

October 2020

On this page

• I. Background
• II. Objective and Scope
• III. Results
• IV. Simplification
• V. Conclusion
• VI. Organizational Response
• VII. Acknowledgement
• Annex A: Status of Review Recommendations

I. Background

1. The Office of the Procurement Ombudsman (OPO) conducted a review of procurement activities at Employment and Social Development Canada (ESDC).

2. In accordance with paragraph 22.1(3)(a) of the Department of Public Works and Government Services Act, the Procurement Ombudsman has the authority to review the procurement practices of departments to assess their fairness, openness and transparency.

3. This review is based on issues and complaints brought to OPO’s attention by stakeholders, both in general and in regard to specific solicitations by various federal organizations. Based on this information, OPO has identified the 3 highest-risk procurement elements as: (1) the establishment of evaluation criteria and selection plans; (2) the bid solicitation process; and (3) the evaluation of bids and contract award. For the purposes of this review, these elements are defined as follows:

• Evaluation criteria and selection plans—the development of mandatory and point-rated evaluation criteria, and the identification of the selection method to determine the successful bid. 
• Solicitation—the design and execution of the solicitation process, including the clarity and completeness of solicitation documents. 
• Evaluation of bids and contract award —the establishment of a process to ensure the consistent evaluation of bids in accordance with the planned approach, including an evaluation plan and instructions to evaluators, and the adequacy of documentation to support the selection of the successful supplier.

4. ESDC was selected for review as one of the top 20 federal departments and agencies in terms of the value and volume of their annual procurement activity. OPO is in the process of conducting similar reviews of the other top 20 departments and agencies over 5 years.

5. As the fourth largest federal department with over 25,000 employees, ESDC has a broad mandate to build a stronger and more inclusive Canada, to support Canadians in helping them live productive and rewarding lives, and to improve Canadians’ quality of life. ESDC is listed under Schedules I and VI of the Financial Administration Act (FAA) and reports to Parliament through 4 Ministers: Minister of Employment, Workforce Development and Disability Inclusion; Minister of Families, Children and Social Development; Minister of Labour; and Minister of Seniors. ESDC’s procurement activity is carried out by a centralized Procurement Team within the Chief Financial Officer Branch that is located in the National Capital Region.

II. Objective and Scope

6. This review was undertaken to determine whether ESDC’s procurement practices pertaining to evaluation criteria and selection plans, solicitation, and evaluation of bids and contract award, supported the principles of fairness, openness and transparency. To make this determination OPO examined whether ESDC’s procurement practices were consistent with Canada’s obligations under applicable sections of national and international trade agreements, the FAA and regulations made under it, the Treasury Board Contracting Policy (TBCP) and, when present, departmental guidelines.

7. The following 3 lines of enquiry (LOE) were used to assess the highest-risk procurement elements identified above:

• LOE 1: Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies;
• LOE 2: Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies; and
• LOE 3: Evaluation of bids and contract award were conducted in accordance with the solicitation.

8. OPO’s review covered the period from August 1, 2017, to September 30, 2019, and excluded non-competitive contracts, low dollar value contracts below $25,000, construction contracts, acquisition card activity and contracts for which ESDC was not the contracting authority. While ESDC remains in the top 20 federal departments and agencies in terms of the value and volume of the procurement activity, a preliminary review of the contracting data for the review period revealed that the majority of this procurement activity involved contracts for which Public Services and Procurement Canada (PSPC) or Shared Services Canada (SSC) acted as the contracting authority, or consisted of call-ups against a standing offer, which precludes OPO from assessing ESDC’s practices against the lines of enquiry mentioned above.

9. As a result of the small number of contracts that fall within OPO’s review program, OPO elected to conduct a micro-review of ESDC’s procurement practices, which involved an assessment of 10 procurement files that included: 2 contracts issued under standing offers established by ESDC, and 8 contracts awarded through competitive solicitation processes, including 1 contract issued under a supply arrangement established by PSPC.

III. Results

10. ESDC’s procurement practices pertaining to evaluation criteria and selection plans, solicitation documents, and evaluation of bids and contract award were assessed against the 3 LOEs noted above. OPO made 3 recommendations to address issues identified in the review.

LOE 1: To determine whether evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies

11. OPO examined a total of 10 files awarded through competitive processes, 8 of which involved a solicitation process, and 2 were call-ups issued against standing offers. The 8 files that involved a solicitation process were examined to determine whether evaluation criteria and selection methodology were clearly communicated in the solicitation, not overly restrictive, and were aligned with the requirement. The method of allocating points to weighted criteria was also assessed to determine whether instructions were clearly communicated and reflected the relative importance of the criteria. The results of this assessment are presented below.

12. Of the 8 solicitations OPO reviewed, evaluation criteria were not overly restrictive, were aligned with the requirement, and ESDC consistently advised bidders of the manner by which the contract would be awarded through the communication of the selection methodology in the request for proposals (RFP). However, several instances were noted in which evaluation criteria were not defined in a clear, precise and measurable manner.

13. In order to ensure fairness and transparency in the award process, the TBCP requires contracting authorities to communicate the criteria needed to meet the requirement and the weighting assigned to them. Section 10.7.25 states that “these criteria should identify accurately all the performance elements significant to the success of the project and should measure both the competence of the firm and the worth of its particular technical approach.” Using clear and precise language to define the evaluation criteria and selection method helps bidders to prepare a responsive proposal and evaluators to ensure that criteria are applied equally to all bidders.

14. The following instances were noted in which the evaluation criteria were not communicated in a clear, precise and measurable manner:

• 1 solicitation contained a mandatory criterion that required bidders to provide 3 project examples and stated that “at least one (1) of the three (3) projects should be with a government department, agency or a large organization.” [emphasis added] Non-definitive terms, such as the word “should,” must be avoided when defining mandatory criteria as it can create confusion for suppliers in determining what is required of them and for evaluators in assessing proposals. Mandatory criteria must always be defined using definitive terms such as “must” and “shall.”
• 2 solicitations contained definitive language in the description of point-rated criteria, which has the effect of making these criteria mandatory requirements. For example, 1 solicitation contained a point-rated criterion that stated the bidder “must demonstrate a minimum of 10 years’ experience in change management.” [emphasis added] A second solicitation contain point-rated criteria that stated that the bidder “must be fluently bilingual” and “must have the following professional qualifications.” As stated in ESDC’s guide to writing bid evaluation criteria “you must never use the words “must” or “shall” in the rated criteria. Rated criteria are not mandatory, but those words make them mandatory.”
• 4 solicitations required bidders to demonstrate the experience of the proposed resources, but did not define how much experience must be demonstrated. For example, 1 solicitation contained a mandatory criterion that stated “the proposed counsellors must have demonstrated experience in providing phone clinical consultation support services.” [emphasis added] By failing to describe how much experience is required, any amount of experience, even as little as one day, would result in mandatory criteria being met. A similar situation was witnessed with respect to point-rated criteria which allocated points based on a rating scale of “less than satisfactory, satisfactory, and more than satisfactory” without defining what constitutes satisfactory. Another solicitation contained only 2 mandatory criteria; the first was a financial requirement that the proposal not exceed the proposed budget of $75,000.00, and the second was that “the bidder MUST submit resumes for each proposed personnel that will be working on the project.” In this instance, the mandatory criterion was too vague such that the submission of any resume would have resulted in the criterion being met.

15. When evaluation criteria are not communicated in a clear, precise and measurable manner it prevents bidders from knowing the requirements and the methods by which their proposals will be evaluated. Failure to adequately define evaluation criteria at the outset carries the risk that bidders may be required to guess at their meaning and evaluators may struggle interpreting these criteria during the evaluation process. It can also be difficult to defend against external challenges, as it is more difficult to demonstrate that criteria have been strictly adhered to when the criteria are unclear and open to multiple interpretations.

16. ESDC’s Procurement Group has developed a suite of guidance documents, which includes a guide for writing bid evaluation criteria. In addition to providing an overview of the key considerations involved in the development of evaluation criteria and common selection methods, it includes a breakdown of the components of a RFP and clearly assigns responsibility for drafting the Statement of Work and the evaluation criteria to the Project Authority. The document also draws attention to some common pitfalls to avoid, such as including undefined or vague language in the solicitation or including the words ‘must’ or ‘shall’ in the rated criteria. Given the existence and quality of this guidance, it is OPO’s determination that the issues described above are indicative of a failure to implement the appropriate guidance rather than the need to develop such guidance.

LOE 2: To determine whether solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies

17. Of the 10 files reviewed, 3 involved an open solicitation, 5 were awarded through traditional competitive solicitations (open to a limited number of bidders), and the remaining 2 files were call-ups against standing offers. The 8 solicitations were assessed to determine whether the solicitation document was clear and contained complete information, including a clear description of the requirement, and instructions necessary to prepare a compliant bid. The design and execution of the solicitation process was also assessed to determine whether the process supported a fair, open and transparent procurement. This included an assessment of whether ESDC has established a framework to ensure that procurement practices are consistent with laws, regulations and policies; whether the solicitation was open to the appropriate number of suppliers and for the required duration; and whether communications with interested suppliers supported the preparation of responsive bids. The 2 call-ups against standing offers were assessed to determine whether the call-up was clear and contained complete information, including a clear description of the requirement and delivery schedule, and whether the specified contracting procedures were adhered to. The results of this assessment are presented below.

18. In 7 of the 8 solicitation processes reviewed, the solicitation documents and the design and execution of the solicitation processes complied with key aspects of the TBCP and trade agreement provisions, where applicable. For example, solicitation documents contained a detailed description of the requirement, evaluation criteria and selection methodology were clearly identified in the RFP, and instructions for submitting proposals and posing questions during the solicitation period were identified. Where the estimated value of the requirement triggered trade agreements, solicitations were publically tendered for the required duration. For solicitations carried out through limited tendering, the number of invited suppliers complied with departmental policy and PSPC-established Supply Arrangements. Both of the call-ups against standing offers established by ESDC were issued according to call-up procedures.

19. The TBCP sets out detailed procedures to ensure that government contracting is carried out in a manner that enhances access, competition and fairness and results in best value. Section 10.7 includes the minimum requirements to be included in the solicitation document as well as mandatory elements related to the design and execution of the process. Solicitation documents must contain work descriptions or specifications defined in terms of clear outputs or performance requirements, the assessment and award criteria, and, in the case of service contracts, the objectives to be attained and time frame for delivery. These minimum requirements increase for solicitations subject to the trade agreements. The Canadian Free Trade Agreement (CFTA) prescribes certain mandatory elements related to the design and execution of the solicitation process, such as the establishment of a reasonable period of time for suppliers to prepare and submit responsive tenders. Under the North American Free Trade Agreement (NAFTA) and World Trade Organization—Agreement on Government Procurement (WTO-AGP), the open tender period must be no less than 40 days.

20. In 1 instance, the statement of work was found to contain contradictory information and communications with suppliers did not support the preparation of responsive bids.

• In a solicitation for moving and storage services, the statement of work contained contradictory information. Section 3.1.1 of the Statement of Work states that “the Contractor is expected to carry out three (3) large projects in three (3) different locations at the same time, requiring at least 20 installers and 10 movers, in addition to dismantling and installing as many as 100 workstations over a weekend period in each location.” [emphasis added] Section 5. Hours of Work of the Statement of Work states “Normal working hours for the Contractor will be Monday to Friday from 7:30 A.M. to the completion of the work order.” Contradictions, such as this, in the statement of work can make it difficult for interested suppliers to accurately price an opportunity and can lead to disputes in contract administration.
• In the same solicitation, communications with suppliers did not support the preparation of responsive bids. On June 17, 2019, questions were submitted by an interested supplier, who did not end up bidding, and ESDC’s response was not posted until June 27, 2019, one day before bid closing. Since the value of the contract was approximately $1.19 million, the solicitation was subject to trade agreements, including NAFTA, which requires that suppliers be provided adequate time to prepare and submit tenders before the closing of the tendering procedures.

LOE 3: To determine whether the evaluation of bids and contract award were conducted in accordance with the solicitation

21. 8 solicitation processes were examined to determine whether a process had been established, complete with guidance for evaluators, to ensure: the consistent evaluation of bids; that the evaluation of bids had been carried out in accordance with the planned approach; and that files were adequately documented. The results of this assessment are presented below.

22. In order to ensure the transparency and defensibility of evaluation processes, the TBCP requires that evaluation criteria must be adhered to strictly and applied equally to all bidders. Failure to ensure the consistent evaluation of proposals increases the risk that ambiguities in the selection process may result in the contract being wrongly awarded and may call into question the integrity of the procurement process. 

23. ESDC has established a standard process for evaluating proposals. This process is well documented and available on ESDC’s intraweb. ESDC has also developed guidance materials for non-procurement specialists that highlight the responsibilities of evaluators as well as the risks associated with deviating from the planned approach. Prior to the evaluation of proposals, members of an evaluation team are required to review and sign the Bid Evaluation Instructions, which identify the procedures for conducting evaluations, performing reference checks, seeking clarifications from bidders, and the minimum documentation requirements that must be adhered to by evaluators. While these instructions provide a thorough overview of the procedures that must be followed during the evaluation, they could be improved through the addition of a clause referencing the need for evaluators to disclose potential or perceived conflicts of interest in alignment with Section 11.1 Review Mechanism of the TBCP that states “those in a possible conflict of interest situation are to declare themselves and be replaced.”

24. OPO noted several instances across 5 of the 8 files reviewed in which the evaluation and contract award was not carried out in accordance with the planned approach. Some examples are provided below:

• 2 contracts were awarded to non-compliant bidders. In both cases, the suppliers were the sole respondents to the solicitations. In the first case, the supplier failed to attach their financial proposal and the contracting authority permitted the supplier to submit their financial proposal after bid closing. In the second case, the bidder was awarded the contract after failing to meet the minimum threshold for rated criteria, scoring 55 out of a possible 90 points where the minimum threshold was 63; no rationale was found on file to justify awarding the contract to a non-compliant bidder. Section 10.8.7 of the TBCP states that departments may consider proposals that “respond to the mandatory requirements but contain a minor aberration,” provided that the contracting authority’s management determines that the aberration is trivial or negligible, there is no impact on other bidders, and the decision to accept such bids is fully justified on the contract file. Documentation was not on file to demonstrate that ESDC’s management came to a determination that these circumstances applied.
• In 2 instances, documentation issues prevented OPO from verifying whether evaluations were conducted consistently and in accordance with the planned approach. In the first instance, the evaluation was not on file, and in the second instance only the winning bid was on file. In the second file, OPO also noted that ESDC had made changes to the evaluation criteria in response to questions received from interested suppliers, but that some of these changes had not been reflected in the evaluation grids used by evaluators to evaluate bids. Since only the winning bid was on file, OPO was unable to verify that the other bidder’s proposal had been correctly evaluated against the modified criteria. Keeping complete and detailed evaluation records is crucial to demonstrating that evaluation criteria have been applied equally to all competing bids, and demonstrating that the procurement has been carried out in a manner consistent with ESDC’s obligations under the TBCP and applicable trade agreements. Failure to maintain complete records places the ESDC at risk of not being able to defend challenges to the procurement process. Note - In the period since these solicitation were carried out, ESDC has implemented a paperless filing system which is intended to minimize documentation issues.
• In 1 solicitation the points awarded by the evaluation team were not in accordance with the rated scoring grid. The scores were to be awarded on the three tier scale, depending on if the bid was “less than satisfactory”, “satisfactory” or “more than satisfactory.” Each of these tiers had a corresponding allotment of points. For example, for criterion A, the points that could be awarded were 1-3-5; however, the evaluators awarded 4 points. Since the rating scheme applied to each criterion is specific, evaluators do not have the discretion to award scores other than 1, 3 or 5 in this case. For this evaluation, the evaluators incorrectly awarded points that were not in accordance with the rating scale for six of the evaluation criteria.

25. The procurement practices highlighted above demonstrate that ESDC, despite having a well-documented process, had not implemented effective supervision and review mechanisms to ensure evaluations are carried out in accordance with the planned approach and are appropriately documented to support the transparency of the award process.

Employment and Social Development Canada provided detailed debriefings to unsuccessful suppliers

26. Concerns related to absence of debriefings or the insufficient information provided by federal organizations to unsuccessful suppliers is consistently one of the top issues reported to OPO. Section 10.8.21 of the TBCP states “debriefings should be provided to unsuccessful bidders on request and should normally include an outline of the factors and criteria used in the evaluation, while respecting each bidder’s right to the confidentiality of specific information.” The merits of providing unsuccessful suppliers with a debriefing following the results of evaluations is echoed in ESDC’s procurement policy which states “debriefing demonstrates the fairness, openness, and transparency of the federal government contracting process. Procurement specialists can improve future solicitations by using the comments and suggestions provided by bidders / suppliers.”

27. OPO reviewed 8 solicitation processes, 3 of which received responses from multiple suppliers. In 2 of these cases, unsuccessful suppliers were provided with detailed debriefings that included a copy of their consensus evaluation and a written description of how they could have improved their proposal. This same level of transparency was also witnessed in ESDC’s communications with successful bidders. This practice not only supports the transparency of the process, but it assists suppliers in improving future proposals.

IV. Simplification

28. OPO regularly hears from both Canadian businesses and federal officials who believe the federal contracting process is unnecessarily complex. In reviewing ESDC’s procurement practices, OPO sought to identify opportunities to alleviate unnecessary administrative burdens placed on suppliers and federal procurement officials, and draw attention to good practices for simplifying the procurement process. 

Standardization of procurement documents to streamline procurement processes

29. ESDC consistently used PSPC standardized procurement documents that reference the Standard Acquisition Clauses and Conditions (SACC) manual. This contributes to greater simplification by ensuring consistency and uniformity across procurement processes. This practice supports the simplification of government procurement by reducing the burden on suppliers, particularly those who supply the same good or service across multiple departments. Using standardized documents and processes decreases the cost to bid for suppliers and potentially lowers bid prices for departments.

Contracts frequently included option years

30. In the 8 competitive solicitation processes that OPO reviewed, ESDC frequently included options in the contract for requirements that involved recurring needs. This practice supports the simplification of government procurement by reducing the number of solicitation processes and, consequently, reduces the cost to both government and suppliers. This practice allows departments to continue working with well-performing suppliers, while being transparent about the amount and value of work associated with a contract during the initial solicitation process. 

V. Conclusion

31. ESDC’s procurement practices pertaining to evaluation and selection plans, solicitation, and evaluation of bids and contract award were assessed for consistency with Canada’s obligations under applicable sections of national and international trade agreements, the FAA and regulations made under it, the TBCP, departmental guidelines, and to determine if they supported the principles of fairness, openness and transparency.

32. The solicitation documents reviewed consistently identified evaluation criteria and the selection method to determine the successful bid; however, evaluation criteria were sometimes not communicated in a clear, precise and measurable manner. Several instances were noted in which the use of ambiguous or undefined terms may have impacted transparency by hampering the ability of bidders and evaluators to determine how evaluation factors would be used to determine the successful bid.

33. The design and execution of competitive solicitation processes were consistent with applicable rules, regulations and policies in almost all instances.

34. ESDC has established a standard process for evaluating proposals complete with detailed guidance for evaluators. However, numerous instances were noted in which evaluations were not consistently carried out in accordance with the planned approach and in two cases, contracts were awarded to non-responsive bidders.

35. In order to address issues identified, OPO made 3 recommendations. These recommendations can be found in Annex A of this report. OPO will conduct a follow-up review in 2 years to assess the implementation of the ESDC’s action plan to address these recommendations.

VI. Organizational Response

36. In accordance with section 5 of the Procurement Ombudsman Regulations, the Procurement Ombudsman provided ESDC with the opportunity to comment on the proposed recommendations in this review and the reasons for them. ESDC was further given the opportunity to comment on the review’s findings.

37. ESDC is in agreement with the recommendations contained in this report and recognizes the importance of rigorous and well-documented procurement practices as a key factor in delivering its mandate. ESDC will address these recommendations in accordance with the action plan provided below and is committed to ensuring its procurement practices are sound and support the principles of fairness, openness and transparency.

38. ESDC thanks OPO and the review team for its thoroughness and the professionalism throughout the review period.

VII. Acknowledgement

39. OPO wishes to express its appreciation to the management and staff of ESDC for the assistance and cooperation extended to the reviewers during this assessment.

Alexander Jeglic
Procurement Ombudsman

Annex A: Status of Review Recommendations

Date modified:

2020-11-20