Follow-up report to the 2018-19 procurement practice review of the Canadian Food Inspection Agency

December 2022

On this page

Introduction

1. As its standard practice, the Office of the Procurement Ombudsman (OPO) follows up on the recommendations made in a Procurement Practice Review (PPR) two years after those recommendations are published.

2. Follow-up reviews allow OPO to: provide assurance that actions to respond to PPR recommendations have been effectively implemented, or that senior management has accepted the risk of not taking action; and provide an organizational report card that facilitates comparisons across organizations over time.

3. In 2019-2020 OPO conducted a PPR entitled Procurement practice review of the Canadian Food Inspection Agency (CFIA). The final report was issued in February 2020.

4. The initial review was undertaken to determine whether the CFIA’s procurement practices pertaining to evaluation criteria and selection plans, solicitation, and evaluation of bids and contract award, supported the principles of fairness, openness and transparency. To make this determination, OPO examined whether the CFIA’s procurement practices were consistent with Canada’s obligations under applicable sections of national and international trade agreements, the Financial Administration Act and regulations made under it, the Treasury Board Contracting Policy and, when present, departmental guidelines.

5. The following three lines of enquiry (LOE) were used to assess the highest-risk procurement elements:

  • LOE 1: Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies
  • LOE 2: Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies
  • LOE 3: Evaluation of bids and contract award were conducted in accordance with the solicitation

6. Regarding LOE 1, OPO concluded that the solicitation documents reviewed consistently identified evaluation criteria and the selection method to determine the successful bid; however, evaluation criteria and weighting schemes were sometimes unnecessarily restrictive and may have precluded bidders from an equal opportunity to compete. Several instances were also noted in which the use of ambiguous or undefined terms may have impacted transparency by hampering the ability of bidders and evaluators to determine how evaluation factors would be used to determine the successful bid. Likewise, greater clarity was needed in the instructions for awarding points to point-rated criteria to ensure that evaluation criteria were applied equally to all bidders.

7. Regarding LOE 2, the design and execution of competitive solicitation processes was generally consistent with applicable rules, regulations and policies. However, OPO could not conclude that contracts issued against CFIA-specific standing offers had been executed according to the specified contracting procedures as some files were insufficiently documented. In the absence of information on the nature of the services being sought, there was a risk of perceived contract splitting with several cases of multiple, same-day call-ups made to the same supplier.

8. Regarding LOE 3, CFIA has developed standardised guidance for evaluators, and has implemented a consistent process for executing the evaluation of bids. However, several instances were noted in which evaluations were not consistently carried out in accordance with the planned approach and in one case, non-responsive bids were incorrectly assessed as compliant, and a contract was awarded to a bidder that failed to meet the mandatory criteria.

9. In order to address the issues identified during the initial review, the Procurement Ombudsman made 8 recommendations: 2 recommendations under LOE 1, 4 recommendations under LOE 2, and 2 recommendations under LOE 3.

Methodology

10. In March 2022, OPO asked CFIA to self-assess their progress in implementing the 8 recommendations using an established progress scale (ranging from “no progress” (level 1) to “full implementation” (level 5)), as outlined in Appendix 1. CFIA was also asked to provide supplementary information and documentation to explain and support their self-assessment.

11. When reviewing departments’ self-assessments during a follow-up exercise, OPO may conduct additional testing of procurement files, if required, to obtain evidence that demonstrates structures and processes have been implemented and positive results have been identified. During this review, it was determined that additional testing of procurement files was not necessary because CFIA’s self assessment and supporting documentation provided sufficient information regarding the effectiveness of actions taken to resolve the issues detected in the initial PPR report.

12. OPO reviewed and assessed the information provided by CFIA in its self assessment for its overall reasonableness and credibility. This report includes a report card of progress made under the three LOEs resulting from CFIA’s efforts in implementing the Procurement Ombudsman’s recommendations, a summary of their progress made and an overall conclusion as it pertains to resolving issues identified in the initial report.

Management Action Plan Assessment

13. Using the scale provided by OPO, CFIA self-assessed the level of implementation of its actions at a level 5, “Full Implementation”. Substantiating documentation provided by CFIA demonstrated structures and processes that were fully implemented and results were identified as stated in its response. Therefore, OPO is in agreement with CFIA’s self-assessment. A detailed summary of OPO’s overall assessment of CFIA’s response to each recommendation is included in Appendix 2.

14. CFIA developed and implemented both new and enhanced products that support fairness, openness, and transparency in the procurement process. The evidence provided by CFIA demonstrated a series of advancements designed to meet the intent of each recommendation through a series of best practice initiatives. These initiatives included development of new internal policy instruments such as the new Procurement Quality Assurance Process and Compliance Review Directive and the Manager’s Guide in Developing Technical Evaluation Criteria for Requests for Proposals. The Procurement 101 training program was enhanced with a new section on development of technical evaluation criteria. New manual and automated tools were also developed and implemented to better facilitate the procurement quality assurance (QA) process. Examples include the Contract Request Form and Procurement Checklist and the ePro procurement application software system. Some initiatives such as the ePro system for example, were comprehensive enough to cover the intent of multiple recommendations.

Report Card

15. PPR follow-up reviews include a report card with a rating that depicts the organization’s performance with regard to the lines of enquiry. This results in 3 assessments. In determining these assessment ratings, OPO takes into consideration the results from the initial PPR, and the actions taken by the organization to implement the PPR recommendations under each line of enquiry. The assessment ratings are as follows:

  • Satisfactory +
  • Satisfactory
  • Partially Satisfactory
  • Unsatisfactory

Note that a Satisfactory + assessment rating is only possible when the initial PPR resulted in no recommendations being issued under a particular line of enquiry. As the initial PPR report contained recommendations under all 3 lines of enquiry, a Satisfactory + rating was not possible in this follow-up review. Assessment definitions and criteria are outlined in Appendix 3.

16. After a review of progress made toward implementing the action plans for each recommendation, and in consideration of the established assessment ratings, a rating of Satisfactory is awarded for each line of enquiry. CFIA’s descriptions of actions taken, and its supporting documentation demonstrate reasonable and credible progress has been achieved in response to all recommendations.

Table 1: Report Card
Line of Enquiry (LOE) Rating Assessment
1. Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies. Satisfactory
  • Recommendations 1 and 2 applied to this LOE.
  • A series of initiatives and products were developed, adopted, and communicated to address this LOE and help to ensure evaluation criteria and selection plans are in compliance with applicable laws, regulations and policies. These include new guides, directives, checklist tools, upgraded training material and an enhanced monitoring and QA process.
2. Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies. Satisfactory
  • Recommendation 3, 4, 5 and 6 applied to this LOE.
  • Internal policy instruments have been established, adopted, and communicated, including new directives and an enhanced monitoring and QA process that helps to ensure solicitations are conducted in compliance with laws, regulations and policy requirements.
3. Evaluation of bids and contract award were conducted in accordance with the solicitation. Satisfactory
  • Recommendations 7, and 8 applied to this LOE.
  • A series of initiatives and products were developed, adopted, and communicated to address this LOE. These include guides, directives, checklist and mapping tools, advanced approval forms, upgraded training material and an enhanced monitoring and QA process.
  • Roles, responsibilities and procedures for matters pertaining to the award of contracts are sufficiently covered by these initiatives.

Table 1 Note

CFIA’s overall report card rating is “Satisfactory”. The key factors contributing to this rating include:

  • The initial PPR resulted in one or more recommendations under each of the three lines of enquiry;
  • CFIA’s Management Action Plans have achieved level 5 implementation that is deemed to be satisfactory, i.e. significant progress has been made to implement action plans that address recommendations; and
  • CFIA's response is supported with adequate documentation that demonstrate structures and processes are fully implemented and operating as intended.

Conclusion

17. CFIA has created new initiatives and enhanced established products that support fairness, openness, and transparency in the procurement process. The evidence provided demonstrates a series of advancements designed to meet the intent of each recommendation through a series of best practice initiatives. These initiatives included development of internal policy instruments such as guides and directives, updated training programs, new manual and automated tools and development of an enhanced procurement QA process. Furthermore, some initiatives, such as an automated system created to support the overall QA function, were comprehensive enough in their capabilities to cover the intent of multiple recommendations.

18. CFIA considered the Procurement Ombudsman’s recommendations from the initial PPR and provided information on its actions. CFIA stated it had taken action to respond to the Ombudsman’s recommendations and provided documentation to substantiate this statement. OPO’s review found CFIA’s self-assessment and substantiating documentation to be reasonable and credible.

Appendix 1

Table 2: Assessment of implementation level
Implementation level Assessment
Level 1 No progress or insignificant progress. Actions such as establishing a new committee, conducting meetings, and generating informal plans have not advanced, or insignificant progress has been made.
Level 2 Planning stage. Formal plans for organizational changes have been created and approved.
Level 3 Preparations for implementation. Preparations for implementing a recommendation are in progress – e.g. hiring or training staff, developing necessary resources, drafting and adopting policy documents, etc.
Level 4 Substantial implementation. Structures and processes are in place and integrated within at least most parts of the organization, and some achieved results have been identified.
Level 5 Full implementation. Structures and processes are fully implemented, and positive results have been identified.
Obsolete Recommendation is no longer applicable due to new policies, procedures, etc.

Appendix 2

Table 3: OPO detailed assessment summary of recommendations
Recommendation OPO Assessment Rating
1- The CFIA should ensure that:
  1. evaluation criteria are limited to the performance elements necessary to the success of the project and do not favour a particular supplier;
  2. weighting schemes do not disproportionately skew evaluation results; and
  3. minimum thresholds for point-rated criteria are reasonable.
 CFIA has developed and implemented a variety of internal products such as guides, confirmation documents, checklists, training materials and an automated QA system. Together, these products serve to ensure that the multiple parts of the recommendation are addressed by increasing awareness and understanding among employees of the requirements and expectations when developing bid evaluation criteria. Additionally, CFIA employs both manual and automated tools to ensure adherence to policy requirements and enforce roles and responsibilities. Level 5
Full Implementation
2- The CFIA should implement measures to ensure mandatory criteria are clear, precise and measurable, and adequately defined to support the preparation of responsive bids and the evaluation of proposals. CFIA has developed and implemented a variety of internal products such as a managers guide for developing technical evaluation criteria, oversight confirmation/control tool, training program and an automated QA system. Together, these products serve to ensure mandatory criteria are clear, precise and measurable and adequately defined to support the preparation of responsive bids and the evaluation of proposals. Level 5
Full Implementation
3- The CFIA should ensure that procurement policies are reviewed regularly, kept up-to-date, and contain sufficient detail to clarify roles, responsibilities and procedures for awarding contracts. CFIA has developed and implemented a variety of internal products such as a Procurement and Contracting Directive and an automated QA system. Together, these products serve to ensure that procurement policies are reviewed regularly, kept up-to-date, and contain sufficient detail to clarify roles, responsibilities and procedures for awarding contracts. The provision of current guidance information serves to promote awareness and understanding among employees of procurement and contracting policies and procedures. Also, an oversight function is enabled through regular monitoring activity while professional conduct and standard requirements are established to ensure activities are in compliance with policy requirements. Risks around processing procedures are mitigated through the implementation of an automated tool. Level 5
Full Implementation
4- The CFIA should establish appropriate review mechanisms to ensure that information shared with suppliers is accurate and complete. CFIA has developed and implemented a variety of internal products such as a Client Review and Confirmation document, a Procurement and Contracting Directive, updated training material and a new QA process that is supported by a documented directive framework. Together, these products serve as review mechanisms that help ensure information shared with suppliers is accurate and complete. They achieve this by enabling oversight on the preparation of procurement engagements before public release of bid solicitations to ensure information is accurate and complete. They also provide training and guidance documentation on roles, responsibilities and preparation of information related to engagements. A formal quality assurance function supported by an automated tool used to process engagements also meets the intent of the recommendation. Level 5
Full Implementation
5- The CFIA should implement an effective mechanism to ensure that procurement files pertaining to call-ups issued against standing offers are sufficiently documented to facilitate management oversight and establish a clear audit trail. CFIA has developed and implemented a variety of internal products such as a Call-Up Against Standing Offer Checklist, an updated training program and a Procurement QA Process framework that is supported by a Directive which outlines key services from groups within the Contracting and Procurement Policy Division, (CPPD) who will aim to ensure sufficient oversight and file documentation. To support these groups, an automated QA system is also in place. Together, these initiatives serve to ensure that procurement files pertaining to call-ups issued against standing offers are sufficiently documented in a manner that facilitates management oversight and establishment of a clear audit trail. Level 5
Full Implementation
6- The CFIA should develop appropriate review mechanisms to ensure that procurement strategies do not unnecessarily divide aggregate requirements and circumvent approval authorities. CFIA has developed and implemented a variety of internal products such as a mapping tool, planning checklist document, frameworks and directives, as well as an automated QA system. Together, these products serve as review mechanisms to help ensure that procurement strategies do not unnecessarily divide aggregate requirements and circumvent approval authorities. This is achieved by increasing rigour around quality assurance and oversight with manual and automated tools, as well as providing guidance to increase understanding of various procurement strategies through a streamlined mapping process that mitigates risk of contract splitting and circumventing approvals. Level 5
Full Implementation
7- The CFIA should establish a mechanism to ensure bid evaluations are consistent with properly designed and disclosed evaluation procedures and are appropriately documented. CFIA has developed and implemented a variety of internal products such as the Managers Guide in Developing Technical Evaluation Criteria for Requests for Proposals, evaluation guidelines, updated training materials as well as a Procurement QA Process and Compliance Review Directive in support of an enhanced QA process. An automated QA system is also in place to support oversight and reporting activities. Together, these products serve as mechanisms that help meet the goal of the recommendation by serving to promote awareness and understanding among employees of the engagement planning process in relation to evaluation and documentation. They provide guidance resources and formal training which increases understanding of roles and responsibilities and strengthen the monitoring and oversight function through the use of manual and automated tools. Level 5
Full Implementation
8- The CFIA should implement an effective mechanism to enforce the requirement to maintain up-to-date and complete procurement files. CFIA has developed and implemented a variety of internal products such as the Contract Request Form and Procurement Checklist, updated training program and a Procurement Quality Assurance Process and Compliance Review Directive. An automated QA system is also in place to support oversight and reporting activities. These products support the goal of the recommendation which is to reinforce the requirement to maintain up-to-date and complete procurement files. This is achieved by increasing rigour at the beginning of an engagement to ensure documentation is in order and complete, providing training to staff to increase their awareness of their roles and responsibilities around file management and enhancing the monitoring and quality assurance function with manual and automated controls that support procurement file documentation and record-keeping. Level 5
Full Implementation

Appendix 3

Table 4: Overall Performance Assessment Scale
Overall performance Assessment
Satisfactory +
  • Findings from the initial PPR resulted in no recommendations being issued under the line of enquiry
  • A rating of Satisfactory + is based on results from the initial PPR
Satisfactory
  • Initial PPR resulted in one or more recommendations under the line of enquiry
  • Action plan(s) have achieved level 5 implementation or a combination of level 4 and level 5 that is deemed to be satisfactory, i.e. significant progress has been made to implement action plan(s) that address recommendation(s)
  • Organization's response is supported with adequate documentation and/or testing results
Partially satisfactory
  • Initial PPR resulted in one or more recommendations under the line of enquiry
  • Some progress has been made in implementing action plan(s), but not enough to adequately address all recommendations under the line of enquiry, e.g. one or more recommendations achieved level 3, but work remains to be done
  • Organization's response is supported with some documentation and/or testing results
Unsatisfactory
  • Initial PPR resulted in one or more recommendations under the line of enquiry
  • No or insignificant progress made to implement action plan(s).
  • Inadequate supporting documentation provided or testing does not demonstrate that positive results have been achieved
  • Recommendation(s) not addressed and remain(s) outstanding.
Date modified: