Follow-up report to the 2020-2021 procurement practice review of Transport Canada

December 2024

On this page

Introduction

1. In accordance with paragraph 22.1(3)(a) of the Department of Public Works and Government Services Act, the Procurement Ombud has the authority to review the procurement practices of departments to assess their fairness, openness and transparency and to make any appropriate recommendations to the relevant department for the improvement of those practices. Such reviews, which are performed by the Office of the Procurement Ombud (OPO), are referred to as Procurement Practice Reviews (PPRs).

2. In 2020-2021 OPO conducted a PPR entitled “Procurement Practice Review of Transport Canada” (TC), which included four recommendations to address identified issues. A final report was issued in July 2021 and published on OPO’s website.

3. As a standard practice, OPO follows up on recommendations resulting from PPRs to determine whether federal departments have implemented their management action plans in response to the Procurement Ombud’s recommendations. This follow-up review includes a summary of results from the initial PPR and OPO’s assessment regarding progress made by TC in implementing their four recommendations. This report also includes a report card that facilitates comparisons across departments over time.

Results from initial PPR

4. The objective of the initial PPR was to determine whether TC’s procurement practices pertaining to evaluation criteria and selection plans, solicitation, and evaluation of bids and contract award, supported the principles of fairness, openness and transparency.

5. For the PPR , OPO analyzed TC’s procurement practices under three Lines of Enquiry (LOE). Below are descriptions of the LOEs, a brief summary of observations for each LOE, and the Procurement Ombud’s four recommendations included in the initial PPR report.

LOE 1: Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies.

6. For LOE 1, OPO’s initial review found that evaluation criteria and selection plans met requirements set out in applicable laws, regulations and policies and that for the most part, mandatory and point rated criteria were clearly communicated. Instances were noted in which mandatory or rated criteria had vague or poorly defined descriptors and unclear evaluation criteria that appeared to favour a bidder.

7. The Procurement Ombud made one recommendation to the issues identified under LOE 1:

Recommendation1: TC should establish a quality control framework to ensure evaluation criteria are measurable, fair and do not favour a particular bidder/group of bidders.

LOE 2: Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies.

8. For LOE 2, OPO’s initial review found that the design and execution of competitive solicitation processes were mostly consistent with applicable rules, regulations and policies. Solicitations were, in most cases, made available to the appropriate number of suppliers, for the correct length of time and included clear and complete information and instructions for submitting bids.

9. The Procurement Ombud made one recommendation to address the issues identified under LOE 2:

Recommendation 2: TC should review and update its Contract Procedures Manual and any model templates to reflect current procurement policies, especially as it relates to information exchanges with suppliers. In addition, “regret” letters should consistently include an outline of the factors from the bid and criteria that caused the bid to be unsuccessful.

LOE 3:Evaluation of bids and contract award were conducted in accordance with the solicitation.

10. For LOE 3, OPO’s initial review found that several bids were missing complete records about either the individual or consensus evaluations. For the files where there were complete evaluation results, more than half showed inconsistencies in the evaluation of bids, and deviations from the planned approach. OPO also found that in most files, procurement file documentation was incomplete.

11. The Procurement Ombud made 2 recommendations to address the issues identified under LOE 3:

Recommendation 3: TC should establish mechanisms to ensure bid evaluations: (1) adhere strictly to the evaluation criteria in solicitations; (2) are carried out in accordance with planned approaches; and (3) are appropriately documented.

Recommendation 4: TC should establish a mechanism to enforce the TBCP [Treasury Board Contracting Policy] requirement to document every decision of business value and maintain up-to-date and complete electronic procurement files.

Methodology and management action plan assessment summary

12. When launching this follow-up review in March 2024, OPO asked TC to self-assess their progress in implementing the four recommendations stemming from the initial 2021 review using a progress scale provided by OPO. This scale ranged from “No progress” (level 1) to “Full implementation” (level 5), as outlined in Appendix 1. Furthermore, TC was asked to provide OPO with supplementary information and documentation to support their self-assessment. OPO reviewed TC’s self-assessment and supporting documentation for its overall reasonableness and credibility.

13. For the follow-up review, TC self-assessed the level of implementation of their actions at level 5, “Full implementation” for Recommendation 1,2 and 4, and between level 4, “Substantial implementation” and 5, “Full implementation” for Recommendation 3. However, information and substantiating documentation provided by TC did not clearly demonstrate that structures and processes were fully implemented for recommendations 1 and 3. Therefore, OPO assessed TC’s level of implementation at level 4 “Substantial Implementation” for recommendations 1, at level 3 “Preparations for Implementation” for recommendation 2 (as no updates have been made to the Contract Procedure Manual since 2015), and at level 5 “Full Implementation” for recommendation 3 and 4.

14. The follow-up review showed that overall, TC has updated existing tools and implemented new and enhanced products that support fairness, openness, and transparency in the procurement process. The evidence provided by TC demonstrated a series of advancements designed to meet the intent of each recommendation through several initiatives, including:

  • a Procurement Overview training session that included the development of evaluation criteria
  • Bid Evaluation Instructions; a Bid Evaluation Team Conflict of Interest (COI) Certification form; and a Financial Evaluation Template
  • new naming conventions for electronic files
  • a series of post-evaluation template letters to bidders
  • an Evaluation Criteria template
  • a Peer Review Checklist
  • a Procurement Compliance Review Framework

15. The updated information and newly developed products are shared with Procurement Officers via email or posted on TC’s intranet.

16. In addition to these positive advancements, there remains an opportunity for TC to adopt oversight and documentation practices to track the participation and completion rate of the existing training sessions. A tracking and monitoring mechanism would help the department to be better prepared for future audit activities or to defend against external challenges. TC should update their Contract Procedure Manual to reflect the current procurement policies. This will help the department with ensuring compliance with current laws and regulations. Additional details of OPO’s overall assessment of TC’s response to each recommendation are included in Appendix 3.

Report Card

17. PPR Follow-up Reviews include a report card with a rating that depicts the department’s performance with regard to the three LOEs, resulting in three assessments. In determining these assessment ratings, OPO takes into consideration the results from the initial PPR review and the actions taken by the department to implement the recommendations under each LOE. The assessment ratings are as follows:

  • Satisfactory Plus
  • Satisfactory
  • Partially Satisfactory
  • Unsatisfactory

18. Note that a “Satisfactory Plus” assessment rating is only possible when the initial review resulted in no recommendations being issued under a particular LOE. As the initial review contained recommendations under all three LOEs, a “Satisfactory Plus” rating was not possible in this follow-up review, and the highest possible score TC could obtain for each LOE was “Satisfactory.” Assessment definitions and criteria are outlined in Appendix 2.

19. After reviewing progress made toward implementing the action plans for each recommendation and in consideration of the established assessment ratings, a rating of “Satisfactory” was determined for LOE 1 and LOE 3 and “Partially Satisfactory” for LOE 2. TC’s descriptions of actions taken and their supporting documentation for LOE 1 and LOE 3 demonstrate that reasonable and credible progress has been achieved. Moving forward, TC should track and monitor the completion rate of their training sessions. In addition, TC’s descriptions of actions taken and their supporting documentation for LOE 2 demonstrate that preparations for implementing the recommendation are still in progress. Regarding the actions taken for LOE 2, TC should consider updating their Contract Procedure Manual to reflect current procurement policies.

Table 1: Details of the Report Card
Line of Inquiry (LOE) Rating Assessment
1. Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies. Satisfactory
  • Recommendation 1 applied to this LOE
  • A series of initiatives and products were developed and are in the process of being implemented to address this LOE. Proper tracking and monitoring of the implementation of the existing training sessions will further improve the department’s development of evaluation criteria and selection plans. Together, these help to ensure evaluation criteria and selection plans are in compliance with applicable laws, regulations and policies.
2. Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies. Partially Satisfactory
  • Recommendation 2 applied to this LOE
  • This LOE was partially addressed by developing a series of post-evaluation template letters to bidders, which helps to ensure solicitations are conducted in compliance with laws, regulations and policy requirements.
  • Reviewing and updating the internal Contract Procedure Manual to reflect the current procurement policies will further support consistency and compliance in this area to fully address this LOE.
3. Evaluation of bids and contract award were conducted in accordance with the solicitation. Satisfactory
  • Recommendations 3 and 4 applied to this LOE.
  • To address these recommendations, TC designed, communicated, and delivered several actions including new tools, templates and products. Collectively, these initiatives better equip procurement officers to ensure that bid evaluations adhere strictly to evaluation criteria, are carried out based on the planned approach and are properly documented.

Table 1 Note

TC’s overall report card rating is “Satisfactory.” The key factors contributing to this rating include:

  • The initial review resulted in one or more recommendations under each of the 3 LOEs, thus precluding a rating higher than “Satisfactory”;
  • TC’s Management Action Plan achieved a combination of level 3 for recommendation 2, meaning that some progress has been made; level 4 for Recommendations 1 and 3, meaning that substantial progress has been made; and level 5 for Recommendation 4, meaning that structures and processes are fully implemented, and positive results have been identified;
  • TC’s self-assessment is supported with documentation that demonstrates structures and processes are in place and integrated within most parts of the organization. However, work remains to be done with respect to updating to the Contract Procedure Manual, and establishing a tracking and monitoring mechanism for training sessions.

Conclusion

20. TC has used a combination of existing and new products and practices to support fairness, openness, and transparency in the procurement process and to address the recommendations of the Procurement Ombud following the release of TC’s initial PPR report in July 2021. The evidence provided showed that TC has taken most of the necessary measures designed to meet the intent of each of the four recommendations stemming from the initial review.

21. The follow-up review concluded that overall TC’s self-assessment, supported by substantiating documentation, demonstrated progress made in relation to the recommendations in the initial PPR report. However, some aspects of the self-assessment have been adjusted following OPO’s review of the self assessment form, substantiating documentation and additional clarifications provided by TC in response to the questions posed.

22. OPO found that while structures and processes are in place, followed and integrated within the organization, there was not always clear evidence showing that the implemented structures and processes are fully operating as intended. OPO’s review found that there is a lack of tracking and monitoring for the existing training sessions and there have been no updates made to the Contract Procedure Manual since 2015.

23. These additional steps will provide Procurement Officers with appropriate training and guidance, which will better prepare the Department for the scrutiny of future internal audit activities or to defend against external challenges.

24. Finally, the Procurement Ombud commends TC for their commitment to supporting fairness, openness, and transparency in the procurement process and the progress noted to date. TC’s timely responses to OPO’s requests and follow-up questions pertaining to this review were appreciated.

Appendix 1

Table 2: Assessment of implementation level
Implementation level Assessment
Level 1 No progress or insignificant progress. Actions such as establishing a new committee, conducting meetings, and generating informal plans have not advanced, or insignificant progress has been made.
Level 2 Planning stage. Formal plans for organizational changes have been created and approved.
Level 3 Preparations for implementation. Preparations for implementing a recommendation are in progress – e.g., hiring or training staff, developing necessary resources, etc.
Level 4 Substantial implementation. Structures and processes are in place and integrated within at least some parts of the organization, and some achieved results have been identified.
Level 5 Full implementation. Structures and processes are fully implemented and operating as intended and results have been identified.
Obsolete Recommendation is no longer applicable due to new policies, procedures, etc.

Appendix 2

Table 3: Overall performance assessment scale
Overall performance Assessment of performance
Satisfactory Plus
  • Findings from the initial PPR resulted in no recommendations being issued under the line of inquiry.
  • A rating of Satisfactory plus is based on results from the initial PPR.
Satisfactory
  • Initial PPR resulted in one or more recommendations under the line of inquiry.
  • Action plan(s) have achieved level 5 implementation or a combination of level 4 and level 5 that is deemed to be satisfactory, that is significant progress has been made to implement action plan(s) that address recommendation(s).
  • Organization's response is supported with adequate documentation and/or testing results.
Partially satisfactory
  • Initial PPR resulted in one or more recommendations under the line of inquiry.
  • Some progress has been made in implementing action plan(s), but not enough to adequately address all recommendations under the line of inquiry, for example one or more recommendations achieved level 3, but work remains to be done.
  • Organization's response is supported with some documentation and/or testing results.
Unsatisfactory
  • Initial PPR resulted in one or more recommendations under the line of inquiry.
  • No or insignificant progress made to implement action plan(s).
  • Inadequate supporting documentation provided or testing does not demonstrate that positive results have been achieved.
  • Recommendation(s) not addressed and remain(s) outstanding.

Appendix 3

Table 4: OPO detailed assessment and summary recommendations
TC Action Plan OPO Assessment Level
Recommendation 1: TC should establish a quality control framework to ensure evaluation criteria are measurable, fair and do not favour a particular bidder/group of bidders.

TC will implement a training activity with Procurement Officers to review these observations in detail and ensure Procurement Officers understand how to apply the existing guidance. TC will ensure all TC Procurement Authorities, including new hires, complete the training on Bid Evaluation and Contractor Selection Methodologies.

TC will also provide department-specific training to its Procurement Officers on best practices for drafting evaluation criteria.

To further support these objectives, new hires will continue to enroll in an additional training course “Bid Evaluation and Selection Methodologies” offered by PSPC based on course availability.

Timeline for Implementation:

Discussions and meetings with procurement management and staff regarding OPO’s observations and recommendations have already begun and will continue over the coming months.

Additional training is expected to be made available by Q3 of 2021-22 through both government and contracted sources.

TC will update its internal TC training materials for its “Contracting for Managers and Project Authorities” course to reflect the observations noted by OPO.

Timeline for Implementation:

Course materials will be updated by August 2021.

TC will enhance its Contracting Peer Review Process to ensure further quality control is applied to evaluation criteria.

TC will also implement increased internal monitoring activities.

Timeline for Implementation:

Both the TC Peer Review Process and the Monitoring of Contracting Activities Framework will be updated, and changes implemented by Q4 of 2021-22.

TC developed and implemented:

  • a training deck titled “Bid Evaluation Criteria – Information Session” to provide Procurement Officers with information regarding best practices in developing evaluation criteria, and to highlight the findings and recommendations from audit reports,
  • a Procurement Overview training session to provide an overview of Canada’s legislative framework and procurement procedures during each phase,
  • a Peer Review Checklist to ensure compliance with the Government Contracting Regulations (GCRs), Trade Agreements, and established approval processes,
  • a Procurement Compliance Review Framework to ensure that TC’s contracting activities adhere to federal procurement contracting requirements, including those specified in the Financial Administrative Act (FAA) and GCRs.

TC developed the training presentation and shared it with Procurement Officers in December 2021 via their Common Drive and posted the presentation internally on the intranet. However, TC was unable to confirm if the “Bid Evaluation Criteria Preliminary Information Session” supported by this deck was ever delivered to Procurement Officers. There was no attendance record saved.

The PSPC “Bid Evaluation and Selection Methodologies” training course included in TC’s action plan was not available during the 2-year period following the initial review. The “Developing a Statement of Work and Evaluation Criteria” course (COR407) was therefore recommended as an alternative. This course has recently become mandatory, and TC has started to track the attendance and completion rate.

The Procurement Overview training session is not mandatory, but it is recommended for all Program Managers and Procurement Officers, and advertised on the internal webpage. This training session amalgamated the content of two other courses as an update to the “Contracting for Managers and Project Authorities” course mentioned in TC’s action plan. It is offered every other month in both official languages. There is no attendance tracked but according to TC, there were approximately five participants in the English session and three in the French session each month.

The Peer Review Checklist became mandatory in June 2024, and it was shared with all Procurement Officers via email. This checklist is also accessible for all Buyers in SharePoint. The Peer Review Checklist is required for procurements exceeding $5k for services and all amendments over $5k including taxes. The peer reviews are performed twice per procurement file involving a competitive process and once for non-competitive contracts before contract award. The result of the peer review must be documented on file. The Peer Review Checklist includes a list of items that require Procurement Officers and their peers to review during the planning stage of a procurement engagement to ensure the proper development of evaluation criteria. To provide additional guidance, a training session took place for all procurement staff at the beginning of August, 2024.

The Contract Compliance Review Framework took effect in July 2022, with the intent to address this recommendation. This Framework reviews contracting activities on a quarterly basis with random sampling of one file per Contracting Officer and one file per Low Dollar Value (LDV) Unit. A report is extracted each quarter to identify the percentage of files with irregularities, including a breakdown of the types of irregularities and recommendations for resolving them. The data collected during the review is used to inform tools, processes and a training curriculum offered to Buyers. In addition, a presentation deck was created and presented to introduce the Contract Compliance Review Framework to the Resource Management Committee (RMC) for their adoption. The RMC serves an oversight function for procurement. The presentation was delivered once during a regularly scheduled All-staff meeting to Procurement Officers. The recent report found that the most common infraction during the procurement process is related to the peer review practice. As a result, TC created a working group to finalize the Peer Review Checklist and made its usage mandatory. TC expects to see a decrease in Peer Review related infractions amongst this group by Q1 of 2024-25.

All together, these products serve the purpose of the quality control framework to ensure evaluation criteria are measurable, fair and do not favour a particular bidder/group of bidders. However, OPO’s review found that there is a lack of documentation and oversight in tracking the completion rate of the existing training.

 Level 4
Recommendation 2: TC should review and update its Contract Procedures Manual and any model templates to reflect current procurement policies, especially as it relates to information exchanges with suppliers. In addition, “regret” letters should consistently include an outline of the factors from the bid and criteria that caused the bid to be unsuccessful.

TC will develop and make available standardized templates for Regret Letters. The standardized format will be mandatory for all subsequent procurement activities.

Timeline for Implementation:

New templates (for Letters to Bidders) will be finalized and made available by July 2021.

TC developed a series of post-evaluation template letters to bidders. This new tool includes four templates for different types of results with distinct options for each selection methodology with detailed wording and explanation of factors. The template letters were shared with TC Procurement Community via Common Drive and made available on TC intranet.

The post-evaluation template letters partially address Recommendation 2, which ensures the consistency of "regret" letters and provides the outline of the factors from the bid and criteria that cause the bid to be unsuccessful.

TC noted the Contract Procedures Manual has not been updated since April 2015. Even though TC has referred to official resources including the GCRs and the TBS Directive on the Management of Procurement, and follows rules and procedures for communication, TC should still review and update their internal Contract Procedure Manual to reflect the current procurement policies and recent organizational updates.

 Level 3
Recommendation 3: TC should establish a process to review technical evaluations to ensure they: (1) adhere strictly to the evaluation criteria in solicitations; (2) are carried out in accordance with planned approaches; and (3) are appropriately documented.

In addition to the various training activities identified in response to “Recommendation 1”, TC will develop standardized templates and revised guidelines for conducting bid evaluations, which will be shared with all Procurement Officers.

Timeline for Implementation:

New bid evaluation templates and guidelines will be finalized and made available by September 2021.

TC Procurement will review its information management practices and will provide standardized guidelines to Procurement Officers on which documents must be kept on file.

Timeline for Implementation:

A digital file checklist will be developed and implemented by October 2021.

The TC Monitoring of Contracting Activities Framework will be developed in alignment with the checklist.

TC developed and implemented:

  • updates to their "Bid Evaluation Guidelines". This document details the methods, procedures and reporting structures to be used during an evaluation process. It outlines the steps to be undertaken prior to releasing the bids to the evaluators, which includes:
  • a “Bid Evaluation Team COI Certification” that all evaluators must sign and date prior to the bid evaluation process,
  • the “Financial Evaluation Template” that includes formulas and examples to ensure the calculation of the financial evaluation is accurate and in accordance with planned approach,
  • an “Evaluation Criteria template” to ensure a standardized approach to the assessment of the evaluation criteria,
  • a new naming convention to provide standardized structures to document complex and simple files,
  • a Peer Review Checklist to ensure compliance with the GCRs, Trade Agreements, and established approval process,
  • a Procurement Compliance Review Framework to ensure that TC’s contracting activities adhere to federal procurement contracting requirements, including those specified in the FAA, GCRs and the Treasury Board Secretariat’s policy suite for the management of assets and acquired services.

As noted above in Recommendation 1, the Peer Review Checklist is required for procurements exceeding $5k for services and all amendments over $5k including taxes. The Peer Review Checklist is consistent with a list of items that require Procurement Officers and their peers to review and ensure that the proper documentation is kept on file. A training session was delivered to all procurement staff in August 2024 to inform this practice and provide additional guidance.

The Procurement Compliance Framework took effect in July 2022 to review contracting activities undertaken by TC on a quarterly basis using random sampling, including LDV contracts and regional files. TC further developed a pre-established LDV checklist and Contracting checklist to review the files for their compliance accordingly. These two checklists contain a list of items divided into three levels of risk indicators for reviewers to check whether the contracting activities adhere to federal procurement contracting requirements, and whether the necessary clauses and file are properly documented.

All together, these initiatives and tools better equip Procurement Officers to ensure that bid evaluations adhere strictly to evaluation criteria, are carried out based on the planned approach and are properly documented.

Level 5
Recommendation 4: TC should establish a mechanism to enforce the TBCP requirement to document every decision of business value and maintain up-to-date and complete electronic procurement files.

TC recognizes the importance of maintaining and ensuring adequate file documentation. TC Procurement will review its information management practices and will provide standardized guidelines to Procurement Officers on how to create and manage their digital files.

Timeline for Implementation:

A digital file checklist will be developed and implemented by October 2021.

The TC Monitoring of Contracting Activities Framework will be developed in alignment with the checklist. A guidance document will accompany the checklist.

TC developed, implemented and introduced:

  • a new naming convention to provide standardized structures to document complex and simple files,
  • a Peer Review Checklist to ensure compliance with the GCRs, Trade Agreements, and established approval process,
  • a Procurement Compliance Review Framework to ensure that TC’s contracting activities adhere to federal procurement contracting requirements, including those specified in the FAA, GCRs and the Treasury Board Secretariat’s policy suite for the management of assets and acquired services.

As noted above in both Recommendation 1 and 3, both the Procurement Compliance Review Framework and the Peer Review Checklist require reviewers and Procurement Officers to review and ensure the proper documentation is kept on file.

Collectively, the actions taken fully address the intent of Recommendation 4 which puts the emphasis on the mechanism that enforces proper documentation, up-to-date, and complete procurement files.

 Level 5
Date modified: