Follow-up report to the 2020-2021 procurement practice review of Natural Resources Canada

February 2025

On this page

Introduction

1. In accordance with paragraph 22.1(3)(a) of the Department of Public Works and Government Services Act, the Procurement Ombud has the authority to review the procurement practices of departments to assess their fairness, openness and transparency and to make any appropriate recommendations to the relevant department for the improvement of those practices. Such reviews, which are performed by the Office of the Procurement Ombud (OPO), are referred to as Procurement Practice Reviews (PPRs).

2. In 2020-2021 OPO conducted a PPR entitled “Procurement Practice Review of Natural Resources Canada”(NRCan), which included 8 recommendations to address identified issues. A final report was issued in November 2021 and published on OPO’s website.

3. As a standard practice, OPO follows up on recommendations resulting from PPRs. This is done in order to determine whether the federal departments have implemented their management action plans in response to the Procurement Ombud’s recommendations. This follow-up report includes a summary of results from the initial PPR and OPO’s assessment regarding progress made by NRCan in implementing its 8 recommendations. This report also includes a report card that facilitates comparisons across departments over time.

Results from initial Procurement Practice Review

4. The objective of the initial PPR was to determine whether NRCan’s procurement practices pertaining to evaluation criteria and selection plans, solicitation, and evaluation of bids and contract award, supported the principles of fairness, openness and transparency.

5. For the PPR, OPO analyzed NRCan’s procurement practices under 3 Lines of Enquiry (LOE). In addition, other observations were also noted during the review. Below are descriptions of the LOEs, other observations, a brief summary of observations for each LOE, and the Procurement Ombud’s 8 recommendations included in the initial PPR report.

LOE 1: Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies

6. For LOE 1, OPO’s initial review found that mandatory criteria in solicitation documents were aligned with requirements and were not overly restrictive; however, a majority of files did not define mandatory criteria in a clear, precise and measurable manner. For the most part, point-rated criteria were not overly restrictive, were aligned with the requirement and reflected the relative importance of the criteria; however, point-rated criteria and scoring grids could be improved to increase clarity and minimize subjectivity. The initial review did note that bidders were consistently advised of the manner in which the contract would be awarded.

7. The Procurement Ombud made two recommendations to the issues identified under LOE 1:

Recommendation 1: NRCan should establish a quality control process to ensure mandatory criteria are adequately defined and communicated in a clear, precise and measurable manner.

Recommendation 2: NRCan should update its internal guidance to provide information with respect to the development of evaluation criteria and selection plans and ensure that evaluation criteria, scoring grids and selection methodologies are communicated in solicitations in a clear, precise, and measurable manner.

LOE 2: Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies

8. For LOE 2, OPO’s initial review found that most of the solicitation documents assessed included clear, complete information and instructions to permit the submission of compliant bids. However, several opportunities for improvement were noted regarding the description of requirements and instructions to bidders. There were some solicitation documents which contained inaccurate wording with respect to an Indigenous designation certification required prior to contract award. Further, regarding communications with suppliers, some of the records reviewed were found to be incomplete or did not support the preparation of responsive bids.

9. The Procurement Ombud made two recommendations to address the issues identified under LOE 2:

Recommendation 3: NRCan should correct the wording used in its RFP template(s) so that the purpose of the Indigenous designation certification is clear to all potential bidders.

Recommendation 4: NRCan should establish a process to ensure that information communicated to suppliers prior to bid closing is clear and that adequate documentation of these communications is retained to facilitate management oversight. In addition, NRCan should establish a process to ensure that contract award notices for applicable files are published within the required timeframe and that regret letters are always sent to unsuccessful suppliers in a timely manner.

LOE 3: Evaluation of bids and contract award were conducted in accordance with the solicitation

10. For LOE 3, OPO’s initial review found that inconsistencies in the evaluation of bids and deviations from the planned approach were noted and in several instances contracts were incorrectly awarded. File documentation was also incomplete in several files.

11. The Procurement Ombud made two recommendations to address the issues identified under LOE 3:

Recommendation 5: NRCan should establish processes to ensure that: (1) evaluation instructions are consistently provided to evaluators; (2) non-compliant bids are disqualified and not further assessed; and (3) technical evaluations adhere strictly to the evaluation criteria and scoring grids in solicitations; and are carried out in accordance with planned approaches.

Recommendation 6: NRCan should establish processes to ensure: (1) evaluations are appropriately documented to support the transparency of the award process; and (2) procurement files are complete and kept up-to-date.

12. Other observations made during the review included the perception of contract splitting and internal procedures not being adhered to with respect to a requirement subject to the Nunavut Land Claims Agreement.Footnote 1

13. The Procurement Ombud made two recommendations to address the issues identified under these other observations:

Recommendation 7: NRCan should establish a process to review planned procurements to ensure aggregate requirements are not inappropriately divided to avoid controls or trade agreement obligations.

Recommendation 8: NRCan should update its Contracting Desktop Procedures to comply with the Nunavut Land Claims Agreement and Treasury Board Directive on Government Contracts, Including Real Property Leases, in the Nunavut Settlement Area and establish a process to ensure its procedures are followed with respect to Comprehensive Land Claims Agreements obligations.

Methodology and Management Action Plan Assessment Summary

14. When launching this Follow-up Review in March 2024, OPO asked NRCan to self-assess their progress in implementing the 8 recommendations stemming from the initial 2021 review using a progress scale provided by OPO. This scale ranged from “No progress” (level 1) to “Full implementation” (level 5), as outlined in Appendix. 1. Furthermore, NRCan was asked to provide OPO with supplementary information and documentation to support their self-assessment. OPO reviewed NRCan’s self-assessment and supporting documentation for its overall reasonableness and credibility.

15. NRCan self-assessed the level of implementation of its actions at “Level 4 - Substantial Implementation” for recommendation 1 and at “Level 5 - Full Implementation” for recommendations 2 to 8 inclusive. OPO is in agreement with the department’s “Level 4—Substantial Implementation assessment of Recommendation 1 and its “Level 5—Full Implementation” assessment of recommendations 2, 3, 5, 6, 7, and 8. However, substantiating documentation provided by NRCan demonstrated that structures and processes were only partially implemented for Recommendation 4. Therefore, OPO assessed NRCan’s level of implementation for recommendation 4 at “Level 4 - Substantial Implementation.”

16. Overall, the Follow-up Review showed that NRCan has enhanced existing tools and developed and implemented new ones that support fairness, openness, and transparency in the procurement process. The evidence provided by NRCan demonstrated a series of these advancements designed to meet the intent of each recommendation through several initiatives, including:

  • the design and implementation of a Peer Review process
  • the development of a comprehensive Business Owners Evaluation Guide
  • the update of templates and internal guidance documents; and
  • the delivery of facilitated information sessions to procurement staff regarding procurement best practices and obligations pertaining to the Comprehensive Land Claims Agreements

17. The recent implementation of some of the initiatives to address recommendations 1 and 4 have yet to yield substantial results to fully validate the action plans’ effectiveness. Eventually, time and the adoption of a mechanism to capture trends and outcomes can enable NRCan to demonstrate their overall impacts.

Report Card

18. PPR Follow-up Reviews include a report card with a rating that depicts the department’s performance, taking into consideration the results from the initial review and the actions taken by the department to implement the recommendations under each LOE and other observations. The assessment ratings are as follows:

  • Satisfactory Plus
  • Satisfactory
  • Partially Satisfactory
  • Unsatisfactory

19. Note that a “Satisfactory Plus” assessment rating is only possible when the initial review resulted in no recommendations being issued under a particular line of enquiry. As the initial review contained recommendations under 3 LOEs alongside the Other observations identified, a “Satisfactory Plus” rating was not possible in this Follow-up Review, and the highest possible score NRCan could obtain for each LOE was “Satisfactory”. Assessment definitions and criteria are outlined in Appendix 2.

20. After the review of progress made toward implementing the action plans for each recommendation and in consideration of the established assessment ratings, a rating of “Satisfactory” was determined for each LOE and the two additional observations. NRCan’s descriptions of actions taken and its supporting documentation demonstrated that reasonable and credible progress has been achieved with regard to the 3 LOEs and other observations.

Table 1: Details of the Report Card
Line of Inquiry (LOE) Rating Assessment
Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies. Satisfactory
  • Recommendations 1and 2 applied to this LOE
  • NRCan developed and implemented a 2-phased peer review process and an Evaluation Criteria Guidelines document. Doing so demonstrated the department’s response to ensure effective governance and oversight mechanisms are in place to support this LOE
  • Through its completed actions, the department improved its compliance with the Rules that were assessed in this LOE, and the associated findings made in the Procurement Practice Review of Natural Resources Canada.
Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies. Satisfactory
  • Recommendations 3 and 4 applied to this LOE
  • NRCan updated its RFP template to clarify Indigenous designation certification requirements. Further, the department also updated internal guidance documents to clarify roles and responsibilities pertaining to communication with suppliers and for adhering to documentation requirements throughout the solicitation period
  • These actions improved NRCan’s compliance with the Rules that were assessed in this LOE, and the associated findings made in the Procurement Practice Review of Natural Resources Canada.
Evaluation of bids and contract award were conducted in accordance with the solicitation. Satisfactory
  • Recommendations 5 and 6 applied to this LOE
  • NRCan developed and implemented an evaluation guide. The department also updated existing internal guidance to foster consistent document storage, evaluation and contract award practices
  • The proposed actions completed by the department improved the its compliance with the Rules that were assessed in this LOE, and the associated findings made in the Procurement Practice Review of Natural Resources Canada.
Other Observations Satisfactory
  • Recommendations 7 and 8 applied to OPO’s Other Observations
  • NRCan updated its procurement tracking system and internal guidance documents to provide check points for the Contracting Authority to validate the procurement strategy applied
  • The department also updated its internal guidance, and delivered an information session pertaining to the Comprehensive Land Claims Agreements to its procurement staff
  • Through the actions fulfilled, NRCan improved its compliance with the findings that were made in the Procurement Practice Review of Natural Resources Canada.

Table 1 Note

NRCan’s overall report card rating is “Satisfactory”. The key factors contributing to this rating include:

  • The initial review resulted in one or more recommendations under each of the three LOEs and 2 recommendations under other observations, thus precluding a rating higher than “Satisfactory”
  • NRCan’s Management Action Plans have achieved a combination of level 4 and level 5, which are deemed satisfactory and demonstrate significant progress has been made to implement action plans addressing the recommendations
  • NRCan’s self-assessment was supported with documentation that demonstrated structures and processes were in place and integrated within most parts of the organization

Conclusion

21. NRCan has used a combination of existing and new products and practices to support fairness, openness, and transparency in the procurement process; and to address the recommendations of the Procurement Ombud following the release of NRCan’s PPR report in November 2021. The evidence provided showed that NRCan has taken the necessary measures designed to meet the intent of each of the 8 recommendations stemming from the initial review.

22. Recommendations 2, 3, 5, 6, 7 and 8 have been fully addressed through the implementation of new and enhanced products and processes. The action plans for recommendations 1 and 4 have been substantially implemented and are intended to reach full implementation over time with viable results for effectiveness.

23. The adoption of a mechanism to capture and measure notable outcomes resulting from the applied tools and processes will further support the department in ensuring continuous improvement and adaptability to changes within the procurement realm. In addition, it will better prepare NRCan for the scrutiny of future internal audit activities or to defend against external challenges.

24. This review concluded that NRCan’s self-assessment is reasonable and credible. As such, OPO concludes that all recommendations have been substantially or fully implemented , resulting in an overall report card rating of “Satisfactory.”

25. Finally, the Procurement Ombud commends NRCan for its commitment to supporting fairness, openness, and transparency in the procurement process and for the progress noted to date. NRCan’s timely responses to OPO’s requests and follow-up questions pertaining to this review were appreciated.

Appendix 1

Table 2: Assessment of Implementation Level
Level 1 No progress or insignificant progress. Actions such as establishing a new committee, conducting meetings, and generating informal plans have not advanced, or insignificant progress has been made.
Level 2 Planning stage. Formal plans for organizational changes have been created and approved.
Level 3 Preparations for implementation. Preparations for implementing a recommendation are in progress—for example hiring or training staff, developing necessary resources, drafting and adopting policy documents, etc.
Level 4 Substantial implementation. Structures and processes are in place and integrated within at least most parts of the organization, and some achieved results have been identified.
Level 5 Full implementation. Structures and processes are fully implemented, and positive results have been identified.
Obsolete Recommendation is no longer applicable due to new policies, procedures, etc.

Appendix 2

Table 3: Overall Performance Assessment Scale
Overall performance Assessment of performance
Satisfactory Plus
  • Findings from the initial PPR resulted in no recommendations being issued under the line of inquiry
  • A rating of Satisfactory plus is based on results from the initial PPR
Satisfactory
  • Initial PPR resulted in one or more recommendations under the line of inquiry
  • Action plan(s) have achieved level 5 implementation or a combination of level 4 and level 5 that is deemed to be satisfactory, that is significant progress has been made to implement action plan(s) that address recommendation(s)
  • Organization's response is supported with adequate documentation and/or testing results
 
Partially satisfactory
  • Initial PPR resulted in one or more recommendations under the line of inquiry
  • Some progress has been made in implementing action plan(s), but not enough to adequately address all recommendations under the line of inquiry, for example one or more recommendations achieved level 3, but work remains to be done
  • Organization's response is supported with some documentation and/or testing results
 
Unsatisfactory
  • Initial PPR resulted in one or more recommendations under the line of inquiry
  • No or insignificant progress made to implement action plan(s)
  • Inadequate supporting documentation provided or testing does not demonstrate that positive results have been achieved
  • Recommendation(s) not addressed and remain(s) outstanding
 

Appendix 3

Table 4: OPO Detailed Assessment and Summary of Recommendations
NRCan’s Action Plan OPO’s Assessment Level
Recommendation 1: NRCan should establish a quality control process to ensure mandatory criteria are adequately defined and communicated in a clear, precise and measurable manner.

NRCan will establish a peer review process to review mandatory criteria (prior to solicitation release) in procurement files identified using a risk-based approach. Internal guidance documents (for example, Desktop Procedures) will be updated to reflect the new process.

Timeline for implementation: June 30, 2022 

The scope of the existing Quality Assurance program will be expanded to address evaluation criteria review.

Timeline for implementation: June 30, 2022

NRCan developed and implemented a comprehensive 2-phased Peer Review process (Pre-Solicitation and Pre-award). The peer review is to ensure solicitation criteria are clear to the reader and represent the principles of fairness, openness and transparency. The pre-award peer review process ensures that the evaluation and the contractor selection method will be conducted in accordance with the solicitation and confirms that no anomalies or potential issues exist which could impede the principles of fairness, openness, and transparency of the procurement process. The new process has been in effect across the NRCan Procurement Services Unit (PSU) since April 2023.

Accordingly, all internal guidance documents were updated to include the mandatory peer review, which was disseminated to procurement staff.

A second component of the department’s proposed action plan involved updating the scope of the existing Quality Assurance program to include a review of a solicitation’s evaluation criteria. However, budget constraints resulted in a pause in the program and revisions to the organization structure to be implemented. This impacted the proposed expansion of the department’s Quality Assurance program. Currently, structures and processes are in place pending full implementation (June 2024). Due to the recent change, it is difficult to ascertain substantial results from implementation.

NRCan has fulfilled most components of its proposed action plan relevant to this recommendation. The recent changes implemented as a response to budget constraints affecting the Quality Assurance program demonstrates the department’s adaptability and focus to continually improve their procurement processes. Through the duration of time and adjustment necessitated by the change in organizational structure, the action plan will eventually be fully implemented with some viable results. Therefore, a Level 4—Substantial implementation has been assigned to this element of NRCan’s action plan.

 4
Recommendation 2: NRCan should update its internal guidance to provide information with respect to the development of evaluation criteria and selection plans and ensure that evaluation criteria, scoring grids and selection methodologies are communicated in solicitations in a clear, precise, and measurable manner.

NRCan will update and communicate its internal guidance documents (for example, Desktop Procedures) to ensure evaluation criteria instructions are clear.

Timeline for implementation: June 30, 2022 

NRCan will develop and communicate an Evaluation Guide for use by both the Contracting Authorities and Technical Authorities to assist in developing evaluation criteria.

Timeline for implementation: June 30, 2022 

The implementation of the peer review process in response to recommendation #1 will further strengthen evaluation criteria and selection methodologies.

Timeline for implementation: June 30, 2022

In response to Recommendation 2 for this LOE, the department also developed and implemented a detailed evaluation guide to support Contracting and Technical Authorities in developing and executing effective evaluation methods. It is a comprehensive tool that provides clear and detailed guidelines to consider when defining both mandatory and technical evaluation criteria. The guide is accessible to the Procurement Services Unit (PSU), and Contracting and Technical Authorities are encouraged to share the guide with their clients.

Desktop procedures were updated with guidance for evaluation criteria and information was shared with staff via email. The internal guidance is accessible and applied for departmental procurements.

Finally, the peer review process that was developed and implemented also supported the action plan in response to this recommendation. It has been in use across the PSU for more than one year. NRCan successfully fulfilled all elements of the proposed action plan resulting in a Level 5—Full implementation rating for this recommendation.

 5
Recommendation 3: NRCan should correct the wording used in its RFP template(s) so that the purpose of the Indigenous designation certification is clear to all potential bidders.

NRCan has revised its RFP template to clarify the purpose of Indigenous designation certification.

Timeline for implementation: Completed August 2021

The update made to the RFP template included accurate guidance and instructions. Its implementation supported the wide use of the updated version. There were no documented reports of recent concerns regarding lack of clarity for Indigenous certification in procurement. It would be valuable for NRCan to schedule a regular review of the RFP templates to ensure that all information is consistently up-to-date.

The completed action plan has responded effectively to the recommendation and has resulted in a Level 5—Full implementation rating.

 5
Recommendation 4: NRCan should establish a process to ensure that information communicated to suppliers prior to bid closing is clear and that adequate documentation of these communications is retained to facilitate management oversight. In addition, NRCan should establish a process to ensure that contract award notices for applicable files are published within the required timeframe and that regret letters are always sent to unsuccessful suppliers in a timely manner.

NRCan will update and communicate its internal guidance documents (for example, Desktop Procedures) to a) include instructions clarifying the roles and responsibilities related to communications with suppliers pre-bid closing; and b) add a mandatory requirement to issue regret letters for competitive procurements and provide a standard letter template.

Timeline for implementation: March 31, 2022 

NRCan will review and update its “Online Procurement Checklist” to ensure both the Regret Letters and Contract Award Notices have been sent and posted accordingly within the prescribed time period, as appropriate.

Timeline for implementation: June 30, 2022

NRCan completed the action plan as stated. Internal guidance documents were updated with roles and responsibilities pertaining to communication with suppliers prior to bid closing as well as the issuance of regret letters after contract award. It is important to note that the department’s action plan did not address the element of timeliness and document storage as outlined in the original recommendation. However, NRCan did implement guidance for document storage and structure in October 2023 as part of an action plan in response to another recommendation. The recent issuance of the guidance does not yet allow for sufficient time to fully validate substantial results.

NRCan updated the online checklist to ensure that the issuance of regret letters and award notices were included as part of the process. However, guidance regarding timelines for the issuance of the documents were not outlined as stated in the action plan. Establishing timelines, even as a guide, can foster a more efficient procurement process.

As a result, this action plan is assigned a Level 4—Substantial implementation rating due to the missed element relating to timelines as outlined within OPO’s recommendation and the department’s proposed action plan.

 4
Recommendation 5: NRCan should establish processes to ensure that: (1) evaluation instructions are consistently provided evaluators; (2) non-compliant bids are disqualified and not further assessed; and (3) technical evaluations adhere strictly to the evaluation criteria and scoring grids in solicitations; and are carried out in accordance with planned approaches.

NRCan will ensure the three identified elements are appropriately reflected in internal guidance documents, including the new Evaluation Guide to be developed.

Timeline for implementation: June 30, 2022

Guidance documents were updated to ensure that evaluation instructions provided were consistent, non-compliant bids were disqualified accordingly and technical evaluations adhered to the outlined criteria and scoring grid. These were communicated to the PSU as stated in NRCan’s action plan. The evaluation guide has been implemented for almost 2 years and it provides guidance to contracting and technical authorities regarding various elements of the evaluation process. The updates made to the documents were comprehensive, detailed, and effectively responded to the recommendation.

 5
Recommendation 6: NRCan should establish processes to ensure: (1) evaluations are appropriately documented to support the transparency of the award process; and (2) procurement files are complete and kept up-to-date.

NRCan will update “Instructions to Evaluators” document to state that the Contracting Authority will not proceed with contract award until duly signed technical evaluation documents are received.

Timeline for implementation: June 30, 2022

 Internal guidance documents will be updated to explicitly state the requirement to ensure the procurement file is appropriately documented.

Timeline for implementation: June 30, 2022

NRCan updated its internal guidance by replacing the “Instructions to Evaluators” document with the “Business Owners Evaluation Guide”. This was shared with the procurement staff in October 2022.

The department also developed guidance to folder structure for document storage in response to the recommendation. According to an email sent in October 2023, the guidance to folder structure was implemented.

NRCan fulfilled the elements noted in its proposed action plan. Overall, NRCan completed all elements of its proposed action plan for this recommendation resulting in a Level 5—Full implementation rating.

 5
Recommendation 7: NRCan should establish a process to review planned procurements to ensure aggregate requirements are not inappropriately divided to avoid controls or trade agreement obligations.

NRCan will modify its existing procurement tracking system to help identify similar requirements emanating from the same Branch.

Timeline for implementation: June 30, 2022

NRCan will update its internal guidance documents (e.g., Desktop procedures) to reflect the system enhancement.

Timeline for implementation: June 30, 2022

NRCan updated the automated procurement tracking system to verify if a requirement is recurring or already has an existing contract in place. This serves as a valuable control prompt should the Contracting Authority fail to identify the appearance of contract splitting. NRCan updated the internal guidance documents with considerations to minimize the risk of contract splitting.

The guidance has been in place for over a year and can demonstrate some concrete results should further assessment be required. In addition, the organizational change in the department that has resulted in commodity based teams can effectively foster a system that ensures that risk-based strategies are applied to similar procurements. Finally, NRCan also developed and delivers a virtual, instructor-led “Procurement 101” course for business owners which covers “Practices to Avoid”, like contract splitting. The topic defines the terminology and example scenarios are discussed. There have been a total of 11 sessions delivered to over 900 participants since it’s inception in fall of 2023. NRCan plans on a minimum of 2 sessions to be delivered annually.

The department has completed all elements in its proposed action plan leading to an assigned rating of Level 5—Full implementation.

 5
Recommendation 8: NRCan should update its Contracting Desktop Procedures to comply with the Nunavut Land Claims Agreement and Treasury Board Directive on Government Contracts, Including Real Property Leases, in the Nunavut Settlement Area and establish a process to ensure its procedures are followed with respect to Comprehensive Land Claims Agreement obligations.

NRCan will review the existing NLCA information in its internal guidance documents and ensure alignment with government-wide direction of the NLCA. NRCan will continue to engage with contracting authorities, including a planned Information Session to share lessons learned from recent procurements. This will complement information and instructions previously communicated as part of the policy implementation in 2019.

Timeline for implementation: December 31, 2021 

 NRCan updated the internal guidance documents with accurate information regarding government-wide direction of Nunavut Land Claims Agreement. In addition, an information session on the Comprehensive Land Claims Agreements was delivered to the PSU staff with a deck for future reference. The department successfully implemented their action plan in response to the recommendation resulting in a Level 5—Full Implementation rating. 5
Date modified: