Procurement Practice Review: Follow-up Report on the 2014-2015 Procurement Practice Review “Review of the Procurement Management Framework of Natural Resources Canada”

“Review of the procurement management framework of Natural Resources Canada ”

March 2018

Promoting Fairness, Openness and Transparency in Federal Procurement

Main points

What we reviewed

1. In 2014-2015, the Office of the Procurement Ombudsman (OPO) conducted a procurement practice review entitled Review of the Procurement Management Framework of Natural Resources Canada.

2. In July 2017, OPO asked Natural Resources Canada (the Department) to provide information regarding actions taken in response to the recommendations in the above-noted review.

3. The purpose of the follow-up exercise was to determine whether the Department considered and took action, or developed plans, in response to the Procurement Ombudsman’s recommendations. In this regard, OPO assessed the information provided by the Department for overall reasonableness and credibility. This report provides a summary, as well as specific examples, of progress made by the Department in implementing the recommendations from the original review.

Why it’s important

4. There are three main reasons why reporting on progress made in response to the Procurement Ombudsman’s recommendations is important. First, it informs interested stakeholders of specific actions organizations have taken to improve procurement practices. Second, by sharing information on changes being implemented by the organizations whose practices were reviewed, OPO facilitates other federal organizations’ ability to introduce similar improvements. Lastly, the information on the nature and extent of responses to the recommendations provides an indication of the usefulness of OPO’s reviews in promoting fairness, openness and transparency in federal procurement.

PRB

What we found

5. The Department stated it took seven actions in response to the two recommendations made by the Procurement Ombudsman, including three actions to enhance its monitoring and oversight regime, and four actions to ensure the consistent application of controls. Using a scale provided by OPO^{Footnote 1}, the Department initially self-assessed the level of implementation of these actions as “substantial implementation” (Level 4), with one action assessed as “preparations for implementation” (Level 3). During the report clearance process^{Footnote 2} , the Department revised its self-assessment to “full implementation” (Level 5) for all actions and provided OPO with documentation to substantiate the revised assessment. OPO’s assessment and conclusion in this report are based on the Department’s revised self-assessment.

Introduction

6. OPO published the following report in April 2015:

• Review of the Procurement Management Framework of Natural Resources Canada

Objectives

7. The objectives of this follow-up review were to determine:

• Whether the Department considered the recommendations made by the Procurement Ombudsman in the April 2015 review with respect to its procurement practices;
• Whether action plans to respond to the recommendations were prepared and approved; and,
• What actions were undertaken in response to the recommendations, and the extent to which each action had been monitored and completed.

8. OPO expected the Department to have introduced changes to improve its procurement practices in response to the Procurement Ombudsman’s recommendations.

What we found

9. OPO requested the Department provide information on actions planned or implemented as a result of the recommendations in the original (i.e. April 2015) review. This report reflects actions implemented by the Department up to September 2017.

10. The approach OPO uses for follow-up exercises differs from the approach used in OPO’s procurement practice reviews. The assessment of progress made against recommendations was based upon the Department’s self-assessment and assertions regarding its plans and actions, coupled with supporting (i.e. substantiating) documentation. For each recommendation in the original review, OPO reviewed the information provided for overall reasonableness and credibility. This was done by:

• Verifying whether any contradiction existed between the Department’s assertions and information available from publicly accessible sources or obtained during the original review;
• Analyzing the Department’s responses to understand how its actions addressed the recommendations and whether there were plans to monitor the results or effectiveness of these actions or changes; and,
• Seeking clarification, as required, to ensure a clear understanding of the information and supporting documentation provided by the Department.

11. This report provides an overview of the Department’s assertions, as well as OPO’s assessment, on progress in implementing changes in response to the recommendations contained in the original review.

Assessment of implementation of the department’s actions

Summary of original review findings

12. The original review covered key elements of a procurement management framework such as: organization and structure; procurement strategy and planning; documentation and communication of policies, procedures, and roles and responsibilities; procurement training; monitoring and risk management; quality assurance (QA); and reporting. The original review’s findings for each of these elements are summarized below.

13. The Department had used centralized procurement to service the majority of its procurement transactions. There were, however, three sectors within the organization that had certain delegated contracting authorities, meaning they could undertake some procurements without going through the Department’s central procurement.

14. Regarding procurement strategy and planning, the Department had not provided evidence of a risk-based procurement strategy or planning exercises beyond an annual request to program managers asking them to identify at-risk requirements above $70,000, unusually complex procurements, and special projects.

15. Departmental policies, procedures, roles and responsibilities had been found to align with the Treasury Board Contracting Policy.

16. Procurement training had consisted primarily of on-the-job training, mandatory tool-specific training from Public Services and Procurement Canada (formerly Public Works and Government Services), as well as individual learning plans.

17. Monitoring and risk management of procurement and contracting activities had been accomplished in large part through the Department’s Procurement Review Board (PRB) and the Procurement Policy Analysis and Reporting (PPAR) Unit. These oversight and monitoring entities had been noted by OPO as good elements from a governance perspective. Nonetheless, OPO had identified three areas where the Department’s monitoring regime could have been enhanced:

• Non-centrally processed procurement;
• Departmental Standing Offers and Supply Arrangements; and
• Low-dollar value contracts and amendments.

18. Regarding quality assurance (QA), OPO had noted good practices through the Department’s use of checklists, peer reviews and performance management agreements. While checklists existed in the Department, only two had been noted as mandatory.

19. Regarding reporting, Departmental contracting data used for proactive disclosure reporting had been provided to managers by the PPAR Unit for verification prior to making it public. At the time of the original review, a manager’s confirmation was not required prior to posting proactive disclosure data, increasing the risk of undetected errors in the data. The original review stated the Department had introduced new proactive disclosure training for managers which outlined steps managers needed to take to validate the data.

20. As part of the original review, OPO had examined select procurement files to assess whether procurement practices were consistent with applicable policies and procedures. Good practices had been observed, for example evaluation criteria were reflected in Statements of Work. Nonetheless, deficiencies had been observed in some files, and OPO had noted that files with checklists were more robust and compliant than those without.

Original review conclusion

21. The original review had concluded the Department’s procurement framework had been aligned with procurement related legislation and policy requirements. No major gaps had been found in the framework that would pose a risk to fairness, openness and transparency. OPO had noted good elements from a governance perspective, more specifically the PRB and PPAR Unit as oversight and monitoring entities.

22. OPO had noted, however, that the scope of PRB’s mandate was focused on contract requirements processed by the central procurement unit. In addition, PRB’s mandate had not explicitly included monitoring of contracts valued at less than $25,000. Similarly the PPAR Unit had been mainly focused on procurements processed by the central procurement unit. These limitations had represented potential vulnerabilities to the Department.

23. From an operational perspective, OPO had noted good practices including the use of checklists, peer reviews, and quality assurance of procurement files. However, the sporadic, and in some cases discretionary, use of these effective safeguards could have jeopardized the overall quality of the Department’s procurement.

24. Given the Department’s contracting volumes and values, the original review had suggested that it may have been advisable to review existing awareness and training sessions to ensure the Department’s training regime was sufficiently robust to contribute to fair, open and transparent procurement.

25. The Procurement Ombudsman recommended the Department:

• Take appropriate measures to enhance its monitoring and oversight regime commensurate with senior management sanctioned risk tolerances for the full spectrum of Departmental procurement; and
• Build on the existing good quality control and assurance practices by taking the necessary measures to ensure these are applied consistently and to all sectors of the Department procuring goods and services; regardless of type of procurement and monetary value.

Summary of departmental response to follow-up

26. The Department stated it took actions to address the recommendations made in the original review. The Department had initially stated that six of these actions had been substantially implemented, and one action was at the “preparations for implementation” stage, and had provided some substantiating documentation. During the report clearance process, the Department revised its self-assessment to “full implementation” for all seven actions and provided OPO with additional substantiating documentation.

Response to recommendation 1

27. In response to the Procurement Ombudsman’s first recommendation, the Department stated that enhancements to its monitoring and oversight regime were introduced through three actions.

28. First, the Department stated a management response was required for all contract QA reports, which ensured awareness of findings and enabled corrective measures as appropriate.

29. Second, the Department stated a new risk-based approach to choosing files for QA review, endorsed by the PRB in July 2017, was introduced to enable “flexible responsiveness to arising areas of vulnerability”.

30. Finally, the Department stated it would, where appropriate, include QA findings in bi-weekly national videoconference sessions for procurement professionals, to provide an open forum to focus on solutions to challenges faced from coast to coast.

Response to recommendation 2

31. In response to the Procurement Ombudsman’s second recommendation, the Department identified four actions to ensure the consistent application of controls.

32. First, the Department stated it developed a new QA tool which required the same steps and analysis be performed regardless of which sector in the Department was being reviewed.

33. Second, the Department reported having developed a revised schedule of QA review activities, which included sectors with special procurement delegation for both trend analysis and targeted reviews.

34. Third, the Department stated it increased training and awareness sessions to sectors with special delegated contracting authority, as well as ongoing mentoring of procurement professionals. In addition, sectors with special delegated contracting authority within the Department were asked to provide a record of procurement training taken by officers.

35. Finally, sectors with specialized delegated contracting authority within the Department were required to prepare Standard Operating Procedures to support their procurement activities.

Assessment of departmental response

36. The Department stated it implemented seven actions to respond to the Procurement Ombudsman’s two recommendations. All actions were reported, by the Department, as being fully implemented. The Department provided documentation to substantiate the actions taken.

37. Regarding the first recommendation, the Department provided substantiating documentation to support the assertion that it implemented three actions to enhance its monitoring and oversight regime.

• The first action related to requiring a management response to QA findings. OPO noted, in the original report, that the Department had already begun this action.
• Regarding the second action, the Department provided documentation demonstrating its approach to risk assessment, including a risk reference guide and a risk assessment of contract types.
• Regarding the third action, the Department provided documentation confirming that QA findings were shared with procurement professionals.

38. Regarding the second recommendation, the Department provided substantiating documentation to support the assertion that it implemented four actions to ensure consistent application of controls.

• The first action related to the development of a new QA tool, and the Department provided screenshots of the functions available within this tool.
• Regarding the second action, the Department developed a revised QA monitoring plan. The revised QA plan, among other things, changed the frequency of coverage for the three sectors with special procurement delegation. Documentation provided by the Department indicated the frequency was changed from quarterly across each sector to annual rotations across sectors. This means that each of these three sectors will be reviewed once every three years under the new plan, compared to once every three quarters under the previous schedule.
• Regarding training, the Departmental response specified that sectors with special delegated contracting authority within the Department were asked to provide a record of procurement training taken by officers. The Department provided copies of calendar invitations for procurement training sessions, a copy of the training presentation, and written confirmation that 75% of employees with specialized procurement delegation were among the employees who participated in the training.
• Finally, the Departmental response stated that procurement client sectors within the Department were required to prepare Standard Operating Procedures (SOPs) to support their procurement. OPO requested examples of the procedures, and the Department provided the SOPs for two of the three sectors with special delegated contracting authority.

Conclusion

39. The Department considered the Procurement Ombudsman’s recommendations from the original procurement practice review and provided information on its plans and actions. The Department stated it had fully implemented all actions to respond to the Ombudsman’s recommendations, and provided documentation to substantiate this statement. OPO’s analysis found the Department’s response (i.e. the self-assessment and the substantiating documentation) to be reasonable and credible.

Annex A—Office of the Procurement Ombudsman Recommendations and the Department’s Responses

OPO Recommendations	Department’s Responses
The Procurement Ombudsman recommended that NRCan: 1. take appropriate measures to enhance its monitoring and oversight regime commensurate with senior management sanctioned risk tolerances for the full spectrum of NRCan procurement.	Enhancements to monitoring and oversight regime have been introduced, including: A management response required for all Contract Quality Assurance reports, which ensures awareness of findings and enables appropriate corrective measures as appropriate. A new risk-based approached [sic] in choosing files for QA reviews, endorsed by the Procurement Review Board (July 2017), to enable flexible responsiveness to arising areas of vulnerability. Inclusion of new QA findings in a bi-weekly national videoconference for procurement professionals at NRCan to provide an open forum for solution focused discussion to address challenges faced from coast to coast.
The Procurement Ombudsman recommended that NRCan: 2. build on the existing good quality control and assurance practices by taking the necessary measures to ensure these are applied consistently and to all sectors of the department procuring goods and services; regardless of type of procurement and monetary value.	Consistent application of controls include: The development of a new QA Tool, which imposes the same rigour for a given category of analysis regardless of departmental sector subject to review. A revised schedule of QA review activities, which includes sectors with special procurement delegation. Those QA reviews include both trend analysis and targeted reviews. Increased training and awareness sessions to Sectors with special delegation and ongoing mentoring of procurement professionals. Client sectors were required to prepare Standard Operating Procedures to support their procurement activities.

Annex B

OPO Recommendations	Additional information provided during the report clearance process
The Procurement Ombudsman recommended that NRCan: 1. take appropriate measures to enhance its monitoring and oversight regime commensurate with senior management sanctioned risk tolerances for the full spectrum of NRCan procurement.	Please find attached additional supporting documents that further demonstrate and substantiate NRCan’s compliance with OPOs recommendations made in its 2015-2015 [sic] Procurement Practice Review. QA Tool. Screen shots are attached that demonstrate its existence and use. We remain open to having you come and see it in an operational context. Training of Clients with special delegation. A screen shot of my calendar demonstrating that I delivered the training in both official languages is attached. We have previously provided your office with a copy of the training presentation but are including it again for reference purposes. We can confirm that 75% of NRCan employees with specialized delegation were among the individuals who participated in the procurement training. We are re-attaching the client Standard Operating Procedures which were included in our original response but the draft mentions that none were provided to your organization. Please find the “risk based approach for choosing files” attached.
The Procurement Ombudsman recommended NRCan: 2. build on the existing good quality control and assurance practices by taking the necessary measures to ensure these are applied consistently and to all sectors of the department procuring goods and services; regardless of type of procurement and monetary value.	n/a

Date modified:

2018-11-01