Follow-up report to the 2019-20 procurement practice review of the Department of Fisheries and Oceans Canada

June 2023

On this page

Introduction

1. As a standard practice, the Office of the Procurement Ombudsman (OPO) follows up on the recommendations made in Procurement Practice Reviews (PPRs) approximately two years after those recommendations are published.

2. Follow-up reviews allow OPO to provide assurance that actions to respond to PPR recommendations have been effectively implemented, or that senior management has accepted the risk of not taking action; and provide an organizational report card that facilitates comparisons across federal organizations over time.

3. In 2019-2020 OPO conducted a PPR entitled Procurement practice review of the Department of Fisheries and Oceans Canada (DFO). The final report was issued in February 2020.

4. The initial review was undertaken to determine whether DFO’s procurement practices pertaining to evaluation criteria and selection plans, solicitation, and evaluation of bids and contract award, supported the principles of fairness, openness and transparency. To make this determination, OPO examined whether DFO’s procurement practices were consistent with Canada’s obligations under applicable sections of national and international trade agreements, the Financial Administration Act and regulations made under it, the Treasury Board Contracting Policy and, when present, departmental guidelines.

5. The following three lines of inquiry (LOE) were used to assess the highest-risk procurement elements:

  • LOE 1: Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies
  • LOE 2: Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies
  • LOE 3: Evaluation of bids and contract award were conducted in accordance with the solicitation

6. Regarding LOE 1, which reviewed evaluation criteria and selection plans, OPO’s initial review found issues with half of the applicable procurement files. In two cases (i.e., lowering the security requirement, and obtaining a quote from one supplier before posting the solicitation), one bidder was favoured over others. In the other cases, either the criteria themselves or the selection methodology were unclear or contained discrepancies. These issues were noted as potentially leading to non-compliant bids and improper evaluations, thus impacting the fairness of the procurement process.

7. Regarding LOE 2, which reviewed solicitation documents and DFO’s actions during the solicitation process, OPO found issues with a third of the applicable procurement files, including one case where DFO purposely modified the requirement to remain within the ProServices method of supply and below the Task and Solutions professional services (TSPS) threshold. In 6 cases, DFO appeared to favour one bidder over others or found ways to limit the bidder pool, thereby affecting the openness of those procurements. In 4 other cases (one where the rotation/ proportion was unclear and 3 where DFO did not provide correct information during the solicitation process), the transparency and fairness of the processes were negatively impacted by DFO’s actions.

8. Regarding LOE 3, which reviewed the evaluation of bids and contract award, OPO found issues with 4 of 24 applicable procurement files. Overall, DFO properly informed its evaluators of how to conduct those processes. DFO also sought to ensure that potential conflict of interest could be mitigated by requiring one of the evaluators to be outside the span of control of the project authority. This positive step is significant because this stage of procurement has the greatest potential to affect the overall competitive process given its importance vis-à-vis awarding the contract to the correct bidder in a fair, open and transparent manner. Of concern were 2 cases where the evaluators appeared to be in a potential conflict position, including a case where the evaluation of the lower-priced bid, and the individual evaluations of the winning bid, were missing.

9. In order to address the issues identified during the initial review, the Procurement Ombudsman made 6 recommendations: 3 recommendations under LOE 1, 1 recommendation under LOE 2, and 2 recommendations under LOE 3.

Methodology

10. In May 2022, OPO asked DFO to self-assess their progress in implementing the 6 recommendations using an established progress scale, ranging from “no progress” (level 1) to “full implementation” (level 5), as outlined in Appendix 1. DFO was also asked to provide supplementary information and documentation to explain and support their self-assessment.

11. When reviewing departments’ self-assessments during a follow-up exercise, OPO may conduct additional testing of procurement files, if required, to obtain evidence that demonstrates structures and processes have been implemented and positive results have been identified. During this review, it was determined that additional testing of procurement files was not necessary because DFO’s self-assessment and supporting documentation provided sufficient evidence regarding the actions taken to resolve the issues noted in the initial review.

12. OPO reviewed and assessed the information provided by DFO in its self-assessment for its overall reasonableness and credibility. This report also includes a summary, a report card, and an overall conclusion regarding progress made by DFO in implementing the recommendations made by the Procurement Ombudsman in the initial report.

Management Action Plan Assessment—Summary

13. Using the scale provided by OPO, DFO self-assessed the level of implementation of its actions at a level 5, i.e. “Full Implementation”, for all 6 recommendations in the initial PPR report. However, substantiating documentation provided by DFO did not clearly demonstrate structures and processes being fully implemented for 5 of the 6 recommendations. Therefore, OPO is not in full agreement with DFO’s self-assessment, and assessed DFO’s level of implementation at level 4 (“Substantial implementation”) for 5 of the 6 recommendations.

14. Overall, DFO developed and implemented both new and enhanced products that support fairness, openness, and transparency in the procurement process. The evidence provided by DFO demonstrated a series of advancements designed to meet the intent of each recommendation through a series of initiatives, including: new guides and information resources; updated training programs and checklist tools; a new, standardized electronic file retention methodology through the GCdocs platform; new standardized templates with instructions on how to complete them; and a new oversight guidance tool to assist with quality control and oversight. Best practices such as formal lessons-learned meetings, onboarding programs for new procurement staff and assigning dedicated teams to different client sectors were adopted. DFO has also established a formal Contract Review Committee and is participating in an ongoing procurement streamlining initiative. Some initiatives such as the Contracting Planning and Approval document cover multiple recommendations. Despite these positive advancements, there is an opportunity for DFO to adopt better documentation practices around some of their initiatives and practices. Additional details of OPO’s overall assessment of DFO’s response to each recommendation are included in Appendix 3.

Report Card

15. PPR follow-up reviews include a report card with a rating that depicts the organization’s performance with regard to the 3 LOEs, resulting in 3 assessments. In determining these assessment ratings, OPO takes into consideration the results from the initial PPR, and the actions taken by the organization to implement the PPR recommendations under each LOE. The assessment ratings are as follows:

  • Satisfactory +
  • Satisfactory
  • Partially Satisfactory
  • Unsatisfactory

16. Note that a Satisfactory + assessment rating is only possible when the initial PPR resulted in no recommendations being issued under a particular LOE. As the initial PPR report contained recommendations under all 3 LOEs, a Satisfactory + rating was not possible in this follow-up review, and the highest possible score DFO could obtain for each LOE was Satisfactory. Assessment definitions and criteria are outlined in Appendix 2.

17. After a review of progress made toward implementing the action plans for each recommendation, and in consideration of the established assessment ratings, a rating of Satisfactory is awarded for each LOE. DFO’s descriptions of actions taken, and the supporting documentation provided demonstrate reasonable and credible progress has been achieved in response to all recommendations, although more formal documentation would serve to strengthen DFO’s efforts and help it to be better prepared for the scrutiny of future internal audit activity and to defend against external challenges.

Report Card
Line of Inquiry (LOE) Rating Assessment
1. Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies. Satisfactory
  • Recommendations 1, 2 and 3 applied to this LOE.
  • A series of initiatives and products were developed and communicated addressing this LOE including updated training materials, information resources and a new, standardized electronic file retention methodology through the GCdocs platform to ensure compliance with the intent of the LOE.
  • An oversight function supported by a procurement approval form is in place but, along with other initiatives such as mentorship activities, dedicated teams for clients and specific team lead peer review of oversight activities, it could benefit from a more formalized and documented direction pertaining to specific duties that are taking place. This could include outlining what exactly is being done, how the work is being conducted and where records of activities are stored. This would benefit DFO by preparing the department for the scrutiny of any future audit activity and to defend against external challenges.
2. Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies. Satisfactory
  • Recommendation 4 applied to this LOE.
  • DFO developed and communicated effective measures such as updated training materials, and adopted best practices such as conducting lessons learned meetings to better avoid or address future issues that may impact consistency with the intent of the LOE.
  • As noted in LOE 1, the oversight function taken in tandem with a procurement approval form could benefit from additional formal documentation that establishes and communicates the role of the oversight function and process. This would help demonstrate how the work being conducted better covers the intent of the recommendation and intent of LOE 2.
3. Evaluation of bids and contract award were conducted in accordance with the solicitation. Satisfactory
  • Recommendations 5 and 6 applied to this LOE.
  • DFO developed and communicated a variety of initiatives such as updated training materials, guidelines for evaluators, information resources and supporting tools.
  • The Contract Review Committee serves a high-level oversight function for the Departmental Procurement Plan, as well as ensuring procurement-related policies and procedures are up to date.
  • As noted in LOE 1 and 2, an oversight function supported by a procurement approval form is in place but it could benefit from a more formalized and documented direction pertaining to specific duties that are taking place.

DFO’s overall report card rating is “Satisfactory”. The key factors contributing to this rating include:

  • Initial PPR resulted in one or more recommendations under each of the three LOEs, therefore the highest possible score DFO could obtain for each LOE was Satisfactory;
  • Action plan(s) have achieved level 5 implementation or a combination of level 4 and level 5 that is deemed to be satisfactory, i.e., significant progress has been made to implement action plan(s) that address recommendation(s).
  • DFO's response is supported with some documentation, but work remains to be done with respect to documentation supporting DFO’s oversight function.

Conclusion

18. DFO has implemented new initiatives and enhanced established products that support fairness, openness, and transparency in the procurement process. The evidence provided demonstrates a series of advancements designed to meet the intent of each recommendation through a series of best practice initiatives. These initiatives included development of internal policy instruments such as guides and directives, updated training programs, new manual and automated tools and development of a procurement oversight process, specifically involving a Contract Review Committee that reviews the Departmental Procurement Plan to ensure compliance with policy frameworks and transparency in planning. There is an opportunity for DFO to further develop clear internal policy, procedures and/or guidance documentation that demonstrates support of a formal peer review function and its communication to responsible staff, to fully implement all recommendations stemming from the initial PPR.

Appendix 1

Assessment of Implementation Level
Level Assessment
Level 1 No progress or insignificant progress. Actions such as establishing a new committee, conducting meetings, and generating informal plans have not advanced, or insignificant progress has been made.
Level 2 Planning stage. Formal plans for organizational changes have been created and approved.
Level 3 Preparations for implementation. Preparations for implementing a recommendation are in progress—for example, hiring or training staff, developing necessary resources, drafting and adopting policy documents, etc.
Level 4 Substantial implementation. Structures and processes are in place and integrated within at least most parts of the organization, and some achieved results have been identified.
Level 5 Full implementation. Structures and processes are fully implemented, and positive results have been identified.
Obsolete Recommendation is no longer applicable due to new policies, procedures, etc.

Appendix 2

Overall Performance Assessment Scale
Overall performance Assessment
Satisfactory +
  • Findings from the initial PPR resulted in no recommendations being issued under the line of enquiry
  • A rating of Satisfactory + is based on results from the initial PPR
Satisfactory
  • Initial PPR resulted in one or more recommendations under the line of enquiry
  • Action plan(s) have achieved level 5 implementation or a combination of level 4 and level 5 that is deemed to be satisfactory, i.e., significant progress has been made to implement action plan(s) that address recommendation(s)
  • Organization's response is supported with adequate documentation and/or testing results
Partially satisfactory
  • Initial PPR resulted in one or more recommendations under the line of enquiry
  • Some progress has been made in implementing action plan(s), but not enough to adequately address all recommendations under the line of enquiry, for example one or more recommendations achieved level 3, but work remains to be done
  • Organization's response is supported with some documentation and/or testing results
Unsatisfactory
  • Initial PPR resulted in one or more recommendations under the line of enquiry
  • No or insignificant progress made to implement action plan(s)
  • Inadequate supporting documentation provided or testing does not demonstrate that positive results have been achieved
  • Recommendation(s) not addressed and remain(s) outstanding

Appendix 3

OPO Detailed Assessment Summary of Recommendations
Number Recommendation OPO Assessment Rating
1 DFO should ensure all procurement records are retained, organized and accessible, for the period of time required by Treasury Board record-keeping standards.

DFO has substantially implemented this recommendation through:

  • adherence to policy instruments such as the Record Keeping Directive for Information Management;
  • a comprehensive communications plan; and
  • a formal file structure within the GCdocs system.
While DFO noted that its Contracting Planning and Approval document (CPAA) was intended to serve as an internal tool to help address the recommendation, it does not provide clear links to the goal of procurement record retention to ensure Treasury Board (TB) record keeping standards are being met. Iterating on the document to clearly highlight how it is used to help employees ensure records are retained, organized and accessible per government standards would fully implement the recommendation.		 Level 4
Substantial
Implementation
2 DFO should ensure evaluation criteria are reviewed to eliminate bias and ensure clarity prior to publication.

DFO has substantially implemented the recommendation through adoption of training and client engagement initiatives. Together, these activities help meet the intent of the recommendation by helping employees better understand criteria requirements and how they fit into the overall evaluation process, as well as how to recognize bias and take steps to mitigate the risk of it impacting evaluation results. DFO has done this through:

  • Traditional training formats via courses.
  • Demonstrating best practices through adoption of an onboarding program to assist new staff by linking them with more experienced procurement officers and providing them with links to additional contracting training information via a Procurement Hub resource.
  • Development of internal guidance documentation such as a bid receiving instruction resources.
  • Establishment of a Procurement Hub contracts list with links to call for more information.
  • Adopting the practice of assigning dedicated procurement teams to specific client sectors to build knowledge of needs and enhance relationships. This approach is based on best practice guidelines from the Information Technology Infrastructure Library (ITIL) and is also considered a best practice.

DFO referred to the Contracting Planning and Approval document as their established evaluation process that helps ensure proposed evaluation criteria are aligned with potential procurement instruments. However, there is a lack of clear linkage that demonstrates how exactly it ensures proposed evaluation criteria are aligned with potential procurement instruments. DFO would benefit from reviewing the CPAA to provide an indication as to how assurance is demonstrated that evaluation criteria are reviewed to eliminate bias and ensure clarity prior to publication. Iterating on the document to clearly highlight how it is used to ensure evaluation criteria are reviewed to eliminate bias and ensure clarity prior to publication would help to fully implement the recommendation.

 Level 4
Substantial
Implementation
3 DFO should ensure clarity and consistency in its solicitation documents to avoid discrepancies and ensure transparency in its bidding processes.

DFO has substantially implemented the recommendation through adherence to guidance documentation, active monitoring and oversight and information sharing among employees. These activities serve to meet these goals in a variety of ways:

  • The Contract Planning and Approval document has been developed to review and/or approve proposed procurement strategies prior to posting solicitation documents and contract award. This allows for discrepancies to be addressed early and serves as a peer review function. However, to fully address the recommendation, DFO should clearly indicate how the subjects of clarity and consistency are covered in its solicitation documents through clear instructions based on guidance documents. Additional direction or guidance should be included in the CPAA to ensure discrepancies in DFO’s bidding processes are avoided and full transparency is achieved. In addition, DFO’s mandatory peer review process lacks internal guidance/communication of the peer review process requirement and what it entails.
  • DFO shares information and lessons learned and to better deal with similar issues that may arise in the future. However, while a senior official is identified as being responsible for monitoring and updating templates or documents per changes required by PSPC/TBS, no evidence of their participation in any of the boards they reportedly get these updates from were provided, nor was there any evidence of them disseminating any updates. These issues would need to be addressed to consider the recommendation as being fully met.
 Level 4
Substantial
Implementation
4 DFO should properly document its files to demonstrate how its contracting personnel are giving all qualified firms an equal opportunity for access to government business, and how its contracting personnel are decreasing any possible bias or incumbent advantage.

DFO has substantially implemented the recommendation through:

  • Use of an automated Contract Monitoring Report system that identifies Buyers and Suppliers, as well as other procurement information. This data is scanned for any trends that may hint at contracting personnel favoring certain companies. Final approval to proceed with contract award comes after this report is reviewed to ensure no preference is given to one individual supplier.
  • High-visibility files are reviewed by Senior Management (Director and Director General).
  • Procurement strategy training for clients.
  • Use of standard forms to demonstrate that conditions are laid out in the solicitation phase and that information which may provide an unfair advantage must be disclosed. These activities help to meet the intent of the recommendation by increasing employees’ understanding of their roles and responsibilities around the process of documenting contracting engagements and helping them understand how to provide equal opportunity to bidders for government business while guarding against bias or incumbent advantage. Automated systems help with reporting on trends to guard against favouritism and formal committees use of standardized documentation provide an effective monitoring and oversight function.
  • DFO maintains that they are documenting files to demonstrate how contracting personnel are giving qualified firms an equal opportunity for access to government business and decreasing possible bias through use of a peer review process undertaken in tandem with completion of the Contracting Planning and Approval document. However, there is a lack of supporting documentation of this peer review process and what it entails. The CPAA could benefit from guidance documents. These guidance documents could provide clear instructions on matters to be reviewed and the overall oversight process in more detail. This would provide a degree of assurance that discussions are relevant to the themes of each recommendation to ensure they are being considered and fully meet the intent of the recommendation.
 Level 4
Substantial
Implementation
5 DFO should ensure that a standard “evaluation package” including a Confidentiality form and Conflict of Interest form are signed by all evaluators and recorded on file prior to providing evaluators access to bids.

DFO has fully implemented their action plan to address this recommendation. Their activities include:

  • Dissemination of information among staff, and training and targeted engagements with client groups. Specifically, DFO releases evaluation documentation via email to evaluators with instructions on how to conduct the evaluation work and an outline of roles and responsibilities.
  • The Procurement 101 training package also covers the evaluation process extensively.
  • Prior to starting the evaluation, an evaluation team meeting is chaired by the Contracting Authority (CA). The CA will review the evaluation guidelines to ensure a clear understanding of roles and responsibilities and prior to submitting the technical bids to the Project Authority (PA), the CA must ensure that all participants have signed and returned the Conflict of Interest and Non-Disclosure Agreement.
  • Additional training such as a series of procurement courses, some that focus on bid evaluation, are also offered. Several of these courses are mandatory for functional specialists.
  • Establishment of an onboarding program offered by the Chief Financial Officer (CFO), as well as procurement training for new staff.
  • Assigning dedicated procurement teams to client sectors to build knowledge of needs and enhance relationships is an internal initiative communicated throughout the department and posted on the Procurement Services Intranet site. It is based on best practice guidelines from the Information Technology Infrastructure Library (ITIL). Evidence was provided that showed the new model being communicated to staff, contact names and an email discussing the rationale for adopting this approach.
 Level 5
Full Implementation
6 DFO should put a mechanism in place to ensure evaluation processes are adhered to and contracts are awarded to the appropriate, qualified bidder.

DFO has substantially implemented the recommendation through use of formal oversight bodies, peer review and a procurement streamlining initiative. These activities help ensure evaluation processes are adhered to and that contracts are awarded to the appropriate bidder by providing formal monitoring, oversight and direction to employees to ensure processes are followed throughout the span of the engagement. For example:

  • DFO’s Contract Review Committee maintains several oversight responsibilities related to procurement within the departmental procurement plan. The Committee also approves requests for revisions to policies, guidelines and templates related to procurement. It monitors procurement reports to address issues and promote efficiency and also provides quarterly reports to the Finance Committee. This demonstrates DFO’s intent to ensure processes are adhered to and helps meet the intent of the recommendation. Evaluation guidance documents are provided to evaluators such as overall instructions to evaluation teams, individual instructions and consensus reporting requirements.
  • DFO claims to conduct peer review via discussions held in tandem with the Contracting Planning and Approval document. However, DFO did not provide evidence that would support a formal peer review function or its communication to procurement officers. This would be required to consider the intent of the recommendation to be fully met.
  • As noted in recommendation 4, an automated Contract Monitoring Report system is in use that identifies Buyers and Suppliers, as well as other procurement information. This data is scanned for any trends that may hint at contracting personnel favoring certain companies. Final approval to proceed with contract award comes after this report is reviewed to ensure no preference is given to one individual supplier.
  • DFO is involved with a Streamlining Procurement initiative. The focus is to introduce their commodity risk-based approach model. A high-level summary of recommended automation opportunities was noted that could drive self-service for end users. For example, one automated tool under development could assist with evaluating bids at the point they are submitted to accelerate evaluation time. Any type of automated feature is going to have an optimal chance at ensuring evaluation processes are adhered to and efficiencies are welcome in the process, but it remains unclear how it would determine if contracts are awarded to the appropriate, qualified bidder. The process is still ongoing.
 Level 4
Substantial
Implementation
Date modified: