Procurement Practices Review 2009-2010 - Chapter 4: Environment Canada – Review of Procurement Practices Related to Management Consulting and Other Professional Services
Table of Contents
Executive Summary
4.1 In the spring of 2007, a supplier made a formal complaint to the Canadian International Trade Tribunal (CITT) alleging that Environment Canada (EC) had showed favouritism toward another company in the award of a contract. The CITT reviewed the case and recommended that EC re-evaluate the complainant's submission. The proposal was re-evaluated with the same outcome as the initial evaluation. The supplier then complained to Justice Canada. Justice Canada responded that the CITT had reviewed the case and provided a judgment.
4.2 The complainant further complained to the Information Commissioner and the Office of the Auditor General of Canada. In December 2008, the supplier brought the complaint to the Office of the Procurement Ombudsman (OPO). After some preliminary discussions, the OPO decided to carry out a review of EC's procurement practices to ensure they supported the principles of fairness, openness and transparency. The OPO and Environment Canada agreed that a review would be carried out using a collaborative approach, and EC hired an external audit firm to conduct a procurement audit.
4.3 The scope of the audit included all phases of the procurement process specific to "Management Consulting and Other Professional Services." The scope included procurement planning (e.g. requirements definition), solicitation activities and contract award, contract administration and contract close-out. It was agreed that the objectives of the review would be to:
- assess whether the allegations of favouritism were founded;
- review the procurement process specific to "Management Consulting and Other Professional Services" to determine whether Environment Canada complied with the Treasury Board Contracting Policy and the Government Contracting Regulations; and
- determine whether the procurement business function was based on sound contracting management practices and is fair, open and transparent.
4.4 OPO staff reviewed the objectives, scope and methodology of the audit and was satisfied with the work that was performed by the external audit firm.
4.5 The external auditors have completed their examination and concluded that EC has an adequate management control framework appropriate to the needs of the Department and that overall procurement activities comply with the policies of the Treasury Board, Public Works and Government Services Canada and its own department with respect to contracting for management consulting and other professional services.
4.6 The external auditors also concluded there was no evidence that EC had shown favouritism toward any firms in the proposal evaluation and contract award process during the period examined.
4.7 The audit report noted the following:
- There were six of the fifty files in the statistical sample and four of the seven files in the judgmental sample where the evaluation documentation was incomplete. As a result, the Department may be at risk of being challenged on its decisions;
- While EC does not have a contracting policy, it follows the Public Works and Government Services Canada (PWGSC) Supply Manual. According to the Supply Manual, "All notes taken during the evaluation must be kept in their original form and retained on the procurement file for audit purposes." While not mandatory for departments and agencies, this is clearly a best practice to be emulated. The Supply Manual also states that "evaluators' worksheets are an integral part of the evaluation process and constitute part of the complete record regarding the procurement and part of the written record of all communications substantially affecting the procurement within the meaning of the international trade agreements".
- As a result, in OPO's view, without properly documented forms and complete reports, EC Procurement cannot demonstrate compliance with either the Treasury Board Contracting Policy or PWGSC policy requirements relating to the documentation of files. If there are instances where decisions are not adequately documented, this may give the perception that the procurement practices in the Department are not being conducted in a fair, open and transparent manner.
4.8 The OPO has used the audit findings as the basis for this report. The OPO has also reviewed the auditor's working papers and is satisfied with the results.
4.9 The OPO concurs with the findings and the recommendations of the audit. The OPO has concerns regarding the lack of documentation in the evaluation process and in the process used to engage the co-chairs of the Committee on the Status of Endangered Wildlife in Canada (COSEWIC) at Environment Canada.
4.10 It is recommended that the Department should:
- revise the standard instruction letter that is provided to evaluators to clearly outline what minimum standards of documentation are required. The minimum standards should also be revised on the Web site instructions, and the Director should ensure that these minimum standards are followed;
- ensure that all amendments are properly justified and processed prior to contract expiry dates; and
- obtain from Public Works and Government Services Canada and the Treasury Board of Canada a resolution on how to process transactions similar to those related to COSEWIC in light of legislative obligations.
Departmental Response
Environment Canada concurs with the recommendations contained in this report and the work that was performed by the independent audit firm. It will be preparing a management action plan, which will be presented to its independent External Audit Advisory Committee in June 2010.
Introduction
Context
4.11 The need for this review originated in complaints made by a supplier to the Office of the Procurement Ombudsman (OPO) in December 2008.
4.12 Prior to the OPO's receipt of this complaint, in the spring of 2007, the supplier (firm A) made a formal complaint to the Canadian International Trade Tribunal (CITT) alleging that Environment Canada showed favouritism toward another supplier (firm B) in the award of a contract. CITT reviewed the case and recommended that Environment Canada re-evaluate the complainant's submission. As a result, the proposal was re-evaluated, with the same outcome as the initial evaluation. The supplier then complained to Justice Canada, who responded that the CITT had reviewed the case and provided its judgment.
4.13 The supplier (firm A) then filed Access to Information and Privacy (ATIP) requests related to information specific to the other supplier (firm B). Given the amount of research and documentation required to complete the requests, it took a long time for the Department to respond to all the requests. As a result, the complainant made another complaint to the Information Commissioner and the Office of the Auditor General.
4.14 The allegations deal mainly with the process for awarding competitive contracts over $150K and the access to information process. It was also alleged that Environment Canada was favouring one particular firm in the awarding of contracts.
4.15 The Procurement Ombudsman Regulations provide the Ombudsman with authority to review the procurement practices of departments in order to assess their fairness, openness and transparency and make recommendations for the improvement of those practices.
4.16 After some preliminary discussions with the Department, in May 2009, the OPO advised the Department that a practice review would look at the procurement practices at EC specific to the allegations to determine whether systemic problems exist. It was agreed between the OPO and EC that EC would carry out an audit using a collaborative approach. It was also determined that the Internal Audit Division of EC would commission an external firm to conduct the audit.
4.17 It was agreed that following the audit, the OPO would use the contents of the audit as the basis for the results of this review.
Focus of the Review
Review Objectives
4.18 The three objectives of this work were to
- assess whether the allegations of favouritism were founded;
- review the procurement process specific to "Management Consulting and Other Professional Services" to determine whether Environment Canada complied with the Treasury Board Contracting Policy and the Government Contracting Regulations; and
- determine whether the procurement business function was based on sound contracting management practices and is fair, open and transparent.
Scope and Period of Review
4.19 The scope included all phases of the procurement process specific to "Management Consulting and Other Professional Services" including procurement planning (e.g. requirements definition), solicitation activities, contract award, contract administration, contract close-out and evaluation criteria. The majority of the contracts that were received by firm B were in the category "Management Consulting and Other Professional Services." Hence we focussed the review on this category of expenditures.
4.20 More specifically, the scope included contracts greater than $25,000 issued by EC or by Public Works and Government Services Canada (PWGSC) on behalf of EC from April 1, 2008, to December 11, 2009 (20 months).
4.21 The audit scope also included contracts referring to the firm subject to allegations of favouritism from April 1, 2006, to December 11, 2009 (44 months). The allegations related to favouritism toward firm B regarding both competitive and non-competitive contracts in the past 10 years. However, we limited our scope of this review to the last 44 months as we did not believe it would have been useful to extend the review prior to 2006. This review work involved looking at seven of the eight contracts obtained through the competitive process and eight of the eighteen obtained through the non-competitive process during the period under review.
Review Methodology
4.22 The audit that formed the basis for this review was conducted by an independent firm in accordance with Institute of Internal Auditors Standards and the Treasury Board Policy on Internal Audit. The audit involved an examination of background information, interviews and transaction testing.
4.23 The OPO has used the audit findings as the basis for this review. The OPO has also reviewed the auditor's working papers and is satisfied with the results.
Data Analysis
4.24 Between fiscal years 2006-07 and 2009-10, Environment Canada and PWGSC on behalf of EC issued 46,003 contracts, 58.6% and 41.4% respectively, for a total value of $625,791,347. For the statistical sample, our efforts concentrated on contracts in the "Management Consulting and Other Professional Services" category for the period April 1, 2008, to December 11, 2009. There were 1,337 contracts in the Management Consulting and Other Professional Services category for this period. Of these, 168 were over $25K. The statistical sample was selected from this population as the review was testing the application of the evaluation criteria.
4.25 This sample consisted of 50 contract files for a value of $4,700,122. The sample was based on a random statistical strategy with a 90% confidence level, a ±10% precision level and a 25% coefficient of expected population deviations. The sample includes 9 transactions greater than $100,000 and 41 transactions less than $99,000 but greater than $25,000. The following table illustrates population and selection.
The following table illustrates population and selection.
| Transactions | Population April 1, 2008, to Dec. 11, 2009 (20 months) |
Sample April 1, 2008, to Dec. 11, 2009 (20 months) |
|---|---|---|
| Greater than $100K | 31 | 9 |
| Less than $99K but greater than $25K | 137 | 41 |
| Total value of contracts where the sample was extracted | $13,603,619 | $4,700,122 |
| Less than $25K | 1,169 | - |
| Total | 1,337 | - |
4.26 In addition to the statistical sample that the auditors used to test the overall procurement practices in the Department, they also selected a judgmental sample of contracts awarded to firm B in response to allegations suggesting that EC had given firm B preferential treatment in the evaluation of proposals and subsequent award of contracts.
4.27 As documented in the Environment Canada working papers pertaining to an analysis of contracting information concluded in June of 2009, the firm in question was a large firm in terms of contract value awarded by EC between 2005-06 and 2008-09 (22nd largest firm in terms of contract value over the period under review, for a total of $766,000). It was the seventh largest in terms of number of contracts, with 32. Most of these (27) were in the "Management Consulting and Other Professional Services" category.
4.28 Fifteen files that were awarded during the period of this review (April 2006 to December 2010) were selected for review on a judgmental basis (seven of the eight contracts files greater than $25,000 and eight of the nineteen contracts less than $25,000) for the judgmental sample. The contract for $163,000, which led to the CITT review, was also included in this judgmental sample. The seven files were tested for compliance with the overall procurement procedures, including a review of the evaluation criteria used. The eight files that were awarded on a sole source basis were reviewed for sole source justification.
Review Observations
Fairness and Transparency in Allegations Related to Favouritism
4.29 We reviewed seven of the eight contracts that were awarded to firm B for "Management Consulting and Other Professional Services" through the competitive process. Among these, there was a wide range of services provided to EC. Services ranged from the generic—facilitation and recording of workshops, program evaluations (two occurrences)—to management consulting, Governance Options for Programs and Guidance Documents for Project Management and to the more technical regulatory issues. All were awarded competitively through MERX™ (the Government's Electronic Tendering System) and subject to a documented bid evaluation process.
4.30 The fact that the firm in question was involved in such a wide range of services meant that of the seven contracts looked at, no two contracts, or bids, were managed or evaluated by any of the same individuals. Furthermore, one evaluation was an interdepartmental undertaking and there were evaluators from three departments involved. Hence, there is less of an opportunity for one firm to be favoured if there are different managers and different evaluation teams.
4.31 The audit's analysis of the specific criteria used in the evaluation of bids for these seven contracts awarded through the competitive process to the firm in question (including the re-evaluation related to the complaint that gave rise to this review) noted no apparent limiting or preferential factors neither in evaluation criteria nor in the statements of work. This provides further evidence that there is no favouritism.
4.32 From the available data, the audit was unable to determine how many bids the firm in question submitted where it was unsuccessful. Therefore, the auditors judgmentally selected five contracts awarded through a competitive process to one of the firm's known competitors. Of the five files, there was one for $116,000 where firm B had submitted a proposal and while it did qualify, it was not awarded the contract based on price per point selection methodology.
4.33 With respect to the eight of the nineteen directed contracts under $25,000 that we reviewed, the review revealed that the justification to support the decisions and to demonstrate compliance were reasonable in all cases. This, combined with the fact that these sole source contracts were not contracted by the same responsibility centre manager, provides evidence that there was no favouritism shown toward firm B.
4.34 In summary, the audit concluded that there is no evidence to support the suggestion that EC has shown favouritism to any firm in the proposal review and contract award process.
Management Control Framework
4.35 EC has an adequate Management Control Framework in place, appropriate to the needs of the Department, for the contracting for management consulting and other professional services.
4.36 EC has several areas of strength in its contracting process:
- EC has a rigorous and effective Contract Review Board (CRB) that acts as a key challenge mechanism ensuring that government contacting policy is adhered to;
- Statements of work and associated evaluation criteria were well articulated; and
- General advice and guidance to managers are readily available on the Department Intranet for most aspects of the contracting process.
4.37 However, notwithstanding the above strengths, the audit noted some matters that need to be addressed by management in order to improve procurement practices and ensure they are conducted in a fair, open and transparent manner.
Compliance with Policies
4.38 Overall, procurement activities comply with the Treasury Board Contracting Policy and PWGSC and departmental policies. However, some improvements should be made to more clearly demonstrate that the Department is complying with government contracting policies.
Evaluation process
4.39 Overall, the evaluation processes and criteria were appropriate to the dollar values involved.
4.40 However, the audit noted that there were contracts where the evaluation documentation was incomplete. As a result, the Department may be at risk of being challenged on its decisions.
4.41 In the 50 statistical samples, there were six instances where there was no evidence that EC "Guidelines for Evaluating Proposals" had been adhered to. Some individual evaluation forms were not signed or summary/group assessment forms were not signed by all evaluation committee members contrary to EC policy instructions to evaluators.
4.42 In the judgmental samples, four of seven files reviewed either did not provide comments on how points were gained or lost in individual evaluation forms or did not document how consensus was arrived at in the group evaluation summary report.
4.43 As EC does not have its own detailed contracting policy, we have been informed that it follows the PWGSC Supply Manual in instances where its internal policies are not specific. However, EC is required to adhere to the Treasury Board Policy on Contracting.
4.44 As per the PWGSC Supply Manual, "all notes taken during the evaluation must be kept in their original form and retained on the procurement file for audit purposes." While not mandatory for departments and agencies, this is clearly a best practice to be emulated.
4.45 To quote the Supply Manual further, "Following a relevant Canadian International Trade Tribunal decision, it was found that evaluators' worksheets are an integral part of the evaluation process and constitute part of the complete record regarding the procurement and part of the written record of all communications substantially affecting the procurement within the meaning of the international trade agreements."
4.46 To quote the TB Contracting Policy on documentation, "Procurement files shall be established and structured to facilitate management oversight with a complete audit trail that contains contracting details related to relevant communications and decisions including the identification of involved officials and contracting approval authorities."
4.47 For these reasons, evaluators must ensure that comments are clear and reasons for deducting or assigning points are clearly documented.
4.48 The standard letter forwarded to proposal evaluators by EC procurement officers provides guidance but does not clearly state that all summary evaluation forms must be signed by all evaluators as stated in the guidelines provided to managers on their Web site. Nor does this letter provide guidance on the level of detail required to support eventual decisions and contract award recommendations.
4.49 As a result, in OPO's view, without properly documented forms and complete reports, EC Procurement cannot demonstrate compliance with either the Treasury Board Contracting Policy or PWGSC policy requirements relating to the documentation of files. If there are instances where decisions are not adequately documented, this may give the perception that the procurement practices in the Department are not being conducted in a fair, open and transparent manner.
Recommendation 1
4.50 The Department should revise the letter to evaluators, as well as the Web site instructions and the "Guidelines for Evaluating Proposals," to clearly outline what minimum standards of documentation are required and ensure that these are adhered to. Without proper documentation, it is difficult for management to assess whether decisions have been taken to provide a level of assurance of the effectiveness and objectivity of these decisions. Therefore, proper documentation is critical in demonstrating that the procurement practices are being carried out in a fair, open and transparent manner.
Amendment process
4.51 Within the statistical sample of 50 contracts that we reviewed, in the processing amendments, there was no supporting documentation or explanation for the significant increase (from $36,000 to $72,000) in the cost in one case out of twenty. In one other instance, the amendment was issued even though the contract had expired. These kinds of situations, when not properly documented, may be perceived as unfair, limiting access to other potential suppliers and not being transparent.
Recommendation 2
4.52 The Director General, Assets, Contracting and Environmental Management, should ensure that all amendments are properly justified and processed prior to contract expiry dates.
Other observations – Committee on the Status of Endangered Wildlife in Canada
4.53 The statistical sample included a contract related to the Committee on the Status of Endangered Wildlife in Canada (COSEWIC). Based on the concerns raised with the establishment of this committee, it was felt necessary to bring this to the attention of EC senior management.
4.54 COSEWIC is a body of qualified experts from jurisdictional (provincial, territorial and federal) agencies and non-government organizations, independent scientists, Aboriginal traditional knowledge specialists and members of the academic community. On June 5, 2003, the Species at Risk Act (SARA) was proclaimed. SARA established COSEWIC as an advisory body, ensuring that wildlife species would continue to be assessed under a rigorous and independent scientific process. Section 20 of the legislation also requires the Minister to "provide COSEWIC with any professional, technical, secretarial, clerical and other assistance, and any facilities and supplies, that, in his or her opinion, are necessary to carry out its functions."
4.55 Public notices are posted to the COSEWIC Web site calling for submissions for the various co-chair positions. According to the site, "Submissions will be reviewed by a selection committee of COSEWIC, who will forward the statements of qualifications to COSEWIC and make a recommendation. COSEWIC will, in turn, make a recommendation and forward it to the Canadian Endangered Species Conservation Council (CESCC) and the Minister of the Environment. The successful applicant will be appointed by the Minister of the Environment."
4.56 Compliant with the legislation, EC then issues multi-year contracts to the co-chairs of the various committees. These contracts are in the range of $30,000 annually for a four-year total of approximately $120,000. Since April 1, 2006, 20 contracts have been issued under COSEWIC.
4.57 This procurement process does not currently meet the requirements of the TB Contracting Policy, which requires departments to use MERX™ as the official site for submissions, nor does it meet any of the four criteria for directed contracting.
4.58 Once we brought this to the attention of the Department, it started to take action. In fact, we have been informed that at the time of writing, the Internal Audit group at Environment Canada was consulting with its legal group to determine the most appropriate mechanism for engaging the members of COSEWIC.
Recommendation 3
4.59 The Director General, Assets, Contracting and Environmental Management, in collaboration with Legal, should obtain from Public Works and Government Services Canada and the Treasury Board of Canada a resolution on how to process transactions similar to those related to the COSEWIC transactions in light of legislative obligations.
Conclusion
4.60 The Office of the Procurement Ombudsman concurs with the findings and the recommendations of the audit performed by the independent audit firm.
4.61 There is no evidence that EC has shown favouritism toward any firm in the proposal review and contract award process.
4.62 Overall, the procurement process specific to "Management Consulting and Other Professional Services" within Environment Canada, with the exception of the issues mentioned earlier, complies with Treasury Board, PWGSC and departmental policies regarding goods and services. The procurement business function for those services is based on sound contracting management practices. Specifically, EC has implemented a strong governance regime with the Contract Review Board process, and there are adequate internal controls as demonstrated by the low number of material instances of non-compliance with the Policy and/or Regulations.
4.63 All phases of the procurement process specific to "Management Consulting and Other Professional Services," including procurement planning (e.g. requirements definition), solicitation activities and contract award, contract administration (including amendments) and contract close-out, were complete, appropriate to the dollar values involved and generally well documented. However, improvements should be made in ensuring better documentation is in place to document the evaluation process, which would clearly demonstrate that the principles of fairness, openness and transparency are being adhered to.
4.64 The statements of work, evaluation criteria and eventual proposal assessments demonstrated that standards of fairness and transparency were adhered to.
4.65 There remains some risk with regard to the documentation of proposal evaluations and the process used to engage co-chairs of COSEWIC committees.
Departmental Response
Environment Canada concurs with the recommendations contained in this report and the work that was performed by the independent audit firm. It will be preparing a management action plan, which will be presented to its independent External Audit Advisory Committee in June 2010.
- Date modified: