Procurement Practices Review - Chapter 1: Procurement Challenge and Oversight Function
Table of Contents
- Executive Summary
- Background
- Focus of the Review
- Findings
- General Findings
- Criterion a: Senior Procurement Review Committee Terms of Reference
- Criterion b: Committee Membership
- Criterion c: Appropriate Information for Making Decisions
- Criterion d: Senior Procurement Review Committee Effectiveness (Monitoring)
- Departmental Committees Being Strengthened or in Development
- Conclusion
- Recommendations
- Effective Practices
- Glossary of Acronyms
Executive Summary
According to the 2007 Purchasing Activity Report, departments and agencies of the Government of Canada spent $14,257,457,000 on 339,401 contract awards for goods, services and construction needed to deliver programs to Canadians.
The Treasury Board (TB) Contracting Policy clearly states that departments and agencies, unless specifically excluded by an Order in Council, are responsible for ensuring that adequate control frameworks for due diligence and effective stewardship of public funds are in place and working. More specifically, the Policy encourages contracting authorities to establish and maintain a formal challenge mechanism for all contractual proposals and recognizes that this mechanism could range from a formal central review board to divisional or regional advisory groups, depending on the departmental organization and magnitude of contracting.
The procurement challenge and oversight function is a key component of the broader set of management controls that are used to ensure the sound management of government procurement. In many departments, the principles of fairness, openness and transparency in procurement are safeguarded through oversight, review and monitoring by a senior procurement review committee. Depending on the mandate given to this committee, it can play a role in ensuring that all departmental actions in respect of the procurement process, including selection of the procurement strategy (e.g. use of Advance Contract Award Notices (ACANs)), evaluation criteria, contractual disputes, supplier debriefing and vendor performance, are carried out in accordance with policy and legal requirements.
There are two main reasons why having an effective procurement challenge and oversight committee function is important. First, the committee has a role in assessing corporate risks, which includes ensuring that all procurement activity is compliant with the relevant laws, regulations, trade agreements and policies and fulfilling the government's commitment to fairness, openness and transparency in procurement. Second, for all contract spending from a financial perspective, the committee should ensure that the requirement is justified and represents good value for money on behalf of all Canadian citizens.
The objective of our review was to examine departmental practices related to the committee responsible for the procurement challenge and oversight function at the senior departmental level. Through our review, we also wanted to identify effective practices that could be shared among government departments.
Nine departments and agencies were selected for this review. Eight of these organizations are governed by the aforementioned TB policy requirements relevant to procurement and contracting. Canada Revenue Agency has unique authorities in its enabling legislation and therefore is not governed by these policies.
We focused our review on the organization of and processes used by the most senior committee responsible for the procurement challenge and oversight function within each department. We conducted interviews with senior departmental officials and examined supporting documentation such as the committee terms of reference and sample submissions reviewed by them for the period from April 1, 2007, to March 31, 2008.
The nine departments and agencies included in our review carry out the challenge and oversight function by means of a senior departmental committee or board in combination with other procurement controls and committees. We found the roles of these committees as well as their stage of development varied, and the way they conducted their business differed considerably. There are some essential characteristics that the Office of the Procurement Ombudsman (OPO) recommends be considered in the creation and operation of these departmental committees. These are set out later in this Executive Summary.
Our findings indicate that Agriculture and Agri-Food Canada (AAFC), Environment Canada (EC) and the Canada Revenue Agency (CRA) had well-established senior review committees governing the procurement challenge and oversight function. Further, they demonstrated the use of performance measures to assess their effectiveness and continuing efforts to improve the function within their respective organizations.
The Canadian Institutes of Health Research (CIHR), a much smaller agency, demonstrated a sound procurement management control framework, yet used an entirely different model appropriate for an agency of its size. In this model, the senior procurement review committee is responsible for providing direction and support for the development of CIHR's procurement framework. The review of procurement submissions at a transactional level is completed by the Manager of Procurement and procurement personnel, with transactions over $1 million reviewed by the Chief Financial Officer.
Canadian International Development Agency (CIDA), Industry Canada (IC) and Public Works and Government Services Canada (PWGSC) were operating senior procurement review committees consistent with their existing mandates. Both IC and PWGSC were actively pursuing improvements to strengthen their senior review committees. PWGSC and CIDA have stated that they intend to use the results of the Office of the Procurement Ombudsman review to address areas requiring improvement and incorporate identified "best practices" into their procurement management and oversight frameworks.
Finally, the Department of Justice (DOJ) and Public Safety and Emergency Preparedness Canada (PSEPC), both at the early stages of establishing senior procurement review committees, appear to be on track to have a strong central procurement challenge and oversight function. At the time of our review, both had recently developed terms of reference for their review committees. DOJ has advised that shortly after the completion of our field work their senior review committee became fully operational. PSEPC's senior committee was to be fully operational in May 2009 upon completion of training of its members.
All nine organizations reviewed have established terms of reference for their senior procurement review committees. Of the eight committees that have responsibility for conducting individual procurement submission reviews, six conduct their reviews at the procurement planning stage. One of the committees conducts its reviews at various procurement stages from planning to pre-contract award, depending on potential risks, as determined by the Director of Contracting. One committee conducts its reviews atthe pre-award stage. We consider the completion of reviews at the planning stage to be an effective practice that can help departments reduce procurement risks before publishing solicitation documents that reflect the government's intentions.
Our review revealed that the criteria for submitting procurements for review varies depending on the individual department's risk profile and the existence of other controls such as investment review committees, compliance control functions and internal audits. Common examples of review criteria include dollar thresholds, types of procurement (e.g. goods versus services, competitive versus sole source), risk factors such as changes in scope, potential for disputes and contract ratifications. It was noted that some departments defined their review criteria using terms such as "significant plans" or "significant ACANs." We believe that such criteria are unclear and should be supported by an explanation of the relevant risk factors.
We determined from our review that generally the committee membership comprised senior management and was multidisciplinary, with senior financial and legal representatives participating as regular members of most committees. By having such very senior departmental personnel on the committee, the departments ensure that procurement submissions undergo the type of scrutiny that only senior management personnel with experience and a department-wide perspective can bring.
Based on a detailed examination of information submitted to the senior procurement review committees in six of the nine departments, we concluded that, as a rule, the committees were provided with appropriate information for decision making. The six organizations require that a template/checklist be completed by the submitting branch or directorate to ensure the procurement submissions tabled for review address key departmental risk issues and are recommended by the submitting directorate or branch.
We did note specific procurement issues that can benefit from further attention by these committees to assist in the mitigation of procurement risks. From our review of sample procurement submissions, we observed that information on past vendor performance was not provided to the committee. In addition, in our review of committee meeting minutes and records of decisions, we did not observe records of discussions of past vendor performance during contract periods.
The terms of reference for CRA's senior procurement review committee states that the committee reviews reports from previous procurements to analyse vendor performance.
We also noted, that in cases where poor vendor performance has been confirmed, PWGSC's Vendor Performance Policy (currently under review) calls for reasonable measures to be taken to prevent future problems. The Policy further stipulates that bids received from vendors that are debarred or suspended from doing business due to poor performance will not be considered for evaluation. In our opinion, the senior procurement review committee should be provided with assurances that the terms of this or any similar vendor performance policies are being implemented. More specifically, the committee should be provided with assurances that procurement solicitation documents and evaluation procedures will ensure that the terms of any restrictions or conditions imposed on a vendor as a result of poor performance are being complied with for all suppliers responding to the solicitation.
Further, from the samples reviewed, only one of the procurement submissions that involved more than one department was duly signed off by the departments concerned. We believe that where multiple departments are involved in the procurement, it is important to consider whether the proposed procurement actions are supported by all departments involved.
We also observed that while most committees require that they review procurement submissions where the proposed procurement process includes the use of an Advance Contract Award Notice, some do not. We believe that such procurements pose a special risk and all departments should establish risk indicators based on materiality and complexity, and require that all procurements meeting the risk profile, especially those that use ACANs, be reviewed by the committee.
In conclusion, we are generally satisfied with the progress made to establish effective procurement oversight committees. Through their membership and activities, these committees are working to ensure the openness, fairness and transparency of the procurement system and thereby strengthening the confidence of Canadians in government procurement.
The following practices are viewed as effective means of increasing the confidence of Canadians in procurement by improving oversight in departments. OPO recommends that in the creation and operation of these committees certain essential characteristics be prevalent:
- Committees should have comprehensive and objective terms of reference.
- Committees should include members who are multidisciplinary and who understand the procurement process and have an appreciation of the risks involved.
- Departments should establish risk indicators based on materiality and complexity, and require that all procurements meeting the risk profile, especially those that use ACANs, be reviewed by the committee.
- Committees should conduct their reviews at the outset of the procurement process (planning stage).
- Information submitted to these committees should be sufficient so as to ensure sound and effective decision making.
- Procurement submissions involving more than one department should be duly signed off by the departments concerned.
- Committees should be provided with assurances that procurement solicitation documents and evaluation procedures will ensure that the terms of any restrictions or conditions imposed on a vendor as a result of poor performance are being complied with for all suppliers responding to the solicitation.
- Committees should have the means to ensure they are receiving all procurement submissions included in their mandate.
- Committees should monitor the results of their decisions.
- Committees should have the means to judge whether or not they are operating effectively.
Our review also gave us an opportunity to observe additional effective practices, which departments may find helpful in strengthening their own oversight function. Departments listed in parenthesis are those where we noted these practices:
- Updating the terms of reference on a regular basis ensures that information is always current. (AAFC, CIHR, CRA, EC, IC, and PWGSC)
- The use of a procurement checklist/template ensures that submissions presented for review address key departmental risks. (AAFC, CIDA, CRA, EC, IC and PWGSC)
- A series of supplementary questions, such as those used by Agriculture and Agri-Food Canada, are useful to ensure that the submission is comprehensive and that officials presenting submissions have turned their minds to all considerations. These questions are in addition to what is provided in the procurement submission template.
- A streamlined review process for low risk procurement submissions is very important and incorporates sound risk management processes and appropriate use of resources. (CRA, EC, and, IC)
- The committee is supported through a computerized system that provides a tracking function to determine the status of the decisions it makes. (AAFC, CRA, and EC)
- The committee has a means to "flag" contracts coming up for renewal to ensure renewal of contracts is not automatic, and options are exercised with due diligence. (AAFC, CIDA, CRA, and EC)
- The committee measures whether through its actions there are improvements or deterioration in the procurement activity of the department or agency. (AAFC, CRA, and EC)
- The committee tracks the stage in the procurement review process where the procurement submission is, to apprise clients of the status of their requirements. (CRA)
All departments and agencies involved in this review have been provided an opportunity to review this report and their comments have been taken into consideration in finalizing this chapter.
Background
Context
1.1 According to the 2007 Purchasing Activity Report, departments and agencies of the Government of Canada spent $14,257,457,000 on 339,401 contract awards, including net amendments for goods, services and construction, and excluding call-ups against standing offers. Apart from payroll, grants, contributions, transfer payments and statutory votes, most public funds are spent through the contracting process. The expenditures for goods, services and construction projects are intended to support the programs designed to benefit Canadian citizens.
1.2 In formulating these contracts, unless specifically excluded by an Order in Council, departments must comply with the laws and policies established by the Government of Canada. The Treasury Board (TB) Contracting Policy clearly states that departments and agencies are responsible for ensuring that adequate control frameworks for due diligence and effective stewardship of public funds are in place and working. More specifically, the Policy encourages contracting authorities to establish and maintain a formal challenge mechanism for all contractual proposals and recognizes that this mechanism could range from a formal central review board to divisional or regional advisory groups, depending on the departmental organization and magnitude of contracting.
1.3 The procurement challenge and oversight function is a key component of the broader set of management controls that are used to ensure the sound management of government procurement. In many departments, the principles of fairness, openness and transparency in procurement are safeguarded through the oversight, review and monitoring of procurement by a senior procurement review committee. Depending on the mandate given to this committee, it can play a role in ensuring that all departmental actions in respect of the procurement process, including selection of the procurement strategy (e.g. use of ACANs), evaluation criteria, contractual disputes, supplier debriefing and vendor performance, are carried out in accordance with policy and legal requirements.
1.4 The procurement process is complex, and the control framework for the procurement process has many components. This report presents the results from our practice review of one component: the senior review committee(s) responsible for the procurement challenge and oversight function.
Why It Matters
1.5 There are two main reasons why having an effective procurement challenge and oversight committee function is important. First, the committee has a role in assessing corporate risks which include ensuring that all procurement activity is compliant with the relevant laws, regulations, trade agreements and policies and fulfilling the government's commitment to fairness, openness and transparency in procurement. Second, for all contract spending from a financial perspective, the committee should ensure that the requirement is justified and represents good value for money on behalf of all Canadian citizens.
Focus of the Review
Objectives
1.6 The objective of our review was to examine departmental practices related to the committee responsible for the procurement challenge and oversight function at the senior departmental level. Through our review, we also wanted to identify effective practices that could be shared among government departments. We focused our efforts solely on the most senior committee responsible for the function within each department.
1.7 We expected to find that these committees would have the mandate, the membership and the processes in place to provide an effective challenge and oversight function.
Scope and Period Under Review
1.8 We selected nine organizations for review. Eight of these organizations are governed by the TB policy requirements relevant to procurement and contracting. The Canada Revenue Agency has unique authorities as provided for in its enabling legislation and in section 41(2) of the Financial Administration Act, and therefore is not required to comply with these TB contracting policies.
1.9 We focused our review on the organization of and processes used by the most senior committee responsible for the procurement challenge and oversight function, within each department. The committees reviewed were at Agriculture and Agri-Food Canada (AAFC), the Canadian International Development Agency (CIDA), the Canadian Institutes of Health Research (CIHR), the Canada Revenue Agency (CRA), the Department of Justice (DOJ), Environment Canada (EC), Industry Canada (IC), Public Safety and Emergency Preparedness Canada (PSEPC), and Public Works and Government Services Canada (PWGSC). We conducted interviews with senior departmental officials and examined supporting documentation such as the committee terms of reference and samples of submissions reviewed by them for the period from April 1, 2007, to March 31, 2008.
1.10 We expected large departments and agencies (LDAs)Footnote 8 to have higher contracting volumes than small departments and agencies (SDAs). By definition, LDAs are those organizations that employ 500 people or more and whose annual expenditures exceed $300M. Only one of the organizations we selected met the definition of an SDA, the Canadian Institutes of Health Research. We did not expect it would have as extensive a management control framework as the LDAs. However, we did expect all organizations to have a senior committee or other means to perform an effective procurement challenge and oversight function.
1.11 We specifically wanted to know what departmental practices were with respect to the following:
- What was the senior procurement review committee mandate?
- Did the committee members have sufficient knowledge and experience or access to the right subject matter experts to carry out the committee's mandated responsibilities?
- Did the committee have sufficient information before it to approve or recommend a procurement submission?
- Did the committees monitor the results of its work to assess if it is working effectively?
1.12 Our scope did not include a technical review of the documentation accompanying the procurement submission received by the committees. For instance, we did not validate the details of the draft statements of requirement, requests for proposal, statements of work, technical statements of requirements or sole source justifications. Our focus was on whether the committees received sufficient information and whether such information was duly signed off by the program managers.
Methodology
1.13 We interviewed the departmental and agency officials most closely associated with the work of the committees and obtained documentation to support the questions we asked. For example, we interviewed committee chairs, committee members and personnel working in the secretariats that support the committees.
1.14 We obtained the terms of reference for these committees and established who the members were. Where committees maintained a record of decisions, we obtained them for the period from April 1, 2007, to March 31, 2008. Where applicable, we selected a sample of 5-10 decisions from the population and carried out a detailed examination. The samples were chosen to reflect the variety of the types of decisions that were made by the committees.
1.15 Finally, using the results from our sample, we assessed the effectiveness of these committees against detailed criteria describing expectations in each of four focus areas which are further described in the Findings section of this report:
- Criterion a – Senior procurement review committee terms of reference;
- Criterion b – Committee membership;
- Criterion c – Appropriate information for decision making; and,
- Criterion d – Senior procurement review committee effectiveness.
Findings
General Findings
1.16 We take the opportunity below to describe in more detail four senior procurement review committees exhibiting a certain level of maturity, with a view to illustrating how they function and the variety of models.
1.17 At Agriculture and Agri-Food Canada, the Procurement Review Board (PRB) is the management review and approval forum for high value, high risk or significant plans for the procurement of goods, services and construction. The PRB is concerned primarily with how the planned procurement is proposed to be actioned. All submissions must be pre-approved by the respective team leaders and/or Board of Directors for content, program priority, fund availability and alignment with departmental plans. The PRB is accountable to the Executive Council through the Corporate Services and Systems Board (CSSB). The Executive Council is made aware of procurement activities on a quarterly basis. The PRB reviews procurement submissions at the beginning of the procurement process and makes its recommendations to the CSSB, whose role is to approve the proposals. In almost all cases, the CSSB upholds the recommendations made by the PRB.
1.18 The Environment Canada Procurement Review Board (PRB), which serves as a formal procurement review committee, has the following functions:
- Ensure compliance with administrative, contractual and financial policies, procurement disciplines, trade agreements, legislative regimes and delegations of authority.
- Ensure proposed investments support the Department's programs and policy direction as established by the Executive Management Committee, the Report on Plans and Priorities, etc.
- Establish an accountability framework to support the effective management of procurement.
- Ensure effective reporting and communication plans are in place for these activities.
1.19 The PRB reviews procurement submissions, including requests for proposals, needs justification and statements of work, at the outset of the procurement process and makes its decisions on procurement submissions. The information is then recorded and disseminated to the appropriate regional procurement and contracting units, which relay decisions to the sponsoring manager. The Executive Management Committee receives annual updates of decisions made by the PRB.
1.20 At the Canada Revenue Agency (CRA), there are two senior committees that carry out parts of the procurement challenge and review function. One is the Information Technology Procurement Strategy Committee (ITPSC), which makes procurement strategy decisions on information technology projects. The second is the Branch Management Committee, which is the decision authority for non-information technology procurements. We chose to review the ITPSC which does not report to other levels within the Agency, because we had not reviewed IT procurement in any other department.
1.21 Client sponsors (i.e. those proposing IT acquisitions) and procurement personnel from the Administration Directorate are required to present documents to meet the specifications of the "Request for ITPSC Approval" form. The form identifies the sponsor (whose signature indicates approval), the contracting officer, the particular requirement, the quantity, the estimated value, a description, a risk assessment, the proposed procurement method and an ACAN justification (if applicable) and contains the signatures of the official giving approval on behalf of the ITPSC and the chair. The review and approval by the ITPSC is completed at the outset of the procurement process, and accordingly, the ITPSC can recommend, recommend with conditions or not recommend a procurement strategy. What the ITPSC specifically approves is the procurement strategy. Following approval by the ITPSC, it is the responsibility of the CRA procurement officer and the client sponsor to begin developing the relevant solicitation and resulting contract documents for that strategy. To achieve this, a Request for Proposal Team (RFPT) is established. The members of the RFPT include representatives of various specialty areas. For instance, the IT Branch provides expertise from both security and IT compatibility perspectives. The compliance function performed by the Business and Management Services Division is responsible for ensuring consistency with ITPSC direction.
1.22 The mandate of the senior procurement review committee at the Canadian Institutes of Health Research (CIHR), the Contract Review Board (CRB), is to provide direction and support for the development of CIHR's procurement framework. This includes procurement policies, training for managers, delegations and tools. It also includes regular oversight of all contracting activities to ensure compliance and value for money. The CIHR prepares an annual report to the Extended Executive Management Committee summarizing its assessment of the effectiveness of CIHR's procurement framework and contracting practices, together with recommendations for improvement.
1.23 The CRB's mandate does not include the review of procurement submissions at a transactional level. However, at the onset of the procurement process, all proposed procurements of goods and services are challenged and reviewed by the Manager of Procurement and his personnel. All transactions over $1 million are brought to the attention of the Chief Financial Officer for his consideration and approval. Any salient issues are brought to the attention of the CRB with recommendations for its review and approval. The Standing Committee on Performance Measurement Evaluation and Audit also exercises oversight through its follow-up of management action plans in response to internal audits.
1.24 The following summarizes the results of the review criteria.
Criterion a: Senior Procurement Review Committee Terms of Reference
1.25 In our review of the nine departments and agencies, we expected to see that a senior procurement review committee would have terms of reference (TOR) that incorporate the review elements specified below. The terms of reference would define what the committee is to do and how it is to do it. For instance, the TOR should define membership, its roles and responsibilities, the frequency of meetings, the specific types of procurement submissions it must review, who it reports to and to whom it must report, and the extent of its authority.
Table 1
Legend:
[X] Criterion was satisfied
[ ] Criterion was not satisfied
Table 1 summarizes the results of our review of the senior procurement review committee terms of reference.
Department or Agency | AAFC | CIDAFootnote 1 | CIHR | CRA | DOJ | EC | ICFootnote 2 | PSEPC | PWGSCFootnote 3 |
---|---|---|---|---|---|---|---|---|---|
The authority of the committee is indicated | [X] | [X] | [X] | [X] | [X] | [X] | [ ] | [X] | [X] |
The committee's mandate is specified | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] |
The roles and responsibilities of the committee members are specified | [X] | [ ] | [X] | [X] | [X] | [X] | [X] | [X] | [ ] |
Membership is specified | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] |
The frequency or occurrence of meetings is specified | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] |
All ACAN procurements over a specified threshold are required to be presented for review | [X] | [ ] | N/A | [X] | [ ] | [X] | [X] | [ ] | [X] |
Text description of Table 1 is available on a separate page.
1.26 As reflected in the table above, the terms of reference reviewed met most of our review criteria. We did note areas of opportunity to further strengthen the terms of reference for the senior procurement review committees.
1.27 In one instance the procurement reviews were conducted only at the end of the competitive selection process. We believe that the completion of reviews at the planning stage is an effective practice that can help departments reduce procurement risks before publishing solicitation documents that reflect the government's intentions.
1.28 In some departments the senior procurement review committee's terms of reference and other departmental documentation and communications were inconsistent. This could lead to uncertainty about the roles, responsibilities and practices of the committee. All documentation and communication related to the senior procurement review committee should be aligned with the committee's terms of reference.
1.29 We noted procurement review criteria included in the terms of reference that were defined using terms such as "deemed significant" or "significant scope." We believe that such criteria are unclear and should be supported by an explanation of the relevant risk factors.
1.30 In one department, key submission review criteria focused on only services procurements, and in another department, only competitive procurements were considered by the review committee. We believe that the rationale or relevant risk criteria for limiting reviews to these categories of procurement should be clearly communicated to ensure that identified risks are being mitigated.
1.31 Further, Chapter 3 of the OPO's Annual Report notes the special risks relating to contracts processed through Advance Contract Award Notices (ACANs). Our review revealed that some departments had not established criteria for the review of ACAN contracts by the procurement review committees. Consequently, there is no assurance that all the sensitive contracts awarded using an ACAN are included in their review.
Criterion b: Committee Membership
1.32 When reviewing the committee membership, we expected to find that the committee members would occupy positions that require the skills and expertise to ensure that procurement submissions undergo the scrutiny that only senior management personnel with experience and department-wide perspective can bring. Further, we expected to see that the committee would have advisory members who bring specific skills to the table if needed, including skills in the finance, legal, communication and information systems areas.
Table 2
Legend:
[X] Criterion was satisfied
[ ] Criterion was not satisfied
Table 2 summarizes the results of our review of the senior procurement review committee membership.
Department or Agency | AAFC | CIDA | CIHR | CRA | DOJ | EC | ICFootnote 4 | PSEPC | PWGSCFootnote 5 |
---|---|---|---|---|---|---|---|---|---|
Evidence that committee members occupy positions in the department/agency at a senior level | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] |
Evidence that the CFO or Senior Financial Official is a member of the committee | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] |
Evidence that the committee includes a senior representative from Legal | [X] | [X] | [ ] | [ ] | [X] | [X] | [X] | [X] | [X] |
Evidence that the committee includes a senior representative from Procurement, IT, HR, Risk Management or other relevant sections | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] | [X] |
Text description of Table 2 is available on a separate page.
1.33 We determined from our review that generally the committee membership comprised senior management and was multidisciplinary. By having these very senior departmental personnel on the committee, the departments ensure that procurement submissions undergo the type of scrutiny that only senior management personnel with experience and a department-wide perspective can bring.
Criterion c: Appropriate Information for Making Decisions
1.34 In our review of the appropriateness of information submitted by branches and directorates for decision making purposes, we expected to see that procurement submissions would provide a basis on which to establish effective decisions. For example, the following was reviewed:
- Whether there was a fully completed procurement submission to the committee, including documentation indicating that the procurement submission was approved by appropriate authorities.
- Whether there was sole source justification if a non-competitive process was used or an ACAN was issued.
- Where approval was conditional and/or restricted, whether follow-up to ensure the committee received evidence that the condition or restriction had been met was noted.
- Whether there was evidence of a mechanism in place to monitor past vendor performance prior to the award of a subsequent contract to the same supplier.
1.35 We performed a detailed review of the information submitted for decision making purposes at six departments: AAFC, CIDA, the CRA, EC, IC and PWGSC.
1.36 We were not able to complete detailed sample testing at the other three organizations for the following reasons:
- DOJ and PSEPC had just recently established their senior procurement review committees and terms of reference, and at the time of our review, these committees had not completed any procurement submission reviews.
- At CIHR, the senior review committee does not perform reviews of procurement submissions at the transactional level.
1.37 The six organizations reviewed require that a template/checklist be completed by the submitting branch or directorate to ensure the procurement submissions tabled for review address key departmental risk issues and are recommended by the submitting directorate or branch. From our review of samples tested, we observed that the senior committees of those organizations generally reviewed appropriate information for decision-making purposes.
1.38 From our review of sample procurement submissions, we observed that information on past vendor performance was not provided to the committee prior to the award of a significant contract. In addition, in our review of committee meeting minutes and records of decisions, we did not observe documentation of the review of vendor performance during contract periods.
1.39 It was noted that the terms of reference of CRA's senior procurement review committee did specify that ITPSC reviews reports from previous procurements to analyze vendor performance.
1.40 We have also noted that PWGSC's Vendor Performance Policy (currently under review) states that decisions to apply vendor performance corrective measures should be made where, on the basis of the vendor's past performance, a prudent person acting on their own behalf would not continue to deal with the vendor, or would not continue to deal with the vendor without special conditions being attached. The Policy further stipulates that a debarred or suspended vendor is ineligible to bid on or receive contracts related to certain types of procurements. Under the policy provisions, bids received from vendors debarred or suspended from doing any business with PWGSC will not be considered for evaluation. In our opinion, the senior procurement review committee should be provided with assurances that the terms of this or any similar vendor performance policy are being implemented. More specifically, the committee should be provided with assurances that procurement solicitation documents and evaluation procedures will ensure that the terms of any restrictions or conditions imposed on a vendor as a result of poor performance are being complied with for all suppliers responding to the solicitation.
1.41 Further, from the samples reviewed, only one of the procurement submissions that involved more than one department was duly signed off by the departments concerned. We believe that where multiple departments are involved in the procurement, it is important to consider whether the proposed procurement actions are supported by all departments involved.
Criterion d: Senior Procurement Review Committee Effectiveness (Monitoring)
1.42 In our review of the effectiveness of the monitoring functions performed by the senior procurement review committees, we expected to observe that the terms of reference would be well communicated throughout the department. Well-communicated terms of reference for the senior procurement review committee would assist in ensuring department-wide understanding of its role and the submission requirements to be met by the presenting branch or directorate. Secondly, we expected to find the committee had the means to ensure it was receiving for review all procurement submissions within its mandate. We also considered measures that were in place to inform the committee of its effectiveness in achieving the intended results.
1.43 We were not able to complete detailed sample testing at three organizations for the following reasons:
- DOJ and PSEPC had just recently established their senior procurement review committees and terms of reference, and at the time of our review, these committees had not completed any procurement submission reviews.
- At CIHR, the senior review committee does not perform reviews of procurement submissions at the transactional level.
Table 3
Legend:
[X] Criterion was satisfied
[ ] Criterion was not satisfied
Table 3 summarizes the results of our review of senior procurement review committee effectiveness (monitoring).
Department or Agency | AAFCFootnote 6 | CIDA | CRA | EC | IC | PWGSCFootnote 7 |
---|---|---|---|---|---|---|
Evidence that the committee is reviewing the appropriate information | [X] | [X] | [X] | [X] | [X] | [X] |
Evidence of the use of a "checklist" or other means to ensure the committee's review is comprehensive in terms of addressing key risk issues within the procurement submission | [X] | [X] | [X] | [X] | [X] | [X] |
Evidence of a system to ensure that the committee is receiving all procurement submissions within its mandate | [X] | [X] | [X] | [X] | [X] | [X] |
Evidence that the Record of Decisions is maintained by the committee to provide a basis for the decisions reached, i.e. approved, approved with conditions and/or restriction, or rejected | [X] | [X] | [X] | [X] | [X] | [X] |
Evidence of a manual or computerized system that provides the committee with a tracking function to determine the status of decisions it makes, particularly with respect to those approved with restrictions and/or conditions | [X] | [X] | [X] | [X] | [X] | [X] |
Evidence of the existence of a system to flag contracts coming up for renewal to ensure renewals are not "automatic" and options are not exercised without due diligence in regard to competition | [X] | [X] | [X] | [X] | [X] | [ ] |
Evidence that the committee ensures itself that the decisions it makes are in fact carried out/respected by the appropriate authorities within the department/agency contracting activities | [X] | [X] | [X] | [X] | [X] | [ ] |
Evidence that the committee utilizes performance measures to determine through its actions whether any improvement or deterioration is occurring in the department/agency contracting activities | [X] | [ ] | [X] | [X] | [ ] | [ ] |
Evidence that the committee measures the satisfaction of the program officers with respect to the procurement and contracting activities that support their programs | [X] | [ ] | [X] | [ ] | [ ] | [ ] |
Evidence from samples provided that actual attendance at committee meetings was consistent with that specified in the terms of reference | [X] | [X] | [X] | [X] | [X] | [X] |
Evidence that the terms of reference are well communicated throughout the organization | [X] | [X] | [X] | [X] | [X] | [X] |
Text description of Table 3 is available on a separate page.
1.44 As the foregoing table indicates, Agriculture and Agri-Food Canada, Environment Canada and the Canada Revenue Agency have for the most part met all the criteria and therefore from our review we have concluded they have effective committees. For instance, these committees utilize performance measures to determine whether through their actions any improvement or deterioration is occurring in the department/agency contracting activities.
1.45 We can also conclude that the senior review committee at CIHR has been appropriately established and that the direction it has taken is appropriate for an agency of its size. We can also conclude that the transactional challenge and oversight function of all procurement of goods and services performed by the Manager of Procurement and his personnel, in addition to the approval of all transactions over $1 million by the Chief Financial Officer, is appropriate for an agency of its size.
Departmental Committees Being Strengthened or in Development
1.46 Five of the departments reviewed were in the process of establishing their senior procurement review committee or had indicated that they will be improving their existing oversight mechanisms. A description of these departments' senior procurement review committees and progress towards further improvements is provided below. While senior procurement review committee oversight is highly desirable, departments and agencies that are developing or strengthening their senior review committees do have compensating controls for procurement oversight and review to ensure fairness, openness and transparency in the interim.
Public Works and Government Services Canada
1.47 During the period under review, PWGSC was providing senior level oversight through two committees, the Procurement Oversight Committee (POC) for its own procurement and the Strategic Review Committee for all procurements that exceed the Minister's authority (including those that PWGSC does on behalf of other government departments). In addition, PWGSC has branch level committees and functions (e.g. the Contract Review Board within the Information Technology Services Branch and the Real Property Branch Investment Management Board) that contribute to procurement oversight at the operational level. Our review focused on the functions of the Procurement Oversight Committee.
1.48 The POC began reviewing procurement submissions in November 2007 on the understanding that the Department would revisit its terms of reference after the committee had functioned for a reasonable length of time.
1.49 The POC was responsible for reviewing large and/or sensitive procurements and providing relevant guidance on procurement strategies early in the procurement process. Branch heads were accountable for identifying procurement activities for review by the POC, in accordance with the criteria included in the POC terms of reference.
1.50 The POC conducted follow up on conditional approvals, as required and the branches or regions were required to return to POC with their revised procurement strategy documentation. Once the procurement strategy was approved, the branches and regions were responsible for ensuring that the decisions of POC were carried out.
1.51 Our review confirms that during, its initial five months of operation (from November 2007 to March 2008), the POC was generally operating within its mandate according to criteria provided in its terms of reference.
1.52 In May 2008, the POC developed a departmental procurement planning process involving all PWGSC branches and regions. This process is intended to ensure a systematic enterprise-level approach to managing procurement within PWGSC and has been incorporated into the Department's business planning cycle.
1.53 In November 2008, PWGSC's Departmental Oversight Branch was formally mandated to undertake a review of the terms of reference for the POC, as well as a review of the Department's procurement risk assessment structure and processes. During our review, at the request of PWGSC, we shared effective practices noted in other departments with PWGSC. PWGSC has stated that they intend to use the results of this review to address areas requiring improvement and incorporate identified "best practices" into their procurement management and oversight frameworks. Further, PWGSC has indicated that anticipated improvements include revised terms of reference and criteria as well as a comprehensive and effective risk-focused assessment structure supported by processes common to all branches and regions.
1.54 In January 2009, PWGSC merged the Procurement Oversight Committee and the Strategic Review Committee into the Business Operations Committee (BOC), which meets weekly. PWGSC has indicated that under the BOC the fundamental mandate of the POC will be maintained and procurement issues will be addressed every fourth week. However procurement cases can also be addressed at anytime if deemed urgent.
1.55 Based on our review, we conclude that as the Department's oversight committee continues to evolve, PWGSC will benefit from the establishment of corresponding measures to assess compliance with established terms of reference and procedures, as well as the effectiveness of the procurement oversight committee function.
Justice Canada
1.56 Justice Canada appears to be on track to have a strong central procurement challenge and oversight function with the establishment and operation of the National Contracts Review Committee (NCRC) supported by the Contracting and Materiel Management Division (CMMD) and a secretariat. Further, CMMD performs a strategic role in the completion of the procurement oversight transformation to establish and ensure effective and efficient contracting oversight at Justice Canada.
1.57 The NCRC established in December 2008 is mandated to review procurement submissions of goods and services to be acquired for the Department in excess of the delegated authority of the regional branches. DOJ has advised that shortly after the completion of our field work their senior review committee became fully operational. We were told that their reviews are being done at various procurement stages from planning to pre-contract award, depending on potential risks, as determined by the Director of Contracting.
1.58 An internal audit of the contracting for services was completed in March 2007 by the Audit and Management Studies Division of Justice Canada. The scope of the audit included a review of contracting operations and activities of CMMD's Contracting Unit and the contracting activities for services within the Department's organizations at Headquarters.
1.59 The review of the most recent Management Response and Action Plan presented indicated that substantial progress had been made to address audit recommendations resulting in significant improvements to procurement governance and oversight at Justice Canada. However, there are still management action responses that are currently being addressed.
1.60 We were told that at the time of the review, there was no mechanism to ensure that all procurement submissions within the scope of the NCRC are submitted for review. Further, there is no monitoring and reporting mechanism to ensure decisions of the NCRC are adhered to.
1.61 At the time of our review the NCRC was in development. Therefore, we are not able to comment on committee effectiveness.
Industry Canada
1.62 Industry Canada is on track to strengthen the oversight function performed by the senior procurement review committee, the Contracts Review Programs and Services Board (CR-PSB). The committee performs reviews at the outset of the procurement process and recommends the procurement strategy.
1.63 The approval of the needs justification is the responsibility of the sector head, and the sector head's concurrence is required before a procurement submission is put on the CR-PSB agenda for review.
1.64 We were told that contract renewals are managed by the sector. If the sectors are aware of renewals that will fall due, they can bring it to Contracts and Materiel Management's attention
1.65 During the period under review, the mandate of the senior procurement review committee, the CR-PSB, was under revision as a result of a department-wide internal audit of contracting. The planned management actions to address the recommendations of the internal audit of contracting are in progress, and the revised completion dates range from March to July 2009.
1.66 We were told that a new terms of reference for the CR-PSB is being drafted. We were informed that the new terms of reference will place greater emphasis on contractual risk. Further, the revised TOR will reflect the responsibility of the Contracts and Materiel Management division to perform a monitoring and oversight function regarding the procurements that require CR PSB review.
1.67 The Early Warning System for Contract Proposals (EWSCP), which involves CR-PSB reviews of procurements in excess of specified thresholds, was also under revision as a result of an internal audit. The internal audit of the EWSCP was completed by the IC Audit and Evaluation Branch in September 2008. The corresponding management action plan states that the CR-PSB terms of reference would be revised to include a detailed section on the EWS process. Some of the planned actions for the management response to the EWSCP internal audit have been completed. Other planned actions are still outstanding.
1.68 Currently, the streamlined procurement review process, the Précis System, is used for procurement strategy decisions that are low risk. In the Précis System, the Chair of the CR PSB is provided with the procurement strategy submission. If the Chair has questions in regard to the submission, the client may present the submission before the CR PSB, or the Contracts and Materiel Management division will ask the client for clarifications, which are provided to the CR PSB.
1.69 To support effective monitoring, the IC contracting policy is being revised and the target date for completion is August 2009. At present, the internal guidance document, Contracting and Procurement: Your Role in Industry Canada, has been updated to reflect the latest policy documentation.
1.70 From our review, it is clear that Industry Canada is making progress to strengthen the senior review committee procurement challenge and oversight function.
Canadian International Development Agency
1.71 During the period under review, CIDA's senior procurement review committee, the Evaluation Review Board (ERB), was operating within its mandate.
1.72 As reflected in its terms of reference, the ERB mandate is to provide an independent assessment regarding the fairness and transparency of the solicitation and evaluation process undertaken by the Branch Project Team for contracts exceeding $500,000. ERB is a sub-committee of the Audit and Evaluation Committee that receives secretariat support from the Contracting Management Division.
1.73 The ERB review is done at the end of the competitive selection process, prior to the signature of a contract. The purpose of the review is to determine whether the process is fair and transparent, represents best value and will stand the test of public scrutiny.
1.74 ERB is not required to review submissions for sole source procurements, regardless of whether an ACAN is used or not. We were told that CIDA's current procedures require all ACANs to be approved by the Branch Head as well as the Director General, Contracting Management Division (who is also the Chair of the ERB), before being published. In addition, where the estimated contract value exceeds $1 million, further authorization by the Chief Financial Officer (CFO) is required.
1.75 During our review, we were told that implementation of the Agency Transformation for Results Initiative and the planned decentralization of program resources abroad will eventually have an impact on CIDA's governance model and investment decision architecture. In addition, the recommendations of a review of contracting services conducted in March 2008 were yet to be addressed, and the recommendations of the June 2008 Internal Audit Report of the Management of Competitive Contracting were in the process of being addressed.
1.76 CIDA has confirmed that its senior procurement review committee will remain a cornerstone of the Agency's contracting management oversight and control framework. CIDA has also indicated that it intends to use the results of this review to address areas requiring improvement and incorporate identified "best practices" into the Agency's procurement management and oversight framework.
Public Safety and Emergency Preparedness Canada
1.77 The senior procurement review committee with primary responsibility to ensure prudence and probity in public funds within the Public Safety Emergency Preparedness Canada mandate is the Contract Review Committee (CRC). The terms of reference for the CRC were approved by senior management in December 2008. The review performed by the CRC is done at the planning stage. The mandate of the CRC includes the review of requests for services contracts that are $20,000 and over for sole source, and $50,000 for competitive procurement submissions.
1.78 The training sessions for the committee members were held in March and April 2009. The CRC will be fully operational in May 2009 upon completion of the training of all members.
1.79 As of February 2009, no files had been reviewed or maintained by the CRC. Therefore, we were unable to evaluate the effectiveness of the committee, as it had not come into operation at the time of our review.
1.80 We can conclude that the CRC has been appropriately established.
Conclusion
1.81 The Treasury Board policies require departments and agencies to ensure that adequate control frameworks for due diligence and effective stewardship of public funds are in place and working.Footnote 9 To this end, the senior review committee is a key departmental control to ensure prudence and probity, and compliance with the principles of fairness, openness and transparency in procurement.
1.82 We are generally satisfied with the progress made to establish effective senior procurement review committees. We are generally satisfied that departments, through these committees whether they are currently functioning or are in development, are working to ensure the principles of fairness, openness and transparency are reflected in their procurement activities.
1.83 Our review noted several good practices being used by the departments. The details of these are included in our report.
1.84 We noted areas that require further attention. Only the CRA terms of reference requires that the committee receive documentation on vendor performance.
1.85 We also noted, that in cases where poor vendor performance has been confirmed, PWGSC's Vendor Performance Policy (currently under review) calls for reasonable measures to be taken to prevent future problems. The Policy further stipulates that bids received from vendors that are debarred or suspended from doing business due to poor performance will not be considered for evaluation. In our opinion, the senior procurement review committee should be provided with assurances that procurement solicitation documents and evaluation procedures will ensure that the terms of any restrictions or conditions imposed on a vendor as a result of poor performance are being complied with for all suppliers responding to the solicitation.
1.86 Some committees require that they review procurement submissions done through the Advance Contract Award Notice process. We believe that such procurements pose a special risk and all departments should establish risk indicators based on materiality and complexity, and require that all procurements that meet the risk profile, especially procurements that use ACANs, be reviewed by the committee.
Recommendations
1.87 The following practices are viewed as effective means of increasing the confidence of Canadians in procurement by improving oversight in departments. OPO recommends that in the creation and operation of these committees certain essential characteristics be prevalent
- Committees should have comprehensive and objective terms of reference.
- Committees should include members who are multidisciplinary and who understand the procurement process and have an appreciation of the risks involved.
- Departments should establish risk indicators based on materiality and complexity, and require that all procurements meeting the risk profile, especially those that use ACANs, be reviewed by the committee.
- Committees should conduct their reviews at the outset of the procurement process (planning stage).
- Information submitted to these committees should be sufficient so as to ensure sound and effective decision making.
- Procurement submissions involving more than one department should be duly signed off by the departments concerned.
- Committees should be provided with assurances that procurement solicitation documents and evaluation procedures will ensure that the terms of any restrictions or conditions imposed on a vendor as a result of poor performance are being complied with for all suppliers responding to the solicitation.
- Committees should have the means to ensure they are receiving all procurement submissions included in their mandate.
- Committees should monitor the results of their decisions.
- Committees should have the means to judge whether or not they are operating effectively.
Effective Practices
1.88 Our review gave us an opportunity to observe additional effective practices, which departments may find helpful in strengthening their oversight function. Departments listed in parenthesis are those where we noted these practices.
- Updating the terms of reference on a regular basis ensures that information is always current. (AAFC, CIHR, CRA, EC, IC, and PWGSC)
- The use of a procurement checklist/template ensures that submissions presented for review address key departmental risks. (AAFC, CIDA, CRA, EC, IC, and PWGSC)
- A series of supplementary questions, such as those used by Agriculture and Agri-Food Canada, are useful to ensure that the submission is comprehensive and that officials presenting submissions have turned their minds to all considerations. These questions are in addition to what is provided in the procurement submission template:
- Describe whether the bid solicitation process to be used is competitive or non-competitive including ACANs.
- Will the procurement be effected by a Specified Service Agreement, Memorandum of Understanding, Standing Offer, Supply Arrangement or Contract?
- Will the procurement be done by the department itself or Public Works and Government Services Canada?
- Is this a new project? If so, will there be future work?
- Contractor history: Is there an existing contract in place for the work being requested? If so, how was the previous procurement awarded: competitive vs. non-competitive, call-up against a standing offer, supply arrangement, etc.?
- Provide the name of the contractor, the value of the contract and the previous committee number assigned.
- Were there any amendments? If so, provide the amendment number and value.
- What is the business situation and the associated business challenges or pressures that are creating the need for the procurement (i.e. the context)?
- What are the consequences of or impact to the business if the procurement does not proceed in an efficient manner?
- What is the "big picture" from the business point of view (i.e. is the business trying to deliver a project in multiple phases over multiple years and how does the procurement request fit in to the overall picture)?
- Has the business recently received additional funding or approvals that are now creating the business need to proceed with the procurement?
- A streamlined review process for low risk procurement submissions is very important and incorporates sound risk management processes and appropriate use of resources. (CRA, EC and IC).
- The committee is supported through a computerized system that provides a tracking function to determine the status of the decisions it makes. (AAFC, CRA and EC)
- The committee has a means to "flag" contracts coming up for renewal to ensure renewal of contracts is not automatic, and options are exercised with due diligence. (AAFC, CIDA, CRA and EC)
- The committee measures whether through its actions there are improvements or deterioration in the procurement activity of the department or agency. (AAFC, CRA and EC)
- The committee tracks the stage in the procurement review process where the procurement submission is, in order to apprise clients of the status of their requirements. (CRA)
All departments and agencies involved in this review have been provided an opportunity to review this report and their comments have been taken into consideration in finalizing this chapter.
Glossary of Acronyms
- AAFC
- Agriculture and Agri-Food Canada
- ACAN
- Advance Contract Award Notice
- BOC
- Business Operations Committee
- CFO
- Chief Financial Officer
- CIDA
- Canadian International Development Agency
- CIHR
- Canadian Institutes of Health Research
- CMMD
- Contracting and Materiel Management Division
- CRA
- Canada Revenue Agency
- CRB
- Contract Review Board
- CRC
- Contract Review Committee
- CR-PSB
- Contracts Review Programs and Services Board
- CSSB
- Corporate Services and Systems Board
- DOJ
- Department of Justice
- EC
- Environment Canada
- ERB
- Evaluation Review Board
- IC
- Industry Canada
- IT
- Information Technology
- ITPSC
- Information Technology Procurement Strategy Committee
- LDAs
- Large Departments and Agencies
- NCRC
- National Contracts Review Committee
- OPO
- Office of the Procurement Ombudsman
- POC
- Procurement Operations Committee
- PRB
- Procurement Review Board
- PSEPC
- Public Safety and Emergency Preparedness Canada
- PWGSC
- Public Works and Government Services Canada
- RFPT
- Request for Proposal Team
- SDAs
- Small Departments and Agencies
- TB
- Treasury Board
- TOR
- Terms of Reference
Footnotes
- Footnote 1
-
The information in this table is based on a review of the CIDA senior procurement review committee terms of reference and additional information that is provided in the Contracting Guide for Managers in CIDA.
- Footnote 2
-
The information in this table is based on a review of the IC senior procurement review committee terms of reference in effect during the period under review. A new TOR for the CR-PSB was being drafted but was not ready for dissemination.
- Footnote 3
-
At the time of our review, PWGSC was merging its Procurement Oversight Committee (POC) and its Strategic Review Committee into the Business Operations Committee (BOC). The mandate and ToR of POC was also under review. The information in this table is based on a review of the POC terms of reference that were in effect during the period under review.
- Footnote 4
-
The information in this table is based on a review of the IC senior procurement review committee terms of reference in effect during the period under review. A new TOR for the CR-PSB was being drafted but was not ready for dissemination.
- Footnote 5
-
At the time of our review, PWGSC was merging its Procurement Oversight Committee (POC) and its Strategic Review Committee into the Business Operations Committee (BOC). The mandate and TOR of the POC were also under review. The information in this table is based on a review of the POC terms of reference that were in effect during the period under review.
- Footnote 6
-
AAFC has a Policy, Analysis and Reporting Section that monitors the health of the materiel management function including procurement. Various reports are produced to monitor contracting trends, procurement vehicle use, spending trends, vendor use etc. The Asset Management Team holds annual presentations on key areas such as overall contracting activity, activity by commodity, performance indicators relative to level of competiveness, multiple contracts to the same supplier and amendment activity. These presentations are provided to the Corporate Services Systems Board and subsequently to the Executive Council of the Department. AAFC will consider holding biannual presentations to the PRB.
- Footnote 7
-
At the time of our review, PWGSC was merging its Procurement Oversight Committee (POC) and its Strategic Review Committee into the Business Operations Committee (BOC), and the mandate of POC including the ToR was under review. The above table reflects findings of POC effectiveness during the review period.
- Footnote 8
-
Office of the Comptroller General definition: LDAs – 500 employees and budget greater than $300M; SDAs – anything less
- Footnote 9
-
The CRA has unique authorities as provided for in the Agency's enabling legislation and in section 41(2) of the Financial Administration Act, and therefore is not governed by the TB contracting policies.
- Date modified: