Procurement practice review of Immigration, Refugees and Citizenship Canada
- I. Background
- II. Objective and scope
- III. Results
- LOE 1: To determine whether evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies
- Mandatory criteria
- Point-rated criteria
- Selection methodology
- LOE 2: To determine whether solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies
- LOE 3: To determine whether the evaluation of bids and contract award were conducted in accordance with the solicitation
- LOE 1: To determine whether evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies
- IV. Simplification
- V. Conclusion
- VI. Organizational response
- VII. Acknowledgment
- Annex I: Organizational response and action plan
1. The Office of the Procurement Ombudsman (OPO) conducted a review of procurement practices at Immigration, Refugees and Citizenship Canada (IRCC).
2. In accordance with paragraph 22.1(3)(a) of the Department of Public Works and Government Services Act, the Procurement Ombudsman has the authority to review the procurement practices of departments to assess their fairness, openness and transparency.
3. OPO’s procurement practice reviews are based on issues and complaints brought to OPO’s attention by stakeholders, both in general and in regard to specific solicitations by various federal organizations. Based on this information, OPO has identified the 3 highest-risk procurement elements as: (1) the establishment of evaluation criteria and selection plans; (2) the bid solicitation process; and (3) the evaluation of bids and contract award. For the purposes of this review, these elements are defined as follows:
- Evaluation criteria and selection plans—the development of mandatory and point-rated evaluation criteria, and the identification of the selection method to determine the successful bid
- Solicitation—the design and execution of the solicitation process, including the clarity and completeness of solicitation documents
- Evaluation of bids and contract award—the establishment of a process to ensure the consistent evaluation of bids in accordance with the planned approach, including an evaluation plan and instructions to evaluators, and the adequacy of documentation to support the selection of the successful bidder
4. IRCC was selected for review as one of the top 20 federal departments/agencies in terms of the value and volume of its annual procurement activity. OPO is conducting similar reviews of the other top 20 departments/agencies over a five-year period ending in 2022-23.
5. IRCC is responsible for developing and implementing policies, programs and services that facilitate the arrival of people and their integration into Canada, protect refugees and people in need of protection, enhance values and promote the rights and responsibilities of Canadian citizenship, and foster increased intercultural understanding and an integrated society. IRCC also advances global migration policies in support of Canada’s immigration and humanitarian objectives.
6. During OPO’s review period of August 1, 2018, and July 31, 2020, IRCC’s procurement activity included the award of 5,620 contracts with a total value at contract award of $332.7M. These contracting totals included 307 contracts valued at $279.5M that were awarded on IRCC’s behalf by Public Services and Procurement Canada (PSPC) or Shared Services Canada (SSC). IRCC awarded 5,313 contracts at a value of $53.19M. Of these contracts, 712 were awarded by IRCC’s Procurement and Contracting Services group and had a combined value at contract award of $46.28M.
II. Objective and scope
7. This review was undertaken to determine whether IRCC’s procurement practices pertaining to evaluation criteria and selection plans, solicitation, and conducting evaluations of bids and contract award, supported the principles of fairness, openness and transparency. To make this determination, OPO examined whether IRCC’s procurement practices were consistent with Canada’s obligations under applicable sections of national and international trade agreements, the Financial Administration Act and regulations made under it, the Treasury Board Contracting Policy (TBCP) and, when present, departmental guidelines.
8. The following 3 lines of enquiry (LOE) were used to assess the highest-risk procurement elements identified in paragraph 3 above:
- LOE 1: Evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies
- LOE 2: Solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies
- LOE 3: Evaluation of bids and contract award were conducted in accordance with the solicitation
9. OPO’s review consisted of an assessment of procurement files for contracts awarded by IRCC between August 1, 2018, and July 31, 2020. This review did not include construction contracts, non-competitive contracts, acquisition card activity, contracts awarded through PSPC or SSC standing offers or procurement activity for which IRCC was not the contracting authority.
10. Based on contracting data provided by IRCC, OPO selected 40 competitive procurement files for assessment. Of the 712 contracts awarded by IRCC’s Procurement and Contracting Services group, 148 contracts met the criteria for this review after excluding the above mentioned categories. This judgmental sample was developed with consideration to factors including materiality and risk. The risk of selection bias was minimized through random selection of individual files meeting these pre-established factors.
11. The reviewed files included 13 open requests for proposals (RFPs), 14 contracts issued pursuant to PSPC professional services supply arrangements, 6 solicitations sent to a limited number of suppliers, 2 requests for quotes (RFQs) and 5 call-ups against Standing Offers (SOs).
12. IRCC’s procurement practices pertaining to evaluation criteria and selection plans, solicitation documents, and evaluation of bids and contract award were assessed against the 3 LOEs noted above. OPO made 3 recommendations to address the issues identified in the review, which are summarized in Annex I of this report. The recommendations are based on the analysis of information and documentation provided to OPO by IRCC during the course of the review.
LOE 1: To determine whether evaluation criteria and selection plans were established in accordance with applicable laws, regulations and policies
13. Section 10.7.27 of the TBCP states that “[c]ompeting firms should be told the measurement criteria and the weighting assigned to them. … The courts have ruled that the factors and their weighting must be established beforehand and adhered to strictly. … Fairness to all prospective contractors and transparency in the award process are imperative”. Using clear and precise language to define the evaluation criteria and selection methodology helps bidders prepare a responsive bid and evaluators to apply the same criteria equally to all bidders.
14. This LOE applied to 34 files where evaluation criteria were developed and included in solicitation documentation. Of these, 10 contained only mandatory criteria and 24 contained both mandatory and point-rated evaluation criteria. All files were examined to determine if the evaluation criteria and selection methodologies were: clearly communicated in the solicitation; not overly restrictive; and aligned with the requirement. The method of allocating points to weighted criteria was also assessed to determine whether instructions were clearly communicated and reflected the relative importance of the criteria.
Mandatory criteria were inadequately defined, and were not limited to the essential qualifications
15. Of the 34 files that contained mandatory criteria, these criteria were generally well-defined and could be assessed objectively on a pass-fail basis. However, in 9 files, mandatory criteria were inadequately defined, and were not communicated in a clear, precise and measurable manner. There were an additional 4 files in which the mandatory criteria were not limited to the essential qualifications necessary to fulfill the requirement.
- In 8 files mandatory criteria required bidders to demonstrate ‘experience’ but did not specify how much experience was required. As a result, any amount of experience would have resulted in the criteria being met. As an example, one solicitation that would provide permanent residence status to entrepreneurs who have the potential to build high-growth start-ups in Canada required the bidder to “have experience providing written recommendations relating to business and/or innovation to the Federal Government of Canada, including Crown Corporations, over the past two years from bid closing date.” However, the amount of experience required was not specified in the criterion, as a result, as little as one day of experience would have resulted in the criterion being met. In such circumstances, the requirement should be tailored to specify the amount of experience that was essential to perform the work.
- In 1 file IRCC should have been more prescriptive in defining the mandatory criteria. For example, in a solicitation for reconfiguration and moving services, the criterion stated: "[t]he bidder must possess the equipment as follows: The Bidder must provide all the resources, tools, lifting equipment and supplies, necessary to perform all tasks properly, efficiently and safely, at no additional cost to [IRCC]. Example of what could be required…". The way in which this criterion is framed may create challenges for bidders to demonstrate their compliance and for evaluators to assess that the criterion has been satisfied by the bidder. IRCC should have been prescriptive in defining all of the necessary equipment so that the criterion could be assessed on a pass/fail basis.
16. There were 4 files where the mandatory criteria were not limited to the essential qualifications necessary to complete the requirement. In 1 of these files, the criterion was overly restrictive and may have reduced competition.
- 2 files for translation, editing and review services included a mandatory criterion that required bidders to identify a single point of contact. Mandatory criteria must be limited to the essential qualifications necessary to conduct the work; identifying a single point of contact, while useful from an administrative standpoint, is unrelated to the bidder’s ability to perform the work and is not aligned with requirements, and could result in non-compliance by bidders due to an administrative oversight; this info could be sought differently, and still achieve the desired result
- In 1 file the solicitation for loan collection services contained 1 mandatory criterion that required bidders to attest to their capacity to deliver services. While the intent of this criterion was to ask bidders to certify that they could deliver the services within the specified time period, its inclusion as a mandatory criterion is not essential as the contractor’s certification is achieved through their acceptance of the statement of work on signing the contract. Furthermore, a bidder’s attestation in their proposal does not demonstrate to evaluators that they have the capacity to deliver the services within the required timeframe. The inclusion of unnecessary evaluation criteria, particularly as mandatory criteria, unnecessarily complicates the procurement process by making it administratively burdensome on bidders to prepare their proposal. Further, it creates opportunities for bidders to make mistakes and for their proposal to be deemed non-responsive for failing to meet a mandatory criterion that was ultimately not required.
- In 1 file for expert advice related to a transformation strategy that was subject to the North American Free Trade Agreement (NAFTA) and the World Trade Organization-Agreement on Government Procurement (WTO-AGP), a mandatory criterion required bidders to demonstrate experience in 4 transformation studies “where the Bidder was not involved with the implementation of the solution.” Both NAFTA and the WTO-AGP require that the conditions for participation by suppliers in tendering procedures are “limited to those that are essential to ensure the fulfilment of the contract in question.” In this case, it is unclear why a bidder’s involvement in the ‘implementation of the solution’ would be considered a necessary exclusion. IRCC received numerous comments from interested suppliers regarding the restrictive nature of the criteria. The criteria was not changed and the solicitation process resulted in only 1 compliant bid with the bidder receiving a perfect score in the technical evaluation.
IRCC should establish a quality control process to ensure mandatory criteria are adequately defined and communicated in a clear, precise and measurable manner, and that mandatory criteria are limited to the essential qualifications necessary to fulfill the requirement.
Point-rated criteria and rating scales were not overly restrictive, were appropriate to the requirement, and reflected the relative importance of the criteria
17. Section 10.7.25 of the TBCP states that “criteria should identify accurately all the performance elements significant to the success of the project and should measure both the competence of the firm and the worth of its particular technical approach.”
18. In all 24 files that contained point-rated criteria, point-rated criteria were not overly restrictive, were appropriate to the requirement and reflected the relative importance of the criteria. However, in 2 files, the rated criteria or the scoring grid could be communicated in a more clear and precise manner to minimize subjectivity.
- In 1 file for qualitative research in labour markets, the point-rated criteria did not specify how much experience would result in maximum points being awarded. Rather the criteria stated that the maximum allotment of points would be awarded if “[t]he proposal show[ed] an excellent match between knowledge and experience on the one hand and the research theme on the other.” Terminology such as “excellent match” is highly subjective and is open to multiple interpretations.
- In 1 file for satisfaction surveys the scoring grid for a rated criterion could have been communicated in a clearer manner. For example, the criterion required bidders to submit a workplan and listed 4 criteria that “should at a minimum be included in the Bidder’s work plan.” The scoring grid states that 5 points shall be awarded for a workplan that is “well defined and includes all criteria” while 4 points shall be awarded for a workplan that is “clear and includes 4 of the criteria.” Since the criteria for awarding 4 and 5 points both require the bidder to include 4 of the criteria in their workplan, this could make it difficult for potential bidders to understand how evaluators would distinguish between a workplan that receives 5 points versus one that receives 4 points. This did not impact the outcome of the competitive process.
19. When evaluation criteria are communicated in a clear, precise and measurable manner, it enables bidders to know the requirements and the methods by which their proposals will be evaluated. Failure to adequately define evaluation criteria at the outset carries the additional risk that evaluators may struggle to interpret these criteria during the evaluation process. It can also be difficult to defend against external challenges, as it is more difficult to demonstrate that criteria have been strictly adhered to when the criteria are unclear and open to multiple interpretations.
Bidders were consistently advised of the manner in which the contract would be awarded
20. 33 of the 40 solicitations were carried out as requests for proposals. In all 33 files, the selection methodology was clearly communicated in the solicitation document and reflected the complexity of the requirement.
21. IRCC’s Procurement and Contracting Services group has implemented a number of preventative controls to ensure the overall consistency and compliance of the procurement process. One such control is the completion of the Contract Planning and Advance Approval (CPAA) document. This document is completed by the Contracting Officer following the identification of the requirement, and is also required for each contract amendment. The CPAA provides a comprehensive summary of all relevant information necessary to facilitate management oversight; including a summary of the requirement, estimated costs, related contract history, any contract amendments and the associated rationale, the proposed procurement strategy, and analysis of risks, and general remarks from the Contracting Officer. The CPAA is accompanied by a risk assessment that is completed using IRCC’s Procurement Risk Assessment Tool. The risk rating resulting from the completion of this assessment determines the required approval level for the procurement. High risk procurements, such as those involving an amendment that exceeds 50% of the original contract value are subject to approval by IRCC’s Procurement and Contracting Oversight Committee (PCOC).
22. IRCC’s procurement officers consistently completed the CPAA and provided the necessary context for key decisions, including the rationale for the respective selection methodology. This practice helps to ensure that selection methodologies are appropriate to the complexity of the requirement and that an adequate justification is on file to support decisions and to facilitate management oversight. In instances where common selection methodologies are not chosen, this practice may be useful should the process be subject to scrutiny. IRCC’s CPAA is an effective tool for documenting key decisions and is a good practice that other departments should emulate.
LOE 2: To determine whether solicitation documents and organizational practices during the bid solicitation period were consistent with applicable laws, regulations and policies
23. The TBCP sets out detailed procedures to ensure government contracting is carried out in a manner which enhances access, competition and fairness and results in best value. Section 10.7 of the TBCP includes the minimum requirements to be included in the solicitation document as well as mandatory elements related to the design and execution of the process.
24. Solicitation documents must contain work descriptions or specifications defined in terms of clear outputs or performance requirements. Solicitation documents should include: bidder instructions; bid preparation instructions; clear evaluation procedures; certification requirements; security and financial requirements; resulting contract clauses; and instructions informing bidders they may request information on the results of the RFP and how their bid was evaluated. The clarity of this information is key in supporting the Government of Canada’s obligations that are identified in the TBCP.
25. This LOE applied to 35 of the 40 files reviewed; 13 involved an open solicitation, 14 involved PSPC professional services supply arrangements, 6 solicitations were sent to a limited number of suppliers, and 2 involved RFQs. For these files, solicitation documents (excluding evaluation criteria and selection plans which were assessed under LOE 1 above) were assessed to determine whether they, among other things, contained a clear description of the requirement, and instructions necessary to prepare a compliant bid or submit a quote. The review of organizational practices included an assessment of whether IRCC has established a framework to ensure that procurement practices are consistent with laws, regulations and policies and support fair, open and transparent procurements; whether the solicitation was open to the appropriate number of suppliers and for the required duration; and whether communications with suppliers supported the preparation of responsive bids. The 5 call-ups against SOs were assessed to determine whether the call-ups were clear and contained complete information, including a clear description of the requirement and delivery schedule, and whether the specified contracting procedures were adhered to. The results of this assessment are presented below.
Solicitation documentation was complete; however, several opportunities for improvement were noted regarding the design and execution of the solicitation process
26. All of the solicitation documents reviewed, as well as the 5 call-ups against SOs, included a description of the requirement. All RFPs identified evaluation criteria and selection methodology, along with instructions for submitting proposals and posing questions during the solicitation period. Despite solicitation documents being fairly complete, several opportunities for improvement were noted regarding the design and execution of the solicitation process; these included 1 instance where the solicitation was structured to avoid NAFTA thresholds and 3 instances where communications with suppliers lacked transparency. Further details on these files are presented below.
- In 1 file the requirement was intentionally structured to avoid NAFTA thresholds. The requirement for a social media analytics subscription was initially contemplated as a 5 year contract with an estimated value of approximately $350,000. Emails on the file indicate that IRCC adjusted the requirement so as to avoid NAFTA. Upon receipt of the purchase order, the Contracting Officer advised the Technical Authority that due to the estimated value, the requirement is subject to NAFTA and must be openly competed. In an email response, the Technical Authority stated “this is a situation we want to avoid, especially since as we mentioned the funding isn't even indeterminate, so we are happy to scale back the option years in order to stay under that limit. At what $ value are we subject to NAFTA?” The Contracting Officer correctly advised the Technical Authority of the risks, specifically that “awarding a contract for only one year and following this, awarding another contract for the same services gives the perception of contract splitting, especially if you already know that you will likely need the requirement beyond this time frame. It seems as though trade agreement thresholds are being avoided in order to ensure an expedited procurement process.” The contract was structured as a one year contract with a one year option, the total value falling below the NAFTA threshold. Articles 1001(4) and 1002(4) of NAFTA both state that you cannot structure a procurement to avoid obligations under NAFTA.
Some communications with suppliers lacked transparency and, in 1 instance, disadvantaged unsuccessful bidders from seeking recourse
27. Section 2(a) of the TBCP states that “[g]overnment contracting shall be conducted in a manner which will: a. stand the test of public scrutiny in matters of prudence and probity, facilitate access, encourage competition, and reflect fairness in the spending of public funds”. Section 12.3.1 further states that “[p]rocurement files shall be established and structured to facilitate management oversight with a complete audit trail that contains details related to relevant communications and decisions…”.
28. These requirements apply to all aspects of the procurement process, including interactions with suppliers. During the bid solicitation process, suppliers may communicate with contracting authorities to obtain clarifications or explanations of the content of the solicitation. For procurements subject to the Canadian Free Trade Agreement and the WTO-AGP, section 12.3.2 of the TBCP requires contracting authorities to ensure all communications with bidders are supported by complete documentation and records to demonstrate “that the procurement process was carried out in accordance with the agreements”. Any significant information given by a contracting officer to a supplier with respect to a particular procurement must also be given simultaneously to all other interested suppliers.
29. In 2 files communications with suppliers during the solicitation period did not support the principles of fairness and transparency.
- In 1 file for expert advice services, the Contracting Officer asked a bidder to phone him during the solicitation period and did not document the content of this conversation. This bidder was ultimately awarded the contract. Phone calls with bidders during the solicitation period should be avoided and at the very least, well documented to ensure transparency and that fairness among bidders is respected. As stated in Section 12.3.2 of the TBCP, NAFTA and the WTO-AGP require that contracting authorities “guarantee that complete documentation and records, including a signed and dated record of all communications with suppliers, are maintained to allow verification by the Canadian International Trade Tribunal that the procurement process was carried out in accordance with the agreements.”
- In 1 file to digitize and index a collection of passport applications, there was a lack of transparency regarding the existence of an incumbent. The RFP stated that “[n]o Contractor is presently performing the services”; however, section 4 of IRCC’s internal Contract Planning and Advance Approval document notes that a vendor was, at the time, engaged from 2018-11-09 to 2019-03-31 under a contract valued at $452,000.00. Remarks states “[s]imilar work is currently being performed under the contract outlined above in section 4.” The incumbent was awarded the contract.
30. After the solicitation process is completed and bids evaluated, IRCC informs bidders of the evaluation results via either a regret letter, advising the bidder it was not the successful bidder, or an award letter advising the bidder it won. “Regret” letters are sent to unsuccessful bidders and should offer an explanation of why their bids were not selected. Of the solicitation processes reviewed, a record that the unsuccessful bidder(s) was notified of the results of the evaluation was on file in all but 1 instance. When provided, the regret letters provided a detailed account of why the bidder had not been awarded the contract, and included the results of the technical evaluation as well as the overall score of the winning bidder. However, in 1 file regret letters to bidders contained inaccurate information, which disadvantaged these bidders from seeking recourse.
- In 1 file the regret letters to 5 bidders contained inaccurate information about why their bid had not been selected. In 3 of these letters, the bidder had been informed that their proposal had been responsive, but that they had not been the lowest cost compliant bid. In all cases, these bidders had not been responsive to one or more mandatory criteria. The sharing of information following contract award, such as through regret letters and award notices, is often the first way for a supplier to detect that the procurement process was not conducted appropriately. In failing to provide accurate information to these bidders about why their proposals had not been selected, IRCC disadvantaged them from seeking recourse and challenging the results of the evaluation. The regret letters to the 2 other bidders correctly identified the bid as non-responsive but did not include their non-compliance to certain mandatory criteria. In these instances, IRCC failed to provide unsuccessful bidders with valuable information about how their proposals were evaluated and where they could improve.
IRCC should ensure that the information that is communicated to suppliers both during and after the solicitation process is accurate and that adequate documentation of these communications is retained to facilitate management oversight.
LOE 3: To determine whether the evaluation of bids and contract award were conducted in accordance with the solicitation
31. In order to ensure the fairness and defensibility of evaluation processes, section 10.7.27 of the TBCP requires that evaluation criteria be adhered to strictly and applied equally to all bidders. Failure to ensure the consistent evaluation of proposals increases the risk that ambiguities in the selection process result in the contract being wrongly awarded. Inconsistent evaluations may also call into question the integrity of the procurement process.
32. Section 12.3.1 of the TBCP also requires that procurement files “facilitate management oversight with a complete audit trail” containing “details related to relevant communications and decisions, including the identification of involved officials and contracting approval authorities.” The requirement to ensure adequate file documentation extends to the actions undertaken during the solicitation period as well as the evaluation of bids.
33. 33 solicitation processes were examined to determine whether a process had been established, complete with guidance for evaluators, to ensure: the consistent evaluation of bids; that the evaluation of bids had been carried out in accordance with the planned approach; and that files were adequately documented. The results of this assessment are presented below.
Immigration, Refugees and Citizenship Canada has established a standard evaluation process that is well-documented and was applied consistently across almost all of the files reviewed
34. Of the 33 files for which an evaluation was conducted, the evaluation grid contained criteria that were identical to those found in the solicitation document, and the number and identities of the evaluators had been documented in the file. In all but 1 file, IRCC ensured that the requisite number of evaluators participated in the process, and the evaluation followed the prescribed process (etc. kick off and consensus meetings were held).
35. IRCC has developed a robust suite of process documentation, including guidance for evaluators, to ensure that the bid evaluation process complies with key requirements of the TBCP. IRCC’s Bid Evaluation Procedure sets forth a standardized process for the evaluation of proposals, and provides guidance to contracting authorities on their responsibilities throughout the bid evaluation process. The Evaluation Team Guidelines provide evaluators with a detailed overview of the process that includes a comprehensive summary of their responsibilities, instructions to ensure proposals are evaluated in accordance with the bid evaluation procedures, and are appropriately documented, including the identification of strengths, weaknesses, missing information and are cross-referenced to the bid in the Evaluation Report Form. These instructions also include recommended procedures to mitigate risk during the process such as procedures to protect the confidentiality of the process, and guidance for seeking clarifications, as some examples. These guidelines contain a conflict of interest declaration and are required to be signed by each evaluation team member and returned to the Contracting Officer prior to the receipt of the proposals. In OPO’s review of the 33 files for which a technical evaluation was completed, a copy of these guidelines was consistently on file and completed by evaluators.
Some evaluations were incorrectly conducted
36. In order to ensure the transparency and defensibility of evaluation processes, section 10.7.27 of the TBCP requires that evaluation criteria must be adhered to strictly and applied equally to all bidders. Failure to ensure the consistent evaluation of proposals increases the risk that the contract will be wrongly awarded and may call into question the integrity of the procurement process.
37. IRCC has established a robust evaluation process; however, there were a few instances in which the evaluation was not carried out in accordance with the planned approach. In 2 files, the evaluation team incorrectly deemed proposals that did not meet certain mandatory criteria to be responsive, and in 2 additional files, the evaluation team awarded partial points, which deviated from the procedure identified in the solicitation.
- In 1 file, 2 of the bidders failed to include mandatory information in their proposals yet their bids were incorrectly deemed responsive. The mandatory criterion required bidders to identify, among other things, the name of the organization in a reference letter. The criteria states “[i]f the Bidder cannot provide a reference letter that meets the requirements, [IRCC] will disqualify the Bidder.” One of the bidders did not provide the name of the organization and instead stated “a major investment company”. The bid was incorrectly evaluated as meeting this mandatory criterion. Similarly, the successful bidder failed to provide the job title for a reference. Emails on file indicate that the Contracting Officer deemed the bid non-responsive as a result of this omission; however, the bid was ultimately deemed to have met the criterion at the consensus evaluation.
While these issues should not impact a bidder’s ability to fulfill the requirement, they are nevertheless mandatory elements of the evaluation criteria that must be adhered to strictly. While section 10.8.7 of TBCP leaves room for contracting authorities to consider proposals with minor aberrations, this is only the case where all mandatory requirements have been satisfied. Since the omissions are identified as mandatory requirements, evaluators have no discretion but to deem the bids non-responsive in accordance with section 10.8.6 of the TBCP.
This draws attention to the consequence of including elements in mandatory criteria that are not limited to the essential qualifications necessary to meet the requirement and reflects a broader opportunity for simplification.
- In 1 file for project management services, the proposed replacement resource, at the Task Authorization stage, must be evaluated in accordance with the requirements of the bid solicitation and must have the qualifications and experience that meet or exceed those obtained for the original resource. One of the mandatory criteria of the bid solicitation pertaining to experience was incorrectly accepted. The mandatory criterion required the resource to possess experience “overseeing at least two (2) projects within a Canadian federal government organizations and/or agencies”. The proposed resource demonstrated this experience using 2 projects, 1 of which was a project for the City of Calgary and did not satisfy the criterion. A review of the proposed resource’s resume demonstrated that this individual did not possess experience within the last five years working on two projects within a Canadian federal government organization or agency, and should not have been accepted as a resource.
- In 2 files partial points were awarded by evaluators, which deviates from the planned approach. Details regarding how points will be awarded by evaluators should be disclosed to bidders and should be clear to evaluators to support consistency in the scoring approach. The Canadian International Trade Tribunal (CITT) has ruled that partial points may not be awarded unless specifically contemplated in the solicitation document; the evaluation must be conducted according to the methodology established in the RFP.
IRCC should ensure that evaluations adhere strictly to the evaluation criteria and are carried out in accordance with the planned approach.
Overall file documentation was exceptionally good
38. File documentation was reviewed to determine whether a complete audit trail was retained to support consistent and transparent decision-making. Overall, IRCC maintained detailed records of all procurements.
39. IRCC was given an opportunity to provide OPO with additional documentation that may have been missing from the initial file disclosure. Of the 40 files that were reviewed, only 3 documentation issues could not be resolved. These included 1 instance where it was unclear that unsuccessful bidders had been notified; 1 instance where the invitation to tender was on file, but names of the suppliers were not visible (Blind carbon copy (BCC)); and 1 instance where the file did not contain the Centralized Professional Services System (CPSS) Client Module generated list of the selected suppliers.
40. IRCC has developed and has consistently implemented several tools to ensure that procurement files contain adequate records of decisions to facilitate management oversight and maintain an adequate audit trail. Some examples of note include:
- Contract Planning and Advanced Approval (CPAA) document: This document provides a comprehensive record of all information pertinent to decision-making, and serves to document the approval of key decisions, such as the approval of procurement strategy and any contract amendments. The CPAA includes information such as a summary of the requirement, related contract history, any contract amendments, summary of risk as well as general remarks from the Contracting Officer.
- Risk Assessment Tool: This tool is intended to assist Contracting Officers in the assessment of risks and to support the development of recommendations in the CPAA. The results of the risk assessment determine the level of approval required for the procurement. For example, a low to medium risk rating requires signature from the delegated contract authority, whereas a high risk rating prompts review and approval by IRCC’s Procurement and Contracting Oversight Committee (PCOC). Some examples where PCOC review is required include all sole-source contracts greater than 10k, unplanned contract amendments where the total value exceeds 50% of the planned contract, and where there is a second (and all subsequent) unplanned amendment.
- Evaluation Summary Report: This report is completed by the Contracting Officer following the conclusion of the bid evaluation process. It provides a comprehensive record of the results of the evaluation including the time each bid was received, the results of the technical and financial evaluation, and the recommendation for contract award. This document is signed and saved to file by the Contracting Officer.
- Procurement File Checklist: This checklist includes a detailed list of all steps and considerations for each procurement phase: pre-contracting, contracting, contract management and close out. The Checklist is saved to each procurement file and requires the Contracting Officer to complete each step by checking “Yes”, “No”, or “N/A” depending on the requirement and method of solicitation. A common element across all phases is the prompt to save all pertinent correspondence and decision making emails to file.
41. OPO regularly hears from both Canadian businesses and federal officials who believe the contracting process is unnecessarily complex. In reviewing IRCC’s procurement practices, OPO sought to identify opportunities to alleviate unnecessary administrative burdens placed on bidders and federal procurement officials, and draw attention to good practices for simplifying the procurement process.
42. In 1 file the solicitation bundled three distinct projects to simplify the contract management as the same Technical Authority was responsible for managing all 3 projects. Bundling similar requirements can often have the effect of simplifying the procurement process for both suppliers and buyers. In this case, bundling of these requirements may have made it challenging for smaller firms to compete, as bidders needed to be successful in qualifying highly specialised resources in all categories in order to be successful. Alternatively, the solicitation could have been structured to allow for the award of up to 3 contracts. This method would have allowed IRCC to benefit from the increased efficiency of a solicitation process that bundled multiple projects, without restricting competition. OPO did observe at least 2 instances where IRCC bundled similar requirements. This practice reduces the amount of time and effort required to run a solicitation process for IRCC, and to respond to it on the part of suppliers.
43. As noted under LOE 1, multiple instances were noted in which evaluation criteria were either inadequately defined, or included factors that were not essential to fulfilling the requirement. These evaluation criteria are often unnecessary and can make it administratively burdensome for bidders to prepare proposals, while increasing opportunities for mistakes. The inclusion of an excessive number of mandatory criteria can similarly increase the administrative burden on suppliers, and may result in reduced competition. For example, 1 solicitation to develop e-learning products included 45 mandatory criteria each asking for specific projects, dates, tasks, etc. While these amounted to roughly 5-6 mandatories per resource, which is not unreasonable, it does make for a significant amount of work for bidders to prepare a proposal. This may have limited the number of bidders, as evidenced by the solicitation process resulting in only 1 bid. Limiting criteria to essential qualifications, while ensuring that they are communicated in a clear, precise and measurable manner supports the principle of simplification.
44. In 1 instance, efforts undertaken by the Contracting Officer during the planning stage supported the principle of simplification. The Contracting Officer examined questions from bidders from a previous process to enhance the clarity of the solicitation and reduce the amount of Q&A during the solicitation period. In the same process, the Contracting Officer elected to remove the rated criteria on the rationale that including rated criteria for solicitations with a financial limitation increases the cost of preparing a proposal and may discourage bidders from participating.
45. Opportunities for simplification extend to the manner in which information is communicated to prospective bidders. For example, in 1 file, mandatory criteria were presented in a bulleted list in the body of the RFP, instead of the traditional grid format, and one bid was declared non-responsive because the bidder missed the mandatory criteria and thought they only had to submit a quote.
46. Similarly, the format that is used to respond to questions during the solicitation period can also be simplified to reduce the risk of error. One effective example of this was witnessed during the course of this review, wherein the Contracting Officer used a single document to track questions and answers during the solicitation period. As questions were submitted, a single Q&A document was amended. As a result, bidders needed to consult only 1 document for all of the information needed to prepare a responsive proposal. This eliminates the risk that a bidder may unknowingly miss one of the responses were they provided in a piecemeal fashion via separate amendments to the RFP.
47. Finally, another effective example of simplification that was witnessed included the requirement that bidders provide a reference letter or client satisfaction survey with their bid. This practice simplifies the more common approach to references, which is to require bidders to provide contact information for each of the projects they identify in their proposal. Depending on the number of references requested, this can sometimes require bidders to provide up to date contact information for references on projects dating back several years. This increases the level of effort and time required to prepare proposals. On the evaluation side, the process of contacting these references to validate the information provided in each bidder’s proposal can be quite time consuming. The practice of requiring a reference letter or client satisfaction survey, rather than requesting contact information for references, makes it easier for bidders to prepare their proposals, expedites the evaluation process for buyers, and removes the risk that the information provided by a reference may be used to repair the bid.
48. IRCC’s procurement practices pertaining to evaluation and selection plans, solicitation, and evaluation of bids and contract award were assessed for consistency with Canada’s obligations under applicable sections of national and international trade agreements, the Financial Administration Act (FAA) and regulations made under it, the TBCP and departmental guidelines, and to determine if they supported the principles of fairness, openness and transparency.
49. The solicitation documents reviewed consistently identified evaluation criteria and the selection method to determine the successful bid; however, in some limited circumstances, evaluation criteria were not communicated in a clear, precise and measurable manner.
50. The design and execution of competitive solicitation processes were consistent with applicable rules, regulations and policies in all but 1 file. However, in some limited circumstances, communications with suppliers lacked transparency, both during the solicitation period and when advising bidders of the evaluation results via regret letters.
51. IRCC has established a standard process for evaluating proposals complete with detailed guidance for evaluators. However, in some limited circumstances, evaluations were not consistently carried out in accordance with the planned approach and in two files, contracts were awarded to non-responsive bidders.
52. In order to address issues identified, OPO made 3 recommendations. These recommendations can be found in Annex I of this report. OPO will conduct a follow-up review in 2 years to assess the implementation of the IRCC’s action plan to address these recommendations.
VI. Organizational response
53. IRCC would like to thank the Office of the Procurement Ombudsman for conducting this independent and objective examination of IRCC procurement practices. IRCC recognizes the importance of sound and rigorous procurement practice supported by fairness, openness, and transparency in delivering our mandate.
54. We have carefully examined the draft report and are in agreement with the content of the report and recommendations. OPO feedback will be a valuable tool in our continuing approach to improve our processes. As such, IRCC will address all recommendations and is committed to the action plan stated in the Annex I of the report.
55. OPO wishes to express its appreciation to the management and staff of IRCC for the assistance and cooperation extended to the reviewers during this assessment.
Annex I: Organizational response and action plan
Procurement practice review of evaluation and selection plans, solicitation, and evaluation of bids and contract award at Immigration, Refugees and Citizenship Canada.
|Number||Recommendation||IRCC response/ Action plan||Timeline for implementation|
|1||IRCC should establish a quality control process to ensure mandatory criteria are adequately defined and communicated in a clear, precise and measurable manner, and that mandatory criteria are limited to the essential qualifications necessary to fulfill the requirement.||IRCC agrees with the recommendation and will update its solicitation procedures to reinforce its quality control process by transitioning current review best practices into a mandatory contracting officer peer review of solicitation documents. Among other aspects, this peer review will seek to ensure that mandatory criteria are adequately defined and clearly communicated, and that these criteria are limited to the essential qualifications necessary to fulfill the requirement.||Q1 2022-2023|
|2||IRCC should ensure that the information that is communicated to suppliers both during and after the solicitation process is accurate and that adequate documentation of these communications is retained to facilitate management oversight.||IRCC agrees with the recommendation and will update its solicitation procedures with communications standard templates and protocol for contracting officers. They will notably ensure the information that is communicated to suppliers both during and after the solicitation process is accurate, and that adequate documentation of these communications is retained. IRCC will also develop a protocol for contracting officers to record on file relevant events such as meetings and phone calls throughout the life-cycle of the contract. Once developed, contracting officers will be trained on new protocol.||Q1 2022-2023|
|3||IRCC should ensure that evaluations adhere strictly to the evaluation criteria and are carried out in accordance with the planned approach.||IRCC will update its bid evaluation procedures to include roles and responsibilities for Contracting Officer’s during bid evaluations and consensus including a dispute resolution/escalation process to management and the Procurement and Contracting Oversight Committee. This will aim to ensure that evaluations always adhere strictly to the evaluation criteria and are carried out in accordance with the planned approach. Once developed, contracting officers will be trained on new procedures.||Q2 2022-2023|
- Date modified: