Procurement practice review of the National Research Council Canada

Mandatory criteria

Mandatory criteria were not overly restrictive and, for the most part, were aligned with requirements; however, a majority of files did not define mandatory criteria in a clear, precise and measurable manner

18. Of the 24 files that contained mandatory criteria, these criteria were not overly restrictive, did not unnecessarily favor or penalize any particular bidder, and in all but 3 cases, were aligned with requirements. However, in 14 of 24 files, mandatory criteria were not communicated in a clear, precise or measurable manner. Such evaluation criteria can: undermine the transparency of the bid solicitation process; cause bidders to submit non-compliant or sub-optimal proposals; and result in unexpected or inconsistent evaluations between evaluators. Examples of unclear mandatory criteria included the following:

1. In 5 files, mandatory criteria included non-mandatory language when describing the requirement (for example, using the word “should” instead of the word “must”), making it unclear whether the criteria would be met if the requested information was not provided or if the requirement was not met. For example, in a file for the services of a business transformation architect, a mandatory criterion stated that the “resource should be certified in ADKAR Management of Change”. Using non-mandatory language such as the word “should”, rather than “must”, results in less certainty for suppliers and evaluators to determine what aspects of the requirement must be met to be considered compliant. To avoid confusion, only mandatory language should be used when describing mandatory requirements.
2. In 6 files, mandatory criteria required bidders to demonstrate ‘experience’ but did not specify how much experience was required. As a result, any amount of experience would have resulted in the criteria being met. For example, in a file for the services of a software architect, a mandatory criterion required “experience mapping Microsoft Dynamics CRM OOB functionality to business requirements, utilizing multiple modules (Sales, Marketing, Case Management, Unified Service Desk) within Dynamics.” However, the amount of experience required was not specified in the criterion. As a result, as little as one day of experience would have resulted in the criterion being met. 2 additional mandatory criteria in this file also failed to specify the amount of experience required. In such circumstances, the requirement should be tailored to specify the amount of experience that is essential to perform the work.
3. In 5 files, NRC should have been clearer in defining the mandatory criteria. For example, in a file for the development of a technical housing guide, 3 of the mandatory criteria required demonstration of experience by providing examples of projects or previously published guides, but it was not clear whether these criteria applied to the bidder (firm) or to the resources to be proposed. Another mandatory criterion required the bidder to “identify personnel who are participating in the development of the current National Building Code, National Fire Code and Canadian Standards as part of the proposed team.” However, the number of personnel to be proposed to be considered compliant was not specified. In another mandatory criterion, bidders had to submit curriculum vitae (CV)s of engineering staff demonstrating 10 years of experience. The criterion further specified that a minimum of 1 CV for each of the 6 engineering disciplines listed in the criterion was required. As the number of required resources wasn’t specified, it wasn’t clear if a minimum of 6 resources had to be proposed, each with 10 years of experience in their specified discipline, or if fewer resources with experience in multiple disciplines could be proposed. It cannot be left up to bidders to guess what is acceptable nor for evaluators to determine if the response noted in the bid is “acceptable” based on their individual subjective interpretations of the mandatory criteria.

19. In 4 files, the solicitations contained mandatory criteria that could not be evaluated on a pass or fail basis at the time of bid closing. For example in a file for janitorial services, the solicitation included mandatory criteria requiring that bidders “provide notification to building management of any cleaning products used in the building”, “communicate to the management or owners of the building the presence of pests and any maintenance issues discovered while performing cleaning operations”, and perform other tasks such as requesting that facility managers identify building occupants with sensitivities. Such requirements belong in the contract awarded to the successful bidder as they relate to the performance of the required services in the future, whereas mandatory criteria are used to confirm skills or experience held by the supplier at the time of bid closing, prior to contract award.

20. In 3 files, mandatory criteria were not in alignment with requirements, resulting in mandatory criteria that were not significant to the success of the project as described in the solicitation. For example, in a file for environmental services, the Statement of Work (SOW) specified that the field technician must have a minimum of 3 years of experience in environmental site assessment under the Canadian Standards Association standard. This mandatory experience requirement was not assessed in mandatory criteria. As another example, in a file for database modelling consulting services, the SOW indicated that the proposed resource “must have a minimum of 8-10 years [experience] of which five (5) must be with the federal government, of demonstrated experience…” in 12 different areas. The experience requirements in the mandatory criteria for this solicitation did not correspond to those listed in the SOW. For example, mandatory criteria required 2 years of general experience within government (whereas the SOW required 5 years in all areas specified) and other criteria required more or less experience than the amounts indicated in the SOW. Since the purpose of a SOW is to describe the work required, criteria identifying minimum bidding requirements (such as the experience requirements in the examples above) should not be found in a SOW. Further, including bidding requirements in the SOW brings on issues of consistency and efficiency. For example, when amending criteria, multiple sections of solicitations need to be reviewed to ensure there is no conflicting or contradictory information. This is also inefficient since bidders and evaluators need to look to multiple areas of a solicitation to view all mandatory criteria.

Point-rated criteria

Point-rated criteria and rating scales were not overly restrictive, were appropriate to the requirement, and reflected the relative importance of the criteria; however, most files contained point-rated criteria and scoring grids that lacked clarity and were unnecessarily subjective

21. Section 10.7.25 of the TBCP states that “criteria should identify accurately all the performance elements significant to the success of the project and should measure both the competence of the firm and the worth of its particular technical approach.”

22. In all 22 files that contained point-rated criteria, these were not overly restrictive and were appropriate to the requirement. However, in 19 files, the criteria or scoring grid was not communicated in a clear and precise manner to minimize unnecessary subjectivity.

23. Examples of point-rated criteria or scoring grids containing incomplete or unclear information included the following:

1. In 15 files, point-rated criteria or scoring grids used undefined terms or were otherwise not clearly written. For example, in a file for audit services, certain criteria awarded points based on whether the bidder’s internal process was “sound”, whether the number of back-up resources was “reasonable” or whether the proposed plan was “plausible”, without defining these terms. As another example, in a file for recruitment services, a point-rated criterion required bidders to “provide average time to complete search service request as well as timelines for all associated activities.” This criterion was worth 15 points but the scoring grid only indicated “Poor”, “Average” and “Excellent” with no associated number of points for each category or timeline details (for example hours or days) demonstrating where the band for each category begun and ended.
2. In 7 files, solicitations provided no scoring breakdown for some or all point-rated criteria. For example, a file for a senior project leader using the ProServices SA included 4 point-rated criteria worth 10 points each requesting that bidders describe their experience in different areas. However, there was no scoring breakdown detailing how points would be awarded within each criteria. Further, the flexible grid for the level of resource required was not included in the solicitation (i.e. Stream 10.3 Project Leader/Executive). Under the ProServices SA, NRC was required to include the applicable flexible grid to ensure the proposed resource met the minimum level of points by evaluating education, professional certification and relevant experience.
3. In 3 files, point rated criteria contained definitive language (i.e. “must”), making their evaluation as point-rated criteria unclear. For example, in a file for the services of a business transformation architect, a rated criterion stated that the proposed resource “must have 8+ years of experience…”, contradicting the scoring breakdown which still awarded points for 5 to 8 years of experience. To avoid confusion, mandatory language should only be used when describing mandatory requirements.

24. When evaluation criteria are communicated in a clear, precise and measurable manner, it enables bidders to know the requirements and the methods by which their proposals will be evaluated. Failure to adequately define evaluation criteria at the outset carries the additional risk that evaluators may struggle to interpret these criteria during the evaluation process. It can also be difficult to defend against external challenges to the evaluation process, as it is more difficult to demonstrate that criteria have been strictly adhered to when the criteria are unclear and open to multiple interpretations.

Selection methodology

The selection methodology was clearly communicated and aligned with the requirement in most solicitations, with some exceptions

25. In almost all files, common selection methodologies were used such as “lowest priced responsive bid” and “highest responsive combined rating of technical merit and price”. However, there were 6 files that did not explain the selection methodology in the solicitation and in 1 file, the selection methodology was not aligned with the requirement. Details are presented below. Further, in 1 file, NRC could not produce the RFP and OPO could therefore not confirm whether the selection methodology was communicated in the solicitation.

26. In 6 RFQ files, there was no mention of the selection methodology (for example “lowest priced responsive proposal”) in the solicitation and there is no evidence solicited suppliers were informed of the competitive nature of the process. For example, in a file where quotes were requested for the purchase of gas chromatography systems, neither the solicitation email or document informed suppliers how the winning bid would be chosen or that the process was competitive. 5 of these files also contained deficiencies including the lack of a complete description of the requirements and instructions to bidders.

27. In 1 file for construction materials plant inspection services, the selection methodology used was not aligned to the requirement. In this RFSO, a selection methodology of highest rated responsive proposal based on technical merit was used. The solicitation document also stated up to 10 standing offers would be issued. A financial criterion included in the point-rated technical criteria table awarded 20 points to the lowest cost proposal, 10 points to the second lowest cost proposal and 5 points to the third lowest cost proposal. As this approach does not consider cost proposals after the third lowest cost proposal, it does not appear aligned with the intention to issue up to 10 standing offers. A more appropriate approach would have been to use a basis of selection of highest combined rating of technical merit and price, or to allocate points for the 4th through 10th lowest cost proposals. Further, financial criteria should not be found with technical evaluation criteria as this threatens the integrity of the evaluation process by blurring the separation that should exist between technical and financial evaluations.

Solicitation documents and processes

Solicitation documentation was in most instances complete; however, several issues were noted regarding providing a clear description of the requirements and instructions for submitting bids

31. In 25 of the 40 applicable files, solicitation documents reviewed included a clear description of the requirement, although as noted in paragraph 17, there were 4 files, which appeared to be RFQs, where NRC could not produce solicitation documentation. There were also 3 RFP files where NRC could not produce solicitation documentation. The remaining 8 files were found lacking in this area. Further, 5 files did not contain instructions for submitting bids. Details from OPO’s review of solicitation documents are presented below.

32. In 5 files, there were no communicated solicitation periods or instructions for submitting bids and in 2 of these files, the description of the requirements was incomplete. All 5 files were informal RFQ processes with limited information on file. For example, in a file for invasive plant removal services, an email on file provided a summary of the process but since the process appears to have been conducted by telephone or otherwise not documented, the description of the requirement and any instructions for submitting bids provided could not be reviewed. As another example, in a solicitation for the programming of a data collection application, some technical specifications were provided to potential bidders but the requirement was also discussed in separate conference calls with each supplier. As such, OPO could not confirm what details regarding the requirement were provided to each supplier during these calls or if the same information was provided. Further, the solicitation email briefly described the requirement and requested availability for a call to discuss further but did not establish a bid closing date, provide information as to where and how to submit bids or indicate a format or timeframe for posing questions. In this case, it appears NRC was negotiating with 2 suppliers since multiple amended quotes were accepted from both suppliers over a 5-month period. Section 10.6.6 of the TBCP stipulates that “[w]hen negotiating with more than one firm, care should be taken that all are treated fairly and impartially. The negotiations should not become an auction of the contract, as firms progressively improve their proposals in the light of information about the position of other firms. The confidentiality of each firm’s negotiating position is to be assured.” File documentation did not permit OPO to confirm that bidders were treated fairly and impartially and, based on the documentation reviewed, this file raises concerns regarding fairness and transparency.

33. In 1 file for construction materials plant inspection services, the requirement was not clearly described. In this case, the SOW indicated that completing an inspection entailed completing a plant inspection form and following inspection requirements and that both documents were “attached” to the SOW, however these documents were not found in the solicitation document or published with the tender notice.

34. In 5 files, security requirements were not clearly communicated to suppliers. According to the Treasury Board Secretariat’s Instructions for completion of a Security Requirements Check List (SRCL), in instances where a procurement contains “Protected” and/or “Classified” requirements, requisitions and subsequent tender and contractual documents must be accompanied by a completed SRCL. In 3 files, a security requirement was identified in the solicitation document but there was no SRCL in the solicitation, resulting contract or otherwise on file. In 2 other files, a SRCL was on file but it was not included in the solicitation or resulting contract.

35. When using PSPC’s SA for Laboratory and Scientific Equipment (E60PV-19EQUI), user departments are required to use the simple, medium or high complexity bid solicitation templates based on the estimated dollar value and complexity of the requirement. In 1 file that used this SA that resulted in an awarded contract valued at $79,310.61, NRC did not follow all bid solicitation documentation requirements. In this case, there was no estimate of the value of the requirement on file and NRC did not use a PSPC bid solicitation template (it appears the medium complexity template should have been used). As noted in paragraph 40 below, estimates should be established and documented before soliciting bids. Further, the solicitation document did not include the required Standard Instructions, evaluation procedures and basis of selection, certifications (i.e. Integrity Provisions) or the conditions of the resulting contract.

36. Ensuring information is accurate and complete is important as it eliminates ambiguities, contradictions or other discrepancies in solicitation documents. This helps ensure suppliers have the information they need to prepare and submit responsive bids and helps reduce the number of questions from suppliers regarding such discrepancies. In turn, this reduces the effort required on the part of the department to respond to these questions during the solicitation period.

Most solicitations were open to an appropriate number of suppliers and for an appropriate period of time, with exceptions

37. When PSPC’s methods of supply are used, such as the Professional Audit Support Services (PASS) or Task-Based Informatics Professional Services (TBIPS) SAs, certain procedures must be followed including performing a search in the Centralized Professional Services System (CPSS) to identify the qualified suppliers to be invited to bid. In the case of requirements valued above the Canada-Korea Free Trade Agreement (CKFTA) threshold—or North American Free Trade Agreement (NAFTA) threshold for solicitations prior to July 1, 2020—a Notice of Proposed Procurement (NPP) must also be published on the government electronic tendering service (BuyandSell.gc.ca) and suppliers must be provided with a minimum of 15 calendar days to bid. Similarly, when the estimated price is greater than $25,000 and less than $400,000, PSPC’s SA for Laboratory and Scientific Equipment (E60PV-19EQUI) requires user-departments to consult the list of categories to determine which qualified suppliers to be invited (minimum of 3), publish a NPP on BuyandSell.gc.ca and provide suppliers with a minimum of 15 calendar days to bid.

38. In 8 files where suppliers were solicited by direct invitation, OPO could not confirm the number of invited suppliers as NRC could not produce the solicitation emails. In 5 of these files, OPO could also not confirm the duration of the solicitation period. In 1 file, which used the Laboratory and Scientific Equipment SA, it appears only 1 supplier was solicited. Under this SA, this is allowable when the estimated value of a requirement is less than or equal to $25,000. In this case, the estimate was $32,644, as evidenced in a requisition predating the supplier’s quote. As such, a minimum of 3 suppliers were required to be invited to bid.

39. In 4 files where the TBIPS or Laboratory and Scientific Equipment SAs were used, NRC could not provide evidence that it had conducted the CPSS searches or, in the case of the Laboratory and Scientific Equipment SA, consulted the list of categories to identify the qualified suppliers to be invited. In 1 file that used the Laboratory and Scientific Equipment SA, the bid solicitation period was 8 calendar days which did not meet the required minimum of 15 calendar days. Further, in 5 files where the PASS or TBIPS SAs were used for requirements above the CKFTA or NAFTA thresholds or where the Laboratory and Scientific Equipment SA was used, a NPP was not published on Buyandsell.gc.ca as required. There were no documented rationales on file as to why NPPs were not published in these instances.

40. In 1 file for aircraft engine maintenance services, quotes were requested by email from 3 suppliers with 1 supplier that responded. No solicitation period was established and the resulting contract was valued at $146,680.40 USD. There was also no indication in the file that an estimate had been established before bids were solicited. Section 10.5.1 of the TBCP stipulates when estimates should be established, and reads in part, “[r]equirements should be defined and specifications and estimates established before bids are solicited and contracts let, so that all prospective contractors are treated equally.” Since the estimated value of a requirement helps inform what tendering process must be followed, estimates should be established and documented before soliciting bids. In this case, given the dollar value and nature of the requirement, it is likely the Canadian Free Trade Agreement (CFTA) and NAFTA, which was in force at the time, applied to this procurement. If these agreements did apply, then this process should have been an open tender with a solicitation period of at least 40 days. As it was not an open tender, the openness of this process was negatively impacted by not advertising this opportunity publicly for potential bidders.

41. In 1 file, NRC did not run a fair and transparent process. In a file for language training services, a previously awarded contract was already in place for semi-private learning services with the selected supplier but there was a need to change to private lessons and add additional hours. A quote of $32,660 dated January 17, 2020 was provided by the selected supplier/ incumbent for the updated requirement. The Contracting Authority instructed an NRC employee to request an additional quote from a specific supplier so that the process would become competitive and would not require a sole source justification. There is also an email on file outlining the benefits of continuing to obtain the services from the incumbent. On January 24, a quote was requested by email from the other supplier and a quote was received on the same day. The solicitation email requested a quote and briefly identified the language training requirement but provided no other information or instructions for submitting bids (for example bid closing date, bid format, etc.). File documentation does not support that the second supplier solicited had a fair chance at being awarded the contract. Inviting a second supplier in the manner described above is a breach of section 2 of the TBCP which requires that government contracting be conducted in a manner that will “stand the test of public scrutiny in matters of prudence and probity, facilitate access, encourage competition, and reflect fairness in the spending of public funds.”

Communication with suppliers

Most communications with suppliers were appropriate and supported the preparation of responsive bids; however, issues regarding fairness, documentation and the communication of solicitation results were found

42. Section 2 of the TBCP states: “Government contracting shall be conducted in a manner that will: (a) stand the test of public scrutiny in matters of prudence and probity, facilitate access, encourage competition, and reflect fairness in the spending of public funds.” Section 12.3.1 of the TBCP also states: “Procurement files shall be established and structured to facilitate management oversight with a complete audit trail that contains contracting details related to relevant communications and decisions…” These requirements apply to all aspects of the procurement process, including interactions with suppliers.

43. In 20 of 24 applicable files (for which there were communications with suppliers during the solicitation period), communications with suppliers supported the preparation of responsive bids. However, this was not the case in the 4 remaining files, for example:

1. In 1 file for environmental services, a supplier requested clarifications regarding how points would be distributed for 2 point-rated criteria (no scoring grid was included in the solicitation document). NRC responded that the point distribution would be based on the descriptions provided in the solicitation, which did not address the supplier’s question and provided no additional clarity to help suppliers understand how their bids would be evaluated. As stated in section 10.7.27 of the TBCP: “Competing firms should be told the measurement criteria and the weighting assigned to them. …The courts have ruled that the factors and their weighting must be established beforehand and adhered to strictly.”
2. In 1 file for the purchase of gas chromatography systems, several questions were asked by 2 of the 5 invited suppliers. The responses provided clarification on the requirement and terms and conditions applicable to the resulting contract. Responses were sent to the suppliers asking the questions but there is no evidence that all responses were shared with all invited suppliers.
3. In 1 file for the purchase of laboratory refrigerators and freezers, the successful supplier’s bid indicated communications between NRC and the supplier during the solicitation period regarding a modified delivery date and allowing a financial proposal that did not use the required Basis of Payment form from the solicitation. Other than the references to these communications in the successful supplier’s bid, there was no evidence on file of these enquiries or responses to this supplier or other invited suppliers.

44. The record of communications with suppliers during the solicitation period was also found to be incomplete in 6 files. This included missing documentation showing when all questions were asked and by whom. Without a complete record of communications, OPO is unable to determine if all suppliers’ questions were answered, answered in a timely manner, and whether all suppliers were treated equally.

45. Further, communication of solicitation results did not occur in all applicable files. In 6 files where the solicitation was posted on Buyandsell.gc.ca, a contract award notice was not published as required. In 1 file, a contract award notice that was required to be published no later than 72 days after contract award was published 174 days after contract award. Further, in 10 files, there was no evidence any regret letters were sent to unsuccessful suppliers. As stipulated in NRC’s RFP checklist document, contracting authorities are required to publish contract award notices and issue regret letters to unsuccessful bidders. However, this RFP checklist was not observed in any of the files reviewed. Regret letters are an important tool as they alert bidders to the fact that they were unsuccessful in the selection process, should they wish to seek a detailed debriefing from the contracting department or avail themselves of potential recourse mechanisms such as OPO or the Canadian International Trade Tribunal.

Bid evaluation

Inconsistencies in the evaluation of bids and deviations from the planned approach were noted and in 3 instances a contract or standing offer was incorrectly awarded or issued

48. The requirements of the TBCP referenced above emphasize the importance of having a well-planned and documented process for bid evaluations. For a majority of the 25 files reviewed that included a technical evaluation, OPO observed that an evaluation plan or instructions to assess bids was provided to evaluators, evaluation grids were used with criteria that matched those found in the solicitation document, and the number and identities of the evaluators had been documented in the file. A notable good practice found at NRC is the requirement to have each member of the evaluation team sign an Evaluation Team Sign-off Form, acknowledging their responsibilities to protect the confidentiality and integrity of the process and certifying that there is no conflict of interest in participating as an evaluator. However, in 7 files, there was no evidence that this form was sent to evaluators for signature and in 4 files where the form was sent, there was no evidence that it was signed by all evaluators.

49. In 25 of 37 files reviewed, excluding the 3 call-ups issued under standing offers, technical and financial bids were evaluated in accordance with the planned approach. However in several files, addressed in the documentation section below, the absence of key documents including the solicitation and complete bid evaluations meant file documentation did not always support the evaluation of bids and the selection of the successful supplier. There were also multiple instances in which the evaluation was not carried out in accordance with the solicitation, as presented below.

50. In 2 files, non-compliant bids were accepted and a contract was awarded or standing offer issued to a non-compliant bidder. In another file, an error made in the financial evaluation resulted in a standing offer being issued to the wrong bidder:

1. In a RFSO for fume hood performance testing services, a mandatory criterion required bidders to be certified by a recognized testing, adjusting and balancing organization and provide a copy of the certification. The solicitation document also stated that proposals failing to meet the mandatory criteria would be deemed non-responsive. The evaluations completed stated that the successful supplier did not provide the required certificate. The evaluations also noted that 3 other bidders also did not provide the requested certificate as part of the same or other mandatory criteria requiring certification. While only 1 of 5 bids appears to have met all mandatory criteria, all 5 bidders were deemed responsive since they met “the intent of the technical requirements”. Section 10.8.7 of the TBCP provides for instances where bids “that respond to mandatory requirements but contain a minor aberration may be considered…” However, as providing the requested certificates was a mandatory requirement, NRC was required to deem bidders not meeting all mandatory criteria non-responsive.
2. In a file for building cleaning services, several trade agreements applied (for example NAFTA, CFTA, etc.) and an open tendering process was used. Bidder instructions required bidders to submit their proposals electronically no later than the time and date indicated and that no supplementary information would be accepted after the closing deadline unless NRC requested a clarification. The solicitation also had a security requirement with a mandatory criterion requiring that bidders hold a valid Designated Organization Screening (DOS) at bid closing. The only bid received was received after the bid closing date and did not demonstrate a valid DOS. An email on file explained that since this was the only bid received, it was accepted late and the security requirement was waived. A contract should not have been awarded as part of this process since the solicitation closed with no responses at the time of bid closing, and because a mandatory criterion was not met. As stated in the solicitation’s Standard Instructions and Conditions, NRC’s policy was “to return, unopened, bids delivered after the stipulated bid solicitation closing date and time”, which was not done in this case.
3. In a RFSO for architectural services, 5 standing offers were to be issued based on the 5 highest scoring bids using a selection methodology of highest combined rating of technical merit and price. Regarding price, the total weighted rate to be provided in financial bids consisted of the sum of resource hourly rates for each year requested multiplied by the factor indicated. In the financial evaluation, the total weighted rate used for one of the bidders was incorrectly input as $100 an hour less than the total weighted rate found in the corresponding financial bid. This bidder was ranked 5th and was issued a standing offer. With the rate corrected, this bidder ranks 6th and a different bidder should have been issued a standing offer.

51. In 3 files, the evaluation grid used by evaluators was not identical to the criteria found in the solicitation document. For example, in 1 file for construction materials plant inspection services, a point-rated criterion which was to assess the bidder’s understanding and ability to perform the required services was not included in the evaluation grid used by evaluators and was not evaluated.

52. In 1 file for smart building commissioning and monitoring services, certain evaluator comments indicated that bids were compared against each other instead of against the evaluation criteria. Instructions provided to evaluators noted that bids were to be evaluated on how they respond to the solicitation, using only the evaluation criteria and weightings detailed in the solicitation document.

53. In 1 file for architectural services, the evaluations of certain bids for a point-rated criterion noted that some or all references could not be reached while others were silent as to whether references were contacted or not. As such, the evaluations did not support that reference checks were consistently performed and evaluator comments did not provide the rationale of the point-rating for this criterion. In addition to a lack of consistency and fairness with this approach, if references were not verified for certain bidders, this may have placed an unnecessary burden on bidders that were required to provide up to date contact information for project references dating back up to 5 years, increasing the level of effort and time required to prepare proposals.

54. Evaluations not conducted consistently and in the manner prescribed by the solicitation call into question the integrity of the procurement process. By not adhering strictly to the evaluation criteria, NRC has exposed itself to the risk that ambiguities in the selection process result in the contract being wrongly awarded. It also means NRC may not be able to defend and explain its decisions and actions and prove that they were made in accordance with the applicable laws, regulations, and policies.

File documentation

Procurement file documentation was incomplete

55. Section 12.3.1 of the TBCP requires that procurement files facilitate management oversight with a complete audit trail containing details related to relevant communications and decisions, including the identification of the involved officials and contracting approval authorities. The requirement to ensure adequate file documentation extends to the actions undertaken during the solicitation period, the evaluation of bids, as well as the overall procurement file.

56. File documentation was reviewed to determine whether a complete audit trail was retained to support consistent and transparent decision-making. As was highlighted throughout this report, there were many files in which key documents, including solicitations, correspondence with prospective bidders, regret letters, and Evaluation Team Sign-off Forms were missing.

57. Regarding technical evaluations, documentation was found to be incomplete in 13 of the 25 applicable files reviewed. This included instances where individual or consensus evaluations were not on file, where evaluations were incomplete (for example no or limited comments supporting the evaluations performed or no evidence supporting the contacting of references), and where consensus evaluations or evaluation summaries were not signed or dated. For example, in 1 file for impact analysis services, evaluations completed by 1 of 2 evaluators were not on file. The only evaluation document on file from 1 evaluator indicated that both bidders met mandatory criteria but the mandatory criteria were not listed in the evaluation and there’s no evidence they were evaluated. Further, the overall technical score achieved by each bidder was noted but no scores or comments were provided for the individual point-rated criteria.

58. Further, in 8 files, NRC could not provide some or all technical or financial bids and in 6 files, NRC could not provide evidence that bidders submitted proposals before the solicitation closing date and time. Other instances of incomplete documentation included missing confirmation of attendees for mandatory bidders’ conferences and missing confirmation of integrity verifications.

59. OPO considers the inability to demonstrate and document decisions to be a concern, and a breach of the TBCP and Treasury Board Information Management Guidelines. Without proper records, NRC cannot document its business decisions, inhibiting its ability to demonstrate good stewardship of Crown resources or that the procurements were conducted in a fair, open and transparent manner. Keeping complete and detailed evaluation records is crucial for demonstrating that evaluation criteria have been applied consistently to all competing bids, and demonstrating that the procurement has been carried out in a manner consistent with NRC’s obligations under the TBCP and applicable trade agreements.

60. Upon bringing the documentation issues to the attention of NRC officials during a review of preliminary observations, the response provided in certain instances was that documents could not be provided due to the departure of NRC employees during the pandemic and that inconsistencies with the use of the Evaluation Team Sign-off Form were amplified at a time when employees had to work from home and move from manual paper based processes to fully digital processes. NRC also noted that the review period included the first 18 months of the COVID-19 pandemic where employees were working remotely without fully developed and implemented digital procedures and were experiencing challenges with technology including digital signatures and electronic record keeping.

Establishing standing offer agreements

62. NRC frequently utilized RFSO processes to establish standing offer agreements with suppliers for various recurring requirements such as architectural, environmental, and inspection services. Call-ups against standing offers alleviate the need to conduct a full procurement process (development of bid evaluation criteria, solicitation period, evaluation of bids, etc.) since suppliers are pre-qualified and pricing and terms and conditions are pre-determined. Establishing standing offers and issuing call-ups for recurring needs promotes simplification and alleviates burdens on suppliers and federal procurement officials by reducing the cost and timeline implications associated with full procurement processes.

Service contract information form

63. NRC has developed and implemented a Service Contract Information Form as a best practice tool to assist procurement and project authorities when planning a procurement. The form captures various details regarding a requirement such as Statement of Work overview, estimated procurement values and dates, security requirements, mandatory PSPC methods of supply, basis of selection, intellectual property considerations, accessibility considerations, etc. When used, procurement officers will request that project authorities (i.e. requestors) complete and submit the form before the requirement proceeds to the solicitation stage. This promotes simplification by reducing the amount of back and forth between procurement and project authorities when defining requirements and determining procurement strategies. It also supports effective record-keeping and facilitates management oversight by capturing key planning details and decisions. Use of the form is not mandatory but an encouraged option made available to all NRC employees through the NRC intranet.

Standardization of procurement documents

64. Using standardized procurement documents contributes to greater simplification by ensuring consistency and uniformity across procurement processes. Certain NRC files reviewed were found to use non-standard procurement documents, which can place an unnecessary burden on bidders. Opportunities for simplification were also identified in the manner in which information is communicated to prospective bidders. Examples are detailed below:

1. In 2 files, information regarding the basis of selection, evaluation procedures or bidding instructions were provided in the SOW. For clarity and consistency, information provided in the SOW should be limited to the description of the requirement. Information such as bidding instructions, evaluation procedures and basis of selection should be found in their own sections in a solicitation document.
2. In 1 file for road repair services, the solicitation document presented clauses applicable to the solicitation mixed with clauses applicable to the resulting contract, resulting in a confusing presentation of terms and conditions. Terms and conditions applicable to the solicitation and those applicable to the resulting contract should be separate and clearly identified in solicitation documents.
3. In 2 files, the presentation of mandatory and point-rated criteria could also be simplified. For example, in a file for smart building commissioning and monitoring services, there was no separate SOW and evaluation criteria. The description of requirements and evaluation criteria were combined into one statement of requirements as opposed to the traditional approach of having a separate SOW and evaluation criteria table. This statement of requirements included over 100 mandatory criteria or sub-criteria along with approximately 50 point-rated criteria regarding various tasks, deliverables and project components which had to be demonstrated by bidders to receive consideration, making for a significant amount of work for bidders to prepare a proposal. Limiting criteria to essential qualifications, while ensuring that they are communicated in a clear, precise and measurable manner supports the principle of simplification.

Non-contracting officials conducting contracting activities

65. In 5 files, it appears that non-contracting NRC officials were engaging in contracting activities, up until the point of contract award. A number of avoidable errors were made during these solicitation processes that compromised the procurement principles of openness, fairness and transparency. Examples of this were previously noted in paragraph 32 and 40, as NRC did not provide any instructions for bid submission in its solicitation documents and did not provide a firm closing date for the submission of bids. Additionally, there was almost no formal documentation on these files and in most cases, whatever documentation there was, was sent to the Contracting Authority as a fait-accompli.

Emergency contracting

66. In 3 files related to the Government of Canada’s response to COVID-19, NRC advised OPO the procurements were treated under emergency contracting authorities it received from Treasury Board to combat COVID-19. File documentation to support the use of that emergency process was not consistent in the 3 files and not all trade agreement obligations were respected:

1. In 1 file, which used an open solicitation process with a solicitation period of 20 days, there was no indication of an emergency process on the file and the tender notice stated that several trade agreements applied, including the World Trade Organization Agreement on Government Procurement (WTO-AGP). While trade agreements do allow for modifications to standard procurement processes in certain circumstances, for example Article XI, paragraph 4c of the WTO-AGP requires an “urgency duly substantiated by the procuring entity” to reduce the solicitation period to no less than 10 days; such a substantiation was not on file. In addition, as an open solicitation process, a contract award notice was also required to be published in accordance with WTO-AGP Article XVI, paragraph 2; this was not done.
2. In the 2 other files, the solicitation document or tender notice indicated that the pressing emergency exclusion or exception had been invoked and that the procurement was “excluded from all of the obligations of all the trade agreements”. This statement is inaccurate. While there are exceptions to the requirement to solicit bids in a pressing emergency situation set out in the Government Contracts Regulations and several trade agreements, a department cannot set aside all trade agreement obligations unless the exception to competition has been triggered.

67. NRC’s wording that the procurements were “excluded from all of the obligations of all the trade agreements” is inaccurate. According to section 3.105.5e of PSPC’s Supply Manual and as noted in the COVID-19 Instructions to PSPC Buyers, such wording is to be used only when a national security exception (NSE) is invoked, which was not the case. It appears NRC may have interpreted this wording to apply to all COVID-19 related emergency procurements, even if a NSE did not apply. Care must be taken to not treat emergency procurements the same as procurements where a NSE is invoked as they each trigger different obligations.