Procurement practice review of Innovation, Science and Economic Development Canada

Mandatory criteria

Overall, mandatory criteria were not overly restrictive and were aligned with requirements. There were, however, a number of files that did not define mandatory criteria in a clear, precise and measurable manner.

17. In 29 of the 30 applicable files, mandatory criteria were aligned with the requirements as described in the solicitation. The 1 exception was a solicitation for services to understand Canada’s strategic advantage within critical mineral supply chains. The solicitation, which was subject to several trade agreements, included a mandatory criterion requiring bidders to have a physical presence/location in Canada. This criterion was not aligned with the requirements and provided preferential treatment to domestic suppliers over suppliers from other nations with whom Canada has signed trade agreements.

18. Mandatory criteria were clear, measurable and demonstrable in 23 of the 30 applicable files. However, in 7 files mandatory criteria were not communicated in a clear, precise or measurable manner. Such evaluation criteria can cause bidders to submit non-compliant or sub-optimal proposals, result in unexpected or inconsistent evaluations between evaluators and generally undermine the transparency of the bid solicitation process. Examples of unclear mandatory criteria included the following:

1. In a file for Information technology (IT) security vulnerability analysis specialist services, a mandatory criterion stated that the proposed resource must have “a university degree from a recognized university with acceptable specialization in computer science, IT security or in any other field relevant to the work to be performed or equivalent based on their education, training and/or experience.” There was, however, no indication as to what levels of education, training or experience would be required to be considered as “equivalent” to a university degree with acceptable specialization. OPO noted that ISED deemed that the eventual successful supplier met this mandatory criterion based on the undefined ‘equivalent to a university degree’ provision.
2. In another file, a solicitation for economic analysis services, a mandatory criterion required bidders to demonstrate “significant experience delivering presentations of developed products.” While the criterion did clarify that these were to be presentations to Senior Officials (Deputy Minister level or higher) or Chief Executive Officer (CEO)s, it did not specify how much experience was required to be considered “significant”, meaning that delivering as few as 2 presentations to Senior Officials would meet the requirement.
3. In a solicitation for protective apparel rental services, a mandatory criterion stated bidders “must supply all testing and safety information related to the fabric, construction, and design of apparel options” without specifying what was meant by “all testing and safety information.” In their consensus evaluation of the successful supplier’s bid, evaluators commented that it was “not clear if [there was] ARC protection for all items.” It should be noted that ARC protection refers to flash protective clothing.
4. In a solicitation for economic analysis services, it was not clear what was required for a bidder to meet two of the nine mandatory criteria, requiring a subjective assessment by evaluators. For these two criteria, bidders were required to demonstrate that planned resources and the proposed team collectively were qualified for the level of work proposed. The criteria stated, “considered criteria include education, professional designations, and work experience.” However, there was no indication as to the level of education required, which professional designations would be accepted or what work experience was required to meet these mandatory criteria.

Point rated criteria

With limited exceptions, point rated criteria were not overly restrictive and rating scales reflected their relative importance. Overall, point rated criteria were clearly communicated in solicitation documents; however, issues were identified with the associated scoring grids in one-third of applicable files.

19. In addition to Section 10.7.27 noted above, Section 10.7.25 of the TBCP states that “criteria should identify accurately all the performance elements significant to the success of the project and should measure both the competence of the firm and the worth of its particular technical approach.”

20. In 22 of 24 applicable files, the point rated criteria were not overly restrictive and did not unnecessarily favour or penalize particular bidders. In one file, however, a solicitation for evaluation consultant services issued under the PSPC ProServices supply arrangement, a point rated criterion was designed to favour repeat ISED suppliers over other qualified suppliers that had not previously performed work for the department. This criterion awarded points for each completed evaluation project, up to five projects, that the proposed resource performed for ISED. In another file, for IT project executive services, 2 of the 10 point rated criteria related to experience managing sprints and sprint planning sessions (a sprint is a component of agile project management). These two criteria included weighting scales that awarded the maximum points available for 36+ months of experience, which differed greatly from other rated criteria that required 14+ years of experience to achieve full points. The one bid received included a proposed resource who had been working at ISED at the time of the solicitation. It indicated the resource had 3 years and 4 months experience with sprint giving the impression that the two sprint-related point rated criteria rating scales were designed to favor, or at least not penalize, this resource.

21. In 21 of applicable 24 files, point rated criteria were appropriate to the requirements and weighting factors reflected their relative importance. The remaining 3 files had point weighting schemes that could yield irrational results. In these files, for certain criteria the points increased rapidly bringing into question the relationship between years and months of experience and points to be awarded. For example, in a solicitation for IT Help Desk specialist services, a point rated criterion indicated a proposed resource with 5 to 5.5 years of experience providing IT user support in MS Windows 7 or later operating systems would receive only 4/20 points, while a proposed resource with greater than 7 years of experience would receive the full 20/20 points available. As a result, as little as 18 months experience beyond 5.5 years yielded 5 times the points.

22. In a similar case, in its determination in P&L Communications Inc. v. Department of Public Works and Government Services (PWGSC) (PR-2005-026), the Canadian International Trade Tribunal (the Tribunal) found that the government “unduly and inappropriately allocate[d] points to bidders with more experience.” In that case, the Tribunal noted that for multiple categories used in the RFP, points per year in rating scales were more than, or almost double, for bidders that scored in the highest-rated level, compared to those that just met the threshold. (In the ISED example cited above the difference was five-fold.) The Tribunal was of the view that PWGSC had “properly demonstrated the importance of establishing minimum levels of experience to ensure that the chosen supplier has a satisfactory level of stability and experience. However, it [was] of the view that PWGSC [had] failed to explain how an additional year or an additional project or two could warrant the award of the number of points contemplated in the RFP.” The Tribunal provided an example, similar to the ISED IT Help Desk specialist services example above, where the relationship between years and months of experience and the points to be awarded could “illustrate the unreasonable results that would flow from the application of the … point-allocation scheme.”

23. In 22 of 24 applicable files, the point rated criteria were clearly communicated in solicitation documents. In the other two files, point rated criteria used ambiguous, unclear or non-specific language and in one of those cases an element evaluated for a point rated criterion was not disclosed to bidders. For example:

1. In a solicitation for the conduct of a price comparison study of telecommunication services, bidders were asked to "provide examples of projects for which the proposed project lead demonstrated professional work experience in producing analytical research reports on price comparisons of goods and services." Written comments from the bid evaluators indicated they only considered projects for which the proposed lead had also functioned as a project lead. That element, however, was not stated in the criterion, meaning the evaluators were applying an undisclosed criterion.
2. In a solicitation for services related to intellectual property in innovation programming, there were two similarly worded point rated criteria, one of which clearly stated what was required while the other was open to interpretation. For both criteria, bidders were required to provide project examples to demonstrate professional work experience synthesizing information on public sector intellectual property policy. While one of the criteria specified that bidders were to provide examples of “foreign (non-Canadian)” projects, the other did not state a preference for domestic or foreign projects. Presumably ISED intended to assess experience with different types of projects in each of these criteria, for example one assessing foreign (non-Canadian) project experience and the other assessing domestic (Canadian) project experience; however, that was not clearly stated. Given the lack of precision in one of the criteria, a bidder with only foreign (non-Canadian) experience could have submitted the same foreign projects for both criteria.

24. Of the 24 files with point rated criteria, 10 had issues with respect to clarity of the scoring grids, for example, limited or no breakdown for some or all point rated criteria. Examples of scoring grids containing unclear information included the following:

1. In a solicitation for spectrum auction verification services, it was not clear how the 30 possible points for a point rated criterion would be awarded. The criterion mentioned two elements: 1. demonstration of an understanding of the scope and objectives of the work; and 2. understanding of the clock auction format, clock round price incrementing rules and bid processing, and assignment stage winner and price determination. However, there was no indication how points were to be assigned for these two elements. The scoring grid stated that 27-30 points would be available if the bidder demonstrated an “excellent” understanding of the scope and objectives of the Work, 21-26 points for a “good” understanding or 0-20 points if the bidder “failed to demonstrate” an adequate understanding. However, it was unclear whether the maximum number of points available for each of the two elements was 15, or if a bidder could obtain as few as 0 points on one element and up to 30 points on the other. Details regarding how points will be awarded by evaluators should be disclosed to bidders in advance and should be clear to evaluators to support transparency and consistency in the evaluation process.
2. In a solicitation for IT security analysis services, a point rated criterion listed four areas in which the bidder’s proposed resource should have experience, including identifying gaps in cybersecurity processes and preparing IT security risk briefings. This was to be demonstrated by the bidder providing 3 projects showing where experience in those four areas was obtained. The criterion stated, “points will be allocated as follows: up to 10 points for each area, up to a maximum of 40 points.” ISED, however, did not provide an explanation to assist bidders or evaluators in assessing how the points were to be awarded. For example, did a resource require experience in all 4 areas in all 3 projects to obtain the maximum 40 points, or was it possible to obtain 40 points by: (a) demonstrating experience in all 4 areas, but less than 3 projects; or (b) demonstrating experience in all 3 projects, but less than 4 areas? Without a clear explanation of how points were to be allocated, potential bidders would not know which projects/ areas to identify, and evaluators would not know how to award points.
3. In a solicitation for spectrum auction services, bids were assessed against 8 point rated criteria labeled as RT1 – RT8. The first criterion (RT1) assessed past experience and was worth up to 40 points. For RT1, bidders were asked to provide information on three projects demonstrating relevant experience and a description of proposed team members’ experience producing written reports. From the RFP it was unclear how the maximum 40 points would be awarded. The remaining seven point rated criteria (RT2 – RT8) had maximum point values of either 5 points or 10 points each. There was no scoring grid in the RFP to indicate how points would be awarded for these criteria. The grid used by ISED bid evaluators, however, did provide greater clarity including a breakdown of the 40 points for RT1 showing 36 points for the three projects (maximum 12 points each) and 4 points for team members’ experience producing a written report. Further, the evaluation grid included a qualitative scale for each criterion that included “Excellent”, “Good”, or “Inadequate” ratings with accompanying descriptions and point ranges. For greater transparency, this information should have been provided to potential bidders in the solicitation document.
4. In a solicitation for protective apparel rental services, a point rated criterion worth up to 50 points stated, “the Bidder should identify the type and the composition of material for each item being offered.” Up to 10 points was available in each of five categories: FR/ARC ratings, colors available, sizes available, thickness, and durability. There was no indication, however, as to what would merit 10 points for each category versus a lower score.

25. Clear, precise and measurable evaluation criteria and scoring grids enable bidders to know the requirements and the methods by which their proposals will be evaluated. Failure to adequately define evaluation criteria at the outset carries the risk that evaluators may struggle to interpret these criteria during the evaluation process and apply the same tests in assessing all bids. It can also be difficult to defend against external challenges to the evaluation process and demonstrate that criteria have been strictly adhered to when the criteria are unclear and open to multiple interpretations. Clear, precise and measurable evaluation criteria and scoring grids support fair and transparent procurement.

Selection methodology

In all but one file reviewed, the selection methodology clearly communicated the manner in which the contract would be awarded. Similarly, with the exception of one file, the selection methodology also aligned with the requirements.

26. Common selection methodologies such as “highest responsive combined rating of technical merit and price” and “lowest price responsive bid” were used in almost all applicable files reviewed. In two files, both solicitations issued under the PSPC temporary help services (THS) supply arrangement, a less-common “right-fit” selection method was used. In both cases, use of the “right-fit” method was appropriate to the method of supply and respected the rules for these THS solicitations.

27. In one file, for research services to better understand Canada’s strategic advantage within critical mineral supply chains, the selection methodology was not clear in the original solicitation. It stated, “the contractor will be selected on the basis of highest responsive combined rating of technical merit and price.” This was partially clarified through a solicitation amendment that indicated 75% of the score would be based on technical merit and 25% would be based on price; however, the amendment did not explain how financial points would be calculated.

Solicitation documents and processes

With minimal exceptions, solicitation documents provided a clear description of the requirements.

31. In 37 of 39 files reviewed, solicitation documents included a clear description of the requirements. In the other two files, there was insufficient documentation to conclude on clarity of the requirements. Details regarding the two files are provided below:

1. In a file for mapping services, ISED requested bids from two suppliers over the phone. According to ISED, "the project had gone out to a company that could provide this type of service to see what is available and possible. Once we knew what was available, we were able to find a second company who could provide the same service to make this competitive. There are very few companies, that we were aware of, that could provide this sort of service." Without a properly documented file, OPO could not determine if a clear description of the requirements was provided to the 2 bidders.
2. In a second file, ISED was required to issue an RFQ to seven pre-approved resellers listed in the applicable software licensing supply arrangement (SLSA). While there was an internal ISED email stating all resellers had been contacted, the actual solicitation email including a description of the requirements was not found in the file. Without the solicitation document OPO cannot conclude whether or not the description of the requirements was clear.

Solicitation documents from all applicable files included clear instructions for posing questions and submitting bids; however, certain solicitations included instructions to bidders for responding to mandatory and point rated criteria that conflicted with what was stated in the actual criteria.

32. In 10 files reviewed, solicitations were issued under the PSPC Task-Based Informatics Professional Services (TBIPS) supply arrangement. These followed a common structure and included mandatory and point-rated criteria. In all 10 files, the instructions preceding the mandatory technical criteria, stated:

Bidders must provide only the following information in their technical grid response to the mandatory resource criteria listed below. Failure to follow the instructions will render the proposal non-compliant.

• Project #,
• Client Organization/Project Name,
• Duration (month/year to month/year),
• Total level of work effort (# years/# months).

However, the mandatory instructions noted above were inconsistent with some of the mandatory criteria that followed, making it impossible for bidders to comply with both the instructions and the actual criteria themselves.

33. For example, an RFP for Senior Java Developer services under the Programmer/Analyst TBIPS category included eight mandatory criteria, two of which required the Bidder to provide details beyond the information listed in the instructions above. One mandatory criterion stated: “[T]he Bidder MUST indicate the resource category for which the proposed resource was contracted under.” If a bidder were to provide this information to comply with the mandatory criterion, it would simultaneously fail to follow the above-noted instructions and therefore be deemed non-compliant. Another mandatory criterion stated: “Bidders must include the brief client [project] description in the technical grid response or map the description in the proposed resource’s c.v.” Again, if a bidder were to include such a description, it would comply with the mandatory criterion but fail to follow the instructions.

34. Ensuring information is accurate, complete and consistent (i.e., not contradictory) eliminates ambiguities and helps ensure suppliers have the information they need to prepare and submit responsive bids. It also helps reduce the number of questions from suppliers regarding such discrepancies, which in turn reduces the effort required on the part of the department to respond to these questions during the solicitation period.

Overall, solicitations were open to an appropriate number of suppliers and for an appropriate period of time.

35. The duration, or period of time, a bid solicitation must remain open varies depending on the nature of the solicitation. When PSPC methods of supply are used, such as the TBIPS, ProServices, or THS supply arrangements, user departments must respect the bid solicitation rules published by PSPC. For example, for solicitations issued under the TBIPS supply arrangement, for requirements between $100,001 and $3.75 million, the minimum bidding period is 15 calendar days. User departments do have the option to extend the time beyond the minimum based upon a requirement's complexity. For solicitations not issued under a supply arrangement, but covered by one or more trade agreements, the rules of the applicable trade agreement(s) will apply. While exceptions exist, generally, trade agreements such as World Trade Organization-Agreement on Government Procurement (WTO-AGP), the Comprehensive Economic and Trade Agreement, and the Canada-Colombia Free Trade agreement require a 40-day bidding period.

36. In 31 of 33 applicable files, OPO was able to confirm that the duration of the bid solicitation period and the number of suppliers invited to bid was appropriate given the solicitation method used and any applicable trade agreements. For the two remaining files, there was insufficient information to support that finding. One file was a verbal solicitation, and the other was a solicitation issued under an SLSA where ISED was required to issue an RFQ to the 7 pre-approved resellers and provide at least 5 working days for a response. While there was an internal ISED email stating all resellers had been contacted, the number of days bidders were given to submit bids is not known as the actual correspondence was not found in the file.

Communication with suppliers

Communications with suppliers appeared to be appropriate and supported the preparation of responsive bids; however, documentation of communication was incomplete in many files.

37. Section 2 of the TBCP states: “Government contracting shall be conducted in a manner that will: (a) stand the test of public scrutiny in matters of prudence and probity, facilitate access, encourage competition, and reflect fairness in the spending of public funds.” Section 12.3.1 of the TBCP also states: “Procurement files shall be established and structured to facilitate management oversight with a complete audit trail that contains contracting details related to relevant communications and decisions…”. These requirements apply to all aspects of the procurement process, including interactions with suppliers.

38. In 22 of the 39 files reviewed, there was an indication of communications (i.e., questions and answers) with suppliers during the solicitation period. Overall, the responses provided by ISED appeared to adequately address supplier questions. Depending on the solicitation method used, responses were either posted to Buyandsell.gc.ca or provided by email to all invited suppliers. However, file records of communication with suppliers were incomplete. For example, 7 of the 22 files that included responses from ISED to supplier questions did not include the email in which the supplier posed the question. A complete file record would include all relevant correspondence from the email with the original supplier question through communication of the final response from ISED to suppliers. Without a complete record of communications, OPO is unable to determine if all suppliers’ questions were answered, answered in a timely manner, and whether all suppliers were treated equally and received the same information at the same time.

Regret letters to unsuccessful bidders were generally adequate; however, award notices were not posted for a significant number of solicitations posted to Buyandsell.gc.ca.

39. In 22 of 32 files reviewed, there were multiple bidders that responded to the solicitation. In 19 of those 22 solicitations, the files contained regret letters or similar correspondence to unsuccessful bidders. In the remaining three files with multiple bidders, there was no indication that regret letters were sent to all unsuccessful bidders. Regret letters are an important tool as they alert bidders to the fact that they were unsuccessful in the selection process, should they wish to seek a detailed debriefing from the contracting department or avail themselves of potential recourse mechanisms such as OPO or the Canadian International Trade Tribunal. Regret letters also provide valuable information that may be useful to improve the bidders’ chances for success in future contracting opportunities.

40. In 17 of the 39 files reviewed, the solicitation or tender notice was posted publicly on Buyandsell.gc.ca. In only 2 of those 17 solicitations did ISED post an award notice within a reasonable timeframe to notify stakeholders that a contract had been awarded as a result of the procurement process. In the other 15 solicitations, 4 award notices were posted several months after the contracts were awarded and 11 solicitations had an expired contract status giving the false impression that a contract had not been awarded as a result of the solicitation process.

Bid evaluation process

In nearly half of the applicable files, the bid evaluation procedures deviated from the procedures described in the solicitation document resulting in several instances where bidders were wrongly deemed non-compliant or the contract was awarded to the wrong bidder.

43. The requirements of the TBCP referenced above emphasize the importance of having a well-planned and documented process for bid evaluations. In nearly all of the 30 files reviewed that included a technical evaluation, OPO observed that evaluation grids were used with criteria that matched those found in the solicitation document, and the number and identities of the evaluators had been recorded in the file. The limited exceptions included two files that were missing documentation of the technical evaluation. One of those two files included an evaluation grid template; however, it did not include all mandatory criteria that appeared in the solicitation. For this file, OPO was unable to determine if all criteria were evaluated and if the contract was properly awarded, because the completed evaluations were not included in the file.

44. The evaluation of bids must be performed with integrity in a fair, honest and transparent manner. While the contracting authority is ultimately responsible for the process, the evaluation of the technical portion of bids is the responsibility of the evaluation team. Based on the files reviewed, ISED’s evaluation team is typically comprised of employees from the technical authority’s unit. Most often, ISED used a 3-person evaluation team, which included the technical authority and two other employees from that unit with subject matter knowledge.

45. Generally, while bid evaluation team members have the technical knowledge required to evaluate bids, they rely on guidance provided by the contracting authority to ensure procurement rules are followed. In 25 of the 30 applicable files, there were written instructions from the contracting authority to bid evaluators. A good practice observed in 19 files was the use of a “notice of non-disclosure” or equivalent guidance document. The notice of non-disclosure is a form that must be signed by each member of the ISED bid evaluation team prior to the start of the bid evaluation process. By signing the form, evaluators confirm that they would only divulge information to departmental or other officials authorized to participate in the bid evaluation process. The form also includes guidance to evaluators. This guidance stresses the importance of consistency in the process and full documentation of results on the evaluation grid. In 5 of the 30 files that included a technical evaluation, there was no record of instructions to the evaluators located in the file.

A significant number of files had issues with the bid evaluation procedures resulting in contracts being wrongly awarded or bidders incorrectly being found non-compliant.

46. In 18 of 33 files, file documentation demonstrated that technical and financial bids were evaluated in accordance with the planned approach. In 5 files, the absence of key documents including the successful supplier’s technical proposal, quotes in response to an RFQ, and completed bid evaluation grids meant the file did not contain sufficient information to demonstrate that evaluations followed the planned approach or that the successful supplier was selected as per the methodology specified in the solicitation. In the remaining 10 files, documentation showed the bid evaluation procedures that were applied had deviated from the procedures described in the solicitation document, which resulted in 4 contracts being awarded to the wrong bidder, and 6 bidders incorrectly deemed non-compliant. For example:

1. In one file, a solicitation for a multi-year contract valued at over $3.3 million for IT help desk services with work authorized through a task authorization process, the contract was awarded to a non-responsive bidder. The basis of selection in the RFP was highest combined rating of technical merit and price. To be deemed responsive, a bid was required to comply with the requirements of the bid solicitation and meet all mandatory criteria. One such requirement stated, “any bidder proposing a total cost that falls between the “Median” minus 15% and the “Median” plus 15% will be considered financially responsive. Any bidder proposing a total cost that falls outside the “Median” minus 15% and the “Median” plus 15% will be considered non-compliant and no further consideration will be given to that bid.” There were 4 bids received, one was eliminated because it failed to meet all mandatory technical criteria and the other 3 were deemed responsive. All 3 responsive bids received full points for the point rated criteria, meaning the bid price was the determining factor in identifying the successful supplier. ISED did not apply the requirement for bids to be within 15% of the median price and awarded the contract to a non-compliant bidder with a bid price more than 15% below the median. Approximately two months after contract award, ISED discovered that it had wrongfully awarded the contract. Despite the fact the contract would be in place for 4 years with option periods and that the vast majority of resources who would eventually work on the contract had not yet been identified through the task authorization process, ISED decided to continue with the contract, rather than cancel it and re-solicit the requirement, citing a concern over interruption in service if it were cancelled. A note was added to the file to document the error.
2. In two files, both solicitations for IT services issued under the TBIPS supply arrangement, bids were incorrectly rejected for not meeting all mandatory criteria due to ISED’s method of calculating experience of the proposed resource. In one file, an objection was received from a bidder that had been deemed non-compliant. The evaluators had found that 4 of the mandatory criteria (M2, M4, M5, and M6) were not met because they did not believe the proposed resource met the years of experience requirements. The RFP stated, "in situations in which a proposed resource worked at the same time on more than one project, the duration of any overlapping time period will be counted only once toward any requirements that relate to the individual’s length of experience.” The evaluators’ notes showed the decision was made because one project from the proposed resource's CV overlapped with other projects. However, project X was not cited by the bidder in the mandatory technical grid in response to the 4 above noted mandatory criteria and should not have factored into the evaluation of those criteria. Project X was only cited in the bid under one of the point rated criteria. There was no reason given to explain why evaluators considered experience from project X over projects that were cited by the bidder for the 4 criteria in the mandatory technical grid. Additionally, the Mandatory Evaluation Criteria instructions in the RFP stated, "only information cited in the Mandatory technical grid response will be evaluated." OPO cannot determine where the bid would have ranked if it had passed the mandatory criteria because ISED did not evaluate the point rated criteria; however, if it had passed the technical evaluation, it would have been the lowest priced compliant bid. In response to the bidder’s objection to being deemed non-compliant, ISED stated that they stood by their decision. According to ISED’s response, it was the supplier’s “failure to account for the overlap and to ensure an accurate level of work [that] rendered its bid non-compliant.” By not being able to demonstrate evaluations were conducted in accordance with the terms of the solicitation (i.e., "only information cited in the Mandatory technical grid response will be evaluated"), the integrity of the procurement process is brought in to question.
3. In one file for IT Project Executive services, the contract valued at over $1.7 million was awarded to a bidder that should have been deemed non-responsive. The solicitation, which was issued under the TBIPS supply arrangement, included 8 mandatory criteria and 10 point rated criteria. Though the successful supplier’s bid was deemed to have met all mandatory criteria, the bid did not actually demonstrate that the proposed resource met the requirements of mandatory criteria M2 and M3. Criterion M2 stated, “the proposed resource must have greater than ten (10) years professional work experience with-in the last fifteen (15) years working as a project executive.” And M3 stated “the Bidder must provide two (2) client references … where the proposed resource was contracted as a Project Executive within the last five (5) years from the date of bid closing.” However, in response to another mandatory criterion (M1), pertaining to work conducted at ISED in the last 5 years, the bid stated the “resource category for which the proposed resource was contracted under [as] Project Leader”. Project Executive and Project Leader are separate categories under TBIPS, meaning the individual’s recent experience at ISED as a Project Leader should not have been considered towards meeting M2 and M3 requirements for Project Executive experience, resulting in the criteria not being met. There were no notes in the evaluation grid to explain or clarify why the bid was deemed to have met all mandatory criteria.
4. In one file, for economic analysis services, the solicitation had 5 mandatory and 5 point rated criteria and a basis of selection of highest combined rating of technical merit and price. Two bids were received. A 3-person bid evaluation team conducted the technical evaluation and determined that both bids met all mandatory criteria and exceeded the minimum points requirement for point rated criteria. The contracting authority then completed the evaluation process by conducting the financial evaluation, which determined the lower ranked technical bid was ranked 1st overall based on its lower price and superior financial score. After the results were shared with the bid evaluation team, the technical authority said he was concerned that they may have overlooked something in the mandatory criteria and wanted to re-evaluate the 1st overall ranked bid. The contracting authority initially told the evaluators “unfortunately, since the financial evaluation has been completed and we had already come to a consensus on the technical evaluation we cannot go back now.” The response from one of the evaluators stated he would be “happy to delete the financial evaluation email.” Ultimately, ISED did revise its evaluation of the 1st ranked bidder and deemed it non-compliant for not meeting all mandatory criteria. While some aspects of the procurement process were well documented, there was a significant shortcoming with respect to documentation of the decision to allow the evaluators to change their evaluation of the 1st ranked bid after they learned the results of the process. The actions taken and ultimate results of this evaluation process leave ISED open to the perception that this contract was not awarded in a fair and transparent manner.

47. Bid evaluations not conducted consistently and in the manner prescribed by the solicitation call into question the integrity of the procurement process. Failure to strictly adhere to evaluation criteria increases the risk that non-compliant bids will be considered, and in some cases as seen above, awarded government contracts.

File documentation

Completed bid evaluation documentation was missing from a number of files. This was particularly evident among solicitations that pre-dated the COVID-19 pandemic.

48. Section 12.3.1 of the TBCP requires that procurement files facilitate management oversight with a complete audit trail containing details related to relevant communications and decisions, including the identification of the involved officials and contracting approval authorities. The requirement to ensure adequate file documentation extends to the actions undertaken during the solicitation period, the evaluation of bids, as well as the overall procurement file.

49. File documentation was reviewed to determine whether a complete audit trail was retained to support consistent and transparent decision-making. As was highlighted throughout this report, there were multiple files in which key documents, including correspondence with prospective bidders, instructions to bid evaluators, and an RFQ solicitation were missing. OPO recognizes that ISED’s ability to access hard-copy files may have been impacted by COVID-19 pandemic-related office closures and that some of the missing documentation may be located in the ISED offices.

50. Regarding technical evaluations, 7 of 30 files that included mandatory or point rated criteria were missing evaluation documents. Three of those files were missing completed individual and consensus evaluations, and the others did include completed consensus evaluations, but the individual evaluations were not present in the files. Additionally, one file was missing the technical proposal from the successful supplier.

51. OPO considers the lack of documents on file to support decisions to be a concern, particularly with respect to decisions regarding contract award. For files with incomplete records, ISED cannot fully explain its business decisions, inhibiting its ability to demonstrate good stewardship of Crown resources or that the procurements were conducted in a fair, open and transparent manner. Keeping complete and detailed evaluation records is crucial for demonstrating that evaluation criteria have been applied consistently to all competing bids, demonstrating that the procurement has been carried out in a manner consistent with ISED’s obligations under the TBCP and applicable trade agreements, and that the contract was awarded in accordance with the procedure set out in the solicitation documents and to the actual winning bidder.

52. It should be noted, however, that the scope period for this review was October 1, 2019 to September 30, 2021. In March 2020, employees at ISED were forced to pivot to a remote working model due to the COVID-19 pandemic. Six of the seven files with missing evaluation documents had solicitation periods that closed before March 2020, i.e., pre-pandemic and one closed in April 2020, shortly after the lockdown began. Prior to the pandemic, when employees were working from the office, paper-based evaluation processes were more commonly used. It is important that information management practices, whether paper-based or digital, are resilient to threats posed by dramatic events such as the COVID-19 pandemic as well as more predictable events such as the departure of contracting authorities.