Procurement practice review of the Royal Canadian Mounted Police

Mandatory criteria

Mandatory evaluation criteria were clearly communicated, not unnecessarily restrictive and aligned with the requirement in most files reviewed.

17. Mandatory criteria identify the minimum requirements for bids to receive further consideration. They permit the evaluation team to screen out bidders that do not have the minimum necessary competence and capability to undertake the work. As a general rule, the number of mandatory criteria should be minimized to those components that are essential to meet the operational requirements in order to increase the probability of receiving responsive bids. The sample included 19 files with solicitations that included mandatory evaluation criteria. In all files, the criteria aligned with the requirement and the successful completion of the work and, in all but 1 file, were communicated in a clear, measurable and demonstrable manner. In that 1 file, mandatory criteria required a subjective assessment by evaluators to determine whether a bidder had sufficient experience with "policing models" or "community consultation." OPO also noted that neither term had been defined in the solicitation. With this one exception, overall, mandatory criteria were based on technical specifications of goods being acquired or qualifications of resources proposed for services contracts that could be assessed on a pass/ fail basis.

18. In 17 of 19 files, the mandatory criteria were not overly restrictive and did not unnecessarily favour or penalise any particular bidder. In 2 files, however, as described below, the mandatory criteria appeared to give preference to the incumbent supplier. By their nature mandatory criteria are restrictive, however, criteria that are overly restrictive can threaten the fairness of the procurement process by favouring or penalizing particular bidders.

1. In 2 files, the RCMP issued solicitations for consultant services under the Task-based Informatics Professional Services (TBIPS) supply arrangement. In each case there was an incumbent supplier. Under both solicitations, to be deemed responsive a bidder had to meet 6 mandatory criteria and achieve a minimum score for a series of point-rated criteria. For example, in 1 of these solicitations, the mandatory criteria stated the proposed resource:
   • Must have a minimum of 2 years of experience within the last 5 years leading collaborative working groups across multi-disciplinary units within a Canadian Federal government department
   • Must have a minimum of 5 years of experience within the last 10 years fulfilling the role of a Project Manager or Strategic Advisor responsible for at least 2 Information Technology Community Generics (ITCG) implementations in a large Government of Canada department with greater than 50% employees outside the National Capital Region
   • Must have at a minimum, demonstrated experience with 1 project in the past 3 years involving implementing a Government of Canada wide mandate from a central agency perspective
   • Must have a minimum of 5 years of experience within the past 10 years preparing business cases and budgetary estimates for senior management
   • Must have a minimum of 2 years of experience in the last 5 years working in conjunction with human resource staffing and classification departments within a large Government of Canada department with greater than 50% employees outside the National Capital Region

Individually, these mandatory criteria were not overly restrictive but when considered collectively it would have been highly unlikely that any bidder other than the incumbent could have met all mandatory criteria. In both cases, the incumbent bidder used their RCMP project experience to help meet the mandatory criteria. Ultimately, for each solicitation the only bid received was from the incumbent supplier and in each case the incumbent was awarded the contract.

Point-rated criteria

When point-rated criteria were used in solicitations, they were communicated in a clear manner, aligned with the requirement and were not overly restrictive.

19. Section 10.7.25 of the TBCP states that “criteria should identify accurately all of the performance elements significant to the success of the project and should measure both the competence of the firm and the worth of its particular technical approach.” The findings presented below pertain to the 7 files that contained point-rated criteria.

20. In 6 files, the point-rated criteria were not overly restrictive. The remaining file was 1 of the 2 files that also contained overly restrictive mandatory criteria, as described in paragraph 18 above. For that 1 file, in response to the point-rated criteria a bidder was required to demonstrate their proposed resource had:

• Experience in the development of a certain business intelligence reporting system in the past 10 years for a multi-jurisdictional (municipal, provincial, federal) organization, and
• Experience in the development of a certain business intelligence reporting system for law enforcement users (points were based on number of years of experience)

Similar to the previous observation regarding overly restrictive mandatory criteria, on their own these point-rated criteria may not be overly restrictive, but when viewed collectively, and in addition to mandatory criteria, it would have been highly unlikely that anyone other than an incumbent bidder would have been deemed responsive. The incumbent bidder was able to achieve full points for the point-rated criteria based on project experience obtained at the RCMP over the previous 13 years.

21. In 6 files, the point-rated criteria and the methods for scoring those criteria were clearly communicated in the solicitation document. In the 1 remaining file, there were inconsistencies between the criteria and the descriptions of how points would be awarded.

1. In the 1 file cited above, a solicitation process issued under the supply arrangement for Task and Solutions Professional Services (TSPS), was run twice because the first process did not result in the award of a contract. The first time the solicitation was issued it included 5 point-rated criteria. One criterion was for “Canadian Rural Policing experience;” however, the associated allocation of points table indicated that points would be awarded for knowledge of related topics rather than experience in Canadian rural policing. Another criterion was for “Demonstrated knowledge of Canadian Demographics and Trends.” For this criterion points were to be awarded for experience rather than knowledge in these areas. The second time the solicitation was issued the point-rated criteria were revised and the number of criteria was reduced from 5 to 3. Efforts were made to re-word and clarify the point-rated criteria in the second solicitation; however, inconsistencies between wording of these criteria and the allocation of points descriptions remained. For example, the 1st criterion titled “Canadian Rural Policing experience” was modified by indicating that points would be awarded for knowledge or experience. The title of the 2nd point-rated criterion was modified to “Demonstrated knowledge of Canadian Demographics;” however, the allocation of points continued to state that points would be awarded for experience rather than knowledge of Canadian demographics.

22. In 5 files, the weighting factors reflected the relative importance of the criteria and were aligned with the requirement. In the remaining 2 files, as described below, the method for allocating points combined with minimum score requirements identified in the solicitation resulted in point-rated criteria that did not reflect their relative importance or that were not aligned with the requirement. Both solicitations were issued under the TBIPS supply arrangement.

1. In 1 file, a solicitation for business process consultant services included 3 point-rated criteria with a potential value of 10 points each, for a potential maximum score of 30 points. A minimum score was set at 21/30 (70%). Any bid receiving a score below the minimum would be deemed non-responsive. One of the criteria was for experience with a particular software. For this criterion, bids could score either 10 points or 0 points, depending on whether the bid did or did not demonstrate the proposed resource had the relevant experience. In effect, the scoring approach changed this point-rated criterion into a mandatory criterion as not having experience with the software would result in a bid being declared non-responsive.
2. In 1 file, a solicitation for business consultant services that also included 6 mandatory criteria, bidders were required to achieve a minimum score of 64/80 (80%) for 4 point-rated criteria. One of those criteria, was for experience working within Canadian federal government departments. This point-rated criterion was, in effect, another mandatory criterion based on how the points were weighted. A bidder needed to demonstrate that its proposed resource had a minimum of 9 years of experience working with Canadian federal government departments. If not, it would have been impossible to achieve the minimum 80% threshold regardless of its scores on the other point-rated criteria. The successful supplier, which was also the incumbent supplier and the only bidder, was able to meet this 9-year experience requirement based on having 10 years of experience working within the RCMP and other federal government departments.

Selection methodology

With exceptions described below, in the majority of files reviewed the selection methodology clearly communicated the manner in which the contract would be awarded and reflected the complexity of the requirement.

23. OPO’s review of selection methodologies applied to 34 files. It excluded 5 files that were for call-ups under standing offers. Solicitations were missing from 3 of the 34 files. For those 3 files, there was insufficient documentation to demonstrate that the selection methodology was clearly communicated and aligned with the requirement.

24. In the remaining 31 applicable files for which solicitations were present, the selection methodology published in the solicitation document aligned with the requirement. Additionally, in all of these cases the RCMP used common selection methodologies such as lowest priced responsive bid (with or without mandatory technical criteria) or highest combined rating of technical merit and price to select the successful supplier.

25. In 28 of the 31 files, the selection methodology (i.e. basis of selection) was clearly communicated in the solicitation document. In 2 files, the solicitation indicated the contract would be awarded to the responsive bidder with the lowest evaluated price, however, there was ambiguity as to what was required to be deemed a responsive bidder. In 1 file, the basis of selection stated in the solicitation document differed from the selection methodology used by evaluators. Examples of these exceptions included:

1. In 1 file, for the acquisition of firing range ammunition, the solicitation stated that “[a] bid must comply with the requirements of the bid solicitation and meet all mandatory technical criteria to be declared responsive” and “[t]he responsive bid with the lowest evaluated price will be recommended for award of a contract (1 contract only).” It also stated that “[d]elivery must be made by March 31, 2019 following acceptance of all applicable documentation […]. Should it be impossible for the Bidder to deliver the entire firm quantity by March 31, 2019, the Bidder is to complete the table below […]”. The usage of the word “must” implied that the delivery date was a mandatory criterion, however, the following sentence indicated that bids that did not meet the delivery date would still be considered. There were no other mandatory technical criteria specified in the solicitation.
2. In 1 file, for architectural and engineering services, the basis of selection stated the responsive bid with the highest combined rating of technical merit and price would be recommended for award of a contract. However, the bid evaluation guide provided to evaluators added an additional criterion that had not been disclosed to bidders. This undisclosed criterion stated that any bids with price proposals that were greater than 25% above the average would be deemed non-responsive and receive no further consideration. In this case, the additional criterion did not result in any bids being eliminated, however, solicitation documents must include all evaluation criteria that will be used to evaluate proposals and adhered to strictly. Use of this undisclosed criterion breached the transparency and threatened the fairness of this procurement process.

26. A contributing factor to the overall positive results observed under this LOE was the oversight provided by the RCMP’s Contract Quality Control (CQC) function. CQC reviews are conducted prior to issuing a solicitation for services valued over $100,000 and goods valued over $26,400, excluding those issued under a standing offer or supply arrangement.^{Footnote 1} The stated objective of the CQC function is “to ensure that the RCMP's procurement and contracting activities are carried out in accordance with the applicable legislation, policy and procedures.” As described through the examples provided in the paragraphs above, when issues were identified with technical evaluation criteria, these were often from solicitations that were issued under a supply arrangement and were not subjected to CQC review.

Guidance documentation

The RCMP has guidance in place to support fair, open and transparent procurements.

30. The RCMP’s Procurement, Materiel and Assets Management Branch is responsible for developing national policies and procedures for RCMP procurement and contracting, providing advice on contracting and procurement and, among other responsibilities, implementing quality assurance functions for procurement and contracting nationally. The Branch maintains a GCpedia ^{Footnote 2} page where key guidance documents are updated regularly and made easily accessible to RCMP employees and other employees from across the public service. Within its Procurement and Contracting Officer Tool Kit, the RCMP posts procurement notices, general procurement information for clients and procurement officers, quality control checklists, solicitation templates, information about its contract quality control and assurance process, terms of reference for its Contract Review Committee, and other useful guidance to support the procurement process.

Solicitation documents

Most solicitations contained clear and complete information and instructions necessary to prepare a compliant bid.

31. In 33 of 39 files, solicitations were clear and contained complete information and instructions to bidders, supporting transparency in the procurement process. Of the remaining 6 files, 4 did not include sufficient documentation to demonstrate that the solicitation contained a clear description of the requirement. In 2 files, there were inconsistencies between the contract requirement and the description of the requirement in the solicitation. In 1 of those files, the requirement changed after the original solicitation had been issued, but not all prospective bidders were informed of the revised requirement. In the 2nd file, as described below, information in the solicitation may have misled prospective bidders about the scope of work and potential value of the resulting contract.

1. In 1 file, a solicitation issued for refuse and recycling services, the estimated volume of work, i.e. the number of times per month refuse and recycling was to be picked up, did not provide an accurate depiction of the requirement. While the solicitation did clearly describe the locations where the services would be required and the types of containers to be provided by the supplier, the volumetric data that bidders were given and required to use to prepare their financial proposals considerably understated the actual volume of work under the contract. Using these low estimates, the evaluated bid price submitted by the successful supplier was just over $11,000; however, the actual contract value was set at $425,000, an amount reflective of the true requirement. Notwithstanding the fact that the basis of payment in the solicitation stated that the volumetric numbers were “estimated usage for cost evaluation purposes only and do not constitute a guarantee or commitment on behalf of Canada of the quantity or amount to be used under the Contract,” there was no information in the solicitation that would have signaled to bidders that this contract was potentially worth more than $400,000. Bidders would have benefitted by having a clearer description of the work they were bidding on and additional bidders might have submitted bids had they known the potential volume of work and value of this contract.

32. In 30 of 34 applicable files, excluding 5 files for call-ups under standing offers, there were clear instructions to bidders for submitting bids and for posing questions during the solicitation period. For files where these instructions were not present: in 1 case, the file included an email template with the appropriate instructions, but there was no indication the email had been sent to potential bidders; in 1 case, the solicitation included instructions for submitting bids, but not for posing questions during the solicitation period; and in 2 cases, the file did not contain the solicitation or any additional documentation with instructions to bidders. Failure to document solicitations or instructions to bidders increases risks to fair, open and transparent procurement and inhibits oversight by management, internal audit and external assurance providers.

33. Ensuring information is accurate and complete is important as it eliminates ambiguities, contradictions or other discrepancies in solicitation documents. This helps ensure suppliers have the information they need to prepare and submit responsive bids and helps reduce the number of questions from suppliers regarding such discrepancies. In turn, this reduces the effort required to respond to these questions during the solicitation period.

Solicitation processes

Overall, the design and execution of solicitation processes supported fair, open and transparent procurement.

Bid evaluation process

While a majority of files included instructions to support the technical evaluation of bids, inconsistent methods were sometimes used to record the results from financial evaluations. Additionally, a number of files showed that the bid evaluation process deviated from what had been stated in the solicitation.

41. The requirements of the TBCP referenced above emphasize the importance of having a well-planned and documented process for bid evaluations. For a majority of the 19 files reviewed that included a technical evaluation, OPO observed that an evaluation plan or instructions to assess bids was provided to evaluators, evaluation grids were used with criteria that matched those found in the solicitation document and the number and identities of the evaluators had been documented in the file. A notable good practice observed in some files was the use of a declaration form signed by members of the bid evaluation team acknowledging their obligations and responsibilities to protect the confidentiality and integrity of the process.

42. In 5 of the 19 files that included a technical evaluation, an evaluation grid matching the criteria found in the solicitation was not included in the file. In 4 of those cases the number and identities of the evaluators was also not recorded in the file. For example:

1. In 2 files, both for the acquisition of uniform components, as part of the bidding process bidders were required to submit pre-contract award (PCA) samples using detailed specifications included with the solicitation. There was no documented plan for evaluating PCA samples. Based on the evaluation results included in the files, it was not possible to determine if all PCA samples were assessed against all specifications. While the evaluation results were signed-off by a director-level technical authority, it was not clear if that one individual conducted evaluations alone or if others were involved. Completed evaluation grids including specifications that were evaluated and a record of individuals who conducted the evaluations could have demonstrated that bids were evaluated consistently and that all bidders were treated equally.
2. In 1 file, for architectural and engineering services, the solicitation included 1 mandatory and 7 point-rated criteria. The bid evaluation document (i.e. grid) identified the members of the bid evaluation team and the 7 point-rated criteria. Instructions to evaluators focussed on scoring bids against the point-rated criteria. Neither the bid evaluation document nor the instructions to bidders mentioned the mandatory criterion and there was no record in the file demonstrating that bids had been evaluated against the mandatory criterion.

43. With respect to financial evaluations of bids, OPO observed that a variety of methods were used to record results. The best of these featured evaluation tables or grids signed by the contracting authority that recorded financial information from each bid, confirmed whether or not each bid was deemed responsive and ranked responsive bids. In some other cases, there was a table or grid with financial information from bids received; however, it was not always indicated whether each bid had been deemed responsive and the results were not signed off by the contracting authority. Finally, there were a number of files that did not include a table or grid to document financial evaluations. Some of these files included bid prices in a bid registry, while others did not include documentation of the financial evaluation.

44. In 26 of 34 files reviewed, excluding the 5 call-ups issued under standing offers, technical and financial bids were evaluated in accordance with the planned approach. In 4 files, the absence of key documents including the solicitation and bid evaluations meant the file did not demonstrate that evaluations followed the planned approach or that the successful supplier was selected per the methodology specified in the solicitation. In the remaining 4 files, documentation showed the bid evaluation procedures that were applied had deviated from the procedures described in the solicitation document. For example:

1. In 1 file, a solicitation for business process consultant services, a mandatory criterion stated that proof of education demonstrating that the proposed resource possessed a degree in computer science or a related field must be included in the bid. The one bid received, which was from the incumbent supplier, did not include the required proof of education. The solicitation instructions were clear. “At the time of bid closing, the proposed resource must demonstrate compliance with the … mandatory criteria. If the proposed resource fails to demonstrate compliance with each of the … mandatory criteria, the bid will be deemed non-compliant and will not be given any further consideration.” After the period for submitting bids had closed and bid evaluation had begun, the contracting authority contacted the supplier to request the proof of education that had not been included with its bid. While contracting authorities are permitted to seek clarification of some existing aspect of a proposal that does not amount to a substantive revision or modification of the proposal, they are not permitted to facilitate or encourage the repair of a flawed proposal. This request to the bidder to provide the missing proof was an example of the latter, a practice commonly referred to as “bid repair.” Though a copy of the resource's proof of education was sent that day, the bid should have been deemed non-compliant because it did not meet the mandatory criterion at bid closing.
2. In 1 file, a solicitation was issued under the TSPS supply arrangement for professional services under Stream 2—Needs Analysis and Research Consultant(s). When issuing solicitations under the TSPS there are certain rules that user departments must follow. For example, the TSPS requires that proposed resources be assessed against a “flexible grid” to determine if they possess the education, experience and relevant professional certification for the stream of service and level stated in the solicitation. In this case, the solicitation included the Needs Analysis and Research Consultant flexible grid; however, only 2 of the 5 resources proposed by the successful supplier were assessed against the flexible grid.

Procurement documentation

11 of the 39 files reviewed were missing key documents such as the solicitation and bid evaluations.

45. Section 12.3.1 of the TBCP requires that procurement files facilitate management oversight with a complete audit trail containing details related to relevant communications and decisions, including the identification of the involved officials and contracting approval authorities. The requirement to ensure adequate file documentation extends to the actions undertaken during the solicitation period as well as the evaluation of bids.

46. File documentation was reviewed to determine whether a complete audit trail was on file to support consistent and transparent decision-making. As was highlighted throughout this report, there were many files in which key documents, including solicitations, correspondence with prospective bidders and bid evaluations, were missing. This issue appeared to be particularly acute when the contracting authority became involved later in a procurement process that had already been initiated by a technical authority (i.e. client), and when the requirement pertained to the 2018 G7 summit.

47. Incomplete procurement files resulted in inadequately supported procurement actions that risk undermining the fairness, openness and transparency of the procurement process. Keeping complete and detailed records is crucial to demonstrating that evaluation criteria have been applied equally to all competing bids, and demonstrating that the procurement has been carried out in a manner consistent with the RCMP’s obligations under applicable trade agreements, the Financial Administration Act and regulations made under it, and the TBCP. Failure to maintain complete records also places the RCMP at risk of not being able to defend challenges to the procurement process.