Acquisition of business process consultant services by Correctional Service of Canada

April 2019

Review of a Complaint: Acquisition of business process consultant services by Correctional Service of Canada (PDF, 348KB)

On this page

The complaint

1. On October 15, 2018, the Office of the Procurement Ombudsman (OPO) received a written complaint from a supplier (the Complainant) regarding a contract awarded by Correctional Service of Canada (the Department). The contract was for the provision of business process consultant services to “review a number of proposed business areas including the overall human resources model of its food services organization.”

2. The contract, valued at $52,841.95 (taxes excluded), was awarded on August 7, 2018.

3. The Complainant raised concerns about the way the Department conducted the evaluation process as well as the results of that process. This report examines the following key issues raised by the complaint:

  • Were there errors in the Department’s calculation of a proposed resource’s months of relevant experience?
  • Were projects incorrectly assessed as not relevant to the consultant category?

4. On October 16, 2018, OPO confirmed the complaint met the requirements of the Procurement Ombudsman Regulations (the Regulations) and it was considered filed.

Mandate

5. This review of complaint was conducted under the authority of paragraph 22.1(3)(b) of the Department of Public Works and Government Services Act and sections 7 to 14 inclusive of the Regulations.

6. Pursuant to subsection 9(2) of the Regulations, the Procurement Ombudsman requested the Department provide documents and information necessary for the review. The request was for all departmental records associated with the award of the contract in question, as well as the Department’s procurement policies and guidelines in effect at the time of the solicitation. The Procurement Ombudsman also requested the Complainant provide any additional information not submitted as part of the complaint.

7. The Department provided OPO with records associated with the award of the contract in question. It did not provide any departmental procurement policies or guidelines; however, its policies were accessible on the Department’s web site. The Complainant provided copies of additional administrative correspondence between it and the Department, which had not been submitted as part of the complaint.

8. The findings in this report are based on the records provided by the Department and the Complainant, as well as relevant publicly available information. The failure by either the Department or the Complainant to disclose any relevant records or information could impact the findings of this report.

Background

ProServices supply arrangement

9. In June 2018, the Department issued a request for proposal (the “Solicitation”) under the ProServices Supply Arrangement (ProServices SA). The ProServices SA is a federal government-wide mandatory procurement tool for the provision of professional services with a value that falls below the North American Free Trade Agreement (NAFTA) threshold. It is a method of procurement that enables identified users to solicit proposals from a pool of pre-qualified suppliers for specific requirements.

10. The ProServices SA has 13 streams covering 158 professional service categories. Each ProServices SA stream and many of the professional service categories have their own scoring grid, also referred to as a “flexible grid.” Departments use a flexible grid to evaluate resources proposed by a bidder in response to a solicitation. Points are allocated for relevant education, professional certification and experience. The point total determines whether a proposed resource is considered to have a junior, intermediate or senior level of expertise in a given category.

11. When soliciting proposals under the ProServices SA, purchasing departments specify the category and level of expertise (that is junior, intermediate or senior) needed for their requirement. In response, bidders propose resources with education, professional certification and experience relevant to the category. To be considered for contract award, a proposed resource must achieve the minimum number of points specified in the flexible grid for the required level of expertise.

The solicitation

12. The Solicitation for the contract at issue specified that the Department required a senior level resource under ProServices SA category 9.5 – Business Process Consultant.

13. The Solicitation included a single mandatory technical criterion, M1, which was derived from a ProServices SA flexible grid. To meet criterion M1, the bidder’s proposed resource was required to qualify as a senior level business process consultant by achieving 95 points as described below:

Mandatory technical criterion, M1

The Bidder’s proposed resource must meet the flexible grid under 9.5 Business Process Consultant, Senior Level, as per the ProServices Supply Arrangement

Human Resources Services Stream Flexible Grid 9.5 Business Process Consultant

Levels of Expertise

  • Senior: Minimum 95 points (pts)
  • Intermediate: Minimum 70 points
  • Junior: Minimum 50 points

Relevant Education to the Consultant Category

  • University (PhD, Graduate, Undergraduate, degree): 35 points
  • College or CEGEP Diploma/Certificate: 25 points
  • High School Diploma: 20 points

Professional Certification

  • Relevant Professional Certification: 15 points

Relevant Experience in Consultant Category

  • ≥1 year and <2 years: 12 to 23 months—10 points
  • ≥2 years and <4 years: 24 to 47 months—20 points
  • ≥4 years and <6 years: 48 to 71 months—30 points
  • ≥6 years and <8 years: 72 to 95 months—40 points
  • ≥8 years and <10 years: 96 to 119 months—50 points
  • ≥10 years: 120 + months—60 points

14. Evaluation procedures for this procurement were described in section 4.1 of the Solicitation, as follows:

  1. Bids will be assessed in accordance with the entire requirement of the bid solicitation including the technical and financial evaluation criteria.
  2. An evaluation team composed of representatives of Canada will evaluate the bids.

15. The basis of selection, section 4.2 of the Solicitation, stated “[t]he responsive bid with the lowest evaluated price will be recommended for award of a contract.”

Chronology of events

16. The following chronology was developed based on the documentation provided by the Department and the Complainant. It outlines key events leading up to the filing of the complaint.

17. On June 13, 2018, the Department sent the Solicitation by email to 14 suppliers pre qualified under the ProServices SA, including the Complainant. The deadline for submitting proposals was June 20, 2018 at 2:00 PM.

18. On June 20, 2018,

  • at 10:39 AM, the Department received a proposal from one of the pre-qualified suppliers that had been invited to bid.
  • at 12:17 PM, the Department issued an amendment by email to extend the solicitation period by one day. According to an internal departmental email, this change was made in the hopes of receiving more proposals. The new closing date was set as June 21, 2018 at 2:00 PM.
  • at 1:25 PM, the Complainant submitted its proposal.

19. On June 21, 2018, at 2:00 PM, the solicitation period closed. Two proposals had been received by the Department.

20. Between June 27 and July 18, 2018, members of the Department’s three-person evaluation team individually evaluated the proposals received.

21. On July 27, 2018, the Department’s evaluation team completed a consensus evaluation of the proposals received. It determined the Complainant’s proposed resource failed to achieve the minimum number of points required to qualify as a senior level business process consultant.

22. On July 30, 2018,

  • at 12:15 PM, the Department sent a regret letter by email to the Complainant indicating the contract was being awarded to another supplier (the Successful Supplier). The letter provided the name of the Successful Supplier and the contract value. The letter also indicated the Complainant’s proposed resource received 75 points, and did not achieve the minimum requirement of 95 points as set out in the Solicitation.
  • at 3:18 PM, the Complainant replied to the Department’s email. The Complainant asked for a breakdown of how points were awarded for its proposed resource.

23. On August 1, 2018, the Department responded to the Complainant by providing a flexible grid with the evaluation results for its proposed resource. The flexible grid showed the proposed resource received a total of 75 points: 35/35 points for education, 0/15 points for professional certification, and 40/60 points for experience. Written comments for the experience component of criterion M1 stated two projects included in the proposal (#20 and #21) were not relevant to the business process consultant category and were not counted.

24. On August 7, 2018, the Department awarded the contract to the Successful Supplier for $52,841.95, excluding applicable taxes. This contract value was higher than the amount submitted in the Complainant’s financial proposal.

25. On August 15, 2018, the Complainant sent an email to the Department requesting an in person debrief to discuss the evaluation of its proposal.

26. On August 28, 2018, the debrief meeting was held.

27. On September 19, 2018,

  • at 7:51 AM, the Complainant emailed the Department to summarize the debrief meeting and request a recalculation of points for its proposed resource. The email indicated:
    • the Complainant did not believe it received “clear or reasonable justification” as to why projects #20 and #21 were deemed not relevant;
    • the Complainant believed the Department had miscalculated the points for relevant experience;
    • that during the debrief meeting, the Department advised there were other projects, in addition to #20 and #21, that were deemed not relevant;
    • the Department had committed to re-evaluate the Complainant’s proposal and the results of this second evaluation would be provided to the Complainant;
    • the Department would not provide the Complainant with a list of other projects that were deemed not relevant in the original evaluation.
  • at 10:13 AM, the Department replied with an apology for the delay in getting information back to the Complainant and indicated it was preparing a response and would be in contact as soon as possible.

28. On September 21, 2018, the Department sent an email to the Complainant with the results of the second evaluation of its proposal. The email included a new flexible grid and an evaluation summary. The flexible grid showed points awarded for education and professional certification remained unchanged; however, the score for relevant experience had increased from 40 to 50 points. Overall, the Complainant’s proposed resource scored 85 points. This result was 10 points higher than the first evaluation, but still below the minimum 95 points required for the proposal to be considered responsive.

29. On September 24, 2018, the Complainant sent an email to the Department requesting clarification on the results of the second evaluation of its proposal. Specifically, the Complainant sought to confirm its understanding of how project numbers identified in the flexible grid and evaluation summary aligned with its proposal, and to clarify whether certain projects were or were not considered relevant experience.

30. On September 26, 2018, the Department sent an email to the Complainant with a revised flexible grid and responses to each of the questions it had raised. The revisions did not affect the overall score, which remained 85 points.

31. On October 5, 2018, the Complainant emailed the Department regarding the results of the second evaluation of its proposal. It disputed the Department’s count of the number of months of relevant experience. The email also expressed the Complainant’s disagreement with the Department’s determination that six of its projects did not represent experience relevant to the business process consultant category.

32. On October 15, 2018, OPO received the Complainant’s written complaint.

33. On October 16, 2018, the Department wrote to the Complainant in response to its October 5th email. The Department advised that senior management had reviewed the file and the proposal was still deemed non-compliant.

34. On October 16, 2018, OPO confirmed the complaint met the requirements of the Regulations and it was considered filed.

The evaluation process

35. For the first evaluation, technical proposals were evaluated individually by members of a three-person evaluation team. Results from these individual evaluations were recorded on individual evaluation forms that were signed and dated by the evaluator. For the Complainant’s proposal, all three individual evaluations assigned 40 points for “Relevant Experience in Consultant Category,” and 75 points overall.

36. While there was consistency in scoring, the form and substance of written comments for the relevant experience component of criterion M1 varied among the three individual evaluations. One evaluator simply identified the total number of months of relevant experience, another listed the projects that were deemed relevant experience, and the third stated projects from a certain point onward in the proposal were not counted because they were not consultant experience. The evaluators’ handwritten comments from the individual evaluation forms have been represented below:

Individual evaluation results from first evaluation—Relevant Experience in Consultant Category
Individual Evaluations Points Written Comments
Evaluator 1 40 74 months of project experience, given some project overlap
Evaluator 2 40 78 months – current + #'s 2, 3, 4, 5, 6, 8, 9, 11, 12, 14, 15, 17, 19
Evaluator 3 40 EXPERIENCE: 6 to 8 years: 40 points
Starting project 20 on page 38 of the resumé not consultant experience

37. The Department’s response to OPO stated the individual evaluations were followed by a consensus evaluation. The response also included a document identified as the Consensus Evaluation Report that included overall consensus results from technical evaluations. The Consensus Evaluation Report, dated July 27, 2018, indicated the Complainant’s proposed resource received a total of 75 points; 20 points below the minimum threshold to qualify as a senior business process consultant. Written comments stated “78 months of relevant experience, projects 20-23 are disqualified as it is [sic] not relevant to Business Process Consultant category.”

38. On August 1, 2018, the Department provided the Complainant with results from the first evaluation of its proposal. The document sent to the Complainant, however, was not consistent with the Consensus Evaluation Report. While the results continued to reflect that 40 points had been awarded for relevant experience, there were differences regarding the projects deemed not relevant. Rather than identifying projects #20-23 as not relevant, the results sent to the Complainant stated “[p]rojects 20, 21 (starting on page 38 of technical proposal) are not relevant to Business Process Consultant category and were not counted.” The Department’s response to OPO did not provide an explanation for the variation between the two documents. The Complainant stated it was told “when the form was filled out, the person completing it had simply included what they had in front of them, which was what they had been given by the reviewers.”

39. The Complainant reviewed the evaluation results it received and determined, even without counting projects #20 and #21 as relevant experience, its proposed resource “still had sufficient months of experience to have been awarded the full 60 [points] and so should have been awarded the contract based on our lower pricing.” The Complainant then requested an in-person debriefing with the Department to review the evaluation results and “to gain an understanding of how calculations and decisions were made.”

40. The debrief meeting was held on August 28, 2018. At this time the Department disclosed that there were other projects, in addition to projects #20 and #21, that were deemed not relevant to the consultant category. According to the Complainant’s summary of the debrief meeting, it requested “documentation detailing which projects had actually been disallowed during the initial review process and the reasons given at that time for their disallowance, [however] no evidence was produced. … [N]obody in the room was able to provide a complete or definitive answer as to which projects had actually been disallowed or why.”

41. According to an internal departmental email, the Contracting Authority did not have all the details about which projects had been deemed relevant and which had not at the time of the debrief meeting. The Department’s response to OPO stated it “created an action item for a follow-up evaluation to be performed and documentation to be provided to [the Complainant] with further details of the evaluation and a full list of included and excluded projects in order to share all critical information, clarify any confusion, and to provide full transparency and openness into the evaluation process.”

42. The Department conducted its second technical evaluation of the Complainant’s proposal in the weeks following the debrief meeting. This evaluation was performed by the Contracting Authority and another contracting official, neither of whom had been part of the team that performed the first technical evaluation. Results from the second evaluation were sent to the Complainant on September 21, 2018. The email from the Department read:

[The Department] has completed an in-depth evaluation of the proposal …. As a result of this, I have included the Flex Grid Evaluation which contains a list of projects that were included and deemed relevant to the consultant category and also the ones that were excluded/not relevant to the consultant category. In addition to this, I included an Evaluation Summary report that will show in more detailed [sic] the justification and rationale as to why the projects were excluded and not counted in the evaluation. Please review the attached information and if you have any questions or concerns, feel free to contact me.

43. The Complainant wrote back requesting clarification regarding the second evaluation’s results. Subsequently, on September 26, 2018, the Department responded to the Complainant’s questions and clarified which projects were and were not considered as being relevant to the consultant category.

44. OPO reviewed both the flexible grid evaluation form and the accompanying evaluation summary completed by the Department. Compared to the first evaluation, the Department’s comments provided a more complete and informative depiction of the second evaluation’s results. The flexible grid evaluation form listed all projects that were deemed relevant and not relevant to the consultant category. The evaluation summary provided comments to explain why the Department assessed six projects (#13, 16, 18, 20, 22, and 23) as not relevant.

45. A significant change between the first and second evaluations involved project #21. While the first evaluation determined project #21 did not represent relevant experience in the consultant category, the second evaluation deemed the project relevant. The Department, however, only counted 29 months of experience for project #21, which had a total duration of 66 months. In response to questions from the Complainant about the second evaluation, the Department stated “project 21 was recognized as relevant and 29 months were counted.” The Department did not explain why only 29 of the 66 months were counted.

46. In the second evaluation, the Department determined that overall the proposal demonstrated 101 months of relevant experience, including the 29 months for project #21. Based on this result, the score for relevant experience increased from 40 to 50 points and the total score was revised from 75 to 85 points. There were no changes to the scores for education or professional certification.

47. On October 5, 2018, the Complainant emailed the Contracting Authority regarding the results of the second evaluation of its proposal. The Complainant disagreed with the Department’s count of 29 months of experience for project #21. The email stated:

In our bid we claimed only 29 months for Project #21 because it overlapped Project #22 by 37 months and we were claiming that project as well. However, Project 22 has been disallowed as non-relevant, which means that there is no longer a 37-month overlap between Projects #21 and #22, rendering all 66 months of Project #21 available to be counted towards our bid.

48. The Complainant stated its proposal demonstrated 138 months of relevant experience based on the projects deemed relevant by the Department, not 101 months as was indicated in the flexible grid. Had the Department performed the calculation correctly, the Complainant contended, its proposed resource would have been awarded the maximum 60 points for relevant experience and 95 points overall. The email also expressed the Complainant’s disagreement with the Department’s determination that six of its other projects did not represent relevant experience in the business process consultant category.

49. The Department’s response to OPO indicated that following receipt of the Complainant’s email, senior management undertook an independent evaluation of the proposal “…in order to address [the Complainant’s] identified concerns with the process.”

50. In a letter dated October 16, 2018, a departmental senior manager informed the Complainant:

In response to your letter dated October 5, 2018, this letter is to confirm that I reviewed the file and your comments and the bid is still deemed non-compliant.

Should you require further information please do not hesitate to contact me.

51. The letter did not include any reasoning for the senior manager’s determination, nor did it mention that another evaluation had been performed. Rather, it indicated that the Complainant’s objection had been escalated and the senior manager confirmed the outcome of the second evaluation.

52. Following the initiation of this review, the Department’s response to OPO provided more insight into why the senior manager deemed the Complainant’s proposal non-compliant. It stated “[u]pon further review of the tasks completed on project 20 and 21 in reference to the category, both projects were deemed non-relevant as the tasks and duties performed in this position did not fit within the scope of tasks of a Business Process Consultant.” In this respect, specifically the relevance of project #21, the senior manager did not agree with the results of the second evaluation.

53. Written evidence from the first evaluation did not clearly identify which projects were deemed relevant and which were not. This appears to be the reason why the Department conducted the second evaluation. Written evidence from the senior manager’s review also did not include explanations for counting certain projects and not others. Only the second evaluation included sufficient written evidence to support the assessment. Documentation from the second evaluation not only indicated which projects were deemed relevant, but also identified projects that were not counted as relevant experience and gave reasons for those decisions.

54. Given that there was insufficient written evidence to support both the first evaluation and the senior manager’s review, the following analysis of the issues raised in the complaint is based on the second evaluation.

Analysis of issues and findings

Issue 1—Were there errors in the Department’s calculation of a proposed resource’s months of relevant experience

55. The Complainant stated:

The calculation of months awarded to [the Complainant] for our bid contained at least one very significant error and, seemingly, a few other more minor errors. While some of the minor errors were in our favor, the most significant error was to our detriment … The [most significant] error lay in [the Department] incorrectly accounting for project overlaps. More specifically, [the Department] mistakenly discounted months from an allowed project that overlapped with a disallowed project. In our proposal we had properly accounted for all overlapping months in the projects we were putting forth to demonstrate our experience.

56. The Department’s response to OPO stated:

An element in the calculation of months of experience and project overlap and relevance was the inconsistency of information in the bid. The evaluation committee noted that the proposed projects numbered in [the Complainant’s] proposal (pages 5 to 8) did not match the project numbers in the bidder’s technical proposal (pages 20 to 43). The evaluation committee had to account for this inconsistency in project numbering, and made a determination to use the projects numbered and referenced in the technical proposal from pages 20 to 43. This project numbering and months of experience referenced in section 4.0 Mandatory Technical Criteria of the proposal was inconsistent with the references in other sections of the proposal as well as with the project dates and number of months provided.

Requirements Issue

57. The Government of Canada’s commitment to promoting fairness, openness and transparency in the bidding process is affirmed in section 40.1 of the Financial Administration Act (FAA).

58. The Treasury Board Contracting Policy (TBCP) reinforces the commitment made in the FAA. Section 10.7.27 of the TBCP states in part:

Competing firms should be told the measurement criteria and the weighting assigned to them. … [Evaluation] factors and their weighting must be established beforehand and adhered to strictly. They are to be recorded along with the requirements of the contract and included in the bid solicitation. … Fairness to all prospective contractors and transparency in the award process are imperative.

59. The Standard Acquisition Clauses and Conditions (SACC) Manual 2003 (2017-04-27) Standard Instructions - Goods or Services - Competitive Requirements, were incorporated by reference and formed part of the Solicitation. The following articles from the Standard Instructions are relevant to this issue:

5.2(f) It is the Bidder's responsibility to provide a comprehensible and sufficiently detailed bid, including all requested pricing details, that will permit a complete evaluation in accordance with the criteria set out in the bid solicitation.

16.1(a) In conducting its evaluation of the bids, Canada may, but will have no obligation to … seek clarification or verification from bidders regarding any or all information provided by them with respect to the bid solicitation.

60. According to the ProServices SA website, which was accessible by a link in the Solicitation:

The experience must be demonstrated and be directly related to the consultant category. Offerors/Suppliers should provide complete details as to where, when, month and year, and how, through which activities/responsibilities, the stated qualifications/experience were obtained …. The month(s) of experience listed for a project whose time frame overlaps that of another referenced project, will only be counted once.

Analysis Issue

61. The Complainant’s proposal made reference to project experience in two places. Section 4.0 of the Complainant’s proposal provided a table summarizing completed projects, and Annex 5 included more detailed project profiles.

62. The table at Section 4.0 of the proposal listed 19 projects that collectively suggested the proposed resource had more than 10 years (120 months) of relevant experience in the consultant category. For each project, the table identified:

  • client name
  • role or title of the proposed resource
  • project start and end dates (month and year)
  • project duration (number of months)
  • number of months intended to count as experience (N.B. OPO noted this was only specified for projects whose time frame overlapped with that of another project.)

63. The table also referred to Annex 5, where further details on the proposed resource’s project experience were provided. Annex 5 included 23 numbered projects, which included all 19 projects listed in Section 4.0 as well as 4 others. Annex 5 included a project summary for all 23 numbered projects and a list of deliverables and tasks for most projects.

64. OPO observed inconsistencies with project numbering in the Complainant’s proposal, as mentioned in the Department’s response. Project numbers in Section 4.0 did not align with project numbers in Annex 5. This impacted the Department’s ability to perform an evaluation in accordance with the Solicitation on the grounds that the Complainant did not provide a comprehensible proposal, as required by the Standard Instructions referenced in paragraph 59. The Department had the option, but not the obligation, to seek clarification regarding any information in the proposal, including project numbering. It did not request such clarification from the Complainant. Rather, it stated the proposed resource’s experience was evaluated based on projects #1-23 as described in Annex 5 (pages 20-43) of the proposal.

65. The flexible grid from the second evaluation of the Complainant’s proposal showed 14 of the 23 numbered projects from Annex 5 had been counted as relevant experience. The evaluation found these projects cumulatively demonstrated 101 months of experience in the Business Process Consultant category, equivalent to 50 points based on the scoring regime as described in the Solicitation.

66. OPO identified multiple examples of the Department having properly accounted for project overlaps when it tabulated the months of relevant experience. For example, the Department counted project #5 and project #6 as relevant experience. It included the full four month duration of project #5, but only eight out of ten months for project #6 to account for project overlap. OPO created the table below to demonstrate how the Department properly accounted for overlap between projects #5 and #6.

Note: Not applicable in the following table refers to the month that is outside the project start or end date.

Overlap between projects 5 and 6
Project number Project dates Months’ duration Deemed relevant Months counted 2014
Jan. Feb. Mar. Apr May June July Aug. Sept. Oct. Nov. Dec.
5 01-2014 to 04-2014 4 Yes 4 1 1 1 1 n/a n/a n/a n/a n/a n/a n/a n/a
6 03-2014 to 12-2014 10 Yes 8 n/a n/a 0 0 1 1 1 1 1 1 1 1

67. In another example, the Department properly tabulated months of relevant experience when two projects’ timelines overlapped, but only one project was deemed relevant experience. In this instance, the Department determined project #13 was not relevant, so it did not count any of its seven months as experience. Project #14 overlapped with project #13; however, because the months of experience listed for project #13 were not being counted, the evaluators correctly included all four months for project #14. This is depicted in the following table, created by OPO.

Note: Not applicable in the following table refers to the month that is outside the project start or end date.

Overlap between projects 13 and 14
Project number Project dates Months’ duration Deemed relevant Months counted 2009
Jan. Feb. Mar. Apr May June July Aug. Sept. Oct. Nov. Dec.
13 01-2009 to 07-2009 7 No 0 0 0 0 0 0 0 0 n/a n/a n/a n/a n/a
14 01-2009 to 04-2009 4 Yes 4 1 1 1 1 n/a n/a n/a n/a n/a n/a n/a n/a

68. With the exception of project #21, the Department counted the full duration of all projects deemed relevant, or the full duration less any overlap with another relevant project. For project #21, the Department only counted 29 of the project’s 66 months. There was nothing in the project’s description in Annex 5 that would explain why the Department counted only 29 months, nor did the Department explain its reasoning in any of the documentation provided to OPO.

69. The table that appeared in Section 4.0 of the Complainant’s proposal included start and end dates as well as duration (measured in number of months) for each project listed. In instances when a project overlapped with another, the Complainant had also specified how many months were intended to count as relevant experience. For project #21 (as it was numbered in Annex 5), Section 4.0 indicated “Dec 1999 - May 005 (66 onths, count 29).” The next project (project #22, as it was numbered in Annex 5) indicated “Sept 2000 - Sept 2003 (37 months).”

70. There were 29 months where project #21 did not overlap with project #22. This is why the Complainant only claimed 29 months of relevant experience for project #21, according to its email to the Department dated October 5, 2018. The Department, however, determined project #22 was not relevant to the resource category. In doing so, it eliminated the potential for double counting the overlapping months between these two projects.

Finding Issue

71. The Procurement Ombudsman found the Department erred in calculating the months of relevant experience in the Business Process Consultant category. Had the Department counted the full 66 months for project #21, the Complainant’s proposed resource would have scored 60 points for relevant experience and achieved the 95 points required to qualify as a Senior Business Process Consultant. This error might have been caused, in part, by inconsistencies between Section 4.0 and Annex 5 in the Complainant’s proposal. The Department could have overcome these inconsistencies by seeking clarification from the Complainant, but instead it chose to evaluate relevant experience based on projects listed in Annex 5. Ultimately, the Department did not explain why it failed to count the full 66 months for project #21 as presented in Annex 5 of the proposal.

Issue 2—Were projects incorrectly assessed as not relevant to the consultant category

72. The Complainant stated:

[During the debrief meeting], [w]e stated our position that we believed the disallowed projects were clearly relevant, that their descriptions in our proposal addressed their relevance, and that they had been routinely accepted as relevant to the category when responding to virtually identical RFPs [requests for proposals] under the category in the past. We asked for an explanation as to why these projects were disallowed when they seemed obviously relevant to the category but received no clear or reasonable justification for their disallowance. We were simply told that relevance was left to the reviewer’s discretion.

On September 21st, [the Department] sent us [the results from] their new evaluation…. Along with their list of newly disallowed projects they also included justifications for the disallowances. Those justifications gave the strong impression that, at best, project relevance was assessed solely by comparison with the Category title (Business Process Consultant) rather than by comparison to the 20 individual tasks and services that define that category.

73. The Department’s response to OPO stated:

[The Department’s] contracting officer and evaluation committee members attended the debrief meeting, during which [the Complainant] asked [the Department] to explain why projects 20 and 21 were disallowed. The evaluation committee indicated that these were not relevant as the roles and tasks did not fall within the scope of the work of a Business Process Consultant. [The Complainant] defended that the tasks were relevant to the Business Process Consultant position and that these projects had not been excluded before from other Departments in past evaluations. [The Department] explained that other processes were not relevant in this evaluation process.

Requirements Issue

74. The Solicitation indicated the Department had the responsibility to assess proposals to determine if the experience, education and professional certification of proposed resources were related to the Business Process Consultant category.

75. According to the ProServices SA website, “[t]he experience must be demonstrated and be directly related to the consultant category.”

76. The ProServices SA website has a listing of all ProServices’ streams and categories. Each category includes a non-exhaustive list of services that may be required under the consultant category. For category 9.5 – Business Process Consultant, the list includes 20 services.

77. As mentioned under Issue 1, the SACC Manual 2003 (2017-04-27) Standard Instructions - Goods or Services - Competitive Requirements, were incorporated by reference and formed part of the Solicitation. Article 19 from the Standard Instructions stated:

The bid solicitation documents contain all the requirements relating to the bid solicitation. Any other information or documentation provided to or obtained by a bidder from any source are not relevant. Bidders should not assume that practices used under previous contracts will continue, unless they are described in the bid solicitation. Bidders should also not assume that their existing capabilities meet the requirements of the bid solicitation simply because they have met previous requirements.

78. In assessing the evaluation of the Complainant’s proposal, the Procurement Ombudsman is bound by subsection 12(2) of the Regulations, which states the “Procurement Ombudsman shall not substitute his or her opinion for the judgment of the persons involved in the acquisition process for the contract in relation to the assessment of any bid unless there is insufficient written evidence to support that assessment or the assessment is unreasonable.”

79. Sufficient written evidence was provided to support the second evaluation; however, the reasonableness of the Department’s assessment of six additional projects as being not relevant to the consultant category required additional analysis. To determine if an assessment is unreasonable, the Procurement Ombudsman is guided by the Canadian International Trade Tribunal’s (Tribunal) well-established standard of review, which has been stated as:

[T]he Tribunal has been clear that it will find an evaluation to be unreasonable and will substitute its judgment for that of the evaluators when the evaluators have not applied themselves in evaluating a bidder’s proposal, have ignored vital information provided in a bid, have wrongly interpreted the scope of a requirement, have based their evaluation on undisclosed criteria or have otherwise not conducted the evaluation in a procedurally fair way.

Analysis Issue

80. The Department’s second evaluation found that nine of the numbered projects from Annex 5 of the proposal did not provide relevant experience in the Business Process Consultant category. In its October 5, 2018 email to the Department, the Complainant indicated three of those projects (#1, 7, and 11) had not been put forward as relevant. Therefore, OPO limited its analysis to the six projects (#13, 16, 18, 20, 22, and 23) that were not counted as relevant by the Department, which the Complainant did put forward as relevant experience in the consultant category.

Results from previous solicitations

81. While the Complainant indicated that disallowed projects from its proposal “had been routinely accepted as relevant … under the category in the past,” the Standard Instructions referenced in paragraph 77 informed suppliers that they “should not assume that practices used under previous contracts will continue.” Evaluation results from prior solicitations, including those from prior solicitations under this SA, could not influence the evaluation of proposals under this Solicitation.

Reasonableness of deeming the six projects not relevant

82. Following the second evaluation, the Department provided the Complainant written comments to explain its decision not to count the six projects noted in paragraph 80 as relevant experience in the consultant category. The evaluation summary prepared by the evaluators explained their reasons for not counting these six projects and the flexible grid evaluation form included a complete list of assessment results for all 23 numbered projects in the Complainant’s proposal. The documentation from the second evaluation did not show the evaluators had failed to apply themselves, nor did it show they had ignored vital information in the proposal.

83. Evaluators’ written comments expressed a common theme for five of the six projects not counted as relevant experience. For these five projects (#16, 18, 20, 22, and 23) evaluators determined the work performed was not directly related to services that define the resource category. This line of reasoning showed the evaluators had properly interpreted the scope of the requirement.

84. The reasons given for not counting project #13, however, were inconsistent with the scope of services for the Business Process Consultant category on the ProServices SA website. In explaining why project #13 was not counted, the evaluators stated they “found it aligned to a consultant reviewing a previous established business model and overseeing Human Resources (HR) planning and development strategy….” The scope of services for Business Process Consultants listed on the ProServices SA website, however, includes both “reviewing existing work processes and organizational structure” and “advising HR Systems groups and Information Technology (IT) groups on discipline requirements.” Had project #13 been counted as relevant experience, after accounting for project overlap, the Complainant’s proposed resource would have been awarded three additional months of experience. This would have had no impact on the number of points awarded.

85. The Solicitation included only one mandatory technical criterion, M1, which included three components: relevant education, professional designation, and experience. Evaluators based their assessment on criterion M1 as described in the Solicitation. There was no evidence of any undisclosed criteria being used in the evaluation of the Complainant’s proposal.

86. At the debrief meeting following the first evaluation of the Complainant’s proposal, the Department was unable to fully explain the evaluation results. The second evaluation was ordered by the Department to improve the transparency and openness of the evaluation process. While evaluations are not typically performed after contracts have been awarded, the evidence does not suggest the evaluators had conducted the evaluation in a procedurally unfair way.

Finding Issue

87. OPO found there was sufficient written evidence to support the Department’s second evaluation of the Complainant’s proposal and, with the exception of the assessment of project #13, that evaluation was not found to be unreasonable. The Procurement Ombudsman will not substitute his opinion for the judgment of the evaluators for five of the six projects deemed not relevant experience in the consultant category. The assessment of project #13 was found to be unreasonable, and as a result the Complainant’s proposed resource should have been awarded three additional months of relevant experience. These additional three months would not have had an impact on the number of points awarded.

Conclusion

88. The Department erred in calculating the total number of months of the Complainant’s proposed resource’s relevant experience in the Business Process Consultant category. The proposed resource would have qualified as a senior level Business Process Consultant if the Department had properly accounted for project overlap. The Complainant should therefore have been recommended for contract award as the lowest priced responsive bidder.

Compensation

89. In order to recommend the payment of compensation to the Complainant, subsection 13(2) of the Regulations requires the following:

If a competitive process was held, the complainant must have submitted a bid in respect of the contract to which the complaint relates, unless it was prevented from doing so because of the actions of the contracting department.

90. As a competitive process was held and the Complainant submitted a proposal, the Ombudsman may recommend payment of compensation in accordance with subsection 13(1) of the Regulations.

Recommendation

91. In accordance with subsection 13(1) of the Regulations, the Procurement Ombudsman recommends the Department pay compensation to the Complainant in an amount equal to 10% of the value of the contract awarded. The Procurement Ombudsman recommends the Department pay $5,284.

Other observations

92. As per subsection 12(1) of the Regulations, the Procurement Ombudsman is required to consider any relevant factors related to the procurement process in question when conducting a review of a complaint.

93. OPO identified an additional issue regarding the flexible grid used in the Solicitation.

94. The ProServices SA website provides mandatory requirements to departments regarding the use of flexible grids in solicitations issued using the tool, as follows:

Each Stream of services has its own flexible grid. The flexible grid indicates the minimum level of points required to qualify for each Level of Expertise. Various amounts of points are given for relevant education, professional certification and relevant experience. The flexible grid must be used for all categories in a Stream unless otherwise specified.

95. The Solicitation incorrectly included the ProServices SA flexible grid for requirements under stream 8 – Human Resources Services. For requirements under category 9.5 – Business Process Consultant, the Department was required to use the flexible grid for stream 9: Business Consulting/Change Management.

96. Fortunately, the differences between the stream 8 and stream 9 flexible grids don’t appear to have influenced the results of this Solicitation; however, that would not always be the case. Given the large number of streams and categories within the ProServices SA, Departments must be attentive when selecting the appropriate flexible grid.

Date modified: