Link: GAO Opinion
Agency: Department of Health and Human Services
Disposition: Protest sustained.
- Protest challenging agency’s cost realism evaluation is sustained where the agency concedes an error in accepting awardee’s proposed labor rates for its existing staff based on market surveys, rather than the actual labor rates for its staff, and where the agency unreasonably ignored assumptions made in awardee’s proposal regarding indirect costs.
- Protest challenging agency’s technical and past performance evaluations is sustained where the record does not demonstrate that the agency reasonably evaluated offerors’ proposals.
- Protest is sustained where agency failed to conduct discussions regarding adverse past performance that the protester had not previously had an opportunity to address
General Counsel P.C. Highlight:
WPS first raises four primary challenges to CMS’s evaluation of the offerors’ proposed costs. The protester argues that the agency failed to reasonably evaluate NGS’s indirect cost rates, its direct labor rates, and its overall evaluated costs on a cost-per-claim basis. With respect to its own proposal, WPS argues that the agency unreasonably evaluated its costs for postage. GAO states that when an agency evaluates a proposal for the award of a cost-reimbursement contract, an offeror’s proposed estimated costs are not dispositive because, regardless of the costs proposed, the government is bound to pay the contractor its actual and allowable costs. Consequently, the agency must perform a cost realism analysis to determine the extent to which an offeror’s proposed costs are realistic for the work to be performed. An agency is not required to conduct an in-depth cost analysis, or to verify each and every item in assessing cost realism; rather, the evaluation requires the exercise of informed judgment by the contracting agency. Further, an agency’s cost realism analysis need not achieve scientific certainty; rather, the methodology employed must be reasonably adequate and provide some measure of confidence that the rates proposed are reasonable and realistic in view of other cost information reasonably available to the agency as of the time of its evaluation.
First, WPS argues that CMS’s evaluation of NGS’s proposed indirect rates was unreasonable because it ignored assumptions made by the awardee in its proposal regarding the impact of the award of additional MAC contracts on its indirect costs. CMS requested during discussions that NGS and WPS explain their assumptions concerning the award of other MAC contracts, and the effect that the awards would have on the offerors’ indirect rates. As relevant here, scenario 1 assumed that NGS would win a MAC contract for only jurisdiction 8, while scenario 5 assumed that NGS would win MAC contracts for jurisdictions 6, 8 and 15. GAO finds CMS’s evaluation of NGS’s indirect costs was unreasonable because the agency knew that the assumptions underlying scenario 5 would not be met, i.e., that NGS would not win MAC contracts for jurisdictions 6 and 15. In this regard, NGS’s FPR explicitly stated that NGS’s indirect cost proposal assumed award of three jurisdictions, and stated that, in the event NGS was awarded only jurisdiction 8, there would be a $[deleted] increase in its indirect costs. Despite this statement, the agency concluded that NGS would not experience an increase in its indirect costs, and made no adjustment to NGS’s costs. While GAO does recognize that CMS relied on the similarity between the scenario 5 rates and the rates experienced in jurisdiction 13, it concludes that the agency could not ignore the plain language of NGS’s proposal, which offered higher indirect rates if it received only one contract. On this record, GAO thinks the agency had no reasonable basis to assume that NGS would experience the lower indirect costs identified under scenario 5, and failed to make an appropriate adjustment to the awardee’s proposed costs.
Next, WPS argues that CMS’s evaluation of NGS’s proposed direct labor rates was flawed because, although the awardee proposed to perform the work with its existing staff, NGS proposed, and the agency accepted, labor rates that relied on market surveys, rather than NGS’s actual labor rates. CMS concedes that its supplemental report on the protest did not accurately escalate NGS’s labor costs. The agency states that its revised calculations show that NGS’s actual labor rates were 14.85% higher than its proposed rates, resulting in an overall increase to NGS’s probable costs of $[deleted]. CMS agrees with the protester that in light of NGS’s proposal to use incumbent labor, the agency should have evaluated NGS’s actual labor rates, as opposed to surveys of the average labor rates for the geographic area. The agency argues, however, that its method of calculating the impact of using NGS’s incumbent labor rates would result in a $[deleted] cost adjustment, and would not allow WPS to overcome the $32.3 million difference between NGS’s and WPS’s probable costs. GAO states that while it has concerns regarding the reliability of CMS’s post hoc cost calculations, GAO thinks that CMS’s calculations demonstrate that WPS was prejudiced by the agency’s error in evaluating NGS’s labor rates. In this regard, the agency’s “conservative” estimate of a $[deleted] cost impact, in addition to the $[deleted] adjustment based on CMS’s flawed evaluation of NGS’s indirect costs, makes NGS’s probable cost higher than WPS’s probable cost. Further, as the protester notes, application of the higher scenario 1 indirect rates requires an additional increase to NGS’s probable costs because those higher rates would be applied to an additional $[deleted] in direct labor. GAO agrees, and notes that applying the scenario 1 rates to CMS’s calculations of NGS’s higher direct labor results in $[deleted] higher indirect costs.
Next, WPS argues that CMS’s evaluation of NGS’s proposed costs was flawed because the agency did not consider the realism of the awardee’s overall cost on a per-claim basis. In this regard, the protester argues that if CMS had considered NGS’s costs-per-claim, the agency would have seen that they are below the current figure for NGS’s other Medicare contracts. The agency states that it did not evaluate the offerors’ proposed costs by calculating the costs-per-claim under each proposal. Instead, the agency evaluated the awardee’s proposed costs on a component-level basis, that is, the agency evaluated whether the individual cost elements were realistic to perform the offeror’s technical approach. As part of its evaluation, the agency considered historical information cited in NGS’s proposal regarding, for example, claims processing and customer service. GAO states that there is no requirement that agencies follow any particular cost realism evaluation method, or evaluate offerors’ proposed costs using every possible method of analysis. Here, the record shows that the agency evaluated the individual cost elements of NGS’s proposal and concluded that the costs were reasonable, overall. The protester’s argument that its analysis of NGS’s costs, i.e., the per-claim costs, suggests that the awardee’s costs are too low, does not clearly demonstrate that the agency’s evaluation was flawed.
Finally, WPS argues that the agency unreasonably adjusted its proposed costs for postage by $[deleted]. The protester argues that the agency’s cost realism adjustment was improper because the protester provided sufficient information to justify its proposed costs. In its initial proposal, WPS stated that its historical data showed that the company could process [deleted] claims per mailing of quarterly Medicare summary notice (MSN) statements. By the time WPS submitted its FPR, the protester stated that its updated data showed that the company was processing [deleted] claims per mailing; as a result, WPS reduced its proposed costs by $[deleted]. The protester explained that the costs were revised because WPS “has a greater amount of actual experience now, as opposed to our original Business Proposal,” and that the “actual [MSN] savings were greater than WPS estimated previously.” The agency concluded that the protester had not adequately explained the basis for its cost reduction. Specifically, the agency stated that WPS had not provided adequate historical information “illustrating the change in claims per mailing.” GAO finds that the protester’s argument is based on its disagreement about the adequacy of the data supporting WPS’s change in its proposed costs. In GAO’s view, the agency’s concerns here were reasonable, and WPS’s disagreement with the agency provides no basis to sustain the protest.
Next, WPS argues that CMS’s evaluation of both offerors’ technical proposals was flawed in a number of areas. GAO agrees with the protester’s arguments concerning the Healthcare Integrated General Ledger Accounting System (HIGLAS), as well as its arguments concerning performance risks posed by NGS’s proposed labor rates.
First, WPS argues that NGS’s proposal showed a lack of experience with and understanding of HIGLAS, a specialized dual accounting and financial management system that will replace and modernize a number of existing Medicare systems. The protester contends that an offeror’s understanding of HIGLAS was directly relevant to the claims processing subfactor evaluation, and that therefore WPS should have been favorably evaluated, and NGS unfavorably evaluated under this subfactor. As a preliminary matter, all MACs will be required to utilize HIGLAS in performance of the MAC contracts, although at the time of the issuance of the RFP not all jurisdictions had been converted from the legacy systems to the HIGLAS. As relevant here, all of the jurisdiction 8 requirements have been transitioned to HIGLAS, with the exception of Medicare Part B requirements for Indiana.
In the protester’s view, experience with HIGLAS should have been considered under the claims processing subfactor of the technical understanding evaluation factor, which stated that offerors must demonstrate “the ability to maintain the proper systems infrastructure to appropriately interact with CMS’s standard systems.” GAO agrees. In this regard, the statement of work (SOW) stated that “[f]inancial reporting for the existing Medicare contractor workloads is performed using HIGLAS or CAFM. Eventually, HIGLAS will be used by all Medicare contractors.” In answering the protester’s contentions, CMS does not dispute the protester’s characterization of NGS’s experience with HIGLAS. Instead, CMS initially argued that an offeror’s experience with or understanding of HIGLAS experience was not relevant to the evaluation of offerors’ proposals. The agency noted that the RFP instructed offerors not to propose costs for HIGLAS transition. The agency further argued that experience with HIGLAS was not required under any evaluation factor, and that therefore “HIGLAS was not to be considered by CMS in the evaluation of proposals under the Technical Understanding factor.” The agency subsequently acknowledged, however, that an offeror’s understanding of HIGLAS was relevant to certain SOW requirements.
GAO finds that the record supports the protester’s contention that an offeror’s understanding of HIGLAS is relevant to the claims processing subfactor of the technical understanding factor. The record, however, is not clear as to whether the agency considered offerors’ understanding of HIGLAS relevant to the evaluation of offerors’ proposals, or whether the agency evaluated offerors’ understanding of HIGLAS. Since the documentation in the record does not demonstrate that the agency’s evaluation in this area was reasonable, GAO sustains this basis of protest.
Next, WPS argues that the agency failed to adequately evaluate the effect of NGS’s decision to utilize its existing staff to perform the contract requirements. WPS argues, and the agency concedes, that the agency’s cost realism evaluation of NGS’s proposal was unreasonable because it accepted labor rates based on market surveys, rather than NGS’s actual labor rates. WPS further argues that the agency should have concluded that there would be a negative effect on NGS’s ability to provide its existing staff because of the apparent reduction in labor rates. WPS contends that this negative effect should have resulted in lower technical ratings for NGS under a number of technical evaluation factors, particularly those where CMS credited the experience of NGS’s existing staff, such as the implementation factor. GAO has held that where a solicitation advises offerors that the agency could consider the results of its cost realism evaluation in the evaluation of technical proposals, the agency may make a cost realism adjustment for unrealistic proposed personnel costs as well as assess the impact the proposal of unrealistic rates could have upon technical performance.
Here, the solicitation stated that the agency would consider technical, cost and risk factors in conjunction with each other. A separate risk evaluation factor also stated that the agency would consider risk in all areas of the agency’s evaluation, including the selection decision, Finally, the solicitation states that “[i]f a cost realism analysis is performed, cost realism may be considered by the source selection authority in evaluating performance or schedule risk.” On this record, GAO thinks that the agency is incorrect in its assertion that it was barred from considering the effect of NGS’s low proposed labor rates on its technical proposal. At a minimum, the agency was permitted under the terms of the solicitation to consider whether NGS’s labor rates posed a performance risk under any of the technical evaluation factors and subfactors which addressed offerors’ ability to provide adequate staffing and services to meet the requirements of the SOW, e.g., the technical understanding, implementation, and personnel evaluation factors. Further, GAO thinks the record is unclear as to what NGS intends to pay its existing workforce. In this regard, while the agency in its supplemental report on the protest states that it now believes an adjustment to NGS’s labor rates is appropriate, NGS contends that it would have proposed the same direct labor rates that CMS believes do not reflect its actual labor costs. Because GAO sustains the protest on the basis of the flawed cost realism evaluation, GAO concludes that the agency should also reevaluate NGS’s proposed labor costs and consider whether this situation–i.e., uncertainty about the rates NGS will pay its existing staff–will require reopening discussions.
WPS argues that CMS’s evaluation of the offerors’ past performance was unreasonable because it ignored relevant information concerning the offerors’ past performance. As a general matter, the evaluation of an offeror’s past performance is a matter within the discretion of the contracting agency, and GAO will not substitute its judgment for reasonably based past performance ratings. However, GAO will question an agency’s evaluation conclusions where they are unreasonable or undocumented. The critical question is whether the evaluation was conducted fairly, reasonably, and in accordance with the solicitation’s evaluation scheme, and whether it was based on relevant information sufficient to make a reasonable determination of the offerors’ past performance.
WPS argues that the agency’s evaluation of its past performance failed to consider its ratings under the Medicare Contractor Provider Satisfaction Survey (MCPSS), a program which asks Medicare providers to rate the performance of fee-for-service contractors, including MACs, in various functions in the prior year. The protester states that it identified positive MCPSS data in its proposal, and argues that the data were relevant past performance information that should have contributed to a higher evaluation rating. On the one hand, CMS states that it did not review this information because it was not considered reliable, e.g., the agency could not always attribute the survey results to a specific contractor. On the other hand, the agency states that it would have considered an offeror’s MCPSS results if the offeror had cited it in its proposal, in which case “the PPEP would confirm this information, but otherwise the MCPSS results were not used.” CMS also concedes that there is no indication in the contemporaneous record that the agency considered this information with regard to WPS’s proposal.
While the agency appears to raise a reasonable concern regarding the relevance of the past performance data, and would otherwise have the discretion to disregard this information in its evaluation of the offerors’ past performance, the agency nonetheless suggests it did in fact consider the information for certain offerors, i.e., those who chose to specifically reference it in their proposals. GAO finds that the record is not clear as to whether the agency considered the MCPSS data relevant to offerors’ past performance, and whether the agency evaluated the MCPSS data identified in WPS’s proposal. On this record, GAO is unable to conclude that the agency’s evaluation was reasonable, and sustains the protest on this basis.
Finally, WPS argues that CMS failed to conduct reasonable discussions in three areas, one concerning the program management subfactor of the technical understanding evaluation factor, and two concerning past performance. GAO states that it is a fundamental precept of negotiated procurements that discussions, when conducted, must be meaningful, equitable, and not misleading. Discussions may not mislead offerors and must identify deficiencies and significant proposal weaknesses that could reasonably be addressed in a manner to materially enhance the offeror’s potential for receiving award. Agencies are also required to provide an offeror with an opportunity to address adverse past performance information to which the offeror has not previously had an opportunity to respond.
First, the protester argues that the agency conducted unfair discussions because it conducted two rounds of discussions with NGS concerning organizational alignment, an element of the program management subfactor of the technical understanding evaluation factor, but did not provide a similar opportunity for WPS. GAO finds no merit to this argument. GAO does not think that there was any requirement for the agency to conduct discussions with the protester regarding this matter because WPS was not in a situation similar to that of NGS. WPS did not have a similar approach or feature concerning organizational alignment that required discussions. Instead, its proposal had a “meets requirements” rating, with no distinguishing features. In this regard, while discussions must be meaningful, leading an offeror into the areas of its proposal requiring amplification or revision, the agency is not required to “spoon-feed” an offeror as to each and every item that could be raised to improve its proposal.
Next, WPS contends that CMS identified two weaknesses concerning its past performance, but did not conduct discussions regarding these issues. The protester argues that the agency’s failure to provide discussions was unreasonable because the agency conducted discussions with NGS regarding one of these issues, and because both issues were areas of adverse past performance that the protester had not had an opportunity to address. GAO disagrees with the protester concerning the systems security weakness, but agrees concerning the appeals weakness.
With regard to the systems security weakness, the PPEP stated that WPS had a “High Overall Risk rating in systems security,” and that there was a trend over the prior three fiscal years of performance which “suggests a lack of effective management and business practices and systems.” The agency did not conduct discussions with WPS regarding this weakness. The agency conducted discussions with NGS, however, and provided NGS an opportunity to address a similar weakness concerning a “High Overall Risk rating in systems security . . . [which] suggests a lack of effective management and business practices and systems.” The agency does not dispute that it conducted discussions with NGS and not with WPS regarding systems security as part of the jurisdiction 8 competition. The agency argues, however, that WPS was not prejudiced by this unequal treatment, nor was the agency obligated to identify this as adverse past performance during discussions, because the agency would not have given WPS a higher evaluation rating even if the protester had received an opportunity to resolve this weakness.
While GAO agrees that CMS did not provide both offerors the same opportunity for discussions in jurisdiction 8, the record shows that WPS was not prejudiced by the agency’s actions. In this regard, the record here shows that the PPEP viewed this matter as one that offerors could not resolve through discussions because CMS had not yet completed the reviews necessary to validate offerors’ corrective action plans. More importantly, WPS does not explain how it would have responded differently or more thoroughly if it had been provided an opportunity to address the issue again in the jurisdiction 8 procurement. At best, the protester argues that the jurisdiction 8 CO might have asked different discussions questions, leading WPS to respond in a different manner. GAO thinks that the protester’s argument posits a scenario too remote and speculative to demonstrate that it was prejudiced here.
Next, with regard to the appeals weakness, the PPEP stated that WPS’s past performance raised a number of concerns. The PPEP found that WPS had performance issues regarding WPS/Mutual Part A appeals processing “that have required CMS oversight.” Next, the agency found that although WPS met timeliness standards for Medicare Part B appeals reports in FY 2007, WPS “submitted several requests for relief for accuracy of reporting on the 2592 report [concerning appeals activities] and hearing officer standards.” The evaluation also cited concerns regarding the quality of decision letters in FY 2007. Finally, the agency cited ongoing concerns regarding Medicare Part B appeals in FY 2008 that were reported by agency staff in July 2008. Based on these concerns, the agency assigned WPS a weakness under the quality of service subfactor of the past performance evaluation factor.
The agency did not discuss this weakness with the protester. CMS argues, however, that WPS was “well aware that most of the issues existed” as a result of exchanges concerning WPS’s performance on other Medicare contracts. In this regard, CMS contends that it was not obligated to raise this concern with WPS during discussions because the protester submitted requests for waivers of the accuracy standards relating to the 2592 reports in 2006 and 2007. However, WPS submitted an additional waiver request in July 2007, which was denied on August 17, 2007, and a January 2008 request that was denied on February 8, 2008. With regard to the denied waiver requests, the agency advised WPS that it was not excused from the requirement for timely processing of appeals, advised WPS of the specific bases for the denials, and provided a point of contact for additional questions. CMS also argues that it discussed concerns regarding WPS/Mutual’s appeals processing during a 2007 report of contractor performance (RCP) review, which is annual performance feedback provided by CMS to contractors. In the 2007 RCP for WPS/Mutual, the agency states that it noted concerns regarding some timeliness issues, while rating the contractor as meeting expectations overall. Although agencies must advise offerors during discussions of adverse past performance regarding which the offeror has not yet had an opportunity to respond, GAO has held that an offeror is not entitled to discussions if it has previously had an opportunity to address the issue during other exchanges with the agency.
Here, two of WPS’s requests for waivers of the 2592 report standards were denied by the agency in writing prior to the submission of FPRs, with written explanations for the denials, and a point of contact for additional questions. On this record, GAO agrees with the agency that it was not required to provide WPS an opportunity to address this adverse past performance. However, the record shows that the weakness assessed in WPS’s past performance regarding appeals was broader than the issues discussed in the waiver requests or the 2007 RCP reviews. Specifically, although the PPEP report stated that “the quality of the decision letters was poor in FY 2007” for Medicare Part B, the agency does not dispute that this issue was not identified in any prior exchanges between the agency and WPS. Next, although the agency states that it raised some concerns regarding WPS/Mutual’s appeals processing in the 2007 RCP, the agency acknowledges that the past performance information here is based on “more recent information.” Finally, the agency concedes that it did not note any problems with WPS’s performance of Medicare Part B appeals in its 2007 RCP (which was separate from the RCP for WPS/Mutual), and did not raise the new concerns regarding appeals performance in 2008 cited in the PPEP report during discussions here. In sum, the record shows that a substantial portion of the agency’s basis for assessing a weakness for WPS’s past performance in the area of appeals was based on adverse past performance information that the protester had not had an opportunity to address, and GAO sustains the protest on this basis. The protest is sustained.