BID PROTESTS: THE COSTS ARE REAL, BUT THE BENEFITS OUTWEIGH THEM
Daniel I. Gordon[*]
[Note for student editors: Will you insert a table of contents and format the headings of the article consistent with PCLJ’s procedures?]
TABLE OF CONTENTS
I.INTRODUCTION
II.ADMINISTRATIVE AND JUDICIAL VENUES FOR FILING PROTESTS IN THE U.S. FEDERAL SYSTEM
III.SPREAD OF THE PROTEST PROCESS OUTSIDE THE U.S.
IV.CORRECTING MISPERCEPTIONS ABOUT PROTEST STATIISTICS
A.Protests are rare events
B.It is rare for a protester to win a protest, and even rarer for a winning protester to go on to obtain the contract at issue in the protest.
V.COSTS OF THE PROTEST PROCESS
VI.Benefits of the Protest Process
VII.Conclusion: The Costs are Overstated, and the Benefits Outweigh Them
I.INTRODUCTION
This article discusses the benefits and costs of bid protests—legal challenges by a bidder to the way the gGovernment has conducted a procurement.[1] In the U.S. [MFM1]federal procurement system, bid protests have existed since the 1920s,[2] and criticism arguing that they are more trouble than they are worth has been around for nearly that long.[3][SM2] The article explains why, in the author’s view, the benefits of the bid protest system substantially outweigh the burdens that it imposes on the procurement system.[NAB3]
II.ADMINISTRATIVE AND JUDICIAL VENUES FOR FILING PROTESTS IN THE U.S. FEDERAL SYSTEM
It was in the U.S. that the first litigation leading to what came to be recognized as a bid protest decision took place.[4] In 1924, a company filed a complaint with the then-new General Accounting Office (“GAO”),[5], alleging that the officials of the Panama Canal had written specifications for a truck that were “wired” to a particular brand name and that unfairly precluded the complaining firm from fair consideration for the contract.[6] GAO, after some internal hesitation, decided to consider the complaint as part of the office’s responsibility to ensure that funds appropriated by Congress were lawfully spent—what is referred to as GAO’s account settlement function.[7] GAO ultimately agreed with the protesting firm.[8]
In the course of the ensuring decades, handling bid protests became a routine function of GAO’s Office of General Counsel.[9] Some protests involved pre-award challenges by potential offerors to solicitation terms;[10] the majority were post-award challenges by firms that had competed for a contract, contesting the award to another offeror.[11]
For many years, courts did not consider bid protests, so that GAO (and the contracting agencies themselves) represented the only place to file a protest.[12] Then, for three decades after the decision of the U.S. Court of Appeals for the District of Columbia Circuit in Scanwell,[13] U.S. district courts had bid protest jurisdiction, until that jurisdiction “sunset” in 2001.[14] From the enactment of CICA in 1984[15] until its jurisdiction was ended through section 5101 of the Clinger Cohen Act of 1996,[16] Pub. L. No. 104-106, there was another administrative forum with jurisdiction over some protests, those related to information technology, and that was the General Services Administration’s Board of Contract Appeals, the GSBCA.[17] In addition, a statutory change in 1996 meant that the Court of Federal Claims,[18] which had only pre-award protest jurisdiction for many years, was given post-award jurisdiction as well.[19]
The result is that, for more than a decade now, the only places outside the contracting agency where disappointed bidders have been able to protest are GAO and the Court of Federal Claims.[20] From time to time there are differences between GAO and the Court, with respect to both process and outcomes.[21] The author views occasional differences between two fora as inevitable, and that is particularly the case here, where one forum is administrative and the other is judicial.[NAB4] In any event, having two fora hearing bid protests may be healthy for the procurement system.[NAB5] While the discussion below regarding the costs and benefits of protests to the procurement system is focused on GAO, the analysis should apply, at least in broad terms, to the Court of Federal Claims as well.[NAB6]
III.SPREAD OF THE PROTEST PROCESS OUTSIDE THE U.S.
Before turning to that analysis, it is worth looking outside the U.S., because the protest process has been receiving substantial attention around the world.[22] More than ever in the past, a protest system has come to be seen as a required part of a good public procurement system.[23] For example, in every free trade agreement that the U.S. has negotiated over the past 20 years, there has been a provision requiring our trading partners to have a protest mechanism.[24] In Canada, for instance, the Canadian International Trade Tribunal was created to satisfy the requirement of NAFTA, the North America Free Trade Agreement, that each partner have a protest forum.[25] Similarly, the World Trade Organization’s Agreement on Government Procurement (GPA) includes a provision requiring WTO members that accede to it to have a protest forum (called a domestic review procedure).[26] In addition, there is a protest provision in Chapter VIII of the model procurement law of the United Nations Commission on International Trade Law (UNCITRAL).[27]
Perhaps most interesting is the attention that protests have received over the past 20 years in the European Union (“E.U.”).[NAB7] Not mentioned in the EU’s Public Procurement Directives,[28] protests were first addressed by the European Commission in what is known as the Remedies Directive.[29] Initially issued in 1989,[30] the Remedies Directive was revised in 2007.[31] The Remedies Directive has had an enormous impact, requiring all member states to have a protest forum,[32] and a good number of EU member states have recently seen a significant increase in the number of protests being filed (whether that is a positive or a negative development is a separate question).[33] The Court of Justice of the European Union has issued decisions that have reshaped the protest process in the E.U., in particular, through the Alcatel decision[34] that led to the requirement (codified in Article 2a of the 2007 revision to the Remedies Directive) that there be a “standstill” period (typically 10 days) between the announcement of the apparent winner of the competition for a contract and the signing of the contract, in order to allow potential protesters to file before the contract is actually awarded.[35]
IV.CORRECTING MISPERCEPTIONS ABOUT PROTEST STATIISTICS
In the circle of people who are interested in federal procurement, there are a number of misperceptions about protest statistics that should be addressed, since they can taint judgments about the benefits and costs of protests.[NAB8] In particular, even people quite familiar with the federal acquisition system believe that protests are more common than they really are, and they often believe, inaccurately, that protesters frequently use the protest process to win contracts.[NAB9]
A.Protests are rare events
The frequency or rarity of protests needs to be judged in context in order for the judgment to be sensible.[NAB10] If a contracting officer is asked whether, in the procurements she or he has worked on over the prior 3 years, protests were frequent, an affirmative answer is often provided;[NAB11] but if one asks how many procurements the contracting officer has worked on during that period, that number often dwarfs the number of protests.[NAB12] Put another way, while the numerator (the absolute number of protests) is important, the denominator (the total number of procurements) is critical to a determination of whether protests can reasonably be said to be frequent.[NAB13] In assessing the frequency of bid protests in the federal procurement system, there are difficulties with both the denominator and the numerator – respectively, the number of federal procurements that occur each year, and the number of protests filed each year.[NAB14]
Regarding the denominator, perhaps surprisingly, there are not good statistics on the number of federal procurements that occur each year.[NAB15] Any such count would need to include, not only the number of contracts awarded, but also any task and delivery orders awarded that can be protested.[NAB16] The latter includes all orders placed under the GSA Federal Supply Schedule and orders issued under multiple-awarded indefinite-delivery, indefinite-quantity contracts, where the orders have a value above $10 million.[36] A recent RAND Corporation study of Air Force procurements that were protested to GAO indicated that approximately 20,000 contracts with a value above $25,000 were awarded by the Air Force in 2008, representing approximately $63 billion.[37] Since overall procurement spending in 2008 was over $500 billion (eight times the amount spent by the Air Force alone),[38] that would suggest that the overall number of contracts awarded was approximately 160,000.[NAB17] The author suspects that the number is low and that a better estimate, including FSS orders and ID/IQ orders above $10 million, would probably substantially exceed 250,000.[NAB18] For the purpose of the analysis in this paper, it will be assumed that the total number of federal contracts and protestable orders awarded in a year is 200,000.[NAB19]
With respect to the numerator – the number of protests filed—there has been confusion,[NAB20] for reasons that can be clarified here.[NAB21] One challenge has been a methodological anomaly that is worth explaining.[NAB22] Since the days when protest filings were tracked on 3” x 5” cards, GAO has used a methodology that can cause people to believe that protest numbers are higher than they actually are.[NAB23] When a company files a protest challenging the terms of a solicitation, GAO assigns it a docket number, referred to as a “B number,” since it begins with a “B” – for example, B-123456.[39] If the protester later learns new information that constitutes a new ground of protest and files a supplemental protest of that same solicitation’s terms, GAO will docket that as B-123456.2.[40] If another potential offeror also protests the solicitation’s terms, GAO will docket its protest as B-123456.3, and if that firm supplements its protest, that will be docketed as B-123456.4.[41] However those pre-award protests are resolved, once the agency awards a contract, a losing competitor may file a protest, which GAO will docket as B-123456.5 and, if there is a supplemental protest once the protester sees the agency report, that will be docketed as B-123456.6.[42] And finally, if another losing competitor files a protest and then supplements it, they will be docketed as B-123456.7 and B-123456.8, respectively.[43] Overall, GAO’s statistics will indicate eight protests, even though only one procurement has been protested.[NAB24] While eight protests of one procurement would be unusual, having two protests is routine, and three would not be out of the ordinary.[NAB25]
What that means is that, when GAO reports the number of protests filed in a year, that number substantially overstates the number of procurements protested.[NAB26] For Fiscal Year (FY) 2008, for example, while GAO reported 1,652 cases filed,[44], elsewhere GAO reported that 1,027 procurements were protested in that period.[45] In other words, GAO indicated that, on average, there are approximately 1.6 docket numbers assigned (“cases filed”) for each protested procurement.[NAB27] Assuming that this ratio is stable over time, it would mean that the 2,353 “cases filed” in FY 2011[46] represented approximately 1,470 protested procurements.[NAB28] While the number of federal procurements was probably not the same in FY 2008 and FY 2011, if we assume, for the sake of simplicity and since this is only a rough estimate, that there were 200,000 procurements in each of those two years, that would suggest that 0.51 percent of procurements were protested in FY 2008 and 0.7 percent were protested in FY 2011.[NAB29] In other words, between 99.3 and 99.5 percent of procurements were not protested.[NAB30]
This calculation has been confirmed through the recent RAND study mentioned above.[47] In that study, researchers at the RAND Corporation found that, in the Air Force, “[t]he number of [GAO] protests as a percentage of total contract awards fell fairly steadily from about 1.7 percent in 1995 to 0.5 percent in 2008.”[48] That is to say, as of 2008, 99.5 percent of Air Force procurements went forward without being protested to GAO.[NAB31] Even if GAO protests of some other agencies’ procurements were twice as common as for the Air Force (and there is no reason to assume that that is the case),[NAB32] it would still be true that 99 percent of their procurements went forward without being protested to GAO.[NAB33]
While the RAND study refers to GAO protests, including the Court of Federal Claims would not alter the picture, since the Court receives fewer than one-tenth the number of protests each year that GAO receives[MFM34].[49] And while protesters can file protests within the Air Force and statistics on the number of those “agency-level protests” are not available,[NAB35] there is no reason to believe that they would increase the total percentage of protested Air Force procurements much above 0.5 percent.[NAB36]
It is, of course, true that very high-dollar procurements are much more likely to be protested—the higher the dollar value, the greater the likelihood of a protest.[NAB37] For a company that loses the competition for a $100 million contract, with all the bid and proposal costs that competing entails, the additional cost of filing a protest may seem minimal, so that filing a protest can be very tempting.[NAB38] That does not change the overall picture, however, especially since very high-dollar procurements are themselves few in number:[NAB39] protests are rare.[NAB40]
What about the mantra we often hear about increases in protest numbers? It is true that GAO has reported a substantial increase in the number of cases filed over the past few years,[50] but even if the numbers doubled, from 0.5 percent of procurements to a full percent, it would still mean that something like 99 percent of procurements are not protested.[NAB41] In terms of absolute numbers, GAO has reported that the number of protests rose from 1,327 in FY 2006 to 2,353 in FY 2011, an increase of more than 70 percent.[51] Adjusted to eliminate the overcounting explained above,[52] that would represent an increase from approximately 830 to approximately 1,470 protested procurements (representing the same percentage increase).[NAB42] During that same period, however, federal procurement spending increased from $432 billion in FY 2006 to $537 billion in FY 2011.[53] Put another way, in FY 2006, there were approximately 1.92 protests for each billion in federal procurement spending, while in FY 2011, there were 2.74 protests per billion.[NAB43] Those figures are similar to the ones that GAO provided in a congressional report in 2009, when it stated that the number of protested procurements per billion dollars in Department of Defense contract spending ranged from 1.4 to 1.9 during the period FY 2004 to FY 2008.[54] While the FY 2011 figure reflects an increase, the number of protests for each billion in federal procurement spending is still extremely low: fewer than three protests for each billion dollars that the government spends on contracts.[NAB44]
B.It is rare for a protester to win a protest, and even rarer for a winning protester to go on to obtain the contract at issue in the protest.
Here again, GAO’s methodology risks misleading observers.[NAB45] GAO reports a “sustain” rate that has ranged (that is, the rate at which GAO rules in favor of the protester and sustains the protest), over the past 5 years, from 16 to 27 percent.[55] That sounds like protesters do fairly well.[NAB46] For better or for worse, the full picture is not so favorable to protesters.[NAB47] First, the sustain rate has been dropping nearly consistently over the past 5 years, falling from 27 percent in FY 2007 to 16 percent in FY 2011.[56] Second, the sustain rate is calculated only among the cases for which GAO issues a merits decision, as GAO explains in its annual reports.[57] That means that, in FY 2010, for example, GAO did not sustain 19 percent of 2,299 cases – it sustained 19 percent of 441 merit decisions.[58] Third, that percentage is distorted by the methodology of counting multiple B numbers separately, because protests that are sustained typically have more B numbers than protests that are denied.[59] Thus, for FY 2010, GAO reported 441 merit decisions, of which 82 were reported as sustained protests.[60] A count of the actual decisions (counting each decision as one, even if it resolves two or more B numbers) shows that there were actually 277 (not 441) decisions, of which 45 (not 82) were sustained.[61][NAB48] That represents a 16 percent sustain rate (not 19 percent).
Thus far, this analysis means that, among the hundreds of thousands of federal procurements that occurred in FY 2010, there were only 45 where GAO sustained protests. The next stage, though, offers even worse news for protesters, and it is surprising how little is reported about it. What happened in those 45 procurements, after GAO sustained the protests? Did the protester that was successful in the GAO litigation succeed in obtaining the contract? The answer: Rarely.[NAB49] The FY 2010 numbers have been selected here for further study, because enough time should have passed for final action in the procurement to be available.[NAB50] That said, discovering the final action can be challenging, because information on what ultimately happened in each one of the sustained protests is not readily available, but enough is known to give a fairly clear picture of protesters often winning at GAO but nonetheless not receiving the contested contracts.[62] In four of the 45 cases, GAO did not recommend any corrective action in the protested procurement, either because the contract had already been performed or for other reasons.[63] In an additional three cases, GAO did recommend corrective action in the procurement, but the agency explicitly declined to follow GAO’s recommendation.[64] In another case, the agency cancelled the solicitation entirely (for reasons, and with impact, unclear).[65] In 16 additional cases (more than one third of all the protests sustained in FY 2010) , GAO recommended corrective action, the agency followed GAO’s recommendation, but the agency then confirmed award to the same company as before (or awarded to a third company, neither the earlier awardee or the protester).[66] In only four cases identified to date did the protester ultimately obtain the contested contract.[67] While the ultimate outcome has yet been determined in the remaining 17 procurements,[68] this much is clear: winning a protest is far from ensuring that a protester will win the contract it is seeking.[NAB51]