Procurement: Focusing on Performance and Results

By Steve Kelman

The federal government spends in the order of $450 billion a year buying goods and services from suppliers,[1] corresponding to about 40% of federal discretionary spending. The large sum of money involved by itself suggests the importance for senior agency leadership of paying attention to how all that money is being spent. In addition, many – though hardly all – products and services bought from contractors are critical to agencies successfully accomplishing their missions. Examples include weapons bought for the military, IT development and operation of services bought for weather forecasting, identification of possible terrorists, or the administration of tax and social security systems.

This report suggests a strategy for procurement management in the new administration that is focused around improving the performance and results of the procurement system. The approach here is to present a modest number of targeted recommendations rather than a long laundry list.

These recommended actions are:

     ·      Develop more information about contract performance;

     ·      Pivot to post-award; and

     ·      Expand forms of contracting that pay for success.

1.     Develop More Information about Contract Performance

The government, not to speak of the general public, has remarkably little information about how well the procurement system is performing in general. A moment’s reflection will explain why: getting generalized information about performance creates insurmountable apples-to-apples problems. The government buys an enormous myriad of products and services, with endless permutations of possible performance indicators. It is thus impossible to develop any global measure of overall performance for such a wide range of products and services. Having said that, the paucity of available performance information is a significant problem for government executives and for procurement managers, as well as for the public, which have little to go on other than journalistic accounts of individual contracts. Even if perfect information is impossible, ways to improve the performance information the system produces should be a priority. There are two reasons for such disclosure, transparency and performance improvement. Such information would help the public (as well as senior government executives outside the contracting system) judge better how well the system is performing. Such information could also be used inside an agency by contracting officers to help them make buying decisions or by a procurement organization for internal performance improvement activities.

                  ·      Look for opportunities to provide pricing information, both inside the government and to the public, on commonly purchased commodity products and services

Where can better contract performance information feasibly be provided? A potential target of opportunity is price information for commodity products or services the government buys in large contracts negotiated for government wide use, by agencies such as General Services Administration and the Defense Logistics Agency (DLA), with a limited number of vendors. Examples would be office supplies, IT hardware, airfares, food for the military, construction materials, industrial hardware, and package delivery. Such commodities represent only a modest slice of the procurement system, but it is a good place for the next administration to start. The new administration should build on efforts begun in the last two years by GSA to improve pricing transparency, as a way to provide the public with information about the performance of the system.

Expanded information disclosure to the public raises two different kinds of issues:

o   Presenting such information in an accessible and non-misleading form; and

o   Political opposition by contractors to information disclosure about their prices.

The contracts being discussed here typically have thousands of line items and thus, in raw form, would be largely useless to most members of the public. So if pricing information were to be disclosed on such contracts, it would need to be limited to a modest number of items, possibly the top-five or top-ten in terms of sale. Insofar as possible items chosen for disclosure should be ones screened for being intuitively understandable to the public. There are also private companies that sell software (currently being bought by GSA) that allows comparison with prices at which a larger number of items on a contract are sold to large corporate customers. However, the GSA arrangements with the data providers do not allow such information to be shared either inside the government or with the public; the right to do so would clearly cost money, which GSA would surely be unenthusiastic about paying unless this were seen as a government-wide priority such that OMB put funds to pay for such access into the president’s budget request. Such data, if it could be disclosed, would be more useful than data on a small number of specific items, and would not need to be limited to a few items, but could be done for a larger market basket.

Additionally, there are a few contracts involving intuitively understandable products where the number of items bid is not too large to be understandable to the public. Three of these are prices for airline tickets and small package delivery services, both negotiated by GSA, where there is a far-more limited array of prices (for airline tickets, one contract per “city pair,” e.g. Washington to Los Angeles, and for small package delivery, a small range of delivery options and package sizes). DLA also has a contract for providing chicken parts to soldiers.

For some commodity categories, any price information that is disclosed is likely to mislead the public, because legislative provisions requiring that items sold to the government must be made in the U.S. (or in countries signatory to the Trade Agreements Act) significantly increase prices the government pays compared to what a member of the public would pay. This is most dramatically the case for IT commodity hardware, much of which, for the non-government market, is produced in China and thus cannot be sold to the U.S. government, and to some extent is also true for office supplies.[2] Side-by-side comparisons of the price for a laptop or other IT commodity would suggest the government is getting a terrible deal, but this is driven by congressional decision, not the poor performance of the procurement system. Such commodities should not be subject to public disclosure. However, it may be possible to develop a subset of commodities where these considerations do not dramatically distort the disclosed prices – a study a few years ago that GSA did for Congress concluded that 40% of office supply item prices were significantly raised because of Trade Agreements Act compliance, meaning 60% were not. GSA should determine whether price data on some subset of IT or office supply products might be disclosed without these distortions.

A second concern is that contractors will oppose public disclosure of their prices, out of a worry that disclosure would reveal information on their government contracts to their own commercial customers. This fear suggests, interestingly, that these suppliers believe the government is getting a very good deal, which they don’t want other customers to know about! The government would need to decide for public disclosure despite industry opposition. [3]

Those consulted inside the government for the preparation of this report believe there would not be significant industry opposition to disclosure of airline ticket prices; the government has already done this to some extent, and the airline industry is vigilant in preventing non-government employees from using these fares. It was also suggested to the author of this report that the small package industry would probably not strongly object to disclosure.

One important way to reduce the intensity of contractor worries about information disclosure to the public would be to have such disclosure occur for average prices for all the relevant contract holders for the item in question, limiting disclosure to contracts with a sufficiently large number of suppliers such that information could be anonymized rather than traced back to a specific contractor. DLA’s contract for chicken parts, which otherwise might be a priority for public disclosure because it is a highly intuitively understandable single commodity, has only one awardee, making anonymization impossible (at least for anyone who accesses the already publicly available information on who a contract holder is) and ensuring the opposition of winners or potential winners of this contract to public disclosure. It then would be a policy decision about whether to move forward despite this, on the grounds this would be helpful information to the public that would also counteract impressions the government is typically overcharged by contractors (additional information-gathering would be necessary to judge the benefits and risks here). Certainly, it is not unknown for the government to make decisions that industry opposes.

It is infeasible without significant further research to specify what commodities, given the considerations mentioned above, would be appropriate for public price disclosure. Based on information available at this point, the first two commodities to subject to price disclosure – used to initiate what might become a larger program over time, should be airline tickets and small package delivery. We also suggest immediately pursuing some limited disclosure of average prices for some commonly bought items where there are a significant number of contract holders and thus anonymization might be a feasible way to reduce industry opposition. The government should thus evaluate for which GSA and DLA contracts, or parts of them (particularly a small subset of most-purchased items), are most appropriate for disclosure given the criteria discussed above. We suggest that OFPP and/or GSA provide some staff assistance to do further research on possible targets of opportunity for disclosures of prices paid for commodities to the public. We hope that, if these initial disclosures work well, there might be opportunities to expand the scope of price disclosure over time.

There are two kinds of pricing data where the government should look for opportunities to improve disclosure both inside the government and to the public. One is contract line-item pricing data, i.e. the prices established in the original contract. For contracts where prices are not adjusted downward for individual transactions (such as airline tickets), such prices might be all that is needed. However, for most GSA and DLA multiple-award contracts, contract prices can be negotiated downward of competed among contract holders for individual transactions, and indeed the regulations encourage agencies buying large quantities at a time to seek to negotiate such discounts. Transactional data indicating actual prices paid for specific buys of some specified commodities would be valuable in such situations, though contractor opposition might be greater to disclosure of transactional than contract prices, and in this case contract prices might be a decent second-best alternative.

In June 2016 GSA, working with the government wide category management initiative led by the OFPP, published a rule establishing a pilot program for the disclosure by a contractor of transactional pricing data for contracts where contract prices can be adjusted downward (https://www.federalregister.gov/articles/2016/06/23/2016-14728/general-services-administration-acquisition-regulation-gsar-transactional-data-reporting). The rule only addresses disclosure inside the government, but GSA announced that it would enter into a dialogue with interested parties about what kind of information should be included in a publicly available “data extract” from submitted transactional data; GSA is strongly considering including basic price information in extracts. These extracts would allow anyone to access and use whatever data is made available.

A different opportunity for performance information disclosure involves the many IT contracts that include service level agreements at the task order level for non-price service features such as system uptime, response speed, and user satisfaction. Such information is currently not readily available either inside the government or to the public. As part of its Acquisition Gateway portal, GSA hopes to develop a repository of such information. Developing such a repository should be a priority for the Gateway, and such information should be released to the public as well.

Finally, there already exists some publicly available data, called the IT Dashboard (https://www.itdashboard.gov/), on how well the procurement system is performing for IT systems development contracts. Again, this is not pricing data, but does have information on cost and schedule performance (although the data must be taken with a grain of salt given changing baselines for many of these projects over time), as well as on a number of other contract performance metrics, although they are generally not standardized across contracts.

·      Use performance information for quarterly performance-improvement reviews on major contracts

A different way to increase the role of performance information in the procurement system would be for the government to use quarterly performance-improvement reviews among those working on some big contracts (contractor representatives should probably be included as well, chaired by more-senior management, to assess and discuss progress being made to meet any performance metrics the contract is using, and to discuss strategies for improvement. Here the performance data made available on a contract, and/or on comparable contracts elsewhere for benchmarking or on a contract over time, is important for learning how to do a better job. These reviews – which are often called “STAT meetings” (Behn 2014) from the famed COMPSTAT system used by the New York City police – would be similar in format and intent to the reviews mandated by the Government Performance and Results Modernization Act to examine agency progress under agency priority performance goals. Most periodic meetings between a contractor and a government customer are currently status reports and show-and-tells, rather than discussions of where progress is being made, where it is falling short, and, in the latter case, what steps can be taken to improve. Those involved in contracting generally have very limited experience with this type of performance-improvement review, and government contract managers would need to be provided training on how to organize them.

 

2.     Pivot to Post-Award

The contracting lifecycle is divided pre-award (acquisition strategy, requirements definition, source selection) and post-award stages. Traditionally, government has devoted much attention and time to the stages of the pre-award process. In contrast, the post-award stage has received relatively scant attention, low visibility, and often insufficient resources. Some even worry that, to borrow the title of a 2008 brief on the topic written by Allan Burman, former Administrator of OMB’s Office of Federal Procurement Policy, that after contract award, nobody is “minding the store,” a view that has also been expressed by outside critics of the procurement system (Burman 2008; Freeman and Minow, editors, 2009). Yet it is during this third stage that the acquisition rubber meets the performance road. This is where contractors perform well or poorly.

The next administration, at the highest procurement leadership level, should announce and help execute, not as a one-time announcement but over a period of years, a “pivot” from time and resources spent on pre-award activities into post-award contract management work. The “pivot” would involve additional bodies being moved into post-award management and monitoring, improved training in post-award management, and establishment of fora for sharing experiences and best practices across an agency by those involved in post-award management. The CAO Council should adopt the improvement of post-award management as a theme and develop initiatives of its own under that theme. Such a pivot would likely be viewed favorably both by senior procurement professionals inside the government and by outside critics worried about “minding the store.”

Post-award management is the responsibility of two kinds of officials with confusingly similar names, “contracting officers” and “contracting officer’s representatives” (COR’s), the latter of whom come out of program offices and are normally subject matter experts on what the agency is buying. Although the division of responsibilities between these two groups varies by agency and situation, generally the COR is the key official for post-award management, working day-to-day with the contractor to track performance through informal feedback to and follow-up with the contractor, and dealing with problems that have emerged or may be emerging.

Currently, the status of COR’s often, unfortunately, reflects the low standing of post-award management more generally. Many COR’s work only a modest fraction of their time at COR responsibilities, giving the job a devastating “other duties as assigned” character. Sometimes the COR’s boss may not even know the employee is a COR. COR’s seldom participate in any COR community of practice or forum designed around their key roles in contract management.

                 The role of the COR in government contracting needs to be significantly upgraded.

·      COR responsibilities should be more concentrated in a smaller number of COR’s working fulltime or most of the time on COR responsibilities

·      Agencies should consider replacing the bureaucratic and uninspiring job title "contracting officer’s representative" with one that is more engaging and mission-oriented, such as "contract performance manager

The COR job description should be revised accordingly to reflect the importance of this key function.

                  ·      COR training currently heavily focuses on formal job duties and regulatory requirements. It needs to move much more to be training on management and leadership skills

This should include training in conflict resolution and in ways government can deal with performance issues, as well as specific training on evaluating contract deliverables, including evaluating performance against metrics. For example, in cost-reimbursement/time-and-materials contracts, there is little the government formally can do in the case of unsatisfactory performance short of the legally very difficult step (almost always legally challenged by the contractor) to terminate the contract, and the government often ends up paying the contractor for work to remedy earlier problems they created or contributed to. But smart contracting officials use more-informal ways, including phone calls from agency leadership to a company’s CEO and agreements with the contractor to provide some free labor (or at least just costs, no fee) or outside consulting help to deal with problems, in “consideration” for not cancelling the contract. Discussing these types of alternatives is an example of something that should be standard in COR training.

     ·      Agencies should establish ways for COR’s to share experiences and best practices, and to get advice from other COR’s

Although COR’s seldom report to contracting officials, the professional community most-interested in an upgraded role for them is contracting. The CAO Council should therefore take the lead, working with the CIO and CFO Councils (organizations for whom many COR’s actually work) and, hopefully, with the President’s Management Council as well, to establish fora where COR’s can interact and discuss good practices. One possible home for such a forum is the Office of Federal Procurement Policy, which already runs a “Frontline Forum” for contracting officers.

Where can the government get resources for additional post-award activities? Some could come from streamlining parts of existing post-award management practices. It is widely believed that government requires contractors to submit too many low-value reports, which uses up considerable contractor time and either causes the government to spend unnecessary time digesting or just generates unused paperwork. More work to process contractor invoices could be given to lower-graded civil servants; more standardization of invoice forms would also reduce government and contractor burden.

In addition to this, the government should aggressively seek opportunities further to streamline (progress was made in the 1990’s) source selection activities. Without such streamlining, the ability to pivot resources to post-award may be limited.

For example:

o   Agencies that award multiple-award task order contracts should be more aggressive about limiting the number of initial awardees

A tendency grew up over the years to give initial awards to almost everyone who bid (contradicting the original statutory basis for the streamlined competition for task orders), in order to avoid complaints from contractors. When there is a more-modest number of awards at the contract level, proposal evaluation for task orders takes less time, and the government may be more apt to take other streamlining steps, such as proposal page limits. At least one agency that awards multiple-award task order contracts looks at past experience in the subareas of the contract, and aims for a number of awards that will typically generate 3-5 proposals instead of double digits. Another agency seeks to limit the total number of awardees to 15. Others should consider these agencies’ lead. Broad scopes could be subdivided into a larger number, with fewer awards per subdivision.

o   Oral presentations should be reinvigorated and modernized

This was a streamlining technique initiated in the 1990’s, which has somewhat fallen out of use. One agency has re-engineered oral presentations to make them more like job interviews, with the government asking questions in real time rather than passively listening to contractor slide shows. They report this has shaven 2-3 months off of source selection and improved the information the government gets.

o   The proliferation of executive orders imposing government-unique requirements and certifications on contractors is a problem

Taken together, these require significant time spent by contracting officials in connection with source selection, while adding no value to improving the procurement process itself. Each of these special requirements is individually politically difficult to resist, and has loud advocacy/interest group supporters, while the only interest on the other side is the diffuse interest of taxpayers.

If an agency assigns contracting officials to work on a contract “cradle to grave,” re-allocating contracting resources from pre-award to post-award is easier, because resources for both come out of the same budget codes. However, COR resources, which make up the majority of post-award management, generally come from agency program budgets for contract performance, and cannot easily be re-allocated from contracting dollars. But paying for COR’s is a tiny fraction of program budgets, and it is more-easily visible that a small redirection of program dollars into contract management could yield a large return on investment (just as funds for IRS tax auditors do).

3.     Expand Forms of Contracting that Pay for Success

Traditional government contracting for services pays either for the efforts the contractor puts into the work (“level of effort”) or, less commonly, a fixed price for satisfying the specifications in the contract. The former creates clear incentive problems for the government. Incentives for the contractor to achieve a certain result for less money are lacking. In level of effort contracting, the contractor is paid even if they accomplish nothing. These kinds of contracts are commonly signed when the risk is high the contractor will fail to solve the problem the government has set out.

There are, however, alternatives to level of effort contracting for risky projects, known under the collective rubric “pay for success.”

                 ·      Challenges/Contests

Challenges, also sometimes known as “contests,” represent the most-important innovation in procurement practice during the last eight years. The basic idea of a challenge is that an agency advertises to the public a problem it wants to solve. Anyone who chooses can enter the contest with a solution. Participation does not require that the entrant be knowledgeable about the procurement system; indeed, experience both with challenges organized by the federal government and by private-sector firms is that a large number of participants are not typical contractors, but rather “garage” players, often quite young (Jeppesen and Lakhani 2010). The government announces a prize, or multiple prizes, for the winners; it then may choose one winner (the first to solve the problem) or several. If there are no winners, the government pays nothing.

An early example of the use of contests in the federal government involved a “Grand Challenge” organized by the Defense Advanced Research Project Agency for an all-terrain vehicle that could successfully navigate an obstacle course the agency had laid out. In 2010 GSA launched a website called challenge.gov to host challenges throughout the government. From 7 challenges in FY11, in FY15 there were 140. GSA has played a role educating government agencies about how to organize challenges, and also to convene a community of practice for those in government working on them. While most government procurements are not appropriate for challenges, it is hard to imagine that the feasible universe has at this point been close to saturated.

                 ·      Payment per transaction (“pay by the drink”) contracts

The government runs many systems (financial management, HR, procurement, claims application) that process large numbers of transactions. The conventional method to procure such systems is to pay for development, and then have either the government or a contractor process transactions. An alternative method would be for the government to contract with one contractor both for developing the system and then for running it for some number of years. What would turn this into pay for success contracting would be if the contractor received no upfront payment, or only a minimal fixed price that would not cover system development costs, and began to be paid, on a pre-negotiated per-transaction basis (“pay by the drink”), only when the system was actually up and running. This would create a powerful incentive for the contractor quickly to develop a working system; keeping the system well maintained and functional would also increase the number of transactions. Although some contractors have promoted this concept in the past, as of yet it has not been used in the federal government. An alternative version of this that has been used by some cities is to pay contractors for number of hours streetlights are working, which gives them an incentive to install longer-lasting bulbs. One consideration discouraging the use of “pay by the drink” contracts is the view of some, but not all, government financial management experts that such contracts are a form of lease, which then is subject to restrictive rules about upfront funding of lease obligations that are designed to inhibit leasing.[4] Perhaps the CFO and CAO councils could work to develop an interpretation of the regulations that would not be overly restrictive.

                 ·      Share-in-savings contracting

This is a kind of contracting appropriate to the limited number of situations where a contractor’s successful efforts produce such significant savings or increased revenues to the government (large return on contractor investment) that the contractor can be paid, all or in part, in the form of a percentage of savings or increased revenue achieved. Share-in-savings contracts are unusual but not unknown in government. The most-prominent federal examples are energy-savings performance contracts, where companies specializing in helping organizations reduce their energy (often heating and cooling) costs are paid a percentage of the energy savings their activities generate. An interesting and successful example in California over a decade ago was use of a share-in-savings contract to pay for modernization of the state’s tax system, with payments to the contractor being a share of increased tax revenues from a modernized system. These contracts require an ability to agree on a baseline and to measure saving. There are also legal issues that arise because these are multiyear contracts, which have special requirements associated with them. (However, suggestions have been made for how to deal with these legal issues so that contractors might be willing to sign such contracts without insisting that the government set aside in advance funds to cover termination liabilities.)

                  ·      Social impact bonds

A social impact bond is one where “government agencies define an outcome they want to accomplish and agree to pay an external organization a sum of money if the external organization achieves that outcome. …All payments are contingent on the outcome being achieved. If outcomes are not achieved, the government pays nothing. Hence, risk is transferred from the government to the external organization or its investors.” Since the contract only specifies an outcome, the provider has maximum freedom to choose a strategy for achieving the result; the contractor is often typically not subject to government auditing or procedural rules (Center for American Progress 2012). Often the money to pay for the effort to deliver the outcome comes from funds raised by units of for-profit firms (often on Wall Street) doing social investments, or from philanthropists.

Social impact bonds were first used for an effort at the Peterborough Prison in the UK to reduce recidivism among newly released inmates. They have had some use in US state/local government, and there has begun to be interest at the federal level. In particular, some of the monetary benefits from a successful intervention may accrue to the federal government rather than state/local governments; for example, interventions enabling individuals with health impairments to remain in the workforce reduce federal spending on Supplemental Security Income, disability insurance, and Medicare/Medicaid.

Providing outcome payments in proportion to federal savings can justify some projects that would otherwise have found it impossible to generate sufficient benefit The Department of Labor is making outcomes payments on two state initiative projects that are producing federal savings. Legislation appearing in the FYxx President’s Budget and passed, in different form, by the House in 2016 would provide a fund for federal co-sponsorship of such projects.

One important issue to keep in mind for various forms of “pay for success” contracting is that, if contractors are being asked to invest in something risky, the government must accept that the payment in the event of success needs to be greater than it would be for a level of effort contract. (The exception might be social impact bonds, whose investors may not be seeking to maximize returns.) The government has not always been aware of this necessity, hoping to bring forth effort on behalf of risky endeavors at the same price it would pay for conventional contracting where there is no requirement to succeed.

The new administration should continue and expand efforts in the previous one to anchor the role of pay for success contracting in the government procurement mix. OFPP should consider issuing a guide to pay for success contracting, and perhaps organize a forum around this topic. Of the various forms of pay for success contracting discussed above, challenges easiest to expand, but they are appropriate for only small projects (people can’t be asked to invest tens of millions of dollars on the chance of winning). Some of the other pay for success methods are more appropriate for larger contracts, but they are also more complicated.

References

Behn, Robert D. (2014). The performance stat potential. Washington: The Brookings Institution.

Burman, Allan. (2008). Post award contract management: Who's minding the store? Washington: IBM Center for the Business of Government.



[1]This report will henceforth use the word “contractors,” which is more common in government. Current spending is down from almost $550 billion in FY2008.

[2] This also applies to items produced by AbilityOne sheltered workshops.

[3] The government would get hurt, however, if there were any significant number of suppliers who withdrew from selling to the government because of required disclosures. It is unlikely that, if these suppliers give the government a better deal than commercial customers, this is because of some special affection for the government; the discounts should be able to be explained to commercial customers as being based on quantities sold or other generic considerations.

[4] These issues have arisen in the context of paying for cloud computing services, which do not involve the same upfront investments as are being discussed here.