Abstract
As state governments expand the use of private contractors to provide public services, they create challenges to performance management and accountability. Using the framework of accountability as a social relationship, we evaluate New Jersey’s oversight practices. We combine data from interviews and observation with a comprehensive analysis of the institutional framework. We raise two key questions. First, what causes New Jersey to neglect its performance management responsibilities? Second, how can New Jersey and other states strengthen their performance management and accountability practices? We posit that the state must retain some level of internal monitoring capacity as a core element of government. Effective oversight requires rebuilding administrative capacity and implementing ongoing, flexible, relational contract management that involves key stakeholders and does not put the entire burden of performance measurement on direct service providers. In this way, performance management and accountability can be linked together as tools for reflection and learning.
Keywords
Managerialism, markets, and the shift to the contracting state
Over the past 30 years, changes in public administration and the way states provide services to their citizens have been influenced by the normative model of New Public Management (NPM). This model is dominated by twin threads of a preference for market-based approaches to provision of public services and a belief that performance measurement and management would make public organizations more accountable. However, the implementation of the first premise, expanding the use of private contractors to carry out a wide variety of government functions, makes it more difficult to carry out the second directive – measuring and managing performance.
The shift from direct service provider to purchaser of services represents at least a partial “reconfiguration of the state’s role in governance,” in which the contracting state emerges (Freeman, 2000: 158). This new role may not require the state to deliver services directly, but it does require the state to have sufficient capacity to oversee its contractors and effective strategies for carrying out that oversight. Democratic society is based on the premise that government must answer to the public for its performance (Radin, 2006). In the public sector, managers and agencies have a responsibility to explain and justify their actions. This is where accountability comes into play.
Although accountability may be conceptualized in various ways, our focus here is on accountability as a social relationship. The model employed by Mulgan (2003) and Bovens (2007) portrays accountability as a relationship between an actor and a forum. We adapt this model to illustrate the challenges posed when the government uses third-party contractors to perform government functions. Under these circumstances, accountability is a relationship between the third-party service providers and the state in which the state must ensure performance to the terms of the contract. A sound relationship requires a non-governmental actor to provide ongoing accounting to the public agency that issued the contract. At every stage, the public agency must closely review and evaluate the non-governmental actor’s performance, providing feedback, suggesting improvements and if necessary, imposing sanctions. In other words, the public agency must conduct oversight.
The accountability chain does not end here. The public agency, not the non-governmental actor, is ultimately responsible for the quality of services delivered. Public agencies must answer to multiple stakeholders including elected officials, the courts, the media, interest groups, clients, and citizens. This layering on of actors and overseers exacerbates the problem of many hands – determining who is answerable for actions (Bovens, 1998; Thompson, 1980) and many eyes – the need to answer to multiple stakeholders with different evaluation criteria (Bovens, 2007).
Our research delves deeper than much of the scholarly literature on contracting, both in order to understand and explicate the specific weaknesses in New Jersey’s contracting for public service delivery and in order to grapple with the broad set of conceptual problems contracting poses. Our institutional analysis looks at the government structures, administrative procedures and legal opinions that together comprise the conceptual framework within which agency personnel are expected to hold contractors to account. This work extends previous scholarship on the contracting process. Although multiple studies have examined the challenges that contracting for public services pose for performance management and accountability, few have combined a comprehensive institutional analysis with an examination of contracting behaviors.
We begin by laying out the challenges of the contracting state as identified in the public management, private supply chain management, and organization theory literature. Next, we evaluate New Jersey’s oversight practices using the framework of accountability as a social relationship. Our analysis uncovers numerous legal, regulatory and procedural failings. Although the role of the state has shifted fundamentally, many of New Jersey’s institutions have not adapted. In fact, the state pays surprisingly little attention to what its agents actually do and often ignores malfeasance on the part of the agents.
We raise two key questions. First, what causes New Jersey to neglect its performance management responsibilities? Second, how can New Jersey and other states strengthen their performance management and accountability practices? Based on our detailed examination of specific administrative failures, we conclude with policy recommendations to help states better exercise their oversight responsibility.
Contracting and accountability: Promises and problems
The widespread adoption of NPM has resulted in dramatic increases in contracting and shrinking governments (Joaquin and Greitens, 2012; Osborne and Gaebler, 1993). NPM reforms rely heavily on a set of assumptions about how market principles apply in the public sector. The overarching premise is that the use of private providers chosen through a competitive process will increase efficiency, almost automatically improving quantity, quality, price, or all three (Donahue, 1989; Kettl, 1993; Pack, 1987; Romzek and Johnston, 1999).
Likewise, the performance movement is purported to provide evidence-based decision-making tools to public managers. It is assumed that performance measurement and management will increase accountability, responsiveness, and operational performance of public organizations (Dubnick, 2005). Moreover, it is argued that performance measurement and management provide value to stakeholders and elected officials, and promote trust in government by making governmental activities more transparent (Radin, 2000; Sanger, 2008).
Both markets and the performance movement have their limitations. For example, markets are only efficient under certain conditions. More specifically, markets discipline producers to improve performance, price, or both only where other producers are available and consumers are well informed and can change to other producers quickly and with negligible cost (Frank and Bernanke, 2013). Unfortunately, these conditions rarely exist in public contracting and market failure is likely to result (Romzek and Johnston, 2005; Sclar, 2001; Smith, 1996).
In fact, it is almost impossible for competitive conditions to exist in contracts for complex social services provided directly to citizens due to the lack of clearly-defined measurable outcomes, long treatment time frames, conflicting goals/values, and the need to build cooperation and coordination among providers when working with vulnerable populations (Brown et al., 2006; Milward and Provan, 2000). The mismatch between market models and the nature of public services makes it difficult to measure and manage the performance of contractors. In such an environment, the risk of shirking is heightened and the need to measure performance and enforce penalties for non-performance is amplified (Brown and Potoski, 2005; Van Slyke, 2007).
Moreover, the provision of public services typically involves multiple principals and stakeholders, political influences and interdependencies, and nonstandard outputs. All of this makes constructing performance benchmarks and accurate measurements in the public sector difficult and costly. As a result, most public sector organizations do not develop the intricate contracts and incentive systems needed to improve performance (Heinrich and Marschke, 2010).
Coping with oversight challenges
Various scholars have noted the shortcomings of the performance movement. Some of the criticisms include: the performance movement reduces the responsibility of elected officials and increases the exposure of public managers (Dubnick, 2005); holding professionals accountable to performance standards can lead to inappropriate and ineffective behaviors (Radin, 2006); states have failed to expand managerial authority needed to achieve results (Moynihan, 2006); and too little attention is devoted to implementing performance management with third-party government (Frederickson and Frederickson, 2006). Yet the ideological push to make government more efficient and business-like remains. What if anything can be done to hold government accountable when at the same time contracting out for service delivery?
The literature on public management and supply chain management offers some guidance. The literature on relational contracting (Sclar, 2001) and collaborative communities (Adler and Heckscher, 2006) focuses on the effort to promote tight, mutually beneficial relationships between the principal and agent. Such relationships help mitigate the high transaction costs, goal incongruence, and limited competition that make governing the contracting relationship exceedingly difficult in most circumstances. Similarly, the accountability and performance management literature also recommends the interactive dialogue model (Moynihan, 2008), learning perspective (Bovens, 2007), managed networks (Frederickson and Frederickson, 2006), and allowing for learning and adjustment, participation of a range of actors, and using a variety of data sources (Heinrich and Marschke, 2010; Radin, 2006) as ways to overcome these challenges.
The approaches discussed above are extremely resource intensive (Heinrich and Marschke, 2010). They require sufficient numbers of highly qualified contract managers to engage in deliberative practices that facilitate positive contracting relationships and outcomes. Contract managers are thus expected to be boundary spanners who understand the public and business arenas and are able to manage the contradictions inherent in contracting (Joaquin and Greitens, 2012).
The existing scholarly literature provides a good starting point for examining the links between contracting, performance management, and accountability. Yet, many of the previous studies provide only a partial picture of the contracting environment and offer limited explanations of the reasons for contracting failures. For example, Dubnick (2005) theorizes about the link between accountability and performance and concludes that it is untested. Heinrich and Marschke (2010) synthesize existing literature to highlight key difficulties in designing public sector performance management systems and make recommendations for improvement, but offer little in the way of empirical support. Radin (2000) analyzes the impact of a single policy, the Government Performance and Results Act of 1993.
Johnston and Romzek (1999) and Sanger (2008) rely primarily on interview data with public managers and other stakeholders. Johnston and Romzek (1999) focus on program implementation. Sanger (2008) focuses on impediments to performance management. Brown and Potoski (2005) and Joaquin and Greitens (2012) draw mainly on survey data. Brown and Potoski (2005) examine perceptions of transaction costs, while Joaquin and Greitens (2012) investigate the manner of service delivery and types of services contracted out.
Moynihan (2006) and Van Slyke (2007) take a more comprehensive approach. Both of these studies draw on multiple sources of data and examine multiple sites. We also take a multi-site, multi-method approach and extend it by conducting an in-depth analysis of the relationship between institutions, implementation, and outcomes involved in contracting. In doing so, we are able to provide a fine-grained examination of administrative failures of the contracting process across multiple agencies and multiple levels of governance.
Methods
New Jersey provides an ideal case with which to examine these accountability problems for a number of reasons. Like other states, New Jersey has adopted the NPM mantra and privatized public services in various areas including: health and human services; corrections; transportation; and environmental protection. In 2010 Governor Christie created the Zimmer Commission to investigate and recommend opportunities for further privatization of state government activities. Between 2007 and 2012 New Jersey eliminated 8,827 non-education jobs in state government, the third largest reduction among the states (Maciag, 2014). Although it is not possible to determine how specific cuts were made, the workforce was reduced through a combination of attrition, layoffs, and outsourcing work to contractors. Regardless of the method used, the end result is a diminished capacity to oversee third-party contracts.
To understand how well New Jersey monitors contractors’ performance and allocates resources to ensure program effectiveness, we combined Fenno’s (1978) soak and poke approach of interviews and observation with a comprehensive analysis of institutions (Hollingsworth, 2000) connected to the oversight process in New Jersey. At the institutional arrangements level of analysis, we examine the offices involved in governance of the contracting process such as the Office of Management & Budget (OMB), Office of the State Comptroller (OSC), Office of the State Auditor (OSA), Division of Purchasing & Property, and the Governor’s Office. At the outputs and performance level of analysis we examine statutes, policies, legal and administrative decisions, and the service delivery agencies.
We conducted semi-structured interviews with current and former state employees from the departments of: Purchase and Property; Human Services; Corrections; Environmental Protection; and Transportation; as well as the OSC. These departments represent “information rich cases”, that are “congruent with the study purpose and yield data on major questions” (Patton, 1999). We purposefully selected interviewees with the knowledge and authority to provide insight and deep understanding (Patton, 2002) of contract monitoring and oversight processes in practice. In total, we conducted interviews with 19 state employees across the above departments. This included 13 current and 6 former employees at nearly every level of authority: from Directors and Chiefs to Analysts and street-level contract managers. In addition, we conducted four interviews with outside experts to supplement our primary informant interviews.
We conducted the majority of interviews in person and they generally lasted several hours. This allowed us to observe the working environment of the agencies, in particular a large number of vacant positions. Seven interviews took place over the phone and lasted roughly an hour each. Interviews were documented and coded throughout the process, so later interviews were informed by themes gleaned from interviews and other information gathered earlier in the process.
Similarly, interviewees frequently pointed us in the direction of other sources of information, and we used other sources to validate interviewees’ claims. For example, our analysis of statutes raised questions about implementation that we addressed with interviewees. Conversely, interviewees identified other legal documents we may have missed without their input, most notably a not easily discovered Attorney General’s opinion that formed the basis of a huge hole in the oversight framework. At the stage of final analysis, interviews were integrated into a holistic analysis of all available sources.
We supplemented our interview data with review of public documents. We drew on a wide range of data sources including: applicable statutes; administrative code provisions; executive orders; government circulars; Attorney Generals’ opinions; organization charts; budgets; service contracts; audits; requests for proposals (RFPs); and news articles. Seventeen RFPs were purposively selected from all departments to ensure at least one small to medium and one large contract (defined according to total value). In addition, we examined reports and data available on the yourmoney.nj.gov website.
The legal documents provided the foundation of our assessment of the institutional framework governing oversight. Organization charts, budget data, RFPs and articles provided evidence of implementation and outcomes. As noted, these sources were used in conjunction with key actor interviews to build a rounded picture of the relationship between institutions, implementation and outcomes. This proved critical, allowing us to identify the complex, reinforcing relationship between all three.
Neglecting oversight: A recipe for disaster
Over the same period in which New Jersey reduced the state workforce and increased the use of contracting, it experienced contract oversight failures that had drastic consequences including: a halfway-house escapee murdering a resident of Newark and extreme delays’ denial of services, and waste by the state’s largest contractor in the Hurricane Sandy Relief efforts. 1 To illustrate, the OSC’s 2011 review of the Residential Community Release Program (RCRP), the state’s halfway house program, concludes: “DOC [Department of Corrections] has not developed performance indicators against which it can evaluate the performance of the RCRP’s thereby bringing into question exactly what the State is receiving in exchange for the more than $60 million expended on this program annually” (Boxer, 2011).
The failure of state oversight also extends to vulnerable populations served by the Department of Developmental Disabilities (DDD). A 2009 audit concluded that the agency modified and renewed third-party contracts with little if any review of performance criteria. Specifically, the report pointed to oversight failures that resulted in payment for services that may not have been provided.
Similarly, New Jersey’s response to Hurricane Sandy Relief efforts reflected major concerns with contract oversight. Documentary data (Fair Share Housing Center, 2014; Katz 2014; O’Dea, 2014) have shown that New Jersey lacked the ability to effectively monitor contractors hired to oversee Sandy Relief efforts. For example, although the state’s RFP required contractors to submit weekly progress reports and monthly program status reports, the lead contractor operated for over eight months without filing a single report. Moreover, the contractor billed the state over $51 million in the first eight months even though it had proposed a three-year contract for a total of $67 million. The state’s ability to oversee Hurricane Sandy Recovery efforts was further complicated by contracting out the monitoring responsibility to third-party service providers. This arrangement required the coordination, compatibility, and integration of systems and personnel, for which the state provided few resources.
Delving deeper: Evaluating New Jersey as a contracting state
Despite a strong preference for contracting over in-house service delivery, oversight of contractors is simply not institutionalized as a core element of the state’s contracting process. The state has little capacity for meta-oversight. There is no requirement for thorough contract costing or planning and no institutionalized requirement for clear performance standards. Although existing regulations require the assignment of a contract manager to all contracts, the state lacks adequate numbers of qualified, trained staff to execute the role. As a consequence of these shortcomings, a great deal of oversight is post-hoc, catching the problems only after they arise and after they have become quite severe. These findings are discussed in more detail in the sections that follow.
The institutional framework of oversight
Statutes and regulations provide the broad framework for all government contracting. This framework has been shaped significantly by an Attorney General’s Opinion (Hyland and Pizzuto, 1976), a State Contract Manager circular (Desai-McCleary, 2014), and Executive Orders. Overall, we found formal rules to be lacking in at least three ways leading to substantial deficiencies in state oversight. First, many of the most significant oversight decisions and processes are not subject to any formal rules. The burden falls to the individuals within state agencies to ensure that contractors are performing honestly and delivering services well and cost-effectively.
There are many reasons for this but two structural issues stand out. To begin with, the auditing agency which has primary responsibility for procurement in New Jersey, regulates and enforces only one part of the process: bidding. Despite a few regulations designed to protect the state from poor performance, the terms contained in the RFPs that get bid upon and the oversight that ensures contractors are properly performing under contract are left almost entirely to the state agencies themselves. In addition, contracts for services provided directly to New Jersey’s citizens – complex services that the literature suggests require the most diligent oversight – are exempt entirely from the auditing agency’s oversight, including bidding requirements (Hyland and Pizzuto, 1976). This is not to say that centralized oversight of these decisions is non-existent; the independent State Comptroller and the legislative OSA both have authority to review decisions and audit processes. Their role, however, is limited by resources and regulations to retroactive analyses for all but the largest contracts.
Second, there are no institutionalized mechanisms to ensure that sufficient resources exist for the individuals responsible for the majority of true oversight to do the job well. The budgetary process impedes departmental decision-makers’ abilities to ensure sufficient resources for oversight. In fact, departmental decision-makers face strong incentives to cut oversight while maintaining or increasing contracted services. This leads to a predictable lack of qualified personnel and support systems.
Third, there does not appear to be any agency within the state with the capacity or competence to conduct meta-analyses of the overall efficiency or effectiveness of resources allocated to contractors. The State Comptroller and the OMB are prime candidates and possess relevant competencies. However, neither currently has a mandate or the resources to do so.
Contract costing and design
We found that the state failed to perform routine contract costing and/or included minimal specification of contract terms prior to the issuance of RFPs. This leaves the foundation upon which oversight might be built extremely weak. In most cases the bidding process locks in place the costs and requirements associated with contracting for a service. An RFP that is based on a poor (or non-existent) estimate of the costs and lacks thorough, clearly defined contract terms makes it extremely difficult to ensure that services are being delivered well and cost-effectively.
The “make or buy” decision is fundamental and must be made with great care and deliberation. Yet, every official we interviewed confirmed that to their knowledge costing was not done in any systematic way. We found two concrete examples of an agency comparing the costs of keeping a service in-house with the costs of contracting out for the purpose of making a decision. The first was in corrections and involved assessing how to best provide education services to inmates. The second was in transportation, and involved a comparison of the cost of replacing highway lighting in-house against the cost of contracting for it. In both cases, costing was not done as thoroughly as best practices might suggest, in particular ignoring the cost of oversight in the calculation of the cost of contracting. The bottom-line is that contracting units are making the decision to contract out without having a complete picture of all the costs associated with doing so.
Street level oversight capacity: Inadequate staffing and training for contract managers
The institutional limitations of the state’s oversight apparatus are compounded by a much larger issue that is arguably the most important finding of this study: neglect of resources to conduct the kind of collaborative, on-the-ground oversight called for in the literature. New Jersey is the poster child for the concerns raised by Brown and Potoski (2003) and Joaquin and Greitens (2012) regarding decreasing oversight capacity in the face of increasing demand for oversight. In all three stages of the process, internal capacity has been significantly reduced by attrition. The attrition of experienced state contract managers has led to the intensification of work, stress, and burnout among those who remain. This occurred in all four of the departments for which we were able to obtain such information.
The most prevalent theme that came through in the interviews we conducted is that attrition of contract management staff has significantly hampered the state’s oversight capacity. Examples from the Office of Information Services and the Office of Auditing within the Department of Human Services (DHS) are illustrative. Information Services is responsible for many of the largest and most critical contracting projects in the state. There are two large and vital projects currently underway. The first is the Consolidated Alliance Support System (CASS), a data management system used to consolidate data from welfare, Medicaid, child care and food stamps, and integrate these across agencies. The second is a digital imaging system designed to digitize paper documents and ultimately integrate them with CASS. Both projects are driven by federal data management requirements and necessitate enormous staffing. These demands are layered on top of the ordinary, ongoing responsibilities of Information Services.
Nevertheless, Information Services’ workforce has dropped from 82 in 2003 to 54 in 2012. A 2011 request by administrators for an increase of 28 workers to handle the additional work of CASS was denied. The auditing office is suffering from a similar problem. It is responsible for ensuring adherence to contract requirements of the over 400 vendors hired by DHS, most of which provide services to clients (e.g., individuals with disabilities, mental health patients, and welfare recipients). More specifically, the auditing office conducts full contract audits, desk reviews of the contractually required Certified Public Accountant (CPA) audits of all 400+ vendors (obtained and paid for by the vendor), and risk reports. This work is carried out by 30 employees. 12 years ago, there were 60.
The result of the decline in staffing is a concurrent decline in the number of audits conducted. There used to be roughly 150 full contract audits annually. Now only about 125 audits are conducted a year, split 50/50 between full contract audits and consulting reviews. This is problematic because full contract audits are thorough rather than perfunctory, and the only ones that cover any programmatic checks (e.g., number of clients served, number of beds provided, etc.). Consulting reviews are lower-level audits looking at particular issues agreed upon between the auditors and the agency. In other words, the most thorough audits have been reduced by almost 60%. Furthermore, the more than 400 annual desk reviews are conducted by only 1.5 staff members.
The DDD contract managers have suffered a similar fate. The contract managers for the fiscal side of the agency play an essential role in oversight because these managers know how much each residential facility receives to operate and they have the ability to stop payments. Knowledge about a facility’s budget means that their site visits can be especially important because their inspections can be particularly thorough. For example, inspectors can tell when they walk into a house and examine refrigerators and pantries whether funding is being appropriately channeled. Unfortunately, these site visits are taking place less frequently. Managers spoke candidly, “They don’t get out nearly as often because we are buried in paper.” While a decade ago they had 12–15 contract managers, they are now down to eight people with one supervisor responsible for overseeing all contracts at the agency.
Although attrition affects the number of people available to conduct oversight, the quality of oversight also depends on the skills of those people that remain. The State Contract Manager Circular (Desai-McCleary, 2014) attempts to ensure a qualified contract manager for at least state end-user contracts. However, its requirements seem to bear little relation to what is actually happening in practice.
According to officials from every department studied, the requirements simply cannot be fulfilled by the individuals who are being designated as contract managers (see Appendix). There are two reasons for this. First, according to officials in multiple departments, the requirements necessitate extremely high levels of skill, experience, and sophistication. This is consistent with what the literature suggests: contract management is extremely difficult. Second, New Jersey shares the common practice of assigning program staff to be contract managers (Johnston and Romzek, 1999; Romzek and Johnston, 2005).
Officials from every contracting unit studied echoed the sentiment from a DHS official that agencies “work with what [they] have.” Unfortunately, they are not given the intensive training recommended. With the exception of the Department of Transportation (DOT), which has created its own internal training program, the only training most of these default contract managers receive is a three-hour online tutorial. Thus, a duty that is “essential for efficient and effective State Contract Management” to ensure “that the State is not wasting taxpayer dollars but is spending them prudently” and consequently justifies demanding qualifications is being entrusted to people who lack those skills (Desai-McCleary, 2014).
Contracts lack adequate performance requirements and standards
The contracts we reviewed lacked detailed and meaningful performance requirements and standards. The overall results of our analysis are presented in Table 1. A minority of contracts had outcome-based performance measures and there was little evidence of performance targets being integrated into a comprehensive oversight system. Only the Division of Mental Health and Addiction Services (DMHAS) had clear, outcome-based performance measures in contracts combined with a comprehensive system of oversight. Additionally, very few contracts required specific data collection and reporting, outcomes-based benchmarks with clear performance measures and milestones tied to payment, despite these being widely accepted best practices. Similarly, very few contracts had automatic sunset provisions and requirements that contractors would have to reapply in a competitive bidding process. Contracts lacked specific evaluation procedures in cases where, given the specialized nature of the service or facilities being provided, vendors were not going to be competing against others for a contract.
Comparison of requests for proposals (RFPs) to best practices in contract design and oversight.
Notes: High: meets three criteria – the category must be mentioned in the RFP narrative, must contain a description of the item, and must be quantified and measurable.
Medium: the category is both mentioned in the RFP narrative and contains a description of the item being measured.
Low: the category was mentioned in the RFP narrative but contains no further description.
NP: not present, the category was not mentioned in the RFP.
Termination of the contract for cause was the typical sanctioning tool. The criteria for these sanctioning efforts should be based on the information gathered from monitoring and oversight of service inputs, outputs, and outcomes consistent with specific standards and measures previously developed and communicated to the contractor (Adler and Heckscher, 2006; Lambright, 2008; Weil, 2014). In general, we found neither the capacity to identify whether any outcome targets were being met nor the appetite to take action when they were not. In keeping with previous research (Girth, 2014), our interview data suggest that the reluctance to implement sanctions appears to be due to dependence on the contractors and a burdensome sanctioning process. In addition, the decisions not to impose sanctions were influenced by the institutional culture and a political environment that does not support sanctioning service providers.
Only four cases identified that the contracting unit would monitor and oversee service provision, via a formal monitoring system that was specified to some degree in the RFP. The majority of contracts we reviewed did not convey the information necessary for contractors to effectively perform their duties. Specifically, the RFPs failed to indicate how the contractors would be evaluated/monitored.
Additionally, every RFP reviewed called for a pre-award or bidders’ conference to discuss the particulars of the service being required. Yet, analysis of the RFPs presents little evidence to suggest that contracting units and vendors engage in any robust collaborative processes to jointly develop contract performance measurement and monitoring systems. For example, 14 out of 17 RFP’s analyzed did not require or identify whether the contracting unit worked with or collaboratively developed service procedures and protocols.
Finally, no language in the RFPs or subsequent contracts attempts to internalize costs of oversight. It did not appear to be a consideration for many contracting units. The only explicit indication of a cost of oversight being built into an RFP was fairly limited. It required vendors to include the $30,000–$80,000 costs of CPA audits as line items in their budgets. However, CPA audits cover only the financial integrity of the vendors, not the specific contracts, nor the vendors’ performance under them. This audit cost is only one small element of the overall cost of oversight.
Structural flaws
There are numerous different ways in which oversight can be systematically improved with a given level of employees. To the extent that systems can compensate, there appear to be very few in place. Executive Order (E.O.) 8 which requires the Treasury to implement performance-based budgeting is an attempt to create systems that can improve oversight across the board (New Jersey Executive Order 2010-8, 2010). It should theoretically impact state agencies by requiring them to obtain performance data and incentivizing the inclusion of performance benchmarks in all contracts. Meanwhile, the publication of data on a free website (http://www.yourmoney.nj.gov) should increase accountability (State of New Jersey Transparency Center, n.d.).
Although performance-budgeting reports are available for 22 Departments, the reports provide aggregate information on the quantity of services provided, but no information on the outcomes for those served or the performance under particular contracts. E.O. 8 also requires the Treasury to fund only those new or expanded programs that sunset after four years (New Jersey Executive Order 2010-8, 2010). This should similarly encourage state agencies to limit the number of new long-term contracts and automatically-renewing contracts that exist.
It is too soon to know if E.O. 8 has had any impact on state agencies in practice. However, the sunset requirement was not mentioned by a single official from any state agencies, suggesting that it has not yet become a key part of decision-making. For its part, the agency charged with implementing the order (OMB), struggles to fulfill its role because of “old and creaky” data systems.
Within state agencies themselves, we came across only two explicit oversight systems. The oversight system in the DMHAS is fully institutionalized. It integrates people, systems, and contract terms. Of particular interest is the “360° Contract Review” process. At the heart of the system is a data tool that displays key indicators for all providers contracted for each service. Fiscal, program, licensure, and performance data are compiled by the relevant units within the Division then aggregated by the system for review along different axes, such as level of care. This aggregation facilitates systemic review, while the review of each contract generates a dashboard report that facilitates contract-specific decisions, such as the type of pay structure to use and what performance benchmarks to include.
The DMHAS also generates Provider Performance Reports. That is, it creates fiscal and calendar year reports for every contractor that displays performance on key client outcomes and compares that performance to peers. The “360° Contract Review” data system interfaces with the Quality Assurance Management System, which collects data on adherence to contract requirements. The sharing of information across these systems further facilitates data-driven program performance evaluation.
The other oversight system in DHS’s Information Services is not institutionalized. Although it consists of a very elaborate governance structure to ensure proper execution of responsibilities under the CASS contract, the system seemed to rely more on the particular professional integrity and experience of administrators who went above and beyond expectations than to any regulatory requirements. Such actions cannot be relied upon as a way to institutionalize better performance oversight practices, unless the state makes greater efforts to professionalize its contract management staff.
Conclusion and policy recommendations
In this study, we sought to identify why New Jersey neglects it monitoring and oversight responsibilities and how New Jersey and other states can strengthen their oversight policies and practices. Given the difficulty of being a good contracting state, especially when it comes to complex services provided directly to citizens, it is not surprising that prior research has found so many instances of governments at all levels doing a poor job of it. It is worth noting that the problem is not limited to governments (Hodge, 1998; Weil, 2014), furthering the theory that some of the challenges are a function of the under-appreciated difficulty inherent in contracting out. Perhaps the most dominant theme, and the one to which other failures can be attributed, is that of insufficient capacity (Brown and Brudney, 1998; Joaquin and Greitens, 2012; Milward and Provan, 2000).
Our findings in New Jersey are in line with those of prior research. We document a lack of street-level oversight capacity due to inadequate staffing and training for contract managers. Beyond the street-level, our review of the institutional environment in which oversight takes place offers additional insights into the nature of the problem and possible solutions. We provide empirical evidence of failures in the institutional framework of oversight, contract costing and design, performance requirements and standards, and structural flaws in performance-based budgeting and monitoring systems. Due to ideology or ignorance, much contracting is done with the assumption that the use of private contractors alone is sufficient to increase efficiency. As a result, little capital is devoted to creating the institutions necessary to deal with cases in which that assumption fails. This is unfortunate, because that assumption fails in most cases and especially in cases where complex services are provided to citizens.
Workers charged with ensuring the public benefits from contracts are forced to operate in an institutional environment that all but guarantees failure. They are undertrained, understaffed, and overloaded with other responsibilities. Moreover, they face mixed signals, inconsistent incentives, and lack access to the type of systems that facilitate strong oversight.
Cooperative contract management is impossible. Instead, management frequently takes the form of muddling through or satisficing. As a result, the state does not have the information it needs about what contractors are doing and whether end-users are receiving high quality services. It also has limited capacity to correct the inadequacies it does uncover.
In New Jersey, as elsewhere (Girth, 2014; Johnston and Romzek, 1999; Van Slyke, 2003) contracting is often undertaken for political reasons as opposed to being driven by rigorous cost–benefit analysis. Given that such decisions may be rooted in political responsiveness to the policy preferences of the electorate, public agencies must adapt. The guiding theme in our theory of the case is that strong public service values should be the core rationale that underpins government delivery of social services (Denhardt and Denhardt, 2000; Dicke, 2002). These values include: a high regard for community service; democratic processes; responsiveness; justice and fairness; and a commitment to increasing citizen trust.
As demonstrated in the supply chain management literature (Weil, 2014), even businesses recognize that certain core functions should not be outsourced. Contract monitoring and oversight falls into this category. In order to hold contractors accountable, government must require contractors to explain their actions, review and evaluate the evidence provided, and impose sanctions where necessary.
Drawing on the academic literature and the problems identified in our study of New Jersey’s oversight policies and practices, we propose a set of recommendations that combine social accountability to stakeholders (Bovens, 2007), administrative accountability to auditors, inspectors, and controllers (Bovens, 2007), and professional accountability that allows managers to exercise discretion (Romzek and Dubnick, 1987). Our biggest priority is to ensure the institutionalization of oversight as an unseverable element of the contracting process.
Systemic oversight is critical
Meta-oversight is necessary to ensure that institutional requirements are being properly executed by the respective contracting units (state agencies). It also provides information that can be useful in making changes to the system. In order to do this, the structural framework for contracting needs to be strengthened. First, there needs to be one agency that is assigned responsibility for conducting meta-oversight of contracts across all state agencies. Second, New Jersey should eliminate the exemption from oversight for services provided directly to its citizens through third-party contracts.
Third, contract costing should be compulsory and the cost of contract management should be built into the contracting process. This recommendation is supported by our analysis of contracting in New Jersey. Previous research also demonstrates that the administrative costs of oversight are rarely included in the cost–benefit analysis of privatization (Van Slyke, 2003) and that governments often rely on the mistaken assumption that “contract management and accountability will take care of themselves,” (Johnston and Romzek, 1999: 394).
Rebuild administrative capacity
Our study confirms and builds on previous findings that governments at various levels lack sufficient contract management capacity (Joaquin and Greitens, 2012; Johnston and Romzek, 1999; Van Slyke, 2003). Analogous to corporations that devolve their supply chains, the state must closely manage its contractors to ensure quality control. Given that most government contracting occurs in a non-market environment, rigorous performance management and accountability structures are needed.
Contracting out for public services increases the need to retain in-house expertise in substantive policy areas and invest in training contract managers in a variety of skills including ethics, negotiations, political communications, procurement, program oversight, and auditing (Joaquin and Greitens, 2012; Van Slyke, 2003). Reporting requirements should not fall entirely on line workers. It is always important for service providers to document their work. At the same time, resources devoted to contract management could free frontline service providers from some of the administrative burdens they currently handle by shifting these tasks to contract management staff for whom this is their primary competence and duty.
In sum, our recommendation is that the decision to contract out must be contingent upon a fully resourced and fleshed out system of oversight. This requires the restoration, articulation, and preservation of the contract development and management role in state government. To ensure proper oversight, the state needs experienced managers with deep knowledge about the services being delivered and the appropriate monitoring tools. At its core, the work entails ongoing, active, flexible, relational management of the contract that respects, supports, and consults the direct service providers.
Many contractors do excellent work and have institutional knowledge and expertise that the state lacks. Under a relational contracting model, providers would have opportunity for input in the contracting and performance management processes. Likewise, no one in the state knows more about service quality than the workers who have direct contact with clients and providers. Their voices must also be included in relational contracting.
In this way performance measurement/management and accountability are linked together as tools for reflection and learning. The relationship between government and non-governmental service providers is reciprocal. However, it is not a relationship of equals. Government can and should influence the behavior of service providers.
This study makes some key contributions to our understanding of the contracting process. First, it uses a multi-method approach that combines interviews and observation with comprehensive institutional analysis. This approach allowed for a deeper examination of the institutions, structures, and street-level oversight practices in the state than would be possible with surveys or interview data alone. Second, it is commonly assumed that contracting failures are largely due to shirking/drifting agents. This study demonstrates that both principals and agents may be guilty of shirking their responsibilities. Third, it suggests how to restore state-level oversight capacity using the collaborative learning approach.
Building on these conclusions we lay out a plan for future research. While the institutional frameworks for contracting will vary across states, we posit that other states can benefit from policy borrowing and tinkering (Weimer, 1993) with our recommendations to fit their specific needs. To improve on the generalizability of this single case study, we recommend that future research concentrates on repeating similar in-depth studies combining “soaking and poking” with institutional analysis in other states. These additional cases would facilitate multi-state comparisons of oversight practices and strengthen recommendations for policy improvements.
One possible theoretical framework that provides guidance for explaining why oversight is so weak in New Jersey and elsewhere is the model of drifting principals (Schillemans and Busuioc, 2015). This research could be extended by further examining the underlying assumptions that states actually care about tasks delegated to contractors, want to hold their contractors accountable, and are willing to correct and discipline failing contractors. In addition, in cases where oversight has been delegated to “others,” future research should examine the extent to which states are able to persuade non-state actors to carry out their accountability responsibilities and non-state actors are empowered to do so.
The twin threads of NPM, contracting out for service delivery and improving accountability through performance measurement and management, place contradictory demands on state governments. When states seek to increase the number of services contracted out while simultaneously eliminating the public managers required to monitor performance and manage contracts, they place citizens and state assets at risk. Under these circumstances, the entire enterprise of accountability and efficiency becomes nothing more than political symbolism, in which ideology trumps reality. States are faced with a choice: either devote the resources necessary to perform effective contract management or keep service provision in-house. The current practice of contracting out while ignoring the structure needed for oversight is at best “un-businesslike” and at worst a recipe for disaster.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: from In the Public Interest and the New Jersey AFL-CIO.
Note
Appendix
Under the circular, Contract Managers have the following responsibilities: “Daily management of the contract, including monitoring and administering the contract on behalf of the using agency; All communications with the contractor; Attempting to resolve all contract issues and problems within the contract terms; Attempting to resolve all contract issues with the contractor; When necessary, working in conjunction with the Contract Compliance and Audit Unit of the Treasury Department (CCAU) to resolve contract issues with the contractor; Ensuring that all tasks, services, products, quality of deliverables. and timeliness of all services and deliverables are satisfied within the contract requirements; Reviewing all contractor billing and assuring that the contractor is paid only for services rendered and goods delivered; Attempting-to recover any and all over-billings from the contractor; When necessary, filing formal complaints as provided under N.J.A.C. et seg. with CCAU; Working with the Division to prepare contract extensions and the Request for Proposal in anticipation of a rebid for the contract services or products; Recommending to the Director, when necessary, contract amendments and/or additional work or reduction of work under the contract; Timely completion and filing with CCAU all required Project Performance Assessment forms. The State Contract Manager shall file a report detailing the final deliverables of the contract with the Associate Director of the Office of Management and Budget (OMB) and provide a copy of this report to CCAU” (Desai-McCleary, 2014). These are merely the base requirements. In many circumstances, such as where multiple agencies are implicated (statewide contracts), there are extensive additional responsibilities. In the case of multiple agencies, for example, Contract Managers must meet with every agency and coordinate among them.