Performance Benchmarking of School Districts in New York State

Abstract

We used data envelopment analysis to measure the relative performance of New York State school districts in the 2011–2012 academic year and provided detailed alternative improvement pathways for each district. We found that 201 of the 624 (32.2 %) school districts with one or more high schools and 28 of the 31 (90.3 %) school districts with no high school were on the performance frontier. Using a mixed orientation, we found evidence that FTE teachers could be reduced by 8.4 %, FTE teacher support personnel could be reduced by 17.2 %, and FTE building administration and professional staff personnel could be reduced by 9.4 %. In addition, we found that the percentage of students who score 3 or 4 on the English exam could increase by 4.9 % points, 5.0 % points on the mathematics exam, and 5.8 % points on the science exam and the average graduation rate could increase by 5.4 % points.

Appendix: The Mathematics of the DEA Model

We use two slightly different DEA models in this chapter, one for school districts with one or more high schools, and one for school districts without a high school. The differences lie in the performance measures (different points at which test scores are measured, and no graduation rate for school districts with no high school). In addition, each model is employed with three different orientations (resource reduction, performance enhancement, and mixed). The text that follows describes the model for school districts with one or more high schools.

Let n = 624 be the number of school districts to be analyzed. The DEA literature refers to units under analysis as decision-making units, or DMUs. Let X _ij be amount of resource i consumed by DMU j, for i = 1, 2, 3, and j = 1, 2, …, 624. In particular, let X _1j be the FTE teachers in DMU j, let X _2j be the FTE teacher support in DMU j, and let X _3j be the FTE building administration and professional staff in DMU j.

Let Y _rj be performance measure r achieved by DMU j, for r = 1, 2, 3, 4 and j = 1, 2, …, 624. In particular, let Y _1j be the percentage of students scoring at levels 3 or 4 in secondary-level English after 4 years of instruction in DMU j, let Y _2j be the percentage of students scoring at levels 3 or 4 in secondary-level math after 4 years of instruction in DMU j, let Y _3j be the percentage of students scoring at levels 3 or 4 in Grade 8 Science in DMU j, and let Y _4j be the 4-year graduation rate as of August in DMU j, for j = 1, 2, …, 624.

Let S _kj be the value of site characteristic k at DMU j, for k = 1, 2, 3, 4, 5 and j = 1, 2, …, 624. In particular, let S _1j be the number of elementary school students in DMU j, let S _2j be the number of secondary school students in DMU j, let S _3j be the percentage of students with free or reduced price lunch in DMU j, let S _4j be the percentage of students with limited English proficiency in DMU j, and let S _5j be the combined wealth ratio in DMU j, for j = 1, 2, …, 624.

13.1.1 The Resource Reduction DEA Model

The resource reduction DEA model with variable returns to scale, for DMU d, d = 1, 2, …, 624, is below. We must solve n = 624 linear programs to perform the entire DEA.

Min E d	(13.1)
subject to
${\displaystyle \sum_{j=1}^n}{\lambda}_j{X}_{1j}\le {E}_d{X}_{1d}$	(13.2a)	FTE teachers
${\displaystyle \sum_{j=1}^n}{\lambda}_j{X}_{2j}\le {E}_d{X}_{2d}$	(13.2b)	FTE teacher support
${\displaystyle \sum_{j=1}^n}{\lambda}_j{X}_{3j}\le {E}_d{X}_{3d}$	(13.2c)	Building administration and professional staff
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{1j}\ge {Y}_{1d}$	(13.3a)	Secondary level English (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{2j}\ge {Y}_{2d}$	(13.3b)	Secondary level math (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{3j}\ge {Y}_{3d}$	(13.3c)	Grade 8 science (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{4j}\ge {Y}_{4d}$	(13.3d)	Graduation rate (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{1j}\ge {S}_{1d}$	(13.4a)	Number of elementary school students
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{2j}\ge {S}_{2d}$	(13.4b)	Number of secondary school students
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{3j}\ge {S}_{3d}$	(13.4c)	Percentage of students with free or reduced price lunch
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{4j}\ge {S}_{4d}$	(13.4d)	Percentage of students with limited English proficiency
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{5j}\le {S}_{5d}$	(13.4e)	School district’s combined wealth ratio
${\displaystyle \sum_{j=1}^n}{\lambda}_j=1$	(13.5)	Variable returns to scale
${\lambda}_j\ge 0\ for\ j=1,\ 2, \dots,\ 624$	(13.6)	Nonnegativity
${E}_d\ge 0$	(13.7)	Nonnegativity

We observe that setting λ _d  = 1, λ _j  = 0 for j ≠ d, and E _d  = 1 is a feasible, but not necessarily optimal, solution to the linear program for DMU d. This implies that E _d ^*, the optimal value of E _d , must be less than or equal to 1. The optimal value, E _d ^*, is the overall efficiency of DMU j. The left-hand-sides of (13.2)–(13.4) are weighted averages, because of (13.5), of the resources, performance measures, and site characteristics, respectively, of the 524 DMUs. At optimality, that is with the λ _j replaced by λ _j ^*, we call the left-hand-sides of (13.2a)–(13.4e) the target resources , target performance measures , and target site characteristics , respectively, for DMU d.

Equations (13.2a)–(13.2c) imply that each target resource will be less than or equal to the actual level of that resource at DMU d. Similarly, (13.3a)–(13.3d) imply that each target performance measure will be greater than or equal to the actual level of that performance measure at DMU d.

The nature of each site characteristic inequality in (13.4a)–(13.4e) depends on the manner in which the site characteristic influences efficiency. Equations (13.4a)–(13.4d) correspond to unfavorable site characteristics (larger values imply a greater need for resources to obtain a given performance level, on average); therefore, we use the greater-than-or-equal to sign. Equation (13.4e) corresponds to a favorable site characteristic (larger values imply a lesser need for resources to obtain a given performance level, on average); therefore we use the less-than-or-equal to sign. Thus, (13.4a)–(13.4e) imply that the value of each target site characteristic will be the same as or worse than the actual value of that site characteristic at DMU d.

Thus, the optimal solution to the linear program for DMU d identifies a hypothetical target DMU d ^* that, relative to DMU d, (a) consumes the same or less of every resource, (b) achieves the same or greater level of every performance measure, and (c) operates under the same or worse site characteristics. Moreover, the objective function expressed in (13.1) ensures that the target DMU d ^* consumes resources levels that are reduced as much as possible in across-the-board percentage terms.

Of course, to proceed we must assume that a DMU could in fact operate exactly as does DMU d ^*. In the theory of production, this is the assumption, made universally by economists, that the production possibility set is convex. In this context, the production possibility set is the set of all vectors $\left\{{\boldsymbol{X}}_i,{\boldsymbol{Y}}_r\Big|{\boldsymbol{S}}_k\right\}$ of resources, performance measures, and site characteristics such that it is possible for a DMU to use resource levels X _i to produce performance measures Y _r under site characteristics S _k . The convexity assumption assures that DMU d ^* is feasible and that it is reasonable to expect that DMU d could modify its performance to match that of d ^*.

We use the Premium Solver Pro^© add-in (Frontline Systems, Inc., Incline Village, NV) in Microsoft Excel^© to solve the linear programs. We use a macro written in Visual Basic for Applications^© (VBA) to solve the 624 linear programs sequentially and save the results within the spreadsheet. Both the Basic Solver^© and VBA^© are available in all versions of Microsoft Excel^©. However, the Basic Solver^© is limited to 200 variables and 100 constraints, which limits the size of the problems to no more than 199 DMU and no more than 99 resources, performance measures, and site characteristics combined. We use the Premium Solver Pro^©, available from Frontline Systems, Inc., for this application.

13.1.2 The Performance Enhancement DEA Model

The performance enhancement DEA model with variable returns to scale, for DMU d, d = 1, 2, …, 624, is below. In this model, we eliminate E _d as the objective function (13.8) and from the resource constraints (13.9a)–(13.9c) and introduce θ _d as the new objective function (now to be maximized) and into the performance enhancement constraints (13.10a)–(13.10d). The parameter θ _d will now be greater than or equal to one, and it is called the inverse efficiency of DMU d.

Max θ d	(13.8)
subject to
${\displaystyle \sum_{j=1}^n}{\lambda}_j{X}_{1j}\le {X}_{1d}$	(13.9a)	FTE teachers
${\displaystyle \sum_{j=1}^n}{\lambda}_j{X}_{2j}\le {X}_{2d}$	(13.9b)	FTE teacher support
${\displaystyle \sum_{j=1}^n}{\lambda}_j{X}_{3j}\le {X}_{3d}$	(13.9c)	Building administration and professional staff
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{1j}\ge {\theta}_d{Y}_{1d}$	(13.10a)	Secondary level English (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{2j}\ge {\theta}_d{Y}_{2d}$	(13.10b)	Secondary level math (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{3j}\ge {\theta}_d{Y}_{3d}$	(13.10c)	Grade 8 science (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{4j}\ge {\theta}_d{Y}_{4d}$	(13.10d)	Graduation rate (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{1j}\ge {S}_{1d}$	(13.11a)	Number of elementary school students
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{2j}\ge {S}_{2d}$	(13.11b)	Number of secondary school students
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{3j}\ge {S}_{3d}$	(13.11c)	Percentage of students with free or reduced price lunch
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{4j}\ge {S}_{4d}$	(13.11d)	Percentage of students with limited English proficiency
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{5j}\le {S}_{5d}$	(13.11e)	School district’s combined wealth ratio
${\displaystyle \sum_{j=1}^n}{\lambda}_j=1$	(13.12)	Variable returns to scale
${\lambda}_j\ge 0\ for\ j=1,\ 2, \dots,\ 624$	(13.13)	Nonnegativity
${\theta}_d\ge 0$	(13.14)	Nonnegativity

13.1.3 The Mixed DEA Model

The mixed DEA model with variable returns to scale, for DMU d, d = 1, 2, …, 624, is below. In this model, we keep both E _d and θ _d in the constraints and we may now choose to either minimize θ _d or maximize θ _d . We introduce a new constraint (13.20) that ensures balance between the goals of reducing resources and enhancing performance.

Min E d  or Max θ d	(13.15)
subject to
${\displaystyle \sum_{j=1}^n}{\lambda}_j{X}_{1j}\le {E}_d{X}_{1d}$	(13.16a)	FTE teachers
${\displaystyle \sum_{j=1}^n}{\lambda}_j{X}_{2j}\le {E}_d{X}_{2d}$	(13.16b)	FTE teacher support
${\displaystyle \sum_{j=1}^n}{\lambda}_j{X}_{3j}\le {E}_d{X}_{3d}$	(13.16c)	Building administration and professional staff
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{1j}\ge {\theta}_d{Y}_{1d}$	(13.17a)	Secondary level English (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{2j}\ge {\theta}_d{Y}_{2d}$	(13.17b)	Secondary level math (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{3j}\ge {\theta}_d{Y}_{3d}$	(13.17c)	Grade 8 science (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{Y}_{4j}\ge {\theta}_d{Y}_{4d}$	(13.17d)	Graduation rate (%)
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{1j}\ge {S}_{1d}$	(13.18a)	Number of elementary school students
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{2j}\ge {S}_{2d}$	(13.18b)	Number of secondary school students
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{3j}\ge {S}_{3d}$	(13.18c)	Percentage of students with free or reduced price lunch
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{4j}\ge {S}_{4d}$	(13.18d)	Percentage of students with limited English proficiency
${\displaystyle \sum_{j=1}^n}{\lambda}_j{S}_{5j}\le {S}_{5d}$	(13.18e)	School district’s combined wealth ratio
${\displaystyle \sum_{j=1}^n}{\lambda}_j=1$	(13.19)	Variable returns to scale
${E}_d+{\theta}_d=2$	(13.20)	Balance resource reduction and performance enhancement
${\lambda}_j\ge 0\ for\ j=1,\ 2, \dots,\ 624$	(13.21)	Nonnegativity
${E}_d,\ {\theta}_d\ge 0$	(13.22)	Nonnegativity