Evaluating Program and Managerial Efficiency: An Application of Data Envelopment Analysis to Program Follow Through

Abstract

A model for measuring the efficiency of Decision Making Units (=DMU's) is presented, along with related methods of implementation and interpretation. The term DMU is intended to emphasize an orientation toward managed entities in the public and/or not-for-profit sectors. The proposed approach is applicable to the multiple outputs and designated inputs which are common for such DMU's. A priori weights, or imputations of a market-price-value character are not required. A mathematical programming model applied to observational data provides a new way of obtaining empirical estimates of extrernal relations—such as the production functions and/or efficient production possibility surfaces that are a cornerstone of modern economics. The resulting extremal relations are used to envelop the observations in order to obtain the efficiency measures that form a focus of the present paper. An illustrative application utilizes data from Program Follow Through (=PFT). A large scale social experiment in public school education, it was designed to test the advantages of PFT relative to designated NFT (=Non-Follow Through) counterparts in various parts of the U.S. It is possible that the resulting observations are contaminated with inefficiencies due to the way DMU's were managed en route to assessing whether PFT (as a program) is superior to its NFT alternative. A further mathematical programming development is therefore undertaken to distinguish between “management efficiency” and “program efficiency.” This is done via procedures referred to as Data Envelopment Analysis (=DEA) in which one first obtains boundaries or envelopes from the data for PFT and NFT, respectively. These boundaries provide a basis for estimating the relative efficiency of the DMU's operating under these programs. These DMU's are then adjusted up to their program boundaries, after which a new inter-program envelope is obtained for evaluating the PFT and NFT programs with the estimated managerial inefficiencies eliminated. The claimed superiority of PFT fails to be validated in this illustrative application. Our DEA approach, however, suggests the additional possibility of new approaches obtained from PFT-NFT combinations which may be superior to either of them alone. Validating such possibilities cannot be done only by statistical or other modelings. It requires recourse to field studies, including audits (e.g., of a U.S. General Accounting Office variety) and therefore ways in which the results of a DEA approach may be used to guide such further studies (or audits) are also indicated.