Robert P. Strauss,** Lori L. Bowes,* Mindy S. Marks,* and Mark R. Plesko* (forthcoming Economics of Education Review) April 25, 1999
** Professor of Economics and Public Policy at the Heinz School; *Student, Carnegie-Mellon University. Pittsburgh, Pennsylvania, 15213-3890. Email: RS9F@Andrew.CMU.EDU; Home Page: http://www.heinz.cmu.edu/~rs9f/. This is a revised version of papers presented to the New York Board of Regents in October, 1997, and at the March, 1998 Annual Research Conference of the American Educational Finance Association in Mobile, Alabama. The authors benefited from the comments of participants at these presentations, and Professor William Vogt. Much of the results are part of a larger project prepared for the Pennsylvania State Board of Education during 1997-8. (Strauss, et al(1998)). Financial support from the Pennsylvania State Board of Education, the Vira I. Heinz Endowment, Grable Foundation and the Frick Fund of the Buhl Foundation is gratefully acknowledged. Responsibility for this paper, its opinions, and any errors rests solely with the authors.
ABSTRACT As national concern over student achievement grows, policy attention has increasingly focused on the quality of the public school teacher force. This paper examines Pennsylvania's teacher preparation and selection processes over the last decade.
Teacher test scores are examined by college of education and extreme differences in median teacher test scores are evident across colleges of education. Extreme variations in districts' median test scores of those actually hired are also evident across and within metropolitan areas.
On average in Pennsylvania, 40% of a district's teachers attended as students the district where they now teach. An exploratory econometric model indicates, after correcting for endogeneity, that the higher the higher the unemployment rate in a district, the more likely it will hire its own former students to teach, and that the greater the fraction of former students now in the district’s teacher force, the lower is student achievement. These results obtain while holding constant the educational status of the community and the poverty status of the students.
The paper concludes with a discussion of conventional and unconventional strategies to improve the quality of teachers actually hired by school boards.
“You're damn right we only hire by nepotism in this district. But there are two kinds of nepotism:
1. Good nepotism is when this Board you and I were elected to is smart enough to hire my son or daughter, and
2. Bad nepotism is when this Board is dumb enough to hire your son or daughter."
--Minutes of New Castle, Pennsylvania School Board Meeting, as reported in the New Castle Gazette.
1 Introduction Some Preliminaries As states move to test their students to find out what they know and do not know, some increasingly realize that there may be limits to what one can reasonably expect from students unless curricula and classroom instruction are adapted to these higher learning standards. Common sense suggests that raising our expectations about what students achieve in the classroom should be accompanied by concomitant policies and resources to improve what teachers know and convey to the students. By and large, however, legislative and regulatory reform of public education has focused on: (1) developing tests or assessment tools for finding out what students know and can do, (2) promulgating information about these results to the public, parents and students, and (sometimes) (3) developing financial rewards and penalties for districts, building level administrators and teachers associated with student achievement or lack thereof. Such accountability models presume that those in charge of local public education, facing financial rewards and penalties, will adjust their activities in order to gain rewards and avoid penalties.
The public education system, however, is a very complicated set of large institutions that may react defensively to external criticism or externally imposed change. The result often is that they remain immune or unresponsive to systems of financial rewards and penalties unless great care (and courage) is taken to place these incentives at meaningful locations in the public education system. The very size, complexity and unresponsiveness of the system probably explains why some favor side-stepping the frustrating problems of redesign, political and administrative implementation and simply giving parents of school-age children vouchers. In this view, empowered parents can buy education services from whomever wishes to sell them. Whether parents will have adequate or sufficient information about these new entrants into the education business to make wise choices for their children is usually not (openly) discussed. Advocates often assert that such alternatives have to be better than the current morass, or that anything will be better that what currently exists.
Legislative battles in many states over the nature of charter schools or vouchers often center on whether or not the teachers in these new forms of schools must be certificated like their public school counterparts. Debate often focuses on whether teacher certification, and education school course-work in particular, is necessary or sufficient to ensure effective classroom teaching. Underlying much of the debate over charter schools or vouchers is often an (unstated) antipathy to teachers' unions, teacher tenure, large and unresponsive bureaucratic school administrations, and unpopular local property taxes, increasingly falling on homeowners, as well as a concern that US secondary students do worse on standardized tests than their counterparts in other parts of the world.
To understand where among and within public education institutions policy change can improve student performance, one must step back and examine its overall institutional architecture. Free provision of public education to school age children, who are required by law to attend some form of school, is typically a state constitutional obligation. State laws in our older, industrialized states typically create local school districts on a parallel basis to municipalities, and empower them to impose local real estate taxes which, in conjunction with state payments to school districts, are used to pay for school costs. Local school districts are also empowered to issue debt for capital purposes, required to balance their budgets, and report to the state on their financial activities. Typically capital activities (debt issuance and school construction) are heavily supervised by state agencies to ensure safety and proper use of funds.
State constitutions also typically require that imposition of any tax be through an elected council or legislative body. In the case of school districts, school directors often serve pro bono, and act as the state legislature's agents in providing a thorough and efficient education. School director elections are often non-partisan in contrast to other local state and local elections which prohibit cross-over voting by voters with expressed political affiliation.
While there are relatively few requirements on who may serve as a school board director, the statutory and regulatory requirements about who may teach in a public school are both very complicated and often very imprecise.
In Pennsylvania, to be eligible to become a member of a school board, one need only be a citizen of Pennsylvania, a person of good moral character, 18 years or older, and have been a resident of the school district for at least one year prior to election or appointment.1 Direct self-dealing is limited statutorily in several ways:
School employees are prohibited in Pennsylvania, under Act 2 of 1980, from serving on a board where they are employed; however, this does not preclude them serving on a board where they live if the district of residence is different than the district of employment.2
2. School board members are prohibited under the School Code, Section 1111, from voting on the appointment of a relative to a teaching position on the board;
3. The Public School Code, at Section 324, directs no school director from being interested in, or doing business, with the school district during the term of office.3
Teachers in most states, however, must earn educational credentials, have a college degree, and pass certain standardized tests. By and large, a college degree, which reflects some sort of course work on pedagogy and the content area in which the prospective teacher will teach, in conjunction with passing scores on standardized tests, are all that are required to become certified. The degree is typically from a state approved program of teacher preparation, and standardized tests are devised by national testing firms such as Educational Testing Service or National Evaluation Systems. These are, as will be indicated below, rather minimal requirements, and often do not attract the most academically talented individuals.
Changes in student and teacher demographics, as well as rising expectations for student performance, are creating new classroom needs and opportunities for substantially renewing many districts' teachers. In the older, industrialized states, school age children will be relatively older in the next ten to fifteen years, thereby requiring relatively more secondary than elementary classroom teachers. At the same time, classroom teachers are, much like the rest of our society, getting older, and retirements will provide an opportunity to hire younger, less expensive teachers, and hopefully those able (or better able) to ensure that students can achieve high learning standards.
Some have commented that these demographic changes should be recognized by teacher preparation institutions so they prepare teachers with the right skills for the classroom needs of the next century. However, higher education faces its own financial incentives, and also has its own rigidities towards change. Colleges and universities with sizable education schools find it difficult to alter the activities of their own highly tenured education faculties, some also unionized, to not only train the right sort of teachers, but also ensure that those trained are able to help students achieve high learning standards. Another aspect of higher education's struggle with its schools of education involves the cross-subsidization which education schools provide for other parts of their campus. Professional schools are often viewed with suspicion by other parts of a university campus, and schools of education perhaps fare worst. They are frequently viewed as profit centers to be taxed to support other programs. Admissions policies are then pursued which encourage many, who would otherwise not attend college, to prepare for a career in education which may never materialize.
Relatively little emphasis has been placed by educational researchers on the role of the local employment decision and role of elected, volunteer school boards in responding to public demands for better student performance, although several recent papers have demonstrated that many local school districts do not hire the most academically qualified applicants.4 Ballou and Podgursky (1997) concluded:
Higher teacher salaries have had little if any discernible impact on the quality of newly recruited teachers.
The failure of this policy can be traced, in part, to structural features of the teacher labor market.
Recruitment of better teachers is further impeded by the fact that public schools show no preference for applicants who have strong academic records.5
While they go on to propose, as solutions to these problems, market-based salaries based on performance, our analysis of the public education problem below focuses on the employment and personnel management decisions and the institutional/legal framework within which they are made for several reasons.
First, Pennsylvania, along with other states, accords a permanent teaching certificate quite early in the career of teachers. Even in states which no longer have life-time certification, recertification typically requires no more than earning continuing education credits through inservice training, or, as in the case of Pennsylvania, college credits without regard to the linkage between teaching content area and graduate courses taken.
Second, evaluation of historical personnel in any professional organization is frequently difficult, and especially so when one can not readily measure outcomes as in the private sector. Simply ascribing student achievement to the efforts of an individual teacher ignores the obvious reality that student achievement is cumulative and dependent on those who taught the student earlier, as well as the student's own intellect, motivation, and home environment. Third, given the aging of the teacher force, there may be an opportunity, if decision makers are selective, to raise the quality of the teacher force by both improving the teacher preparation process as well as improving the teacher selection process.
Thus, while we agree with the Ballou and Podgursky (1997) observations about the results of teacher hiring practices, we arrive at a different set of policy recommendations to remedy them; however, neither set of recommendations precludes the other.
Who a district hires, unless they choose to leave voluntarily, is thus likely to be with the school district for a very long time. The employment decision, because it is therefore a long-term decision, involves the long-term commitment to pay salaries which will rise with if not above the rate of inflation. One commentator6 on an earlier version of this paper observed that the sort of financial commitment made at the time of hiring is on the order of $300,000 to $500,000 per teacher, or well above the national median price of a new home purchase of $125,000. As we shall see below, many school districts in Pennsylvania do not pay enough attention to the personnel process, and make such $300,000 to $500,000 decisions on the basis of no more than an hour of consideration.
The purpose of this paper is:
to provide a variety of facts about public school teacher preparation in Pennsylvania and around the US which are not well known, and will be of general interest; and,
to interpret these facts in relation to the public policy design problem of creating incentives which will promote high learning standards.
The data on the matters of the nature of teacher preparation and certification and selection decisions in Pennsylvania are compelling. They raise serious questions about whether local control as currently construed, the mantra of public education in the US, is capable of doing any more than ensure mediocrity. When one looks closely at who gets hired, how they get hired and retained, to teach in our public school classrooms, much of the mystery and confusion about mediocre student performance disappears. How one changes this, however, is not easy, and will likely be controversial.
Organization of Paper The paper is organized as follows:
Section 2 outlines the major features of the market for classroom teachers in Pennsylvania.
Section 3 explores the quality of teachers prepared in Pennsylvania teacher preparation institutions and the selectivity of school districts in their hiring practices as reflected in the content knowledge scores achieved by prospective and employed teachers;
Section 4 reports the results of an extensive survey of Pennsylvania school district employment practices, correlations across districts between differing personnel practices and average student achievement in mathematics and reading in 1996, and some econometric estimates of an exploratory model of who gets hired and the effects on student achievement.
Section 5 summarizes the stylized facts and their implications for educational policy viz. a viz. teacher preparation standards, program approval, and the standardization of employment practices by local school boards.
2 The Market for Classroom Teachers in Pennsylvania and their Preparation Pennsylvania's 501 school districts currently enroll about 1.8 million students; they have employed about 100,000 classroom teachers annually for the last 15 years; overall there are about 130,000 professional personnel in the public education system. In the 1990's, between 3,700 to 6,300 teachers were hired annually; new hires have increasingly come from the experienced teacher force. In 1984 54% of newly hired teachers had prior experience; in 1996 68% of newly hired teachers had prior teaching experience in Pennsylvania. In 1996 only 1,285 newly hired teachers had no prior teaching experience. Somewhere between 4,000 and 6,000 teachers, administrators and coordinators have been annually hired by all of Pennsylvania's local districts and intermediate units. In 1993 a temporary early retirement window was opened by the Pennsylvania General Assembly. About 10,000 teachers, administrators and coordinators retired, and about 8,000 new hires took place. Hires of inexperienced classroom teachers have been on the order of no more than 2,000 per year, and in the last two years, that number has dropped to no more than 1,200.
Like teachers in other industrialized states, Pennsylvania's teachers are getting closer to retirement: the median age was 45 and the median years of professional experience was 19 years; 25% of Pennsylvania's classroom teachers had 26 or more years of experience.
2.1 Supply of Teaching Certificates from Pennsylvania Colleges and Universities Pennsylvania trains far more teachers than it hires. Pennsylvania currently has better than 90 teacher preparation institutions including 14 state supported institutions which were originally two year normal schools. In the past several years, Pennsylvania certificating institutions have issued better than 20,000 certificates per year of various kinds. Compared to the 1980's, the production of various teaching and administrative certificates is accelerating. It follows, of course, that the vast bulk of newly trained teachers each year are unable to obtain teaching positions in Pennsylvania. Table 1 displays the astounding production of teaching certificates by broad certification area over 5 year intervals.7 (Table 1 about here) 2.2 Supply and Demand through School Year 2005 Demographic analysis of student enrollment by school district, grade and course through 2005, and demographic analysis of possible teacher retirement scenarios by school district and course through 2005 indicate a wide range of teacher replacement needs.8 Table 2 indicates by broad certification area the ratio of hiring needs, under three retirement assumptions and historical quits, in relation to the number of employed classroom teachers in 1996/7.
Three different retirement assumptions are entertained: 1) teachers will retire at age 65; 2) teachers will retire upon reaching 30 years of service; or 3) teachers will retire when they have achieved 27 years of service and age 55 (the incentives in place in 1993)
Columns 8, 9, and 10 of Table 2 show the projected accumulated retirements and voluntary quits between 1997 and 2005 under the three retirement assumptions. Each projection also takes into account changing student demographics and course enrollments. Columns 11,12, and 13 show the projected turnover rate, or the percentage of the currently employed teachers (Column 2) who will be replaced between now and school year 2005.
This analysis indicates:
1. The net number of elementary teachers will decline overall across the next 9 years by 1,388 teachers if elementary school teachers wait until age 65 to retire. Under the other retirement assumptions, around 11-14,000 elementary school teachers will be needed, of whom 5,000 will be due to voluntary quits.
2. If teachers retire earlier, e.g. do not wait until age 65 to retire, the numbers of teachers needed jumps dramatically to somewhere between 46,000 and 50,000 or anywhere from 56% to 61% of the 1996/7 stock of employed teachers.
3. When one combines the predictions with historical teacher production levels, it is difficult to reach the conclusion that there will be teacher shortages. Table 3 indicates that there are vast numbers of certified teachers produced during the 1980's and 1990's who actually outnumber the number of employed teachers by about 2:1 overall. In areas such as elementary education, mathematics, and social studies, vast numbers of teachers were trained. The ratio of hiring needs or demand to this measure of supply is anywhere from 12% to 65%, depending on the certification area in question.
(Table 2 about here) (Table 3: about here)
Several conclusions suggest themselves from this analysis. Tthe public education system has an opportunity under reasonable or accelerated retirement scenarios to employ younger teachers, who presumably will be considerably less expensive than those retiring, and an opportunity, if selective, to employ teachers more able to ensure that students meet high learning standards. The problem local districts will face involves how to choose wisely among many applicants.9
2.3 Pennsylvania's Teacher Certification Framework To be employed as a public school teacher in Pennsylvania, the applicant:
must be of good moral character;
is mentally and physically qualified to perform the duties of a teacher;
is 18 years of age; and,
have earned a baccalaureate degree as a general education requirement in a program of teacher preparation approved by the Pennsylvania Department of Education, and be recommended to the Department of Education for certification by the program.10
Also, under ¶49.18 a) of the Teacher Certification Regulations, the Secretary of Education was required, as of May 9, 1985, to "institute a testing program for candidates for certification designed to assess their basic skills, general knowledge, professional knowledge, and knowledge of the subjects in which they seek certification."
As a practical matter, those interested in pursuing a career in public school teaching must apply to and be admitted to a college or university which has an approved program of teacher preparation. Such programs are approved by area of certification, e.g. elementary education, various types of special education, or areas of specialization at the secondary level (social studies, mathematics etc.). Satisfactory completion of the program's stipulated course requirements, coupled with a letter of recommendation, and passing scores on standardized teacher examinations offered by the Educational Testing Service of Princeton, New Jersey, enables one to obtain teacher certification. In turn, such certification enables the applicant to be legally employed by a public school district.
2.4 SAT Scores of Pennsylvania High School Seniors Interested in Education Careers Pennsylvania is among 19 states that do not require standardized tests for admissions to teacher preparation programs. Some evidence is available from ETS, however, on the comparative scholastic aptitude of those interested in becoming public school teachers compared to other college majors in Pennsylvania. Table 4 displays the mean verbal and math SAT scores for Pennsylvania and the US for 1996, and the combined math and verbal score stated as the percentile in the overall national distribution of scores. Several things are evident. First, Pennsylvania's SAT scores are lower than their counterpart US scores; this has been explained by some observers as due to large numbers of Pennsylvania high school students taking the examinations. Second, Pennsylvania's high school seniors intending to become education majors score substantially below their counterparts interested in pursuing academic majors. For example, the mean math SAT score of an intended education major was 471 compared to a 614 for intended math majors, or a 30% difference. A 471 is well below the median or 50th percentile. Compare the combined percentiles of an intended education major in Pennsylvania at the 35th percentile to the 82nd percentile for an intended math major. When the verbal score of intended education majors, 483, is compared to that of a intended language and literature major of 595, we observe a 26% difference in absolute scores. The relative overall position of intended education majors and language and literature majors is respectively the 35th and 81st percentiles.
Another way to view these SAT scores is to recognize that, 65% of those taking the SAT test in 1996 demonstrated greater scholastic aptitude. than those intending to return to public education upon earning their teaching certificate.
There is national evidence that those who got hired and remain in teaching have SAT scores at about the 25th percentile of all employed college graduates.11 If the academic aptitude level of classroom teachers hovers around the 25’th or 35th percentile of those overall, it is easy to understand why international comparisons of US 14 year olds compares unfavorably with their counterparts around the world. Having classroom teachers with below average aptitude levels themselves can easily dilute the motivation and achievement level of the majority whom they teach, since the majority of students in the classroom display stronger academic aptitude than their classroom teacher did.
(Table 4 about here)
2.5 National Teacher Examination (NTE) Tests and Passing Levels Beginning in the late 1980's, Pennsylvania, along with many other states, began testing the general; and content knowledge of prospective teachers. In 1987, Pennsylvania replaced its own teacher preparation tests with selected ETS examinations. The National Teacher Exam and its successor, Praxis, is designed by ETS to measure competency in the knowledge of core basic skills (reading, writing, and mathematics), competency in core education knowledge (general, professional and communication), and competency in the content knowledge in various specialty areas.
States vary widely in their use of ETS testing products. As of January, 1997 ETS core battery tests in reading and writing were used by 22 states and the Department of Defense, but only 14 purchased the ETS mathematics examination, and fewer (11) purchased the mathematics content knowledge test; only 6 purchased the mathematics test of proofs, models and problems. Only California and Oregon purchased the second mathematics test of proofs, models and problems. It seems likely that states which test their mathematics teachers more widely are more demanding of what their mathematics teachers know about mathematics than states which do not test as widely.
The current versions of the various exams do not purport to measure teacher classroom effectiveness, usually described as pedagogy, although the core battery tests of professional knowledge test for understanding of pedagogy as contrasted with actual effective classroom performance. Also, the NTE/Praxis tests are not validated on the subsequent performance of the teacher's students' performance or academic achievement, rather they have been developed on the notion of developing a pool of competent teachers as determined by practitioners and those in teacher preparation programs. The definition of what constitutes a minimum level of content knowledge is left to the states to determine through annual panels of experienced teachers who review the most recent examinations, and set passing thresholds based on their peer evaluation.12 While ETS is emphatic in their position that NTE/Praxis tests do not measure the competency of a prospective teacher, common sense suggests that the greater the content knowledge competency of a classroom teacher, the stronger the likelihood that the teacher's students will have an opportunity to learn that particular subject matter. That is, subject matter knowledge or content knowledge, as reflected in higher NTE/Praxis tests, is a necessary but not sufficient condition for being an effective teacher. This would seem especially important at the middle school and secondary levels.
In Pennsylvania, prospective teachers must pass ETS's National Teacher Exam core battery tests in Communication Skills, General Knowledge, and Professional Knowledge, and as of 1997 also core battery tests in Reading and Writing (but not Mathematics) and ETS subject matter tests.13 Passing scores are determined by the Department through panels of experienced teachers, and have been implemented over time.
Table 5 shows the passing score (Column 2), test range (Column 3), a calculation of the fraction of questions of average difficulty correctly answered needed to pass (Column 4), and the effective date the passing score was set (Column 5). Since the test range of the tests is centered above zero, typically from 250 to 990, and guessing is allowed without penalty on these examinations, there really are fewer points available to be earned than the top score of, say, 990. If we subtract the lower bound of the range, we note that 740 points are available to be earned since 250 points are available simply for taking the test. If we subtract 250 from the median score and from 990, we can calculate the percentage of questions of average difficulty that must be correctly answered to pass the test.
In general, one need not score better than 50% correct to pass the teacher content knowledge examinations in Pennsylvania, and, in the basic area of reading, one need correctly answer only 26 percent of the questions of average difficulty. The fact that the passing scores or cut scores are set so low necessarily implies very high passing rates for those who take the NTE examination. For Pennsylvania they are, with the exception of Social Studies and the most recent science examinations, on the order of 95% or higher.14 Low cut scores, coupled with vague15 and loosely applied teacher preparation program approval standards, through forgiving state oversight, implies that virtually anyone can become certified to teach in Pennsylvania if they are willing to spend a number of years taking teacher preparation courses and achieve the cut scores. This was especially the case in physics, chemistry, and earth and space science during the period 1987-97 when there was no cut score whatsoever promulgated by the Pennsylvania Department of Education, and remains the case for General Science which continues not to have any cut score.16 An interested general science teacher candidate merely has to take the ETS general science test to pass it.
(Table 5 about here) Table 6 shows all of ETS’ Praxis teacher tests, the range of test scores ( Column (2) and Column (3)), the national scores at the 25’th 50’th and 75’th percentiles of the national distribution (columns (3)-(5)), and the percentage of questions of average difficulty correctly answered for the score at the 25’th percentile (column (6)).
Table 7 carries forward from Table 6 the actual score at the 25th percentile, the percentage of questions of average difficulty answered correctly at that score, and then, in subsequent columns, the passing scores as of March, 1998 for seven comparison states. Very few states set passing scores above the 25th percentile, consistent with the passing scores observed for NTE tests in Pennsylvania discussed earlier.
(Table 6 about here) (Table 7 about here) These low cut scores and high passing rates can also be compared to practices in other countries for public education, and to other professions in the U.S. Bishop(1995) reviews the evidence on teacher preparation standards in the US vs. France and the Netherlands, and the performance of high school students on international achievement tests, and emphasizes the far more selective nature of teacher preparation programs overseas than in the US. In France, for example, only 31% who took the Certificat d'Aptitude au Proessorat de l'Enseignement du Secondaire passed it, while the more rigorous Aggregation Externe had a pass rate of 17.7% rate.
With regard to pass rates in other US professions, in accounting, 20,213 candidates sat the Spring, 1994 CPA examination; passing score standards are set by each state CPA society. Nationally, only 17.6% passed all of the various portions of the exam while 50.4% failed all portions of the examination. In Pennsylvania, 5.2% passed all portions of the 1994 examination, and 62% failed all portions. Overall, 32.0% nationally and 32% in Pennsylvania passed some portion of the overall CPA examination.17 In law, 69.8% of those who took a state bar exam passed it throughout the US in the Winter of 1995; in Pennsylvania the comparable passing rate was 48%.18 By contrast, the passing rates in Pennsylvania and most other states on teacher certification tests is 95% or higher.
2.6 Example of Pennsylvania Program Approval Standard: Mathematics Program Pennsylvania Department of Education regulations governing approved programs of instruction are both quite voluminous and quite vague. As an example, consider those governing the approval of mathematics preparation. They are quoted in their entirety below to acquaint the reader with the nature of the state standards: 19