During a recent school board meeting a recommendation was made that the A&E math program for grades 4 and 5 be eliminated. After reading both the external report, the internal report, and other relevant materials, I found myself in disagreement with many of the statistical analyses and data interpretations from the internal report. Consequently, I do not believe the recommendation of eliminating the A&E math program for grades 4 and 5 has a sound basis.
1. On page 28, the internal report states that “Data collected shows no statistically significant difference in students starting the program in grades 4 & 5 versus students who start the program in middle school”. The footnote on the same page states that “Comparing the semester grades of 9th and 10th grade H&A (A&E) students: students who started the A&E program in grades 4 or 5 average 91.3% at the end of the first semester. Students who entered the program after 5th grade averaged 92.3%. There is no statistically significant difference between the two groups. If WWP were to start A&E math at the 6th grade level, students would meet with the same success.”
The internal report does not mention how many students started the A&E math program in grades 4 and 5 and how many started after grade 5. This information is crucial when presenting statistical findings. More importantly, simply comparing students who started the A&E math program in grades 4 and 5 to those who started after grade 5 does not provide a valid answer to the question of whether or not the A&E math program in grades 4 and 5 is effective. This reasoning is related to the so-called “causal inference” in epidemiological, psychometric, and statistical studies.
Let us first look at a simple example. Suppose we have two groups of cancer patients. Group A received radiotherapy as part of their treatment plan while Group B did not. And when comparing the two groups we found that patients in Group A had a shorter survival time (18 months) than those in Group B (24 months).
Does this mean that the radiotherapy is bad for patients? Without a proper analysis we should not jump into this conclusion. Those patients in Group A were given radiotherapy because they had been in a worse health condition. Had they not received radiotherapy they very likely would have lived even shorter, say, only 12 months. So the radiotherapy, with high probability, may have actually helped improve the survival time from 12 months to 18 months.
As we can see, the point here is that we should not compare Groups A and B directly without considering other confounding factors (e.g., disease stage, overall health condition, response status, etc.), because the two groups are not randomized.
Now let us examine the analysis in the internal report comparing students who started the A&E math program in grades 4 and 5 to those who started after grade 5. Again, to make the discussion easier I am going to assume that there are X students who started the A&E math program in grades 4 and 5 and Y students who started the A&E program after grade 5.
Why hadn’t the Y students started the A&E math program in grades 4 and 5? How about their general performance or scores in math and other subjects in grades 4 and 5? Is it possible that they were actually outstanding math students but chose not to enter the A&E math program in grades 4 and 5 (the internal report mentions that some parents do not sign their children up for the test, I will get to this later)? Have they received out-of-school training in math during the 4th and 5th grades?
Without such information on the confounding issues and other details about the data, we cannot trust any conclusion based on simply comparing the two averages 91.3 and 92.3 percent (and why are the semester grades of 9th and 10th grade H&A chosen as the gold standard for assessing effectiveness of the A&E math program in grades 4 and 5?). The correct question and analysis should be that, had those X students started their A&E Math program after grade 5, would they have had comparable or higher levels of academic achievement in mathematics?
Alternatively we may also ask, had those Y students started their A&E math program in grades 4 and 5, would they have had higher levels of academic achievement in mathematics? If the answer is affirmative for the former question and negative for the latter, then we would agree with the notion of eliminating A&E math in grades 4 and 5. However, current analysis in the internal report does not properly address this issue or support any radical statement such as eliminating the A&E math program in grades 4 and 5. Consequently, the conclusion of “If WWP were to start A&E math at the 6th grade level, students would meet with the same success” has no valid support.
2. During a presentation at the meeting, the issues of gender and ethnic imbalance were raised regarding the A&E math program in grades 4 and 5 because more than 90 percent of A&E students are Asian and more than 80 percent are male. It was also mentioned that the A&E math program in grades 4 and 5 is serving “too few” (fewer than 50 students). Frankly speaking, I am surprised that these observations became part of the analysis.
First, it is not a secret that women are outnumbered by men in the field of mathematics. Take a look at the faculty list of the Department of Mathematics at the Princeton University, and we will find that the majority of the faculty members are male. Should we then eliminate the math program at Princeton University? This year at the International Mathematical Olympiad held in Chiang Mai, Thailand, the U.S. team came out as the best in the world. All six members of this winning team are boys. Two out of top 12 students in the country on the United States Math Olympiad are girls, a fact considered by the head coach as “fortunate” because in many years this number was zero. So should we also eliminate the Math Olympiad program? We certainly need to work on this problem and make changes, but eliminating the A&E math program in grades 4 and 5 offers a terrible solution. We will never be able to solve a problem if we simply try to avoid or delay it. The gender and ethnic imbalance will still be there when students enter the A&E math program in grade 6 or later.
Second, as summarized by the external report, 62 percent of the district students are Asian, 6 percent black, 4 percent Hispanic, and 27 percent white, and less than 1 percent are Pacific Islander or multiracial. If we examine the data more closely, we will find a trend: Over the past 10 years we have had more and more Asian kids in the WW-P schools. The 2013 Demographic Study for the WW-P district shows that during the 2011-’12 academic year, the percentages of Asian students range from 81.6 percent to 45.4 percent for pre-K to grade 12, respectively.
So I do not understand why the majority of the A&E students being Asian becomes an issue. If it does then shouldn’t we also be concerned by the percentage of white students on our high school football teams? Should we then disband the football teams and simply ask them to start in college?
Lastly, I am confused by the assertion that the A&E math program serves “too few.” As pointed out by the district website’s description of the A&E math program, “the research on gifted education suggests that only about 3 to 5 percent of a school population will generally qualify to participate in a gifted program.” Assuming we have a total of about 1,400 students in grades 4 and 5, then there should be about 42 to 70 students enrolled in the A&E math program for grades 4 and 5. Therefore I do not understand why “fewer than 50 students” is a problem. And how does that lead to the recommendation to eliminate the program? Shouldn’t we instead expand the program to accommodate more gifted students if we do believe that it is serving “too few”?
3. At the meeting it was brought up that “half the students in the district’s culminating math course, Multivariable Calculus, were not in the A&E program.” Without more details this statement is quite unclear and could be misleading. How many A&E students are there? How many of them (percentage) took the multivariable calculus course? How many non-A&E students are there? How many of them (percentage) took the multivariable calculus course? How were they compared to each other?
If the total number of A&E students is much smaller than the total number of non-A&E students (which I believe is the case), then we should actually be impressed that up to 50 percent of the multivariable calculus students are (were) in the A&E program. Meanwhile, how about the comparison of course performance (in multivariable calculus) between A&E and non-A&E students? With scores and other information, we can easily conduct some statistical tests, and more comprehensive analyses may be done by including confounding factors. Finally, why does this observation support the recommendation of eliminating the A&E math program particularly in grades 4 and 5?
4. The internal report also tries to identify other problems of the A&E math program in general. These problems include
a. Identification of students for the program begins with a self-selected pool rather than a talent pool.
(i) Creates anxiety for some students who feel the pressure of a high stakes test that is a full grade level above his or her abilities.
(ii) Potentially eliminates some talented students because their parents do not sign them up for the test.
b. The perceived importance of the A&E test
(i) has created an industry of test preparation in the community
(ii) has led to outside-of-school test preparation that is not aligned with the math curriculum, the Standards for Mathematical Practice, and the Competencies
(iii) has produced a cohort of highly skilled students who lack the problem solving ability and conceptual understandings needed to be successful in the A&E program.
Are there any data to support any of these claims? How many students or parents have complained about their “anxiety” (in the external report we see that many responders spoke highly of the A&E math program)? Have we provided professional development for teachers on the affective needs of gifted students, particularly perfectionism, stress, and anxiety as suggested by the external report?
If some “potentially talented” students did not take the A&E test because of their parents’ decision, then what we should do is to reach out to these parents to investigate the cause. And more importantly, we should also try to find out how these “potentially talented” students had been taught in mathematics during the years when they were not in the A&E math program. Did their parents spend a lot of time helping them like the A&E math program would do? Would they have done better if they had been enrolled in the A&E math program (this is related to my comments on causal inference in #1 above)? Once again, I do not see why these claims would lead to the elimination of the A&E math program in grades 4 and 5 in particular, as they, if true, seem to be quite general and should apply to all grades and all tests.
In summary, in the internal report and the presentations during the school board meeting on September 8, the statistical analyses were wrongly performed and most of the numbers were misinterpreted. Thus I did not find the numerical evidence convincing enough for supporting the conclusion that the A&E math program for grades 4 and 5 should be eliminated.
Zhigang Zhang
West Windsor
#b#Logic Lacking in A&E Approach#/b#
After reading through the school district’s reasons for eliminating the A&E program for fourth and fifth grades, I find the logic for eliminating the program rather lacking.
First, the data in the Internal Report do not support the recommendation that the sixth grade level is a better time to start A&E math than the fourth and fifth grade levels. Rather, they provide direct evidence that A&E students, regardless of time of entry, are a homogenous cohort. All A&E students were tested into the program. Once they enter, they receive an identical middle school A&E curriculum. It is only natural that their grades are similar when they get into high school.
What the Internal Report did not mention are the potential negative impacts of eliminating the program for students who would be in A&E in the fourth or fifth grade. The district’s external report of the G&T program has identified a perceived lack of challenges at the primary level. Should WW-P cancel A&E for fourth and fifth grades, these students will not be intellectually challenged by the regular curriculum and are likely to lose interest in math. Thus, a decision to defer offering A&E math until middle school will damage our students’ intellectual advancement in the long run.
Furthermore, the analysis cited in the Internal Report is not statistically sound. A statistical comparison of two groups should be based on the assumption that two groups consist of independent samples. This assumption is not true in analyzing homogenous A&E students. The two groups are highly correlated. In addition, the internal report did not disclose the number of students in each group. This is a piece of public information that is easy to obtain.
The fact is only a small fraction of the students in the A&E program started in middle school. A recent eighth grade A&E class in one of the district middle schools has approximately 28 students, among whom only four started the program in middle school. At the same time, the seventh grade A&E class in the same school has about 35 students, among whom five started in middle school. How could a conclusion that “if WW-P were to start A&E math at the sixth grade level, students would meet with the same success” have been drawn from this flawed analysis?
The same Internal Report remarks that A&E program “supports deep content knowledge; provides a foundation for problem solving, proof and mathematical thinking; is appropriate in content and pace to meet beyond the curriculum learning needs.” To eliminate such an inspiring program, based on insufficient data and reasoning, is irrational and irresponsible. It is NOT the right thing to do.
Yunqing Li, Ph.D.
Princeton Junction
