My notes from Ward 3 community meeting
Below is my review of the W3 community meeting. Note: In trying to reduce the roughly 1-hour of Q and A/comments to something simple and quantitative, it’s important to note how much judgment is required in determining whether a question is a point of clarification or a statement; how to aggregated similar questions; how to paraphrase fairly, how to take account of audience reaction, etc. Here’s my best effort at summarizing with, first, a qualitative summary of the most-raised, most resonant points. Then, a more quantitative tally and tick tock of the questions.
Summary
The issues/suggestions (relevant to accountability portion of the OSSE proposal) that came up more than once and seemed to resonate most with the audience (about 30 people; roughly 21 questions/comments) were these five (not necessarily in order). I was very taken with point one, which hadn’t come up previously. It strikes me as both a way to resolve the disagreements that exist about how to weight different indicators and a way to give parents more nuanced information (though I know some worry that the nuance will make it too complicated; I don’t think it does).
- The 5-star system reduces lots of complicated things to a single score that obscures more than it illuminates. The rating would provide more transparent, useable information if separate measures were provided for proficiency, growth, climate, etc. (Suggestion made by two separate people, with many nods and assents both times.)
“It troubles me because inside each star/box is complicated math, which then gets reduced to just 5 stars. What’s lost is everything that you’re trying to measure and communicate. This measure is going to tell you how many upper-middle class kids to each school. The growth will be invisible, and it will be devastating to schools that ay be doing a good job.”
- Various objections to placing so much weight on reading and math testing vs. other subjects/other measures/other goals. Two revealing (and lightly edited) quotes:
from a parent: “We’re 25 years into education reform. Human reality shows us that we teach to tests. We wish it were different, but it’s not as different as we wish. I’m concerned that while we say that we want kids to be well-rounded in arts, etc., we’re going away from that. This doesn’t right our ship. I really implore you to think deeply about a testing system that only asked schools to focus on math and reading. What this will do is to continue to incentivize schools in a certain direction.”
from a teacher of special needs/English language learners: “When reading and math are the data points, that’s what happens. [There are] student who are enrolled in English as a Second Language and special ed—and have a double period of reading and math. They get no social studies, no science. That wouldn’t be the case if the focus was a broader set of metrics.”
(Three different comments focused on the narrowing effect of the reading/math focus. Two others were focused on how the result of this emphasis is too much testing.)
- Insufficient attention to growth (v. proficiency), with special concern about the lack of any growth at all for high school. (One comment specifically about how growth may be rendered invisible in the score. Two leading/distressed questions about why there is no high school growth; plus, a number of additional questions/comments related to high school growth but/and were also related to the continuing, confusing, frustrating testing situation at Wilson.)
“Does it concern you that we may have schools that are doing very well and get just two or three stars?”
- Two specific requests for a climate survey; two comments/leading questions suggesting that attendance and reenrollment were not great indicators of climate and that we can/should do better. Also, a comment that the consensus of experts is that such a survey should not be used in year 1 to hold schools accountable; year 1 should be a pilot.
- Two questions/comments raising skepticism about whether or not OSSE really planned to make substantive changes to the plan, given how little time remained.
In the final comment of the evening, a mathematician cited Campbell’s Law, an important principle from social science, which captured much of the meeting’s sentiment:
“The more any quantitative indicator is used for social-decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processed it is intended to monitor.”
Other questions concerned how disadvantaged students would be better helped through this plan, how the plan would show the progress of subgroups, the need for PARCC to be tweaked, the race-based character of the annual goals, whether any mathematicians were on staff at OSSE, whether the proposed system measured schools against a standard or relative to other schools, the need to pay more attention to the quality of principals.
Here’s my tally and a tick tock of the gist of the questions and (some) answers: I tried to provide the (paraphrased, shortened) answers to questions to most of the questions, but for the most part, I didn’t try to capture OSSE’s explanations for why a given item was in or out of the proposal, especially if they were long.
| Question | Wgt prof v gro | Wght tsting/read’g-mth | Climate | 5-Star v “dashboard” | timeline/willingness to revise | Clarifying Q’s | Other issues | |
| 1 | Q Can you tell us how much time is taken by the testing?
A No answer at this time.
|
1 | ||||||
| 2. | Q So: Are schools being judged based on an absolute standard of relative to other schools
A Both. (On the proficiency side, while proficiency is the standard, the floors/ceilings means that the initial scores—but not subsequent ones–are in part relative to other schools. On the growth side, growth-to-proficiency is in part relative to other schools (MGP) and in based on the proficiency standard)
|
1 | ||||||
| 3 | Q How did you decide to pick reenrollment and attendance for climate measure
A. mainly because they’re already used by either or both of DCPS and charters.
|
1 | ||||||
| 4 | Q. Is there a measure of equity in here—something that considers how lower performing student groups are faring.
A. We don’t show it here, but 75% of a school’s score is based on schoolwide scores. 25% is based on how subgroups fare.
|
1-tracking of subgroups/measure of equity
|
||||||
| 5. | Q How come there’s no high school growth score?
A. Most students take tests in 8th and 10th. This spring will be first year that we will have PARCC scores for student who took them in 8th and 10th grades. We need to review them. And, all students don’t take the same sequence. Haven’t figured out how to address. Some other states test twice in high school, so easier for them to measure growth.
|
1 | ||||||
| — | Here and later: Set of questions and answers regarding the rules that govern which students take which math tests, which count, who makes rules, etc. Very important, but I don’t record the details/questions/clarifications that aren’t specific to the ESSA plan
|
|||||||
| 6 | Q Wouldn’t it make more sense to make algebra 1 the high school test, so that growth would be measured from math to algebra, not math to geometry?
A. We are open to exploring other options in the future.
|
1 | ||||||
| 7 | Q Is this set in concrete?
A It’s a draft. Point of community engagement is to consider making changes.
|
1 | ||||||
| 8 | Q Does PARCC get tweaked periodically?
A Yes, recently PARCC eliminated one test and reduced overall time of testing by about an hour.
|
1-need to tweak PARCC | ||||||
| 9 | Q Concerned that there are no mathematicians on staff at osse, which is a problem both for improving math ed and for vetting proposal | 1-need mathematicians at OSSE | ||||||
| 10 | Q How does the plan handle long-term goals. Why are they race-based? Can you eliminate them?
A. We have high expectations for all students… (My clarification: The following night, the superintendent confirmed on the record that these goals have no impact on school ratings; they are in effect for use as an internal guide.) |
1-problem with race-based long term goals | ||||||
| 11 | Q Why do we have to have the stars? It troubles me because inside each star/box is complicated math, which gets reduced to 5 stars. The number of stars is what people will see.
What’s lost is everything that you’re trying to get at. I think what the stars will mainly tell us is which schools have high percentages of upper middle class kids Same with attendance. The growth will be invisible, and the result will be low scores that are devastating to schools that may be doing a really good job. Does it concern you that we may have schools that are doing very well and get just two or three stars? A. The report card will provide qualitative context. The reason we have a summative star judgment is because that’s our understanding of what parents want. |
1 | 1 | |||||
| 12 | Q. I agree with that. I have a different point: We’re 25 years into education reform. We have an opportunity to learn from that. ln 25 years, human reality shows us that we teach to tests. We wish it were different, but it’s not as different as we wish. I’m concerned that while we say that we want kids to be well rounded in arts, etc., we’re going away from that. We haven’t righted our ship. I really implore you to think deeply about a testing system that only asks schools to focus on math and reading. What it will do is to continue to incentivize in a certain direction. In 12 years of DCPS, I’ve seen what it does. That’s what it is incentivized. And, look at how PARCC is administered. You aren’t getting good data. It’s very flawed (discusses the problems with testing at Wilson/Walls) You don’t know what happens. No blame here: We should be less defensive and try to understand what’s happening.
A. We agree on need for richness. PARCC is based on more than just reading and math.
I made this comment: There are competing assumptions about whether schools are or shortchanging social studies, science, arts. Highly recommend that a good review/study be done to answer this question with the facts.
|
1 | Parcc data collection is flawed, | |||||
| 13 | Q. What have you done to address the special needs of economically disadvantaged students.
A Title 1 of ESSA provides funding for disadvantaged students. There’s nothing explicitly new in this proposal for economically disadvantaged students; but supporting these students is part of our ongoing effort… Back and forth about the need to provide specific programs to groups of students who are not doing well.
|
1-inadequate attention to providing support for disadvantaged and other subgroups | ||||||
| 14 | Q. The15% this is allotted to climate should be devoted to things besides attendance and reenrollment. Would like a climate survey.
|
1 | ||||||
| 15 | Q. (from a teacher who teaches ELL and special education) When you put math and reading up there as a data point, that’s what gets focused on. Students could be enrolled in English as a Second languages, special ed, double periods of reading and math, and get not social studies, no science. That wouldn’t be the case if the focus was a broader set of metrics.
I see it at schools where my children attend also. In one case, parents bought the school science materials. They sat there gathering dust. Because there wasn’t a priority placed on science. Also: Surveys that go to staff on climate are not good. No way to tease out current issues with the current survey. We should look at what Montgomery County does on this.
|
1 | ||||||
| 16 | Professional consensus is that schools shouldn’t be held accountable for the survey results in first year. Year 1 should be a pilot. | 1 | ||||||
| 17 | Q. I know you’re trying to provide a simple, easy way to understand rating with the stars. But most people would get more from separate measurements for proficiency, growth, climate. Would like to see more effort around improving principals.
|
1 | 1 more attention to supporting/improving principals | |||||
| 18 | Q. I’m incredibly concerned with all the testing. There are many kids who don’t perform well on tests. I was one of them. I was in lowest group all of my life. To measure all these students and schools by testing, including in pre-k, makes me shudder. You’re not getting what can come out of these amazing human
Teachers are being punished if kids don’t well. Other countries don’t do it.
|
1 | ||||||
| 19 | Q I don’t think you will blow this up based on public comment. What flexibility do you really have to change?
A One thing you’ve heard us say is that we would like to use measures that are currently available. To collect new data points will be hard… Yearly, there will be a re-evaluation… we will look at it again in ‘18-‘19 school year.
|
1 | ||||||
| 20 | Q. Just realizing for first time that there’s no science in this system. We already have the test. Why can’t we use it?
A There are technical issues with using the science test and scores. We’re keeping our eye on it for the future. We haven’t been able to release scores yet…
|
1 | ||||||
| 21 | Q. Campbell’s law—Heisenberg Principle for social science. “The more any quantitative indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
|
1 | ||||||
|
Totals |
3—too much prof/not enough growth | 5- too much weight on testing; too little on other subjects | 2-for climate survey
1-Yr1 pilot |
2-for dashboard instead of a single aggregated rating score | 2-concern that revisions won’t reflect comments | 1 | 8 | |
| Question | prof v grwth | Wght tsting/read’g-mth | Climate | 5-Star v “dashboard” | timeline/willingness to revise | Clarifying Q’s | Other issues |