Hi everyone. This may be an interesting post. Now, I am no braggart, not by any means, but I am a relatively sharp guy. I have just read pages 46-48 in the reading for the 4th time. I have highlighted and taken notes, but I am going to admit that I can’t make heads or tails out of what they’re trying to relay in the explanations of probability and effect size. And I’ve been a math teacher for the past 20 years!
The explanation of standards deviation is vague and the rounding of all numbers to a standard deviation of 10 actually confuses the example. Figure 2.4 on page 47, I believe incorrectly displays SD=10, when it is shown as SD=5. Now, how they take that understanding and relate it to measure of probability really threw me. I cannot figure out, by text or graphic, where the example shows p < .05. Page 47 talks about probability being .05 or less but doesn’t relay how that number was calculated. Where does that number come from, and how was it calculated? Can anyone help here?
Then I just about lost it thinking about effect size. Dividing a number by an unrealistic standard deviation of 10 when the artificial difference between the control group and the test group is 10 yields an effect size of 1. ¿Qué? Of course it does. The book rounded everything so much; it’s hard to make real meaning here.
Now, I looked at a recent math test for comparison. The average standard deviation in my math class is around 40, sad but true. In a heterogeneous classroom, this is often the case. I know, I’m looking for a high average and low standard deviation, but reality is reality. For me to have any statistical significance in terms of effect size (according to page 48) using the understanding of standard deviation (using an ES of .5 as moderate), the mean score on my tests would need to yield a difference of 20 points. Wow! Looking at the mean for my tests and quizzes this year, I don’t think trying to yield a difference of 20 points in mean is possible. The mean in my room is often in the low 80s.
But I digress. How do the rest of you use data like this to make determinations about instructional practices? Do any of you find this information complex? How do you interpret research statistics?
John,
ReplyDeleteYou bring up a great point that the majority of consumers glaze over statistics and read results to say "What conclusions did you draw?" I am sure that someone has developed a statistical analysis that was completely wrong, but many people would accept because they don't understand the mathematics. I too struggled to follow their mathematical reasoning. (Now, I'm not stating their conclusions are incorrect, I just think that we sometimes accept results without proper vetting).
Sam, I laughed aloud at your post. I picture a school committee meeting, with a presenter blabbering statistical analysis, and the school committee having no idea what he is talking about.
ReplyDelete...some fancy PowerPoint slide with a bunch of numbers no one has any idea how to interpret... they looked glazed over, nodding like sheep...
Data is going to drive the next big change in education. That is what Race To The Top is going to require. We look at data all the time in my middle school, dealing with corrective action for our subgroups, and meeting average yearly progress for our aggregate population. Nothing worse than being in a staff meeting and having them throw charts and graphs at you showing MCAS scores in every conceivable extrapolation possible. Data is important, but I fear the ramifications to schools who don't quite stack up.
Hi John,
ReplyDeleteI thought it was just me but I felt the same way you did when reading those stat pages. I even Aced stats in undergrad. My school does the same thing with data and what I see happening is what every true educator fears the most, the children are being turned into numbers. Anybody who studies education or humans know that not every child (adults too) learns the same nor do they preform the same when given tests. So, what do all these numbers really mean? Did little Steve really know the answers on test or is he just a good test taker?
Hey Barb: It gets even worse! Race to the Top is going to use those stats to decide upon teacher raises, comparing the growth of kids in my classroom to the kids in your classroom. This is how merit pay will work! This will kill public schools! If I know my merit raise is going to be based on my students test numbers compared to your students test numbers, do you think I am going to work collaboratively with you on anything!? No way! This will pit teachers against teachers, and totally ruin to collaborative community that teaching is founded on. So much for a community of learners! And I'm not just being "Mr. Pessimism" here. Merit pay will ruin public education if teachers don't work together for the common community.
ReplyDeleteI joked with my principal that if Race to the Top does institute merit pay for teachers, then we'll have to have a draft lottery every year for students!! Who gets the special ed kids, the ell kids, the life skills kids?? A draft lottery would be the only fair way to assure equitable classrooms. My principal didn't laugh...
Good post here. It made me laugh. I thought this section was horribly written. Statistics is a very tricky thing. I totally agree with you about the numbers being unrealistic. Now, I know it was just an example, but they did a poor job explaining some of how they concluded their information. I even had our AP statistics teacher look at it. He just laughed.
ReplyDeleteI will have to disagree about your comment about merit pay will kill public schools. I think in the end, it will weed out a lot of bad teachers. I don't know about your school, but at some schools I have worked at in the past, there are teachers that, well, let's say have not been pulling their weight. They occupy classrooms that do not require any kind of end of course exam and really do not teach students anything. Those teachers give us all black eyes. You do bring up some great points about how the students will be assigned. I personally do not have a good answer for that yet. A draft lottery is as good as I have heard. After all, a 10% gain for an honors student may be drastically different than a 10% gain for a special ed student.
I always like to pose this question, not for controversy, but to get opinions. What is your opinion on how teachers should receive their raises? How would you decide which teachers get raises and how much? Currently, where I am employed, raises are determined by years of service and degrees. So, there are teachers at schools making the same amount of money but doing drastically different amounts of work. What is the incentive for teachers to do their job better?
The only fair way is to pay based on years of service and degrees. That brings equity to the environment.You can use standardized test scores for math and english teachers, but what about the art teacher? The spanish teacher? The social studies teacher? What is the measuring stick for them? To start pitting raises based on constantly moving measuring (and often inequitable)devices isn't fair. And in this case, fair is fair. You weren't the only teacher who is responsible for Davie making that progress this year, so why are you getting that raise and the rest of us aren't? Tough stuff!
ReplyDeleteI agree with Derek that there is some dead wood, but less and less each year due to requirements like masters degrees and having to renew certifications based on professional development. Also, your principal should have a professional development plan for every teacher in the building (that is required here in Massachusetts). Each teacher's professional development plan must align with building and district goals and must meet the needs of state recertification protocol. These measures make it much harder to hide bad wood. Also, the push in data analysis will also make those teachers step up to the plate or face repercussions. Principals must hold their staff accountable for proper implementation of building methodologies and protocols. I don't agree that merit pay will get rid of bad teachers. Leadership assures the classroom is successfully run. I know one thing, when I finish my masters this may, I am going to enjoy the column jump in salary scale. And you know what, I deserve it!