Gifted Issues Discussion homepage
Posted By: inky NWEA Letter Help - 01/16/09 04:51 PM
I'm looking for feedback on a letter I've drafted to the NWEA about the erroneous MAP test data. I thought Mr. Boekelheide would be the right person to send it to, since it seems to be a quality issue. Please let me know if I can get my point across more clearly.
I'd rather have direct honest feedback than tactful replies designed to not hurt my feelings. smile
http://www.nwea.org/about/

Quote
Dear Mr. Boekelheide,

We are writing about the MAP for Primary Grades Initial and Preliminary Norms. In October we were given hand written notes for our daughter�s RIT scores and associated percentiles:
Fall 1st Grade: Reading RIT 195 (81%) Math RIT 187 (77%)

These percentiles did not make sense to us, so we started asking questions and digging deeper. Eventually we were told the scores had been renormed using the MAP for Primary Grades Preliminary Norms. We compared the data from the Preliminary Norms with the RIT Scale Norms for Early Primary Grades (RSN for EPG) dated November 29, 2007.

Specifically, the standard deviations (SDs) for the Preliminary Norms appear to be 33 points for math and 31 points for reading. The SDs for the RIT Scale Norms are 12 points for math and 10 points for reading. To put it in perspective, the difference between the beginning of year median for 1st grade and 3rd grade is 32 points in reading and 28 points in math.

After pointing this out to our school officials, they contacted NWEA and found out the Preliminary Norms should not have been used. We are concerned about how the Initial and Preliminary Norms were developed, how they were distributed to school personnel, and how NWEA has communicated with parents and schools about the validity of this data.

Testing gives objective information that is necessary for teachers and parents to make informed decisions about instruction and curriculum in the classroom, school, district, and home. Accurate assessments are critical to the credibility of these tests. Inaccurate data presents a whole host of problems. For example, the Individuals with Disabilities Act requires Individual Education Plans to include measurable annual goals and a description of how the child's progress toward meeting the annual goals will be measured. It is critical to include accurate, objective measurement in IEPs. Changes in test scores over time provide the means to assess educational benefit or regression.

In addition, a number of programs for gifted children require a qualifying test score. For example, the Johns Hopkins CTY Talent Search requires achievement at the 95th percentile or higher on one or more subtests of a nationally-normed standardized test and MAP is one of the tests CTY accepts. When parents are given erroneous data showing their child is at the 81st percentile instead of the 99th percentile, children can lose out on opportunities to participate in these types of education programs.

Educational and psychological tests are designed to present normal bell curve distributions with predictable patterns of scores. When school officials are using data that shows a child is 1 standard deviation from the mean instead of 3 standard deviations from the mean, there should be some concern about whether children at that school are getting appropriate services designed to meet their unique needs. For above the mean, this is a difference between the 84th percentile vs. the 99th percentile. For below the mean, this is a difference between the 16th percentile vs. the 1st percentile. The need for intervention may be obscured if the wrong percentiles are considered. These are some of the impacts of having erroneous data.

We have attempted to address this at the local level and will continue to work with our school district in getting the correct data to teachers and parents. Unfortunately we are concerned that other districts have made similar errors and may also be using the erroneous data. For example, we found the MAP for Primary Grades Initial and Preliminary Norms posted on different districts� web sites:

http://www.bismarck.k12.nd.us/uploads%5Cresources%5C2530%5Cmap_pri_fall_percentiles.pdf
http://www.bismarck.k12.nd.us/uploads%5Cresources%5C2531%5Cmap_pri_spring_percentiles.pdf
http://teachers.greenville.k12.sc.u...20Training%20Reference%20Links091207.doc

We feel the ramifications of inaccurate data necessitate NWEA taking additional steps to ensure districts, teachers and parents have the correct data instead of the erroneous data. Please let us know what NWEA is willing to do to resolve this issue.

Sincerely,
Posted By: 2KMom Re: NWEA Letter Help - 01/16/09 05:26 PM
Inky, thanks for posting this info. I am a newbie here and found this forum searching for info on assessment and achievement testing, specifically the NWEA. I want to now look at my daughter's NWEA results and info on the norms. Her results are all over the map and don't seem to be assessing her ability. Thanks!
Posted By: inky Re: NWEA Letter Help - 01/16/09 05:42 PM
NWEA is the test maker. According to our district's assessment coordinator, NWEA trains the schools on how the test is designed; what data the assessment provides; how to interpret the data; what is appropriate and not appropriate to do with the data; and what additional materials are available to the schools to support the next steps with instruction. They developed the Norms. They made an error developing the Preliminary and Initial norms.

The school made an error substituting the Preliminary Norms in place of the official norms. The district didn't seem to provide any oversight to prevent this and hasn't taken steps to make sure other schools in our district aren't doing the same thing.

Trying to get this addressed at the school and district level, I get the impression that they view erroneous percentiles as no big deal. I included the examples to show why it is a big deal.

The letter I wanted to write channeled the Saturday Night Live comedian:
Identify the problem. FIX IT!! Identify another problem. FIX IT! Repeat as necessary until it's all fixed!



Thanks for the feedback
Posted By: inky Re: NWEA Letter Help - 01/16/09 05:52 PM
2KMom,
If you view my posts, you can find more background under my first post "Weird Scores." Hope it helps you understand her results better.
Posted By: st pauli girl Re: NWEA Letter Help - 01/16/09 06:05 PM
Originally Posted by inky
[quote]
We feel the ramifications of inaccurate data necessitate NWEA taking additional steps to ensure districts, teachers and parents have the correct data instead of the erroneous data. Please let us know what NWEA is willing to do to resolve this issue.
quote]

Hi Inky - I think your letter is good. I'd maybe tone down the last paragraph, because I think NWEA would be more willing to work with someone who is not so obviously angry. (I think you should be angry, but this last part would maybe lead the recipient to put in on the bottom of the pile...)

Maybe "Could you please review the instructions that are given to the districts and perhaps send an addendum explaining more clearly how the results are to be interpreted? As you can see, just a tiny bit of research in our district showed that several schools are using incorrect data, so I feel certain you will find other similar districts nationwide. " or something like this? Just a thought.
Posted By: inky Re: NWEA Letter Help - 01/16/09 06:35 PM
Good point SPG. I'll work on the last part.

Actually, I don't even know if other schools in our district ARE using the incorrect data. Our district seems to have washed their hands of any oversight, saying which reports to use is a school decision. So some of the anger stems from feeling responsible because I know about the situation, but not being able to get the people in charge to take on the responsibility for fixing it. cry
Posted By: inky Re: NWEA Letter Help - 01/16/09 10:54 PM
Thanks kcab and I'll incorporate those suggestions. I definitely want it to go to someone who understands statistics.

In NWEA's Parent Toolkit, it says

Quote
we are unable to discuss any student information, test results, or district assessment programs directly with parents.

and

Quote
Because each district's implementation is unique, parents will need to direct specific questions and concerns to their local school district resources.


That supports your point that NWEA may have limited ability to change the situation. Since they put out the bad data in the first place, I want to emphasize their responsibility to try and correct it.

So your post helps me summarize the problem as: Invalid data is being disseminated by school officials to teachers and parents. It is also being disseminated on the web.

Posted By: Grinity Re: NWEA Letter Help - 01/17/09 12:10 AM
Hi Inky,
I like your letter as a 'phase II' but i would strongly reccomend a phase I as follows.

Phase I:
Part 1: Phone call, 'Hi, I'm calling because you won't believe the trouble we had with your test! I'd love to tell you about it because I'm sure you wouldn't imagine the terrible mess it caused! Who would be the best person to talk to?

Part 2: Talk to the best Person

Part 3: Write a 'thank you' letter to the Best Person, and re-emphasise your main points.

then, if you still feel you are being blown off - Phase II!

KWIM?

Best Wishes,
Grinity
Posted By: chris1234 Re: NWEA Letter Help - 01/17/09 12:43 AM
This is a really strong, super well written letter! I would still probably go with a phone call first - letter in hand.

I think a less formal mode of communication puts people more at ease from the get go, and the back and forth you probably need to engage in is so much more easily accomplished.

Good luck, sounds like a very worthy effort!
Posted By: inky Re: NWEA Letter Help - 01/17/09 02:57 PM
Chris,
Thanks for the feedback and I slept on your advice about calling first. I've been frustrated with trying to advocate by phone about school issues in the past. I've gotten lots of information by phone but it's been difficult to act on it.

Another part of it may be that I don't like being on the phone in general. I like taking time to collect my thoughts and mull over the other person's response. Some of the advantages I've found to writing are:

Quote
Letters create pressure, build accountability, and encourage others to become more productive and responsible.
(from School Advocacy Hamilton)

Quote
If it was not written down, it was not said. If it was not written down, it did not happen.
(from Wrightslaw)

So after giving it serious consideration, I'm going to start with the letter. I appreciate you bringing it up and understand your point. smile
Posted By: inky Re: NWEA Letter Help - 01/17/09 03:21 PM
2KMom,
Another thing to keep in mind with MAP is that it is more likely that your daughter's scores are underreporting her abilities rather than inflating them.
From http://www.nwea.org/support/details.aspx?content=1024
Quote
Is it possible for students to get a falsely inflated score by guessing?
By nature, it's impossible to know when a score is "inflated." However, the number of means by which one can get an inflated score are generally limited to either guessing or cheating. Cheating can be difficult if the room is monitored properly and few students will ever be "lucky" with their guessing.

On the other hand, any number of things can deflate a score. Fatigue, hunger, anger, boredom, or distraction can all have negative impacts on a student's performance. It is more likely that a score that doesn�t reflect a student�s true ability will be deflated than inflated.
DD6 took MAP math last week. I was asking her how it went and was pleased to see she got questions involving multiplying and dividing. She said one of the questions was what's 5 divided by 5 and she'd answered zero. I said, "If you have 5 pieces of candy and divide them among five people, how much would each person get?" She immediately answered one, so I explained that's what 5 divided by 5 means.
Posted By: inky Re: NWEA Letter Help - 01/17/09 03:57 PM
As far as I can tell, the length is determined by how many questions the child gets right. The more questions right, the longer the test takes. I'm guessing DD spent about 20 minutes on the math portion this time.

In K, DD was crying because she was taking the test and was the last one finished. The other kids had left to their next subject, but she was still answering questions. When she looked up and realized she was the only one left, she thought she'd done something wrong. frown
Posted By: Grinity Re: NWEA Letter Help - 01/17/09 04:45 PM
Originally Posted by inky
In K, DD was crying because she was taking the test and was the last one finished. The other kids had left to their next subject, but she was still answering questions. When she looked up and realized she was the only one left, she thought she'd done something wrong. frown

Oh My! Poor Thing! And folks say that being the 'fastest' doesn't get reinforced at school because babies are to young to understand what 'finishing first' means. NOT - this proves it. Were you able to explain to her that with this particular test the one who finishes last, finishes best. Super news about the Mult and Division. I think that gives you some hope that she did really really well, yes?

I agree with 'not written means it didn't happen' which is why I included a 'thanks for the phone call step' to be put in writing. I understand that we all have various strengths and weaknesses, I tend to forget that...opps. I still think a brief phone call to ask 'who' you should address the letter to would be in order. You don't have to go into the whole story, just ask them to help you. I called MAP when I first heard of them, and the person I spoke to sounded really interested and pleased to hear from me. 'N of 1' which is how we say: Your milage may vary!

Smiles,
Grinity
Posted By: JoAnn Re: NWEA Letter Help - 01/17/09 05:18 PM
A very good letter, long but warrants attention. I have my kid privately tested due to thestypes of problems and more with the school screeners. The school will not use them for placement but at leasy we know and I submit them and ask for explanations when there is a discrepency between their screeners and a full WISC or Stanford.

I would have a couple bullets that highlight the main problem and highlaight what you are asking them to do at the end. Our admin. have a difficult time reading anything longerthan a half page. sad but true
Posted By: inky Re: NWEA Letter Help - 01/17/09 05:52 PM
Thanks Grinity,
I'd missed your earlier post and the script you wrote helps. With two people emphasizing good reasons to call first, it looks like I need to get over my phone aversion.

P.S. Yes, I took it as a good sign that she got to the multiplication and division questions.

P.P.S. She understood better this time that taking longer was a good thing. She was just a little disappointed because she didn't have as much time to read her book afterwards as the other students had. wink
Posted By: Grinity Re: NWEA Letter Help - 01/20/09 11:49 PM
Originally Posted by inky
Thanks Grinity,
I'd missed your earlier post and the script you wrote helps.
oh good!

((eyes rolling)) at rushing through tests to have time to read a book - if gifted children wrote country and western songs, I'll bet they would sound like that!

Smiles,
Grinity
Posted By: inky Re: NWEA Letter Help - 01/28/09 02:47 AM
So I called NWEA last Tuesday and spoke with Lori in partner relations. At first she told me I'd have to contact my district about any MAP test information. I explained I had contacted my district but my concerns went beyond my district.

I explained I was calling about the Preliminary norms and was concerned this information was being used by other districts because it was on their web sites. When I said something about inaccurate data and standard deviations, that got her attention. She wanted to know the name of DD's school and said someone would be contacting me. I never got a chance to fully explain the situation.

How long would you wait to hear back before calling or sending a letter?
Posted By: chris1234 Re: NWEA Letter Help - 01/28/09 10:56 AM
Well, since they didn't immediately send out the black helicopters with the zip lines to nab you...I'd give it a couple of weeks. grin

It's been one week already, right?

The sudden shift there does sound promising! Go, go, go! smile
Posted By: Grinity Re: NWEA Letter Help - 01/28/09 11:31 AM
What a drag! IMHO - Maybe wait one more week, and then release the letter. It would be fine to call back, but not gaurenteed, and I know your aren't a phone person.

Thanks for trying. You've fufuilled your obligation.

Grinity
Posted By: inky Re: NWEA Letter Help - 01/28/09 05:04 PM
Thanks and I'll wait another week. I've come to realize my sense of how long things should take in the education world is skewed. I'm used to working on tight deadlines and expect the rest of the world to operate the same way. blush

Quote
Well, since they didn't immediately send out the black helicopters with the zip lines to nab you...I'd give it a couple of weeks
This cracked me up! If you don't hear from me for a while, send out a search party. wink
Posted By: inky Re: NWEA Letter Help - 01/28/09 06:52 PM
Quote
Did you get the email for the person you spoke to?
No, I barely remembered to ask for her name before the conversation ended. blush
Posted By: inky Re: NWEA Letter Help - 02/04/09 04:07 PM
Just a quick update. After waiting another week with no response, I mailed a modified version of the original letter to NWEA. I'll pass on any new info if/when I hear back. Thanks again for the feedback.
Posted By: Grinity Re: NWEA Letter Help - 02/05/09 02:33 AM
Go Inky!
Posted By: inky Re: NWEA Letter Help - 02/18/09 03:39 AM
One of the NWEA Norms Study authors called me today and confirmed the Preliminary Norms should not have been used. He was appreciative that I sent the message that it may not be an isolated case and intends to contact the communications department to get the word out.

It was refreshing to talk to someone who "gets it" after four months of feeling like I'm speaking another language than the school personnel. It really helped to know people on this board "got it." smile It feels like an enormous weight has been lifted off my shoulders. Thank you!!!

Posted By: Grinity Re: NWEA Letter Help - 02/18/09 03:50 AM
((virtual ticker tape parade))

((happy dance))

((rose petals falling from the skies!!))

Yippee!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Go Inky! Go Inky! Go Inky!
Posted By: Mia Re: NWEA Letter Help - 02/18/09 03:57 AM
Wow, that's great, Inky! Congratulations! That will be so helpful for so many students, parents and schools.

Well done!
© Gifted Issues Discussion Forum