Our district has just introduced STAR testing, and I don't know a thing about it yet.

Does anyone have experience with it to share?

DeeDee

Have you read the test website - http://www.renaissance.com/products/star-assessments? Lots of good info there, for example, each of the test areas and domains. What specifically are you looking for?

Thanks, Indigo. But I do need something that goes beyond advertising the test to describe how their "adaptive" model works, and what the data output given to parents actually means. It is oddly presented and not very informative...

This document may give you more info:

http://doc.renlearn.com/KMNet/R003957507GG2170.pdf

You'll have to dig a bit, but I'll summarize some of the relevant highlights:

STAR is computer-adaptive multiple-choice testing intended for use for screening, diagnostic, or progress monitoring functions. Only 24 or 25 items are administered to each child in each subject. Item selection changes from item to item based on performance on the previous items. In reading, in grades 3-12, there are 20 short comprehension questions, and 5 extended comprehension questions, both cloze tasks. In math, there are 8 each of numeration, computation, and problem solving. Each test takes about 10-15 minutes to administer.

Cut sheet for parents:

http://doc.renlearn.com/KMNet/R0054872491706A8.pdf

The parent report is quite sketchy, including only the SS, which you would need the developmental chart to interpret, the grade equivalent (of which anathema I think I need not say more than I have in the past!), the grade-based percentile and range, and the normal curve equivalent (another representation of rank order in the norm population). p. 38-41 of the above document include some interpretive information; the paragraphs on the scaled score are probably the most useful.

The big picture thought to keep in mind is that the instrument is intended for screening and progress monitoring, with an eye to identifying and monitoring at-risk students, not in-depth assessment.

And here is some norming data:

Reading.



Start from p. 98.

Math.



Math appears to be capped at +3 grade levels.

Yes, these documents are found on the website ... as mentioned, a bit of digging may be required to find the information a parent may be most interested in.

Also asking questions on this as well.


Can you figure out if the 1-5 grade tests cross over into grade 6+ content? DS is a 4th grader taking 6th grade math. He reported answering several questions of fractions and decimals and "not much interesting." He says he saw nothing he hasn't seen in class. He's inconsistent with test taking skills, however, and very easily could have messed up several questions.

Thanks, all. This gets me closer, and wading through it will clearly keep me off the streets a few days....

DeeDee

I think the lower grade test must extend to the higher levels because my daughter scored in the 7th grade level in fourth grade. We never received standard scores, only grade equivalents (unfortunately). I'm feeling frustrated with the STAR test right now because my daughter somehow dropped over four grade levels in reading over the summer, leaving her at 8.5, and the school accordingly dropped her work level four grades - based on a 24 question test, if I understand correctly, when she scored higher at the beginning of last year.

aeh, if there is a cap of +3, does that mean that my daughter must be reaching the cap? She is not a mathy kid, so I wouldn't expect a higher score, but it is useful to know the capabilities of the test.

apm, from what I've read in the technical manual, it appears there is a three grade cap in math, based on the start point you're assigned at the beginning (set by the teacher, and presumably based on your nominal grade level in the subject). So, yes, if your daughter's beginning point on the test was fourth grade, seventh grade would be at the ceiling of the test. OTOH, if the school chose to set her starting point higher, the ceiling would be accordingly higher. It's not the test per se that has a ceiling of three grades, but in any given item set, the computer adaptive item selection will not exceed +3 grades.

On the reading levels, I presume you have already had the conversation about the low likelihood of actually losing four grade levels of reading skill, in the absence of brain trauma? Are they willing to let her re-take it? The tricky thing about these CATs is that if you get the first couple of items wrong, you won't be able to work your way back up to your true level before you run out of items.

Aha. DS took his math with his class, likely setting his test at early 4th. Now to figure out if his scores matter. Likely yes since there's another compacted class coming up.

Thanks again for your time and expertise, aeh.

apm, at what points did your daughter take STAR reading over the past year? If a student has not taken the test in over six months, the system automatically resets the start point to 1-2 grades below the grade placement. If a student has taken the test more recently (that is, within the same school year), the start point is based on the last assessment. It is possible that her start point for this year was re-set low enough that she was not able to make it back up to 12th grade in 25 items (34 if the Enterprise version).

AEH, thank you. This is very helpful.

DS7's STAR report came back very odd: giving an overall recommendation of a 6th grade instructional level in reading, but all the goals listed in detail in the report are 8th and 9th grade curriculum goals. I assume the 6th grade instructional level comes from the cap, and if they started him higher, might change.

The math results are beyond strange-- they say he's substantially ahead, but then the subscores don't show that at all, leaving me wondering what if any relation the subscores bear to the overall scaled score and recommendations. I can't quite see how we will get to rational recommendations for instruction from here.

Thanks--
DeeDee

Aeh, they won't change it and say she will be retested later. We've been having a lot of issues so far this year.

apm, so sorry to hear that.

I don't think STAR has enough discrimination to be useful in any way. 25 items across many grade levels just doesn't give enough information. The test took about 10 minutes, and scores were wildly inaccurate returning >12.9 level for reading and math for many second graders in the class.

Disappointing not to get much useful information, but I wouldn't base placement or instruction on STAR results.

That sounds like a valid explanation especially if the do only beginning of the year testing or beginning and middle. After all since someone would have noticed brain trauma they got one of the assessments wrong.

That's helpful to know. I'm not sure exactly when she last took the test, but it was in the spring. So it was probably within 6 months, but maybe not. As I said before, she scored higher on it at the beginning of the last school year, so I think she must have just been distracted or rushing. I'm feeling really frustrated with them for an assortment of issues, including this one. At least she still enjoys the reading work most of the time; the math is a larger problem as she just draws all over the worksheets because she says they are too boring to face. She has been doing mastery tests for over a month now just to show she is capable of starting where she left off last spring and seems to have just had enough at this point. I think they make all of the kids start at the same point regardless of where they left off, so she had to start off showing she could add when she left off last spring getting ready for algebra. She's supposed to be working through fractions now, but is doing art instead. I'm trying to get a meeting scheduled and this information on the STAR will be helpful (for one thing, it sounds like the math version won't even show the right level).

Considering they made her restart at the beginning for math, I wonder if they deliberately reset the STAR to start at grade level?

I think star is only helpful in certain scenarios ...tracking the progress of someone at risk, and deciding who you don't have to worry about (tic). Also, it was helpful to allow ds full acess to any level book in the media center.

This is good to know. It is true for both reading and math? At our first meeting at DD8's new school they said they were planning to assess with the STAR to determine levels for her, but it if starts 1-2 levels below grade and caps at 3 above starting, that won't be useful at all.

Bumping a dead thread, sorry.


I got DS's scores as percentiles.

For the math, on table 58, a 4th grader scoring "99 percentile" has a SS of 838-859. Reading up on Table 57 (pg 128), this puts him at a grade equivalent of 11.7-12.9+. He reports that the test was "just fractions and some decimals." He's not a reliable reporter. There easily could have been algebraic expressions involving fractions and he didn't notice, or know what to call it, or realize that this is supposed to be more advanced.

What does this mean? So is he actually capped in testing +3 grade levels (so 7th grade) or is it telling us something more?

The only reason I care: I suspect these tests will be used to help determine whether or not he can take math 7/8 compacted next year.

The kid is about to undergo a boatload of stressful testing for other things. I can ask for a retest on this if I have to, but I won't if it's not going to have any impact.

The implication would be that he maxed out the test, with skills of at least seventh grade-level. The manual suggests that a fourth grade test would not have information on skills above seventh grade. Keep in mind that the grade-equivalent tells you what the median 11.7+ student would score on the same test, not that there were actual grade 11 & 12 items completed correctly.

Is there any reason to believe his scores would be an obstacle to compacted 7/8 math, at the moment?

Thanks, aeh.

I laughed at the 11.7. We keep getting that number for this kid since before he started kindergarten - never valid, but certainly a recurring level.

So looking at that table more closely, an 838 (his minimum score) is found at the 54th percentile for 10th grade. Does that mean that the 838 number is meaningless as a standalone measure? A 10th grader starts taking the test at the 10th grade level, so is not limited to a 7th grade level of math.

I'm missing something basic. No bother - I'll ask for a retest. Sigh.


The district is very anti-their-own-7/8 course, and do everything they can to limit enrollment in the program. His performance is rather uneven - not for lack of understanding but for lousy test format: it's the online Pearson digits curriculum that the teacher prints out as paper and pencil test - with computer formatting preserved. An 8 year old with a suspected writing disability and increasing attention issues isn't going to perform well there. I see zero problems with his comprehension. I know from experience that he will need to have a rock-solid case built up this year to make next year happen.

Is the 7-8 really compacted (reasonable workload), or just accelerated (do it all faster)? If there are writing issues, one might seem more reasonable than the other for him.

Last year's class did every single lesson, every single problem. This year's class has a different teacher, and I'm watching it closely (thankfully a friend & professional colleague has a kid in the class). So far, it appears compacted in deference to the talent of the kids in the class. This will also most certainly be part of the decision.

Note that writing volume is rather minimal when it's all on the computer. I'm more concerned about making DS hate math.

I want to make the decision together with DS, however, not to have it made for us on account of incomplete/incorrect/misleading testing.

Can you get the gifted ed. teacher leader/ supervisor on the phone and ask point blank whether this test score will matter in selection for this 7-8 class?

A STAR retest isn't that big a deal-- it's 20-30 minutes. But yes, avoiding unnecessary testing is reasonable, especially when the child finds it stressful.

They should NOT be keeping him out of any gifted service he qualifies for, just because of a disability. HK has posted on Lillie-Felton before. I can dig out more documentation if needed.

838 is a scaled score--actually a Rasch score, which means it's continuously scaled across the grade norms, and is supposed to be equal-interval, meaning the difference in difficulty level between an 838 and 638 is equivalent to the difference in difficulty between a 638 and a 438. Obviously, since academic skills don't really grow at a uniform rate, and who knows what "difficulty" really means anyway, this is approximate at best. Now this sounds fabulous as a means of assessing growth, and making scores more comparable across grade levels, but for the question of where 838 comes from. It's derived (like most standard scores and scaled scores) from a raw score, with its requisite basals and ceilings. This would work nicely, if children had nice solid basals and ceilings, with uniform success on all items below the basal, and a smooth, predictable pattern of failures just below the ceiling. Alas, children decline to fit comfortably into this idealized pattern of performance. Consequently, some students' 838 fit the ideal, and probably are better represented by the grade equivalent than others. Others have a lot of holes before the basal level, and receive an overestimate of their actual skill level. One could miss some easy items at an inopportune moment (say early in the item set), and be directed to a lower level set of items, receiving a lowered estimate of their true skills. If you have conceptual skills above your calculation skills, or skills that you figured out on your own, absent instruction, you might be able to do some problems above the level of your ceiling, but we won't know that, because you ceilinged before then. If you start from a higher basal (such as in the 10th grade), the floor of the test will effectively not be as low, which may overestimate skills.

A 10th grader starts at about the 8th or 9th grade level, so, yes, they won't be limited to 7th grade level, and can in fact top out the test, in terms of item difficulty. But the low end of the spectrum will also not be represented as accurately.

The scaled score is not meaningless as a stand-alone measure, it just comes with a collection of caveats, just as the age/grade-equivalent scores derived from it do. It's most useful as what it's designed for, which is a growth score (keeping in mind that, at the upper extreme, it is possible to run out of the pool of high-level items, if too-frequent multiple measurements are taken--even more true of MAP).

I've done some poking around at the MAP scores and tables. My biggest take away is that around half of kids leave school with at best a 7th grade understanding of English and Math.

Just to follow up, I didn't manage to talk to DD's teacher before they did the STAR. I got results for the reading portion which give DD a PR of 99 with the instructional reading level of 11.3. I doubt they started her at grade 8, so it apparently didn't have problem moving upward. No scaled score, unfortunately.

Does anyone know if teachers are able to review the actual questions and answers that resulted in a given score? DD's teacher has sent home her STAR "Instructional Planning Report" which lists as grade 3 (I'm assuming that's the baseline grade?) with a SS of 881.

I haven't the faintest what the benchmark was because this information appears to rely on color viewing, and my printout is black and white. And it rates her as "grade 5" in Operations and Algebraic Thinking, which seems about right, but then "grade 2" for Measurement and Data. The other 3 areas are all either grade 3 or 4. I'm thinking she probably screwed up or wasn't paying attention to some measurement skills because she certainly CAN "draw a picture graph or bar graph to represent set data" well above 2nd grade level, and add/subtract within 1,000 when she bothers to use a pencil on paper and pay attention to what she's doing. But realistically I have no information whatsoever on HOW she got these scores.

I have no idea how she managed to get an apparently above-grade-level-expectations score when her actual results in each category are all over the map....

It is my understanding that no, the teacher does not have that information. Since it is computerized, the computer does some "fancy" math based on which questions were answered correctly versus those answered incorrectly and comes up with the score.
The teacher can see a larger breakdown of scores than many schools send home, but it doesn't really give enough insight to know what the issue is.

I don't think all versions cap at +3 grade level because our DS's scores are 9 grade levels above in math.

Just a reminder that his reported grade equivalents do not mean that he successfully completed items at that level of difficulty, only that his performance on the items administered to him was at the same level as that of the average student of that grade level on these same types of tasks. E.g., a second grade student scores at the eleventh grade grade equivalent. This means that she obtained the same universal scaled score on the second grade test that the average eleventh-grader would have obtained on the second grade test. The test itself may have included items only up into the fifth grade range. It is not the items at different grade levels that are being compared in a grade equivalent, but the performance of students in different grade levels on the same task, which may be well below the instructional level of one of them.

p. 39 of the document cited up-thread clearly states that +3 grade level items are not included in that particular STAR Math assessment. It may be that they have other versions that have higher ceilings. Alternatively, your child may have more frequent assessments, in which case the starting point will have been adjusted based on his prior assessments, which conceivably may make more items available to him, if the whole item set shifts upward. (Although the documentation I've read suggests that it will actually make fewer items available, if the grade-level item set is not shifted, possibly lowering his scores late in the school year.)

Thank you, aeh. I think I understand it better now. I find it confusing, especially since our district does not disseminate any information on what it all means (but does use an average of the STAR scores throughout the year as part of the GT identification process).