Chariho School Parents’ Forum

February 9, 2009

NECAP vs NAEP

Filed under: NECAP — Editor @ 11:54 pm

I have submitted this to the ProJo, but thought I would give you an advanced peek.  I will just say that over the 2-ish years I have been doing this it seems that the more you dig the more you find.

Welcome to Lake Wobegon, RI

After reading the recently released public school achievement test results from the New England Common Assessment Program (NECAP) and then comparing them to the well-respected Nation’s Report Card- also known as the National Assessment of Educational Progress (NAEP), I couldn’t help but think I was in Lake Wobegon because it looks like all of our students are above average. 

Every two years the U.S. Department of Education uses the NAEP test to determine how many students are “proficient” in a given discipline, which is defined at “at or above grade level.” But the Rhode Island Department of Education, as does every State, develops their own test and defines their own grade level expectations. So when a State declares it has achieved an 80% proficiency rate, it is quite likely that the Federal Government would find a proficiency rate far lower.

It is important to remember that the federal “proficiency” defines our progress towards the goals of No Child Left Behind.  

In research published by the Ocean State Policy Research Institute last year, we documented a mapping formula that allows us to estimate NAEP scores for individual schools when only state test scores are available. This allows us to estimate NAEP scores during non-testing years and to compare RI schools to those in other states.

In 2005 the NAEP test found 29% of RI student to be “proficient” in reading but the same year’s NECAP scores labeled 55% “proficient.” When tested again in 2007, the NAEP results went down to 27% while the NECAP rose to 61%. The latest NECAP results continue the rise to 65% but our estimates place the NAEP equivalent down to 26%.

 naep-graph

As you can see by the graph, not only is there substantial disparity between federal and state “proficiencies” but the trends are going in the opposite direction. Its like one is using a yardstick and the other a meter-stick, and one of them is upside down.

Looking at the school level is where we see the impact on shareholders.

Parents of Barrington 8th grade students who are proud to hear NECAP claim proficiency exceeding 90% become less enamored when the federal guidelines place nearly one third of those students below grade level.

And parents in Providence can’t be happy with the NECAP results showing 72% of their 8th grade children below grade level in both math and reading, but they would be justifiably outraged if they knew that NAEP estimates put that number at over 90%.

Our research has identified two problems: the disparity between state and federal “proficiencies” and the trend for 8th grade reading going up according to the NECAP test but doing down according to the NAEP test. We have also reviewed other states and found them all to show inflated scores but only 12 other states have been found with the divergent trends.

The Rhode Island Department of Education openly admits that NECAP and NAEP “proficiencies” are two different measurements. The NECAP “proficiency” more closely aligns with NAEP “basic.” But we believe they should be the same and agree with what Senator Ted Kennedy wrote in the New National Defense Education Act, S. 3502, “Student performance varies greatly between some State-level assessments and NAEP assessments. (sic) [I]n today’s global economy, students must prepare to compete with students from other states and other nations.” 

As to the problem of rising NECAP reading proficiencies and lowering NAEP scores, our research indicates the problem is most likely due to a state curriculum with a narrower content range than that of the NAEP. This theory is further supported by the fact that we find the same phenomenon in New Hampshire, which also uses the NECAP.

Imagine that NECAP teaches from a 100 page book while NAEP teaches from a 500 page book. Since the same number of sample questions are released for both the NAEP and NECAP, the questions for the NECAP represent a substantially larger portion of the total curriculum to be tested. So, over time, teaching to the test (which normally isn’t a bad thing) becomes progressively more effective.  

But that is just a hypothesis. Further study is needed. Consideration should also be given to conducting the assessment function through an independent agency to remove concerns about conflicts of interest. When we published this research in Oklahoma, Senator Clark Jolley introduced legislation to remove testing from the auspices of the Department of Education.

Obviously, Lake Wobegone is a fictitious place if only because it is statistically impossible that “all the children are above average.” Rhode Island, and all states, should measure student performance with the same yardstick so parents know how their children stack up against those they will be competing with, because they won’t just be competing in Minnesota. 

Advertisements

16 Comments »

  1. The state of our educational system can be very frustrating. The reality that so few seem to care is extremely discouraging. Excellent research which will likely be ignored and/or excused.

    Comment by Curious Resident — February 10, 2009 @ 12:28 am | Reply

  2. I should have specified “government educational system”. Private schools seem to get it right.

    Comment by Curious Resident — February 10, 2009 @ 12:29 am | Reply

  3. Since most (all?) private schools don’t give these tests, how can you know?

    Comment by david — February 10, 2009 @ 8:06 am | Reply

  4. One recent study was conducted by the National Center for Education Statistics. In a linear comparison, private school students were found to score significantly higher in the tested subjects of math and English at 4th and 8th grade levels.

    When “adjusted” for student variability, NCES claims little difference between private and government schools. NCES is a government agency with a vested interest in government schools so their results would certainly not be biased in the favor of private schools. The National Catholic Educational Association questions the “adjustments” and even the NCES admits the adjusted results our of minimal value.

    Based on the NCES, the one thing neither side argues about is that private school students test significantly higher than government school students. The debate centers around the reasons why private school students do better. I think this is legitimate, but doesn’t negate the overall consensus of private school students performing at a higher level than government school students.

    The other factor is cost. I don’t have the time now, but I wouldn’t be surprised to discover the average private school student is edcuated at a much lower cost than the average government school student.

    Comment by Curious Resident — February 10, 2009 @ 9:23 am | Reply

  5. The comparison is a very subjective matter based on who is doing the comparison. This is one subject that either side could fins plenty of data to support their argument. The difference is the private sector teachers do not put a stranglehold on the taxpayers through the unions. They can and are fired for failure to perform. In other words, there is accountability.

    Comment by RS — February 10, 2009 @ 11:22 am | Reply

  6. Disagreement based on the agenda of the research organization seems to be the prevailing outcome when looking for school comparisons. Why not let “buyer’s” decide? School choice solves the problem as those who prefer government schools will continue to have the option, and those favoring private schools (turned public schools) will have financial obstacles removed and/or lowered. Everybody wins except government unions and their politician allies…once again, it’s all about the adults with little real concern about children. For the children don’t you know.

    Comment by Curious Resident — February 10, 2009 @ 11:29 am | Reply

  7. “Full price” private schools (Wheeler, Rocky Hill, PCD) have tuition of over $20K per student, plus have fundraising too.

    Private schools that focus on special needs (Landmark, St. Andrews) are over $30K.

    Comment by david — February 10, 2009 @ 2:54 pm | Reply

  8. …..and now we are waiting for their ranking, David.

    Lets see if those folks who can afford the tuition get their money’s worth.

    Comment by RS — February 10, 2009 @ 3:21 pm | Reply

  9. What “ranking” do you mean? As I said up in post #3, private schools don’t use the statewide test or other standardized tests.

    Comment by david — February 10, 2009 @ 4:20 pm | Reply

  10. Some states do, RI doesn’t, but private schools do use the SAT, and many have entry exams as well.

    I still have that lingering question in my mind, if the public school system is a gem, then why did the POTUS put his children in private school?

    Comment by RS — February 10, 2009 @ 4:45 pm | Reply

  11. And if the government schools are a “gem”, why all the fuss about choice? Parents will choose the better government schools. We really don’t need rankings when parents choose. The results of choice take care of rankings for us.

    David can do his own research on costs, but despite anecdotal evidence, I’m pretty confident that overall private schools cost less on average than government schools.

    Since Chariho avoids sharing our special needs costs with those of us paying the bills, we really aren’t sure how much it is costing us, but I suspect we’d be getting a deal if we moved all our special needs students from RYSE to St. Andrews. I suspect the children would be much better off too.

    Comment by Curious Resident — February 10, 2009 @ 6:25 pm | Reply

  12. I’m not sure what’s being responded to here. Certainly I never said that public schools are “gems” (some probably are, but I don’t have the experience base to make such a claim). My only agenda here is to ask about conclusions which are drawn on what appears (to me) to be faulty data.

    I have no problem with people preferring private schooling; heck as a private school parent for 14 years how can I? My personal opinion is that academic success was not an important consideration in that decision; I’m quite confident that my children could have had similarly good preparation for college at public school.

    I also think public school choice is a great idea. I have concerns about vouchers, because I think the money will eventually come with strings attached which will compromise the private schools’ independence, and because I think it will inflate private school costs and therefore fail to provide broadened opportunity for working class people to attend private school.

    I also don’t really agree that private schools *in general* are less expensive on a per student basis. Parochial schools, because they are not independent entites, do not release their finances and have entanglements with the rest of the church which makes their costs murky. Private independent schools run the gamut from relatively cheap (as low as $7K per year) to really expensive (> $25K/year in Rhode Island, more in major cities).

    Comment by david — February 10, 2009 @ 10:40 pm | Reply

  13. If there is a demand for “independent” private schools, they will remain. Not complicated. Free markets work that way. I’ve never heard of any proposal which requires private schools to become public schools. Personally, I’d prefer no strings, but if strings are required to get school choice started, then I can accept it as better than what we have now forcing children from less affluent families into failing government schools.

    As RS notes, it is very common for wealthy politicians to put their own children into private schools while fighting vehemently against giving all families the same possibility. Again, if private schools don’t educate any better than government schools then government schools will do fine. Since school choice can easily be implemented with reimbursement caps, if private schools price themselves out of the market they won’t survive.

    The arguments against choice are specious. When looked at logically, they don’t hold water.

    Comment by Curious Resident — February 10, 2009 @ 11:37 pm | Reply

  14. At least our RI schools didn’t make this list…..sad part is I think there were 11 Chicago schools. Isn’t our new education czar from there? Arne.

    http://www.neighborhoodscout.com/neighborhoods/school-district/ratings/worst100/

    Comment by RS — March 10, 2009 @ 10:44 am | Reply

  15. Now the ACT’s have posted their results…..How can we continually think our RI children are ready to compete in the world. Maybe we should rethink the truck stop at exit 1….at least tere will be a job for those less inclined to go to college.

    http://www.act.org/news/data/09/collegeready.html

    Comment by RS — August 19, 2009 @ 10:07 am | Reply

  16. the public schools on our district can really give some good education to young kids. they have high standards ,”;

    Comment by Letter Tray — December 3, 2010 @ 11:05 am | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: