NECAP vs NAEP

I have submitted this to the ProJo, but thought I would give you an advanced peek.  I will just say that over the 2-ish years I have been doing this it seems that the more you dig the more you find.

Welcome to Lake Wobegon, RI

After reading the recently released public school achievement test results from the New England Common Assessment Program (NECAP) and then comparing them to the well-respected Nation’s Report Card- also known as the National Assessment of Educational Progress (NAEP), I couldn’t help but think I was in Lake Wobegon because it looks like all of our students are above average. 

Every two years the U.S. Department of Education uses the NAEP test to determine how many students are “proficient” in a given discipline, which is defined at “at or above grade level.” But the Rhode Island Department of Education, as does every State, develops their own test and defines their own grade level expectations. So when a State declares it has achieved an 80% proficiency rate, it is quite likely that the Federal Government would find a proficiency rate far lower.

It is important to remember that the federal “proficiency” defines our progress towards the goals of No Child Left Behind.  

In research published by the Ocean State Policy Research Institute last year, we documented a mapping formula that allows us to estimate NAEP scores for individual schools when only state test scores are available. This allows us to estimate NAEP scores during non-testing years and to compare RI schools to those in other states.

In 2005 the NAEP test found 29% of RI student to be “proficient” in reading but the same year’s NECAP scores labeled 55% “proficient.” When tested again in 2007, the NAEP results went down to 27% while the NECAP rose to 61%. The latest NECAP results continue the rise to 65% but our estimates place the NAEP equivalent down to 26%.

 

As you can see by the graph, not only is there substantial disparity between federal and state “proficiencies” but the trends are going in the opposite direction. Its like one is using a yardstick and the other a meter-stick, and one of them is upside down.

Looking at the school level is where we see the impact on shareholders.

Parents of Barrington 8^th grade students who are proud to hear NECAP claim proficiency exceeding 90% become less enamored when the federal guidelines place nearly one third of those students below grade level.

And parents in Providence can’t be happy with the NECAP results showing 72% of their 8^th grade children below grade level in both math and reading, but they would be justifiably outraged if they knew that NAEP estimates put that number at over 90%.

Our research has identified two problems: the disparity between state and federal “proficiencies” and the trend for 8^th grade reading going up according to the NECAP test but doing down according to the NAEP test. We have also reviewed other states and found them all to show inflated scores but only 12 other states have been found with the divergent trends.

The Rhode Island Department of Education openly admits that NECAP and NAEP “proficiencies” are two different measurements. The NECAP “proficiency” more closely aligns with NAEP “basic.” But we believe they should be the same and agree with what Senator Ted Kennedy wrote in the New National Defense Education Act, S. 3502, “Student performance varies greatly between some State-level assessments and NAEP assessments. (sic) [I]n today’s global economy, students must prepare to compete with students from other states and other nations.” 

As to the problem of rising NECAP reading proficiencies and lowering NAEP scores, our research indicates the problem is most likely due to a state curriculum with a narrower content range than that of the NAEP. This theory is further supported by the fact that we find the same phenomenon in New Hampshire, which also uses the NECAP.

Imagine that NECAP teaches from a 100 page book while NAEP teaches from a 500 page book. Since the same number of sample questions are released for both the NAEP and NECAP, the questions for the NECAP represent a substantially larger portion of the total curriculum to be tested. So, over time, teaching to the test (which normally isn’t a bad thing) becomes progressively more effective.  

But that is just a hypothesis. Further study is needed. Consideration should also be given to conducting the assessment function through an independent agency to remove concerns about conflicts of interest. When we published this research in Oklahoma, Senator Clark Jolley introduced legislation to remove testing from the auspices of the Department of Education.

Obviously, Lake Wobegone is a fictitious place if only because it is statistically impossible that “all the children are above average.” Rhode Island, and all states, should measure student performance with the same yardstick so parents know how their children stack up against those they will be competing with, because they won’t just be competing in Minnesota. 

Making tests count

Back in March, we published a request for inputfrom Angus Davis, member of the RI Board of Regents.  The jist of the issue is this:  We graduate 96% of our students, yet only 22% can score a 62.5% on the math test.  This is because the test actually counts for “less than 10%” of the gradation requirements.  The Board of Regents asked for input and the response you provided was magnificent.

Therefore, it should come as no surprise that a group of education professionals has opposed this change.  They want test scores to count for less, not more.  Creating less accountability.

Pasted below is another request from Mr. Davis.  I would also suggest that you could write a letter to the Providence Journal at letters@projo.com.

RI Regents Consider Graduation Requirements and NECAP Test

In recent weeks, a group opposed to testing students has sent several emails asking the Regents “reconsider” our plan to make the NECAP count towards graduation in Rhode Island.

The NECAP is a test of 10th grade skills given in 11th grade, with multiple subsequent opportunities to re-take the test through 12th grade. Our proposed high school graduation requirements would make NECAP just one of three elements in deciding a student’s eligibility for a Regents Diploma, along with grades and a final senior project or portfolio.

This week, an Op-Ed from the “National Center for Fair and Open Testing” (“FairTest”) appeared in the ProJo to oppose the Regents actions. FairTest is opposed to tests, and in the last few years has become, in the words of education policy analyst Education Sector, “an anti-NCLB propaganda machine more than a testing group.”

Opponents decry tests like the MCAS in Massachusetts, claiming if we raise our expectations of students, children will drop out and teachers will be forced to focus on “irrelevant” skills that lack meaning in today’s 21st century workforce. As a reminder, “irrelevant” skills we test for on NECAP are reading, writing, and basic mathematics.

These arguments do not hold water. First, despite the MCAS in Massachusetts, high school graduation rates there have actually gone UP in recent years, not down. Massachusetts outperforms Rhode Island on almost every measure, including the speed at which they are closing the achievement gap. Second, those asking us to lower our bar are mostly education establishment people, rather than parents. I am still waiting for the email that says, “Dear Regent Davis, I would really prefer that my child not learn how to read, so please don’t make that a requirement for her graduation.” (Understandably, some parents of children with special needs are concerned, and our plan calls for exceptions in those cases). Last, the proposal in Rhode Island would make NECAP count, but it would not yet be “all-or-nothing” or “high stakes” as is the case with MCAS; a student could still graduate in Rhode Island even if they fail the NECAP.

Most parents are demanding a more rigorous school for their children, and rightly so, given the increasing competition they face for jobs in the global economy. If you support the Regents effort to make NECAP one of three components of the graduation requirements, alongside grades and a senior project or portfolio, I encourage you to come to tomorrow’s Regents meeting (May 7, 4pm, Shephard Building, 255 Westminster Street, Providence) and give public comment to that effect. You can leave the meeting after making your comments, which normally conclude by 4:30. Your voice would help to counterbalance those who ask us to lower the bar, expect less, and make excuses.

May 7 Board of Regents Meeting Agenda and Details

Let the Regents know what you think!

Repeating an earlier message – Have you ever wondered how we graduate so many students when only 22% of them can pass math?  It’s because the test really means very little to the graduation requirements and the Regents are trying to fix that.

It important that you let the Regents know what you think.  They want  to increase the testing standards, or at least what weight they give those test when determining graduation.  In my opinion, this is an excellent measure.  PLEASE let them know what you think -

Click hereto send an email to all members of the Regents (on the “To:” line) and include members of the ProJo & WJAR staff covering education on the “CC:” line (feel free to add or remove CC’d folks as you see fit!)