Print
Email
by Guest Blogger Dudley Reynolds
Let me begin by making it clear that I am not and never have been:
• a taxpayer of the U.S. state of Arizona,
• a parent of an Arizona school child,
• an employee of a publically-funded university in Arizona, nor
• a researcher of English language instructional program effectiveness for
school-age immigrants.
I live roughly ten time zones away from Arizona in the Persian Gulf country of Qatar, where I research the academic writing development of university-age, second-language students. Nevertheless, I am quite concerned about events surrounding a civil court case currently unfolding in Arizona known as Horne v. Flores.
In June 2009 the U.S. Supreme Court remanded the case to the Arizona Court of Appeals for reconsideration of an earlier decision that had said the state was violating the Equal Educational Opportunities Act of 1974 with respect to opportunities for English language learners. In court the arguments surround whether Arizona’s latest system for instructing English language learners—to segregate them out for four hours of skills-based drills per day—has brought the state into compliance with federal requirements.
What concerns me, however, are the out-of-court arguments. Because the issue is, essentially, program effectiveness, both sides have sought out, commissioned, and examined very carefully the research on English language learners in Arizona schools. That’s good; but unfortunately, things seem to have progressed beyond scholarly debate on research methodologies and interpretations of findings.
At issue in the out-of-court arguments is a series of 9 research studies commissioned by the Civil Rights Project at the University of California Los Angeles (the 2009 winner of TESOL’s Presidents Award). As acknowledged in an August 13, 2010 statement from the Civil Rights Project Directors, “the findings of these studies are extremely critical of the policies and practices currently being required in Arizona.” Not only were they critical, but researchers from the University of Arizona and Arizona State University who were responsible for 2 of the studies were planning to testify about their findings in the case.
Basing their request on the right under U.S. law for both sides in a court case to review beforehand the evidence that the other side will introduce during the case, the state of Arizona demanded that the courts require the researchers to release the names of the school districts, administrators, and teachers who had supplied data for the studies in question. According to a copy of the court order available through Education Week’s Learning the Language Blog, which has written extensively on this case, the judge ruled that the researchers would only have to turn over the names of the districts and anonymous audio files of English language coordinators who had been interviewed.
According to a story published on Inside Higher Ed, the University of Arizona complied with the order, surrendered the district names, and was working on the CDs with the audio files. The researchers at Arizona State University refused to comply and instead took the option given to them by the state’s lawyers to withdraw as expert witnesses for the case.
All sides in the out-of-court arguments are spinning the actions of the other. Supporters of the state of Arizona’s position as evidenced in comments to the Inside Higher Ed article are arguing that researchers who are not willing to expose their data to public review must be hiding something. The Civil Rights Project Directors in their statement describe the state’s approach as “a blatant attempt to get this research out of the trial by making the researchers choose between going to court and putting the districts and schools at risk (we are told the AZ Dept of Education is doing a witch hunt for anyone not 'towing the line' in the state, and cutting budgets of those who don't), or backing down and not testifying.”
I have to say that I agree with the Civil Rights Project Directors’ interpretation. As a graduate student, teacher educator, and researcher I have read, critiqued, and even dismissed as invalid a large number of research studies without ever having had direct access to the original data. In graduate school sociolinguistics, I learned about Grice’s Maxim of Quality, which essentially says that in order to communicate as a species we have to trust that the people we are talking with are making a good faith effort to be truthful and accurate. We have to take published, peer-reviewed research as the authors’ honest interpretation of their data. We can look for evidence of questions they didn’t consider about their data (do the subjects represent a good cross-section of geographic areas?); we can look for mistakes in judgment (does the study report statistical significance?); we can even look for evidence of a political agenda outweighing a reasoned interpretation (why was this explanation not considered?). If, however, we begin by assuming that published, peer-reviewed research cannot be trusted and therefore we have to look at the data ourselves, we are doing more than going down a slippery slope—we’re sailing head first into the abyss of basing program design and implementation on nothing more than gut feelings and attitudes.
Institutional Review Board protocols always include a caveat that says confidentiality will be preserved “to the extent allowable by law.” By even suggesting that this caveat gives them a right to the names of individuals who provided data under the presumption of confidentiality, the lawyers for the state of Arizona have seriously undermined the ability of researchers everywhere to conduct interviews, ask for survey responses, and observe classrooms. Arizona has sown seeds of doubt and mistrust in the researcher-subject relationship that I depend on here in Qatar.
All academic research should be open to debate. Let us remember, however, —and emphasize wherever we can—that it is the methodology and interpretation that we should be questioning, not the names of the subjects.
Dudley Reynolds is a Teaching Professor of English and Director of Research in English Language Learning Carnegie Mellon University in Qatar
