“Uh, sir, phrenology was dismissed as quackery a hundred-sixty years ago.” – Mr. Smithers
“Of course you’d say that; you have the brainpan of stagecoach-tilter.” – Mr. Burns
To get to the part where I talk about phrenology, skip the first 1,600 words or so.
On the off chance someone over 30 stumbled upon this, “I Fucking Love Science” (IFLS) is popular science media page on Facebook with over 19 million “likes”. Its content tends to be composed by more amateur writers than say, Scientific American, but I mean that less as an insult and more of an “I realize they aren’t necessarily professional science writers and maybe should get some slack”. I’m going to go ahead and bitch about one of their articles, but I don’t really want mean for IFLS to catch the bulk of my ire; I intend for that to go to others. But this article caught my eye to the extent that I wanted to bitch:
I’ll try and resist my urge to go fisking, at least on the IFLS article, but I’ll go over my general complaints. I can appreciate the difficulty of sharing science news. It’s time consuming enough to relay press-releases and explain the news accurately and coherently. And there’s nothing wrong with just doing that. Real analysis is insanely time consuming, and if there’s no market for it, then I totally get not wanting to burn the time on it. So again, I my point isn’t to give IFLS crap (although they deserve a bit), my point is that the way science is disseminated these days can be terrifying. There’s very little oversight and critique. I’ll use this as my case in point.
So what’s the big deal? Well, for starters, three of the links to studies or sources were newspaper outlets (WaPo and the NYTimes). In case you didn’t know, newspapers are not in the scientific research industry. Very few scientific experiments are run by journalists. Not really anything wrong with that, seeing as it’s not really their line of work. But the danger is that they are being used as an appeal to authority, and this authority is science. Now, don’t get me wrong, science is a good authority. The scientific method is a pretty solid way of determining things about our universe. But it has limitations in application. The certainty with which we can describe the effect of mixing two chemicals together doesn’t necessarily match the certainty with which we can describe how children perform standardized tests, but when you appeal to science, the public tends to think chemistry and physics-type precision regardless of the validity of that comparison. In this case, we are taking the very nebulous concepts of academic performance and parental income and insinuating that it’s akin to rolling a ball down a slope. And it’s not just IFLS, a lot of science publishing is set up with this in mind.
But the bigger deal is the overall implication: poverty literally changes brain physiology, and this makes kids suck on tests. Science say’s it’s just as much a fact as measuring the pH of a liquid. Then there’s the unstated call to effect political policy. Oh wait, no, it’s the closing paragraph. To reiterate, this is not IFLS’ idea. They’re essentially just relaying an MIT press release. IFLS is culpable for sharing it, sure, but I shouldn’t be too much of a dick to people for sharing science press releases, as much as I may question the content. But MIT is fair game.
You know what’s worse than a science media blog having inadequate references or citations? An extremely well endowed university having no linked citations. To be fair to IFLS, their author did actually link a couple of studies, and you could find more studies in the newspaper links provided. MIT provided no such convenience. “Well, honestly, nobody’s going to read the links anyway. Okay, you are, but that’s because you’re a neurotic asshole who just wants to find a reason to complain.” And that’s where the danger lies. Look, I’m not completely delusional; I get that it’s absurd to try and convince adults to read scientific papers before buying into something they hear about science. But kids still might be receptive to the message, so it’s worth a try…. And I also appreciate that time is valuable. And I get having trouble understanding abstruse, technical papers. I understand the division of labor: someone who’s good at science and communication should be reading the papers and giving a fair report. Well, here’s the thing, the number of people who do that is INSANELY SMALL. This is what I want to warn you all about. There’s very, very little vetting in science publishing today. “What about peer-review? Aren’t articles analyzed by other scientists before publishing?” Scientists like to sell the idea of peer-review as this notion of rigorous academic debate before a wide audience of learned men. Spend 10 minutes with a scientist who has dealt with the peer-review process and ask them what they think of it. Hint: it’s not that. An angry drunk does a good job with a quick criticism. See also.
Here’s the thing: scientists know you’re not going to read the sources. The reality of citing things is that it has very little to do with actually substantiating facts and much, much more to do with appealing to authority. Scientists rarely check sources in submitted papers, and they’re in the industry. I can’t imagine what it’s like for the public at large. Each source exists to say “this is an incontrovertible fact” and cut down on your skepticism. But if you’re a sad and lonely man like me, you’ll find that a surprising amount of cited material doesn’t actually support assertions made. Since the MIT article has the audacity to just forgo citing many of their claims, I’ll go back to the IFLS article, which apparently has more journalistic integrity.
“The children were also given the Massachusetts Comprehensive Assessment System (MCAS). Unsurprisingly, the lower income children performed worse on average, with 57% achieving proficiency compared to 91% of the wealthier students.”
This story has nothing to do with the MCAS; it’s about a study coordinated out of New York and California. The MIT study is what uses MCAS scores. But neither the MIT press release nor the referred article abstract (which IFLS cites, to their credit) gives any number on performance, so the numbers the author gives are dubious. The implication is that this is the discrepancy in score of the students that were involved in the MIT study, but there’s no evidence of that. So maybe it’s from another source. If so, where is that? The numbers may be entirely unsupported, but you see them nonetheless, and they have a conspicuous blue hyperlink that psychologically reinforces their veracity. So the reader trusts their significance regardless of their validity or nuanced meaning.
“Previous studies have linked environmental conditions to changes in the brain, and many more have associated the same conditions with lower academic outcomes, but this is the first to put the three together.”
If you click that link, and click the link in /that/ story, it’ll take you to… the WaPo article linked in the previous paragraph…. Now, to WaPo’s credit, they link to an actual study, which is worth a perusal if you’re interested in the content of these articles.
Both this (and the IFLS article later) link to another actual study, also worth looking at.
IFLS’ penultimate link is to the abstract of the study.
I’d prefer that it’dve been placed earlier in the article, considering that’s the actual study that prompted the discussion, but I’ll let that slide as a minor stylistic complaint. Again MIT has no excuse at all for not linking to it.
Which brings me to my next whiny tangent, but is, I think, a big deal: THE ONLY EVIDENCE WE HAVE FOR ANY OF THIS IS BEHIND A PAYWALL. Why is anyone okay with this? “Do you honestly expect the public to take a huge interest in obscure science topics?” Yes, but I’ll concede that’s untenable. What I refuse to tolerate is that scientists are okay with this. How are you okay with your life’s work being locked away from the public eye? What if Shakespeares’ plays could only have been seen by other playwrights for several years before going public? It’s lunacy. Every goddamn scientist on Earth thinks that the public doesn’t know enough about science, particularly their discipline. Well how the Hell do you expect them too when you charge for a study they can barely understand? Look, I can appreciate that editing, reviewing, and distributing, etc. takes money, but for the good of our civilization, we have got to figure out a different way to get that, because this is absolutely absurd.
A major take away from the Enlightenment is that ideas need to be debated. That’s what we keep telling the children makes science so great: if an idea can’t stand up to scrutiny, it’s discarded, and we keep only those that pass muster. And yet modern science does not lend itself to debate. We are told The Truth by scientists. We take their word for it. The media repeats it and it becomes established in the public as fact. This is bad. Super bad. Did I make that clear enough?
So anyway, back to my titular complaint: this article is selling phrenology as solid science and using it to justify political policies. And holy shit does that scare me. Let’s scroll through the MIT article. But first, let me just restate the ENTIRETY of the new science we have available at our disposal that this article summarizes:
“Cortical gray-matter volume was significantly greater in students from higher-income backgrounds (n = 35) than in students from lower-income backgrounds (n = 23), but cortical white-matter volume and total cortical surface area did not differ significantly between groups. Cortical thickness in all lobes of the brain was greater in students from higher-income than lower-income backgrounds. Greater cortical thickness, particularly in temporal and occipital lobes, was associated with better test performance.”
That’s it. So even assuming that there are no flaws in the data collection, methodology, and analysis, this is all the information we have. Grey matter volume is basically a shorthand for the number of neuronal cell bodies in the brain. The general hunch is that having more cell bodies is good. Can’t emphasize the “hunch” part enough. Cortical surface means the surface area of the cerebral cortex, i.e. the big part of the brain that you think of when you think of a brain. Again, the general vibe is that more surface area= more good, but I mean, that’s why we think we’re smarter than chimps. Human to human (well, modern man to modern man) we have no idea. The precision, let alone the how and why of the value of cortical surface is pretty much unknown. The cortex is divided into 4 lobes, the frontal lobe on the front, the occipital lobe on the back, the temporal lobe on the sides, and the parietal lobe on the top. Each one does a ton of stuff. Often times the lobes work together.
So to summarize, for a handful of people, the brain is a bit thicker in kids whose parents have more money compared to kids in poorer families, and there’s an association between test scores and brain thickness two very broad areas of the brain. If you want to know what the difference is and how strong the association is, well you can fuck right off.
http://en.wikipedia.org/wiki/Cerebral_cortex
So with the extent of what we can know about this new paper, let’s turn to the press-release. I hope you give it a read first so these make sense out of context:
“Many years of research have shown that for students from lower-income families, standardized test scores and other measures of academic success tend to lag behind those of wealthier students.” No fucking citations. This is not sharing evidence, this is a divine dictation from Mount Sinai.
“‘To me, it’s a call to action. You want to boost the opportunities for those for whom it doesn’t come easily in their environment.’” There are two ways I think you can interpret this: this is either a pitch for more grant money and not a genuine plea, or this is a genuine plea and thus the entire reason the study was even done. I’d like to actually get some sleep tonight, so I’m going with the former, but if you’re up for it, consider the latter.
“This study did not explore possible reasons for these differences in brain anatomy.” Because the study is not actually /about/ finding anatomical reasons for academic success/failure. It exists to justify public policy debates as scientifically sound when it is supported by a tiny, tenuous iota of actual science.
“In recent years, the achievement gap in the United States between high- and low-income students has widened, even as gaps along lines of race and ethnicity have narrowed,” Again, no citation, although this may indeed be a reference to the ASCD article posted above.
“Other authors are… a postdoc at Columbia Business School, and Christopher Gabrieli, chair of the nonprofit Transforming Education.” The business major “guided the analyses of race and ethnicity”, according to the paper, which obviously didn’t show up in the abstract. This isn’t to trash business majors, but I have to raise an eyebrow when I see a non-scientist doing science research. Not necessarily a problem in this case, I don’t think. But that Gabrieli is part of an education non-profit to be looks like a gigantic conflict of interest. If this were an article about how a new drug made by Ely Lily upped test scores for low-income students and he was on Lily’s board of directors, there would be a gigantic hissy-fit as people scream “conflict of interest!” And I’m not insinuating this guy will see a dime out of the arrangement; I’m totally willing to assume that everything is above board and without a modicum of malicious intent. But this should still scream “Bias!” He’s looking to fix education. A laudable goal, to be sure, but his science could be tainted. He’s doing science with the intent to fix education. You think he’s using this study to tailor his education ideas? How? There’s nothing in the study to act on! Is he going to suggest some stem cell treatment or growth hormone to up cortical grey matter density in disadvantaged students? Of course not. But if he were just going by what the science told him, that’s what he’d be advocating for. And if they found a correlation on surface area? Or white matter? Or that the only association was in the frontal lobe? He would be doing nothing differently. Because the intent was not to find a science-based solution; the intent was to make a study to get clout to try a policy or sociology experiment. THAT’S. NOT. SCIENCE. It’s phrenology. I grant that noticing that a couple black guys have a slight indentation on a part of the skull and using that as a justification for slavery is shittier than noticing some slightly thinner grey matter in poor kids and using it as a pretext for lunch vouchers or w.e., but from a science standpoint they’re still pretty much on par.
“The researchers compared students’ scores on the Massachusetts Comprehensive Assessment System (MCAS) with brain scans of a region known as the cortex, which is key to functions such as thought, language, sensory perception, and motor command.” The cortex, like I mentioned above, makes up the vast majority of the brain. Saying it’s “key to functions such as thought, language, sensory perception, and motor command” is a gigantic fucking understatement. To the point where that makes a strong case for it being a rhetorical point, and not a scientific one. It’s using jargon to sound smart. The jargon isn’t to communicate better, it’s to show people that “trust me, I’m super smart”. The statement was completely fucking meaningless to anyone with a background on the brain. But if the brain is basically a mystery to you, like it is to the general public, then you’ll think “oh shit, science says poor people don’t have enough of the part of the brain that helps you do well in school.” The reality is, assuming the study is in fact correct, basically that in a pretty small number of kids studied, poor people have a bit less brain mass on average.” Not exactly groundbreaking, or terribly meaningful. They hammer the point in on the next sentence too.
“In fact, differences in cortical thickness in these brain regions could explain as much as 44 percent of the income achievement gap found in this study.” No, you can’t see their reasoning behind this. Just trust them. Nevermind the fact that that sounds completely absurd and doesn’t seem to be justifiable based on anything I’ve heard, and I have a neuroscience degree. I totally get there is a shit-ton I don’t know, but I’m completely skeptical of this and I’m still a neuroscientist. Why the Hell should someone who hasn’t studied neuroscience at all just have to accept this?
“‘A number of labs have reported differences in children’s brain structures as a function of family income, but this is the first to relate that to variation in academic achievement,’ says Kimberly Noble, an assistant professor of pediatrics at Columbia University who was not part of the research team.” She was actually the lead author of the Nature article linked above. Hers was a much larger and probably much more valuable study, even though its conclusions still yielded very little practical information. Her study mostly found that cortical surface area played the biggest role. Even still, the associations found in her study were weak—although there. While she did not directly link her work to academics, she looked at cognitive tests like vocab and reading skills, which is pretty damn similar to an English test to the point where this is kind of weaselly.
“…the researchers found no significant difference [in] overall surface area of the brain cortex.” Which contradicts Noble’s results, interestingly. Curious to see what she’d think of that. It’s a shame MIT couldn’t grab a sound bite from her….
“In a follow-up study, the researchers hope to learn more about what types of educational programs might help to close the achievement gap, and if possible, investigate whether these interventions also influence brain anatomy.” Like I said, the policy is driving the science; the science is not driving the policy.
I’ve seen underprivileged kids get shafted in the education system. It’s fucking heartbreaking, and I completely get wanting to try and make some changes. But damaging the credibility of science is not something we should be doing in the process.