Tag Archives: Default brain network

Functional MRI research is a scientific sewer

First a primer about the science underlying functional Magnetic Resonance Imaging (fMRI). Chemists use MRI all the time, but they call it Nuclear Magnetic Resonance. Docs and researchers quickly changed the name to MRI because no one would put their head in something with Nuclear in the name.

There are now noninvasive methods to study brain activity in man. The most prominent one is called BOLD (Blood Oxygen Level Dependent), and is based on the fact that blood flow increases way past what is needed with increased brain activity. This was actually noted by Wilder Penfield operating on the brain for epilepsy in the 1930s. When a patient had a seizure on the operating table (they could keep things under control by partially paralyzing the patient with curare) the veins in the area producing the seizure turned red. Recall that oxygenated blood is red while the deoxygenated blood in veins is darker and somewhat blue. This implied that more blood was getting to the convulsing area than it could use.

BOLD depends on slight differences in the way oxygenated hemoglobin and deoxygenated hemoglobin interact with the magnetic field used in magnetic resonance imaging (MRI). The technique has had a rather checkered history, because very small differences must be measured, and there is lots of manipulation of the raw data (never seen in papers) to be done. 10 years ago functional magnetic imaging (fMRI) was called pseudocolor phrenology.

Some sort of task or sensory stimulus is given and the parts of the brain showing increased hemoglobin + oxygen are mapped out. As a neurologist as far back as the 90s, I was naturally interested in this work. Very quickly, I smelled a rat. The authors of all the papers always seemed to confirm their initial hunch about which areas of the brain were involved in whatever they were studying. Science just isn’t like that. Look at any issue of Nature or Science and see how many results were unexpected. Results were largely unreproducible. It got so bad that an article in Science 2 August ’02 p. 749 stated that neuroimaging (e.g. functional MRI) has a reputation for producing “pretty pictures” but not replicable data. It has been characterized as pseudocolor phrenology (or words to that effect). Keep reading you’re about to find out just why this was.

What was going on? The data was never actually shown, just the authors’ manipulation of it. Acquiring the data is quite tricky — the slightest head movement alters the MRI pattern. Also the difference in NMR signal between hemoglobin without oxygen and hemoglobin with oxygen is small (only 1 – 2%). Since the technique involves subtracting two data sets for the same brain region, this doubles the error.

Under two years ago, it was shown that 70% of people having functional MRIs (fMRIs) were asleep during the test, and that until then fMRI researchers hadn’t checked for it. For details please see
https://luysii.wordpress.com/2014/05/18/how-badly-are-thy-researchers-o-default-mode-network/. You don’t have to go to med school, to know that the brain functions quite differently in wake and sleep.

Recent work shows that functional MRI work is even worse. A devastating report in [ Proc. Natl. Acad. Sci. vol. 113 pp. 7699 – 7600, 7900 – 7905 ’16 ] showed that certain common settings in 3 software pacakages (SPM, FSL, AFNI) used to analyze fMRI data gave false positive results ‘up to’ 70% of the time. Some 3,500 of the 40,000 fMRI studies in the literature over the past 20 years used these settings. The paper also notes that a bug (now corrected after being used for 15 years) in one of them also led to false positive results.

Here’s a bit more detail on what they did. It turns out that analyzing one voxel (essentially a single MRI pixel) at a time produces valid results. The problem comes when multiple voxels (clusters) are analyzed together. Clusterwise inference considers both the strength of activity at spots throughout the brain as well as the size of the spots. When a parameter called the cluster defining threshold (CDT) is set too low, the analysis is more likely to be false positive. This was true for all 3 packages tested. Parametric statistical methods produce the problem (not for voxels but for clusters). It relies on Gaussian Random Field Theory (RFT) for clusters , which depends on two other assumptions (1) the spatial autocorrelation function has a squared exponential shape — e.g. Gaussian (2) the spatial smoothness of the fMRI signal is constant over the brain. Neither of these assumptions is correct. Those of you who’ve read Nassim Nicholas Taleb about the stock market know about ‘fat tails’. It turns out that the spatial correlation function has them. Here’s what a fat tail is all about. Human height goes fall quite nicely into a Gaussian distribution. There are 7 and 8 footers about but they are rare. If the human height distribution wasn’t Gaussian but had a fat tail, we’d see 12 and 15 footers.

If that wasn’t bad enough,the following is even worse (in my opinion). 40% of 241 recent fMRI studies didn’t report using well known methods for correcting for multiple testing. They may have done so, but every biomedical paper, and drug study says so explicitly. Not only that but drug studies are required to explicitly state the hypothesis (or hypotheses) they are testing.

This is probably why in the early days, fMRI researchers always confirmed their original hypothesis. They could test the massive fMRI for statistical rarity, and since the data was so large, find it and post hoc propter hoc publish it. Possibly they did so out of ignorance, but even so this is inexcusable

Advertisements

The amount of BS you can get published about the brain

Hype about brain therapies isn’t confined to second rank journals The following is a direct quote from Neuron vol. 87 pp. 940 – 941 ’15.

” When a journal of the standing of Nature carries a headline ‘‘Shocks to the brain improve mathematical abilities’’ (Callaway) concerning two studies, we have to ask how this will be perceived. It is doubtful that the non-scientific reader will note that the studies have not been independently replicated, that only one of them tested mathematics, that the gains are as small as being milliseconds faster at some simple sums, and that only six people were tested in follow-up.

When it is claimed that tDCS (transcranial Direct Current Stimulation) can improve problem-solving abilities (Chi and Snyder, Brain Res. vol. 1353 pp. 168 – 175 ’10), the casual reader will not notice that only one-third of subjects improved, that there is no evidence that effects are sustained beyond 3 min, that there was no active control stimulation, and that there was no control for order effects.”

Caveat emptor. The quote is from an excellent review about brain stimulation with either magnetic fields or electrical current (direct, alternating or random) — Neuron vol. 87 pp. 932 – 944 ’15 — e.g. Non-invasive human brain stimulation, by 3 of the adults in the room.

It gets worse — here’s part of an older post — with a link to the whole thing —https://luysii.wordpress.com/2014/05/18/how-badly-are-thy-researchers-o-default-mode-network/

How badly are thy researchers, O default mode network

If you Google “default mode network” you get 32 million hits in under a second. This is what the brain is doing when we’re sitting quietly not carrying out some task. If you don’t know how we measure it using functional mMRI skip to the **** and then come back. I’m not a fan of functional MRI (fMRI), the pictures it produces are beautiful and seductive, and unfortunately not terribly repeatable.

If [ Neuron vol. 82 pp. 695 – 705 ’14 ] is true than all the work on the default network should be repeated.

Why?

Because they found that less than half of 71 subjects studied were stably awake after 5 minutes in the scanner. E.g. they were actually asleep part of the time.

How can they say this?

They used Polysomnography — which simultaneously measures tons of things — eye movements, oxygen saturation, EEG, muscle tone, respiration pulse; the gold standard for sleep studies on the patients while in the MRI scanner.

You don’t have to be a neuroscientist to know that cognition is rather different in wake and sleep.

Pathetic.