Is There a Crisis in Scientific Literature? (1 Viewer)

[GroveDiesel]

Science is amazing, but it seems more and more like there are a whole lot of people who do a really crappy job of practicing it. From poorly designed studies, to journals that don't actually review experiments before publishing, fraudulant experiments that can't be replicated, and even sloppy citations that snowball with terrible consequences.

Like I said, science is great, but it's becoming very clear that there is a WHOLE lot of sloppy/fraudulant work being done that makes it very difficult to know what and who to believe.

Even the seemingly "easy" stuff becomes quickly confusing: are eggs good or bad for you? Is a glass of wine at bed great for your health or will it cause cancer? Is eating organic fruits/veggies good or bad?

I'm not sure what the answer is, but it's no wonder that too many people sneer at "science" and dismiss it all when those who call themselves scientists have done such a piss poor job policing themselves.

 

[RokNRole]

I blame religion

 

[Black Box]

Someone posted an article here about someone using standard social science methods to "prove" ESP exists. That one was pretty eye-opening.

 

[kface]

The other problem is the drive for ideological information.  There are journals popping up all over the place that will publish work without a true peer review for a fee.  It's down to a just a handful of journals being trustworthy.  Which in a way was always the case but the difference is now that the crappy information is being disseminated to the uneducated masses that don't know the difference

 

[Murph]

I don't want to downplay the problem and it's absolutely true that scientists and journals need to do better. But we also need to look at the way non-scientific media reports on studies. When one study finds X is associated with Y, and CNN or whoever reports "X causes Y WE'RE ALL GONNA DIE OMG!!!!1!!!!!!!!!" that's on them not the journals or the researchers.

 

[beer 30]

Think you're asking the wrong crowd

 

[Jayrod]

Similar to the news, the facts have become subjective.

 

[joffer]

Daryl Bem Proved ESP Is Real: Which means science is broken.

Yeah thats been shockingly debunked.  There's a great Skeptics Guide to the Universe podcast that covers it.

 

[D_House]

I don't want to downplay the problem and it's absolutely true that scientists and journals need to do better. But we also need to look at the way non-scientific media reports on studies. When one study finds X is associated with Y, and CNN or whoever reports "X causes Y WE'RE ALL GONNA DIE OMG!!!!1!!!!!!!!!" that's on them not the journals or the researchers.

Yeah I think science reporting is more problematic on the whole than the peer review process. And it stems from the same issue involved with all other kinds of reporting: the drive to sensationalize in order to draw more eyes. That's unlikely to change.

As far as science publication, it is appropriate that fabricators like the group that tried to connect the MMR vaccine and autism are called out and their work retracted, but of course it's still a problem that the study was published in Lancet in the first place and that it took 12 years to correct the record. It sucks that lives were potentially affected by claims made by grousps who pointed to fraudulent work to validate their beliefs; now it's important for the Lancet to continue to review their editorial standards to limit these types of occurrences. 

The one issue that I have with the OP and some of the responses I here is an exaggeration of scale. Yes, peer review is flawed. No, that doesn't mean all science has become "subjective" or that it's in crisis. I think that in actuality the scientific literature is doing better than in the past. For example, authors who submit to biomedical journals are now routinely required to disclose their funding sources and other conflicts of interest. This is a relatively new development that doesn't assure the absence of the problems noted above but does afford readers more information with which to evaluate the validity of the study's results. Before the 90s, these were rarely (if ever) reported. The Lancet MMR vaccine article had issues with its lack of disclosures, as the principal author didn't disclose that he was part of a relevant civil lawsuit or that he had a patent for a new vaccine.

 

[-OZ-]

Sure glad this wasn't a problem for galileo. 

Honestly hasn't this always been a problem but we see it more now, largely due to the availability of information. 

 

[lakerstan]

5 sigma, or it ain't true....

 

[joffer]

The fact that ESP needed to be debunked shows the problem. The study followed all the current norms, did all the right steps, and got an insane result. The problem wasn't the study, it's the method science uses to accept results.

No it didn't. 

 

[wikkidpissah]

It's weird. My old man was a research scientist who invented the grow light as part of an R&D dept for a big corporation that developed a lot of everyday products in the 60s. He took a glamour job in his field across the Hudson from Manhattan in the mid 70s. The times i lived i the city i'd bus out to his plant now & then and ride with him to awful frikkin Jersey burbs for dinner w mom, stay the night, go in with him early next morn. We avoided talking personal on those drives, he didnt consider my work work so we didnt talk about that so we talked about his. His company was ahead of hated GE on two pretty significant products in those days - the halo bulbs for incandescant fixtures that are now the home-lighting standard and one my father cared deeply about, full-spectrum lighting.

Full spectrum lighting duplicates the spectral signature of sunlight. Turns out this is significant because our entire endocrinology is cued by light sensers in our eyes which tell us what season it is by the length of the day. Just so happens that those sensers read candlelight (and incandescent lighting) as darkness. No adjustment for this had been made in the 150 years the human race had just spent moving indoors (i remember him citing a recent - this was early 80s - study of American cities where San Diego led the US in outdoor time with 35 minutes/day, quite a departure from the 10hr average of a century before.

He gave me some reading material and i went full-blown nutty about it. The bodies of anyone my age and younger have had winter levels of melatonin (to make us sleepy & horny for long nights) introduced into our systems because of our indoor lives since we were knee high to a snake. Guess the two diseases which have soared of the charts in globalwarming-type numbers since we moved indoors. If you said cancer & mental illness (including drug addiction) - conditions both deeply connected to our endocrinology - you guessed right. Full spectrum lighting (which you likely only know about from treatment of Seasonal Affected Disorder) in the home reverses that environmental error.

Big stuff, eh?! Came a point i'm literally tugging on his arm like i wanna stop for ice cream imploring him to forget the halo bulb and tell the ####in world about this. He had at least a dozen major theories about life science (not specifically his field) that were damn near as important (even i dont understand his theories on inner tides - yup, the moon's gravitational effect on the water of which we're 4/5ths made) "Can't say a word", he replied. "Gimmick science. Lose my professional standing like THAT, my career's over before my next paycheck."

Because of that, i've always believed in the rigors of science. Nothing approved til it meets the criteria of repeatability & measurability, and, beyond that economy & heuristics (does it point to higher, greater science). I reallyreallyreally hope science is still largely rigorous because, without it, we are more screwed than any politics can cause or cure.

 

[Long Ball Larry]

Even the seemingly "easy" stuff becomes quickly confusing: are eggs good or bad for you? Is a glass of wine at bed great for your health or will it cause cancer? Is eating organic fruits/veggies good or bad?

not sure this has so much to do with science as with media reporting any new "finding", making it seem like it is definitive.

 

[Mister CIA]

Science is amazing, but it seems more and more like there are a whole lot of people who do a really crappy job of practicing it. From poorly designed studies, to journals that don't actually review experiments before publishing, fraudulant experiments that can't be replicated, and even sloppy citations that snowball with terrible consequences.

Like I said, science is great, but it's becoming very clear that there is a WHOLE lot of sloppy/fraudulant work being done that makes it very difficult to know what and who to believe.

Even the seemingly "easy" stuff becomes quickly confusing: are eggs good or bad for you? Is a glass of wine at bed great for your health or will it cause cancer? Is eating organic fruits/veggies good or bad?

I'm not sure what the answer is, but it's no wonder that too many people sneer at "science" and dismiss it all when those who call themselves scientists have done such a piss poor job policing themselves.

That's click-bait, not science.

 

[Galileo]

Sure glad this wasn't a problem for galileo. 

Honestly hasn't this always been a problem but we see it more now, largely due to the availability of information. 

I did #### the right way!

 

[Galileo]

Studies show that if I use the phrase "studies show" when making a point I can get the majority of of people to believe it.

 

[huthut]

Maybe you should try reading PubMed instead of Slate for Science.

 

[Homer J Simpson]

Studies show that if I use the phrase "studies show" when making a point I can get the majority of of people to believe it.

I can buy that.

 

[GroveDiesel]

That's click-bait, not science.

I get that journalists do a terrible job of reporting science accurately and go for the big headlines, but study authors absolutely have an interest in grabbing those headlines and way too often are just fine with allowing journalists to run with headlines that don't match the research outcomes. The article I linked to in the OP quite clearly shows that researchers themselves often distort the the research (intentional or not).

Maybe you should try reading PubMed instead of Slate for Science.

Again, read both of the stories linked in this thread. They're pretty eye opening. Part of the crisis may be driven by poor journalism that focuses on huge headlines, but that's only a small part of the overall problem.

 

[jon_mx]

People are not very good at keeping politics and personal notoriety out of science. 

 

[Da Guru]

Believe....

 

[Sabertooth]

I don't want to downplay the problem and it's absolutely true that scientists and journals need to do better. But we also need to look at the way non-scientific media reports on studies. When one study finds X is associated with Y, and CNN or whoever reports "X causes Y WE'RE ALL GONNA DIE OMG!!!!1!!!!!!!!!" that's on them not the journals or the researchers.

And there it is.....another politics thread.  

 

[Murph]

And there it is.....another politics thread.  

Wat?

 

[Nugget]

Science is fine - it is the people and the bias they bring to the table that jacks everything up.

 

[Riversco]

maybe our expectations are too high.  

 

[Mister CIA]

Getting back to real science:

STUDY FINDS WOMEN WHO DRINK WAY MORE FUN TO STUDY

 

[joffer]

I guess I'm not getting why reproduceability being a problem is a problem. That's part of the process.

 

[peaces]

Yeah, #####! Magnets! 

 

[modogg]

It's been bad and getting worse for about 40 years.

Some of the lynchpin studies in some disciplines can't be reproduced, and may be incorrect.

Add in the preference to publish outliers, and the whole system may be falling down soon.

science has been around for 1000's of years. Something tells me it will perceive. 

 

[cockroach]

"Science is a liar... sometimes"

-Ronald "Mac" McDonald 

 

[James Daulton]

Science is fine - it is the people and the bias they bring to the table that jacks everything up.

Unpack this please.  Doesn't peer reviewed science and the scientific method mostly eliminate this?

 

[Chadstroma]

Even the seemingly "easy" stuff becomes quickly confusing: are eggs good or bad for you? Is a glass of wine at bed great for your health or will it cause cancer? Is eating organic fruits/veggies good or bad?

'Easy' is a comparative term but some of the things you point out here and the constant changing of 'science' specially when it comes to our body- this will cause cancer... no, it is good for you..... no it causes cancer again makes me suspicious of things that are comparatively much more complex and outside our ability to truly grasp but people still foam at the mouth about it being 'settled' science.

 

[kface]

'Easy' is a comparative term but some of the things you point out here and the constant changing of 'science' specially when it comes to our body- this will cause cancer... no, it is good for you..... no it causes cancer again makes me suspicious of things that are comparatively much more complex and outside our ability to truly grasp but people still foam at the mouth about it being 'settled' science.

Health science will very rarely be certain.  The body is so complex and there are so many competing factors that cannot be controlled.  That being said it is still valuable science because after a while a trend begins to show between studies that yes that link might be real.  It will never definitive but the evidence can strong and it's our health so there is nothing wrong with precautionary.  Red meat causes cancer is where the bulk of the literature is. You could probably find a study or 2 that tries to refute that but the bulk points in that direction