Navigation
Friday
May172013

Protocol: Impact of Gray Lit and Unpublished Studies on Meta-Analyses

Great topic.  ETA: mid-2013.  

Title
A protocol for a systematic review on the impact of unpublished studies and studies published in the gray literature in meta-analyses

Authors
Christine Schmucker, Annette Bluemle, Matthias Briel, Susan Portalupi, Britta Lang, Edith Motschall, Guido Schwarzer,Dirk Bassler, Katharina F Mueller, Erik von Elm and Joerg J Meerpohl

Background 
Meta-analyses are particularly vulnerable to the effects of publication bias. Despite methodologists' best efforts to locate all evidence for a given topic the most comprehensive searches are likely to miss unpublished studies and studies that are published in the gray literature only. If the results of the missing studies differ systematically from the published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention's effects.

As part of the OPEN project (www.open-project.eu) we will conduct a systematic review with the following objectives:

  • To assess the impact of studies that are not published or published in the gray literature on pooled effect estimates in meta-analyses (quantitative measure)
  • To assess whether the inclusion of unpublished studies or studies published in the gray literature leads to different conclusions in meta-analyses (qualitative measure) 
More about the protocol (pdf).
Friday
May172013

Problems with Reproducibility

Mobley A, Linder SK, Braeuer R, Ellis LM, Zwelling L (2013) A Survey on Data Reproducibility in Cancer Research Provides Insights into Our Limited Ability to Translate Findings from the Laboratory to the Clinic. PLoS ONE 8(5): e63221. doi:10.1371/journal.pone.0063221

Background

The pharmaceutical and biotechnology industries depend on findings from academic investigators prior to initiating programs to develop new diagnostic and therapeutic agents to benefit cancer patients. The success of these programs depends on the validity of published findings. This validity, represented by the reproducibility of published findings, has come into question recently as investigators from companies have raised the issue of poor reproducibility of published results from academic laboratories. Furthermore, retraction rates in high impact journals are climbing.

Methods and Findings

To examine a microcosm of the academic experience with data reproducibility, we surveyed the faculty and trainees at MD Anderson Cancer Center using an anonymous computerized questionnaire; we sought to ascertain the frequency and potential causes of non-reproducible data. We found that ~50% of respondents had experienced at least one episode of the inability to reproduce published data; many who pursued this issue with the original authors were never able to identify the reason for the lack of reproducibility; some were even met with a less than “collegial” interaction.

Conclusions

These results suggest that the problem of data reproducibility is real. Biomedical science needs to establish processes to decrease the problem and adjudicate discrepancies in findings when they are discovered.

[via Retraction Watch, which also offers some commentary on the study]

Tuesday
Apr232013

What Motivates Us At Work?

The video's worth watching, but you can get the gist from these studies:

1. Seeing the fruits of our labor may make us more productive.

The Study
In a study conducted at Harvard University, Ariely asked participants to build characters from Lego’s Bionicles series. In both conditions, participants were paid decreasing amounts for each subsequent Bionicle: $3 for the first one, $2.70 for the next one, and so on. But while one group’s creations were stored under the table, to be disassembled at the end of the experiment, the other group’s Bionicles were disassembled as soon as they’d been built. “This was an endless cycle of them building and we destroying in front of their eyes,” Ariely says.

The Results
: The first group made 11 Bionicles, on average, while the second group made only seven before they quit.

The Upshot
: Even though there wasn’t huge meaning at stake, and even though the first group knew their work would be destroyed at the end of the experiment, seeing the results of their labor for even a short time was enough to dramatically improve performance.

 

2. Knowing that our work helps others may increase our unconscious motivation

The Study: As described in a recent New York Times Magazine profile, psychologist Adam Grant led a study at a University of Michigan fundraising call center in which  student who had benefited from the center’s scholarship fundraising efforts spoke to the callers for 10 minutes

The Results: A month later, the callers were spending 142 percent more time on the phone than before, and revenues had increased by 171 percent, according to the Times. But the callers denied the scholarship students’ visit had impacted them.

The Upshot: “It was almost as if the good feelings had bypassed the callers’ conscious cognitive processes and gone straight to a more subconscious source of motivation,” the Times reports. “They were more driven to succeed, even if they could not pinpoint the trigger for that drive.”

 

3. The harder a project is, the prouder we feel of it

The Study: In another study, Ariely gave origami novices paper and instructions to build a (pretty ugly) form. Those who did the origami project, as well as bystanders, were asked at the end how much they’d pay for the product. In a second trial, Ariely hid the instructions from some participants, resulting in a harder process — and an uglier product.

The Results: In the first experiment, the builders paid five times as much as those who just evaluated the product. In the second experiment, the lack of instructions exaggerated this difference: builders valued the ugly-but-difficult products even more highly than the easier, prettier ones, while observers valued them even less.

The Upshot: Our valuation of our own work is directly tied to the effort we’ve expended. (Plus, we erroneously think that other people will ascribe the same value to our own work as we do.)

 

4. The less appreciated we feel our work is, the more money we want to do it

The Study: Ariely gave study participants — students at MIT — a piece of paper filled with random letters, and asked them to find pairs of identical letters. Each round, they were offered less money than the previous round. People in the first group wrote their names on their sheets and handed them to the experimenter, who looked it over and said “Uh huh” before putting it in a pile. People in the second group didn’t write down their names, and the experimenter put their sheets in a pile without looking at them. People in the third group had their work shredded immediately upon completion.

The Results: People whose work was shredded needed twice as much money as those whose work was acknowledged in order to keep doing the task. People in the second group, whose work was saved but ignored, needed almost as much money as people whose work was shredded.

The Upshot: “Ignoring the performance of people is almost as bad as shredding their effort before their eyes,” Ariely says. “The good news is that adding motivation doesn’t seem to be so difficult. The bad news is that eliminating motivation seems to be incredibly easy, and if we don’t think about it carefully, we might overdo it.”

There are a few more studies referenced in the story

[via Stephen's Lighthouse]

Thursday
Mar282013

The Perfect Swing (from Michigan Engineering)

Wednesday
Mar272013

User Satisfaction With EHRs

Was just talking about EHR usability with a colleague.  From the ACP:

“Dissatisfaction is increasing regardless of practice type or EHR system,” said Michael S. Barr, MD, MBA, FACP, who leads ACP's Medical Practice, Professionalism & Quality division. “These findings highlight the need for the Meaningful Use program and EHR manufacturers to focus on improving EHR features and usability to help reduce inefficient work flows, improve error rates and patient care, and for practices to recognize the importance of ongoing training at all stages of EHR adoption.”

The findings are from 4,279 responses to multiple surveys developed and analyzed by ACP and AmericanEHR Partners between March 2010 and December 2012. Of the clinicians who responded to the surveys, 71 percent were in practices of 10 physicians or less and 82 percent of respondents intend to participate in Meaningful Use incentive programs, up from 65 percent in 2010.

More.