‘Harming‌ ‌the‌ ‌scientific‌ ‌process‌:’ An attempt to correct the sports science literature, part 3

Matthew Tenan

Why is it so difficult to correct the scientific record in sports science? In the first installment in this series of guest posts, Matthew Tenan, a data scientist with a PhD in neuroscience, began the story of how he and some colleagues came to scrutinize a paper. In the second, he explained what happened next. In today’s final installment, he reflects on the editors’ response and what he thinks it means for his field.

In‌ ‌refusing‌ ‌to‌ ‌retract‌ ‌the‌ ‌Dankel‌ ‌and‌ ‌Loenneke‌ ‌manuscript‌ ‌we‌ ‌showed‌ ‌to‌ ‌be‌ ‌mathematically‌ ‌flawed,‌ ‌the‌ ‌editors‌ referred to “feedback‌ ‌from‌ ‌someone‌ ‌with‌ ‌greater‌ ‌expertise”‌ and ‌included‌ ‌the‌ ‌following:‌ ‌

Continue reading ‘Harming‌ ‌the‌ ‌scientific‌ ‌process‌:’ An attempt to correct the sports science literature, part 3

‘A flawed decision:’ What happened when sports scientists tried to correct the scientific record, part 2

Matthew Tenan

Why is it so difficult to correct the scientific record in sports science? In the first installment in this series of guest posts, Matthew Tenan, a data scientist with a PhD in neuroscience, began the story of how he and some colleagues came to scrutinize a paper. In this post, he explains what happened next.

The‌ ‌journal‌ ‌Sports‌ ‌Medicine‌ ‌is‌ ‌widely‌ ‌considered‌ ‌one‌ ‌of‌ ‌the‌ ‌top‌ ‌journals‌ ‌–‌ ‌if‌ ‌not‌ ‌the‌ ‌top‌ ‌journal‌ ‌–‌ ‌in‌ ‌the‌ ‌fields‌ ‌of‌ ‌sport‌ ‌science,‌ ‌exercise‌ ‌science‌ ‌and‌ ‌physical‌ ‌education.‌  ‌This‌ ‌journal‌ ‌is‌ ‌managed‌ ‌by‌ ‌two‌ ‌professional‌ ‌editors‌ ‌who‌ ‌do‌ ‌not‌ ‌hold‌ ‌PhDs‌ ‌in‌ ‌the‌ ‌journal’s‌ ‌subject‌ ‌area‌ ‌but‌ ‌are‌ ‌generally‌ ‌versed‌ ‌in‌ ‌the‌ ‌topic‌ ‌and‌ ‌have‌ ‌the‌ ‌goal‌ ‌of‌ ‌managing‌ ‌a‌ ‌successful‌ ‌journal‌ ‌for‌ ‌SpringerNature.‌ ‌

The‌ ‌manuscript‌ ‌by‌ ‌Dankel‌ ‌and‌ ‌Loenneke‌ ‌was‌ ‌reviewed‌ ‌by‌ ‌three‌ ‌reviewers.‌  ‌I‌ ‌know‌ ‌this‌ ‌because‌ ‌I‌ ‌was‌ ‌one‌ ‌of‌ ‌the‌ ‌reviewers‌ ‌and,‌ ‌as‌ ‌noted‌ ‌in‌ ‌the‌ ‌first‌ ‌post‌ ‌in‌ ‌this‌ ‌series,‌ ‌I‌ ‌strongly‌ ‌advised‌ ‌against‌ ‌its‌ ‌publication.‌ ‌Greg‌ ‌Atkinson,‌ ‌a‌ ‌practicing‌ ‌scientist‌ ‌in‌ ‌the‌ ‌area‌ ‌of‌ ‌health‌ sciences,‌ ‌has‌ ‌publicly‌ ‌stated‌, in a private Facebook group, that he‌ ‌was‌ ‌one‌ ‌of‌ ‌the‌ ‌reviewers‌ ‌who‌ ‌recommended‌ ‌the‌ ‌paper‌ be‌ ‌published.‌ ‌Both‌ ‌myself,‌ ‌Atkinson,‌ ‌and‌ ‌the‌ ‌senior‌ ‌author‌ ‌on‌ ‌the‌ ‌manuscript,‌ ‌Loenneke,‌ ‌sit‌ ‌on‌ the‌ ‌editorial‌ ‌board‌ ‌of‌ ‌the‌ ‌journal‌ ‌Sports‌ ‌Medicine.‌ ‌And‌ ‌while‌ ‌the‌ ‌paper‌ ‌published‌ ‌in‌ ‌the‌ ‌journal‌ ‌by‌ ‌Dankel‌ ‌and‌ ‌Loenneke‌ ‌proposes‌ ‌a‌ ‌novel‌ ‌statistical‌ ‌method,‌ ‌neither‌ ‌of‌ ‌the‌ ‌two‌ ‌authors‌ ‌on‌ ‌the‌ ‌manuscript,‌ ‌myself,‌ ‌nor‌ ‌Atkinson,‌ ‌have‌ ‌PhDs‌ ‌in‌ ‌statistics.‌ ‌The‌ ‌published‌ ‌paper‌ ‌does‌ ‌not‌ ‌cite‌ ‌a‌ ‌single‌ ‌statistics‌ ‌journal‌ ‌in‌ ‌the‌ ‌course‌ ‌of‌ ‌reporting‌ ‌their‌ ‌“novel‌ ‌method.”‌

‌What‌ ‌could‌ ‌go‌ ‌wrong,‌ ‌right?‌ ‌

Continue reading ‘A flawed decision:’ What happened when sports scientists tried to correct the scientific record, part 2

Why — even after reforms for an episode involving bad statistics — is it so difficult to correct the sports medicine literature? Part 1

Matthew Tenan

Two years ago, following heated debate, a sports science journal banned a statistical method from its pages, and a different journal — which had published a defense of that method earlier — decided to boost its statistical chops. But as Matthew Tenan, a data scientist with a PhD in neuroscience relates in this three-part series, that doesn’t seem to have made it any easier to correct the scientific record. Here’s part one.

In‌ ‌July‌ ‌2019,‌ ‌my‌ ‌colleague‌ ‌‌Andrew‌ ‌Vigotsky‌‌ ‌contacted‌ ‌me.‌ ‌He‌ ‌was‌ ‌curious,‌ ‌he‌ ‌said,‌ ‌whether‌ ‌a‌ paper‌ ‌published‌ ‌in‌ ‌Sports‌ ‌Medicine‌ ‌had‌ ‌undergone‌ ‌statistical‌ ‌review ‌ ‌because‌ ‌he‌ ‌was‌ concerned‌ ‌about‌ ‌some‌ ‌of‌ ‌its‌ ‌claims.‌ ‌The‌ ‌link‌ ‌he‌ ‌sent‌ ‌me‌ ‌was‌ ‌to‌ ‌“‌A‌ ‌Method‌ ‌to‌ ‌Stop‌ ‌Analyzing‌ Random‌ ‌Error‌ ‌and‌ ‌Start‌ ‌Analyzing‌ ‌Differential‌ ‌Responders‌ ‌to‌ ‌Exercise‌,”‌ ‌a‌ ‌paper‌ ‌published‌ ‌on‌ June‌ ‌28,‌ ‌2019‌ ‌by‌ ‌‌Scott‌ ‌Dankel‌‌ ‌and‌ ‌‌Jeremy‌ ‌Loenneke‌.‌

As‌ ‌it‌ ‌happened,‌ ‌I‌ ‌knew‌ ‌that‌ ‌paper,‌ ‌and‌ ‌I‌ ‌had‌ ‌also‌ ‌expressed‌ ‌concerns‌ ‌about‌ ‌it‌ ‌–‌ ‌when‌ ‌I reviewed‌ ‌it‌ ‌before‌ ‌publication‌ ‌as‌ ‌one‌ ‌of‌ ‌the‌ ‌members‌ ‌of‌ ‌the‌ ‌journal’s‌ ‌editorial‌ ‌board.‌ ‌Indeed,‌ ‌I was‌ ‌brought‌ ‌on‌ ‌to‌ ‌the‌ ‌editorial‌ ‌board‌ ‌of‌ ‌‌Sports‌ ‌Medicine‌‌ ‌because‌ ‌the‌ ‌journal‌ ‌had‌ ‌recently‌ received‌ ‌a‌ ‌lot‌ ‌of‌ ‌bad‌ ‌press‌ ‌for‌ ‌publishing‌ ‌a‌ ‌paper‌ ‌about‌ ‌another‌ ‌“novel‌ ‌statistical‌ ‌method”‌ ‌with‌ significant‌ ‌issues and I had been a vocal critic of the sports medicine and sport science‌ field developing their own statistical methods that are not used outside of the field and validated by the wider statistics community.‌ ‌

Continue reading Why — even after reforms for an episode involving bad statistics — is it so difficult to correct the sports medicine literature? Part 1