A researcher in Ecuador has lost a 2019 paper on the application of a widely-used psychological research instrument after the owner of the tool flexed their copyright muscle.
The episode — like another one, recently — echoes the case of Donald Morisky, a UCLA researcher who developed an instrument for assessing medication adherence — and then began charging other scientists small fortunes (and, in some cases, large ones) for use of the tool, or forcing retractions when they failed to comply. (For more on the Morisky case, see our 2017 piece in Science and this recent warning by journal editors.)
Written by Paúl Arias-Medina, of the University of Cuenca, the article, “Psychometric properties of the self-report version of the strengths and difficulties questionnaire in the Ecuadorian context: an evaluation of four models,” appeared in BMC Psychology.
Per the paper’s abstract:
This study evaluates the psychometric properties of four models of the Strengths and Difficulties Questionnaire (SDQ) in a sample of 1470 children and adolescents from Biblián, Ecuador. The instrument has been used by researchers and students. However, there are not reports that show that the instrument is valid or reliable in the Ecuadorian context. …
It concludes:
The four tested models have questionable psychometric properties. Consequently, the use of the SDQ in the Ecuadorian context is not advisable. The three-factor first-order model of the SDQ that shows the best validity and reliability properties does not have undisputed psychometric properties. Comparisons across groups of age and/or sex using the SDQ should not be made.
The SDQ was the brainchild of Robert Goodman, an adolescent psychiatrist at King’s College London. According to youthinmind, a group Goodman helped create which describes itself as “a collection of psychologists, educators, and experts in IT” whose “aim is to promote the psychological well-being of people everywhere through the provision of information, assessment, treatment and research,” the SDQ is a:
brief measure of psychological well-being. It exists in multiple versions to suit the needs of researchers, clinicians, and schools. It is probably the most widely used measure of its kind, and has been translated into over 80 languages.
youthinmind also handles another of Goodman’s screening tools, the Development and Well-Being Assessment (DAWBA), a:
detailed measure of mental health. It exists in two versions for individuals aged 2-17 and 18-65 years old. It can generate ICD-10 and DSM-5 diagnoses and has been translated in over 20 languages.
According to the youthinmind website on the SDQ:
Strengths and Difficulties Questionnaires, whether in English or in translation, are copyright documents that are not in the public domain. As such, they may not be modified in any way (e.g. changing the wording of questions, adding questions or administering only subsets of questions). This is to ensure that the SDQ is fully comparable across studies and settings. Similarly, to ensure high quality and consistency, unauthorized translations are not permitted. Paper versions may be downloaded and subsequently photocopied without charge by individuals or non-profit organizations provided they are not making any charge to families.
Users are not permitted to create or distribute electronic versions for any purpose without prior authorization from youthinmind. If you are interested in making translations or creating electronic versions you MUST first contact [email protected].
Arias-Medina’s article acknowledges the changes, which in most cases appear trivial, although he said they occurred before he joined the research project. From the relevant passage:
The original Spanish translation was slightly modified to make it more comprehensible for Ecuadorian children by three professionals (a psychologist, an anthropologist and an educator). A pilot test was applied to a group of 52 children to guarantee a proper understanding of the questionnaire. As a result, some slight modifications were done to the Spanish version. The word “hiperactivo/a” (hyperactive) was eliminated in item 2 because it was not well understood; “Suelo tener” (I use to have) was replaced by “Frecuentemente tengo” (I frequently have) in item 3; “enfado” (get angry) was replaced by the synonym “enojo” in item 4; “gente” (people) was replaced by “compañeros” (mates/classmates) in item 5 and 14; “A menudo” (Oftentimes) was replaced by the synonym “Muchas veces” (Many times) in items 8, 13 and 20; “enfermo, lastimado o herido” (sick, hurt, or injured) was replaced by “lastimado o enfermo” (injured or sick) in item 9; “me muevo demasiado” (I move too much) was eliminated in item 10; “otros” (others) was replaced by “compañeros” (mates/classmates) and “manipulo” (manipulate) was replaced by “intimido” (intimidate) in item 12; “fácilmente pierdo la confianza en mí mismo/a” was eliminated of item 16; “niño/as más pequeño/as” (younger children) was replaced by “chicos (as) de menor edad que la mía” with the same meaning in item 17; item 19 was changed to “otros chicos (as) de mi edad me agreden o se burlan de mí” (other kids of my age assault or make fun of me) instead of “se meten conmigo” which was confusing for some kids; “Cojo” (take) was replaced by the synonym “Tomo” in item 22.
But youthinmind took a dim view of the edits. We emailed Goodman, and received a response from Helen Hamilton, of youthinmind. She told us:
Prof. Arias-Medina changed the wording of the SDQ without authorization from the copyright holder. This is a serious breach of copyright. No one is permitted to make unauthorized changes to the SDQ.
Every single legal version of the SDQ has a clear copyright notice on it, and there is also a copyright notice on our website: https://sdqinfo.org/
Hamilton added:
Unless the SDQ stays identical for all users, it is not possible to use previous validation or normative studies and assume they apply to SDQs as actually used, and it is not possible to compare or contrast findings across time or space.
If we allowed everyone to make corrections to the SDQ to suit their study, we would end up not with one version of the SDQ, but with thousands of different versions, most of which would be without norms. The SDQ would be much less valuable for everyone if it were not consistent since it would then be inappropriate to combine or contrast studies using different SDQs.
We are very pleased for the sake of our SDQ users that this illegal version of the SDQ has been withdrawn, not least because it will reduce the likelihood of others being misled into making changes they want to without permission from the copyright holder.
In doing so we will have prevented inappropriate modifications of the SDQ which undermines our ‘brand’, leads to non-comparability between studies, and is in line with many thousands of users who respected the copyright.
The retraction notice states:
The authors have retracted this article [1] because the Strengths and Difficulties Questionnaire has been modified without permission from the copyright holder youthinmind. The content of the article is no longer available owing to this breach.
All authors agree with this retraction.
That part about “no longer available” isn’t actually true, at least not at the time of this writing.
Like Retraction Watch? You can make a tax-deductible contribution to support our work, follow us on Twitter, like us on Facebook, add us to your RSS reader, or subscribe to our daily digest. If you find a retraction that’s not in our database, you can let us know here. For comments or feedback, email us at [email protected].
One wonders whether the creators of the questionnaire ever considered the fact that [European] Spanish is different to that spoken in Latin & South America. There will be words that will not be used commonly in the other variants. Did they ever consider this? [Same thing can happen in English – UK, US & Australian versions…] Language is not the same everywhere. If you want to have comparable results across countries then that is an issue that should be considered.
From looking at the SDS website, it does appear that the authors are open to alternative versions. There is a Rio de la Plata version in addition to the standard Spanish. There for instance coger has been replaced with tomar, as we might expect!
https://www.sdqinfo.org/py/sdqinfo/b0.py
So the issue here appears to be more that the authors were not approached for permission, which is simply good practice.
If we base this a reason for retraction 90 percent of survey research should be retracted as no one takes permission from original creators. Retract watch should also try to do it ethically, or highlight all articles which do this.
This is actually not true. I am a scientist at a company that develops survey instruments designed to measure different aspects of people’s health and well-being. These surveys are our intellectual property, and much of our revenue comes from pharmaceutical and medical device companies who license our instruments for use in clinical trials. Because we value research for its own sake, and because we understand that the more our tools are correctly used, the better it is for us, we typically provide licenses to academic researchers (especially students) at no cost.
Our tools have been translated into over a hundred languages. We oversee all of these translations ourselves, using well-established accepted principles for doing so. Because our surveys cover concepts that might vary across languages and cultures, we follow a strict process involving interviews, forward and backwards translations by linguistic experts, etc. It’s serious business, and if it’s done incorrectly, then we cannot vouch for the validity of the translated survey, in which case it’s harmful to our brand.
That said, people misuse our tools all the time. Once you sell someone a car, you can’t stop them from driving it over a cliff.
Basically, people who develop instruments that are reliable and validated need to ensure that people are not modifying or misusing them, and even further, that those people are not disseminating their modifications that will lead to more people misusing them.
So, this is very different from the Morisky case. For Morisky, it’s mostly about money and greed. But in this case, it’s a person who developed an instrument with scientific merit, and then someone created their own ‘version’ of that instrument, some knock-off but with his name on it, which devalues the instrument scientifically. It’s not about money, it’s about protecting intellectual value. Like if you developed a scientific theory, and someone misapplied it and made claims from it that weren’t true, then you’d want to correct them, because they are misusing your ideas and then making it possible for others to follow suit.
I’m 99% sure that if the researchers had gone to the developer first, told him what they wanted to do, the developer would have gladly worked with them to make sure it was done right. People using your instruments correctly is in your best interest. People modifying your instrument in some ad hoc manner but then still calling it your instrument is not in your best interest.
Absolutely ridiculous.