How does it feel to have your scientific paper plagiarized? And what can you do about it?

Marya Zilberberg

Plagiarism is a frequent reason for retraction. Today, we’re pleased to present a guest post by Marya Zilberberg, a physician health services researcher and faculty member at the University of Massachusetts at Amherst School of Public Health and Health Sciences. In this post, she describes what it’s like to find out one of your papers has been plagiarized — and how to get satisfaction. Well, sort of.

Right or wrong, peer-reviewed publications in my trade are academic currency. They provide name recognition, invitations to review, edit and speak, and in general make you feel like a part of the “in-crowd.” Of course the most important metric that publications feed are the infamous h-index, which measures how “influential” your studies are by the number of citations they engender. So, like any other artificial grade, it makes sense to engage in intermittent care and watering of your h-index, and mine is pretty good for where I am in my career. Little did I realize that there is an even more important impact metric than the h-index: plagiarism.

Yes, plagiarism. Let me explain.

About two months ago I was trolling the literature and came upon a review paper on ventilator-associated pneumonia (VAP) by S. Efrati et al, published in 2010 in the Journal of Clinical Monitoring and Computing. I am definitely a VAP nerd, so I started to read the paper. It did not take me long to come upon this passage:

The standard component of IHI’s approach is ‘‘bundles’’ of care, defined as ‘‘a small, straightforward set of practices—generally three to five—that, when performed collectively and reliably, have been proven to improve patient outcomes’’ [9]. The IHI’s ventilator bundle was originally aimed at reducing complications of mechanical ventilation, and not specifically at VAP prevention [9]. Of the four components in this bundle, three—head of the bed elevation, daily sedation interruption and daily screening for readiness to extubate—are aimed specifically at VAP prevention [9]. Each of these three IHI bundle elements has behind it either a level I (head of the bed elevation and stress bleeding prophylaxis) or level II (use of sedation holidays) evidence individually. In view of this, it is difficult to argue that implementing each of the proposed measures does not amount to good care of patients on mechanical ventilation, supporting the original intent of the mechanical ventilation bundle development [9].

My first thought was “damn, this is a well written passage” followed by “looks familiar.” A split second later I realized why it was familiar: these were my words. Consider the similarities from page 1 of this paper of mine from 2009:

The standard component of IHI’s approach is “bundles” of care, defined as “a small, straightforward set of practices— generally three to five—that, when performed collectively and reliably, have been proven to improve patient outcomes” (10).

And then more, on page 4:

In this context, and in the interest of clarity, the IHIs ventilator bundle was originally aimed generally at reducing complications of MV, and not specifically at VAP prevention (10). Of the four components in this bundle, three—head of the bed elevation, daily sedation interruption and daily screen for readiness to extubate, and stress bleeding prophylaxis—are aimed specifically at VAP prevention (10), mirroring some of the recent guideline recommendations from the American Thoracic Society and the Infectious Diseases Society of America (6). Each of these three IHI bundle elements has behind it either a level I (head of the bed elevation and stress bleeding prophylaxis) or level II (use of sedation holidays) evidence individually (6). In view of this, it is difficult to argue that implementing each of the proposed measures does not amount to good care of patients on MV, supporting the original intent of the MV bundle development (10).

Hmmm, I thought, naturally, the authors must have cited my paper where they plucked my words, just neglecting to enclose them in quotes. Reference 9, reference 9, reference 9… Hmmm, reference 9 was to the IHI ventilator bundle implementation page. (Guess what the reference 10 in my paper is.) Scanning the rest of the references, not one of the forty-eight had my name in it.

With a mounting indignation, I went back to the text and read on. A more careful reading revealed yet another place where my words were lifted en bloc without bothering to quote or attribute: The final sentence of the “Definition and Epidemiology” section:

Clearly, the combination of the need to improve patient outcomes, the financial impact of improving throughput, and the proposed cut in VAP medicare reimbursement, is serving as a strong impetus to implement practices and policies aimed at reducing the risk of VAP.

It looked suspiciously like what I said in my discussion section (notice, I did capitalize “Medicare”):

Clearly, the combination of the need to improve patient outcomes, the financial impact of improving throughput, and the proposed cut in VAP Medicare reimbursement, is serving as a strong impetus to implement practices and policies aimed at cutting the risk of VAP.

Up until this point, I had had no experience with such blatant theft of my work. So, for advice I turned to one of the two obsessives who runs Retraction Watch, my friend and colleague Ivan Oransky. First, I wanted to make sure I wasn’t crazy, that this really was what plagiarism looked like — I know, duh! And second, I needed to know how to deal with it and what to expect. Ivan confirmed my suspicions, and advised that I correspond with the journal.

I promptly commenced such correspondence on January 6, 2012, with Laura Walsh, the Senior Editor for Medicine at Springer, the publisher of the journal that had published Erfrati’s et al’s paper. When three days later I had not yet heard from Walsh, I sent another message, this time loaded with a tad less patience, and cc:ing Ivan. This did get a reply the following day, January 10, referring me to Springer’s “Ethics in Publishing Policy.” Walsh added:

These matters are addressed as quickly as possible with due consideration to all parties involved. Our first step is to bring this to the attention of the Editor-in-Chief, which has been done. I expect to have his reply very soon and will keep you informed.

I prepared to wait, but intermittently sent an email to check on how things were progressing. In responses to my messages from January 18 (answered same day), February 2 (answered on 2/7) and March 2 (answered the same day) I was told that the matter was in progress, and the Editor-in-Chief was making contact with the authors. Then, at last, on March 5, I received this e-mail from Chief Editor Stephen Rees:

The status with this situation is that I have written to Dr Efrati passing on our concerns. He has acknowledged these problems as a an unfortunate and undeliberate mistake for which he and fellow authors apologize. In line then with the usual Springer policy, we have offered him the opportunity to write an erratum to the paper to this effect which will then be published. In addition, he has been given your email address and offered the opportunity to write to you directly. I have been asked to be copied in on his correspondence.

I am very sorry for your inconvenience with this matter and hope you feel that we have treated it with the seriousness that it deserves.

It has now been a week, and I haven’t heard anything from Efrati.

So, what do I conclude from this kerfuffle?

  • Plagiarism can happen to any of us.
  • Journals do not seem to do due diligence when accepting papers.
  • No offense to this journal, but exactly how many people are going to read the “erratum” and become aware of these authors’ misconduct? And what are the implications for checking their prior and future work?

But the most important point is this: I have made it! Yes, because when your words so mesmerize that others are compelled to channel them, subconsciously and unstoppably, through their keyboards, it has to be the highest form of praise, even if they “neglect” to attribute. So, no “inconvenience” at all, thank you very much.

86 thoughts on “How does it feel to have your scientific paper plagiarized? And what can you do about it?”

  1. Thank you, Marya, for an interesting read. I’m a prof at a large university and, after reading your post, I thought about how this really doesn’t have to happen any more with the availability of plagiarism detecting software. Are publishers of academic work not using these? And shouldn’t our universities invest in licences for such software as a way for their faculty to ensure that these types of “undeliberate” mistakes don’t happen?

      1. With the orgy of Frontiers this and that, and the rest of the for $ Open Journals, expect this to become a much worse problem…

  2. From my experience, who has also been plagiarized, but in my case not exactly felt like flattered, as others snatched images for reuse after a bit of Photoshop, journals usually try to ignore any claims of misconduct sent via email, and when pushed they try to get the most political, easiest, way out.

    This is not helping increase the reliability of Science at all. Anyone can easily get away with fraud, thus what prevents me from doing likewise? Dishonest practice easily beats hard work when it comes to getting more funding. I have many known good who either succumbed to corruption or have given up sicence altogether.

    Picturing plagiarism as natural or acceptable is not a good way of soothing this.

  3. I’m grossed out!

    What’s up with this?? In the absence of evidence to the contrary, it looks like the offending editors (and I do mean offending) are trying to sweep it under the rug! “Nothing to look at here, folks – move along…”

    I mean, this is wholesale theft, and a total failure by the involved editors to do their job. I can accept that people make mistakes, but the correct response by any such “cop” is to say “Holy crap – we had no idea. Thanks – we’re on it” and yank that sucker immediately. To minimize it is nothing short of corrupt butt-covering, IMO – the editors are failing to do their jobs as custodians of their employers’ brand (particularly the brand’s reliability to their readers).

    Erratum, b.s. The author “undeliberately” made the same omission 48 times? Fixing this is easy: say “We made a big mistake, and that’s important, and it’s been fixed.” Done.

    I might add: so much for the oft-given advice to only trust evidence from peer reviewed journals.

  4. “We have offered him the opportunity to write an erratum to the paper” sounds pretty weaselly to me. I don’t care how “undeliberate” the plagiarism was, the offending review should be retracted.

  5. “Journals do not seem to do due diligence when accepting papers.”

    Unless journals run all submissions to some damn good plagiarism detection software, how could they catch these?

    1. Jon, I don’t know the status these days, but way back in the 1990s I had occasion to use an automated plagiarism site, and I was awestruck by its sophistication. (I have no idea what it was.)

      I’m no academic but to take a stab at it, I just went to Google Scholar and pasted in Marya’s first example sentence, and voila, she shows up first, and Efrati shows up third: http://bit.ly/xEESvM

      I wonder why there isn’t something like a Google Plagiarism Checker. Even if it cost $1 per article, paid by the submitting author, I’d think it’d be a viable business.

      — Hm, I just googled “plagiarism checker,” and in an instant it said “possible plagiarism” and linked to Marya’s paper. (The 1990s service I remember was much more sophisticated.)

      Anyway, Jon, as I said above, I can forgive a mistake, but the job of an “integrity cop” is to JUMP on any discovered accident, not explain it away.

  6. When I was working for About.com (disclosure: before I came to Reuters Health full time to work for Ivan), I was one of several hundred of their writers/bloggers (I managed their celiac disease site), and dozens of us were regularly plagiarized. But outside of academic science and medicine — where researchers can complain to journal editors and department heads, at least — there is virtually no recourse. With online publishing it’s become easier than ever to plagiarize, and the online publishing industry has had its head in the sand. We really need better tools for controlling this problem.

    1. Dr. Zilberberg has given a great description of journal mediated plagiarism issues here, and explains the details in a very solution oriented way. Groups like Cochrane can help, as well as University or online journal clubs for graduate and professional students; also those working on papers themselves can discover instances of plagiarism. As evidenced by Dr. Zilberberg’s description, sometimes clinicians and professors do not have the opportunity to discover the plagiarism at the time it is first published, due to the relatively obscure “erratum” information that varies in availability from journal to journal. Her blog post here will assist others facing the same challenge to make a plan of action. Plagiarism does not equal replication, and curbing this problem can prevent much greater inaccuracies in the medical and scientific literature that can ultimately impact patient care.

      Non-academic plagiarism issues were also well stated by Ms. Lapid. This is a problem that is currently happening in every type of online written science and heath communication. This problem directly changes the quality of information available to patients. The impact on patients can be direct, especially when the original material is used in an inaccurate way by plagiarizers making unproven health claims. That is one risk that academic peer review can eliminate which is not available for health and science writers communicating in non-academic settings. Many health and science writers that publish material for patients share her experience with health communications being plagiarized. What can increase and perpetuate it appears to vary with the individual situation. It needs to be defined clearly. Perhaps Health on the Net can assist with this task.

      There have been instances where original document writers are threatened with plagiarism charges themselves by the plagiarizer after the original material is taken. When combined with other types of threats, along with seeing the plagiarism tactics successfully directed towards other writers by the same plagiarizer in the recourseless realm of non-academic health writing, a robust chilling effect can be the result for the writer that actually created the original material, regardless of the existence of emails and other documents that demonstrate that the original writer did indeed initiate the written communication and publication. Some of it is enabled by lack of available information about writing procedures or workflows that by design minimize the ability of non-academic science communication plagiarizers to get away with these things. Professional science and health journalists have more experience with and can possibly advise the public and non academic science communicators about detecting and correcting this type of online plagiarism.

      1. Hi JudyH. Thank you for bringing that to my attention. “Here” in my first sentence refers to “in the above blog post.” That was an important detail that you mentioned, because the word order could have made the statement somewhat unclear. 🙂

  7. I’m hoping that your final paragraph is sarcasm because I’m troubled by the idea that plagiarism is in any way flattering. What if editors started telling authors that they should be honored, not angry, that their work is good enough to steal? As you point out, the offending author hasn’t contacted you to apologize, an erratum won’t necessarily prevent the offending author’s article from being cited in the future, you’ve spent what appears to be a large amount of time corresponding with the journal to voice your concerns, and there’s no guarantee that the offending author’s past and future work will be scrutinized for other offences – this seems like a major inconvenience to you personally and to the scientific community in general. As a librarian and faithful reader of this blog, this makes me so angry (for you, not with you, of course)!

  8. Crosscheck, a service of Crossref, provides exactly the kind of plagiarism checking software suggested in the comments. Deciding when duplication constitutes plagiarism still requires judgment and appropriate follow-up, of course. The Committee on Publication Ethics has just released guidelines on “Cooperation between research institutions and journals on research integrity cases”.
    http://www.crossref.org/crosscheck/index.html
    http://publicationethics.org/resources/guidelines

  9. One failure to add the quote marks and cite the source could be an “undeliberate mistake.” Multiple quotes? No citation at all of the source? No. Not a mistake at all. And an erratum written by the authors is not the right response. In my opinion, the journal editors should, at a minimum and while communicating with the authors of the paper containing copied and uncited material, write an immediate erratum themselves, stating clearly that the paper contained plagiarism, that the journal does not tolerate even accidental plagiarism, and give the citation for the original paper, and listing the passages in the plagiarizing one that were lifted from the original. It is important to bring to public attention the authors responsible for plagiarism, so that their other papers in other journals can be checked. When a pattern of plagiarism has been found (and the bar for such pattern recognition should be low–multiple plagiarisms in one paper or two papers with one or more instances) all papers by that author or author should be retracted by the journals involved and the author or authors’ institutions/employers notified.

    The use of plagiarism-detecting software can be useful but is not 100% effective in all fields. It would certainly be a good first step for institutions and journals to use in review of papers. However, I would expect scientific plagiarists to spend their brain cells trying to outwit the software rather than review and correct their own papers, as long as the penalty for being caught plagiarizing is “offered a chance to write an erratum.” Automatic retraction of papers–which cuts off the valuable citation index possibilities–is a far more effective negative consequence. Imitation is only a form of flattery when no profit is involved–when someone seeks to profit by imitation, their motive is not flattery but theft…whether it’s theft of money or theft of fame (such as a citation index) is immaterial.

  10. Thank you, everyone, for your comments. Obviously I have had 2 months to live with this, and the acuity of my indignation has subsided somewhat. I would like to think that this is an aberration rather than the rule in the academic circles.

    It is worth mentioning that when familiarizing myself with the issue of plagiarism recently I was surprised to learn that in some cultures it does not carry such serious connotations. In fact, in some cultures it is considered impolite to mention the source. Given that more and more academic publishing is globalized, we, of course, have to come to a uniform agreement.

    I would love to hear more on this cultural aspect of plagiarism and understand what role it may have played in this example.

    Marya

    1. Considering that the authors are from Israel, Italy, UK and US (at least by affiliation) I don’t think they can plead the infamous “cultural differences”. We have just one culture of academic publishing, and I seriously doubt any of the authors managed to get those acronyms besides their names without learning the difference between a quote and a plagiarism. I think you are trying to be too nice by allowing cultural differences into this discourse. These are all supposedly educated people, not a tribe recently spotted from a heli in the Amazon.

      1. I agree. The authors’ last names and current affiliations do not suggest a background in East Asia or in an Islamic country, two places where “cultural differences” might serve as an excuse, although not a justification.

        Look at the possible-conflict-of-interest statement. Three of the authors have stock in a company that makes the AnapnoGuard system touted in the paper and we learn that “The AnapnoGuard system has recently gained CE approval and is currently in use in clinical trials.” I think these guys were hoping to generate some business without taxing themselves too much intellectually.

    2. RE: Cultural aspect of plagiarism
      I taught ESL writing and was astounded sometimes of the attitude of my foreign college students toward plagiarism. It was rather easy to detect passages that were not their own words and had been lifted wholesale from such sources as Time Magazine. When confronted with this fact they became indignant and defensive. For one thing, they couldn’t understand the moral problem with lifting someone else’s words; moreover, they stated that the original had been written so well that there was no way their rewording could be as good. It took a lot of convincing that proper attribution and paraphrase were essential.

      1. The justification that “the original had been written so well” usually means “I have no idea what it says, but they’re the experts, so it must be right”.

      2. I wouldn’t be surprised if the students you dealt with were from Russia/fUSSR. It is not so much in Russian culture, but in culture of Russian educational system at present, to plagiarize other people’s work during the undergrad years in preparation of essays, literature reviews (“referat”s), etc. There are websites like referat.ru where one can search by the topic, or even course/professor, and get a ready-made essay, other free or for a fee. As the system stepped away from elitist state-sponsored education to pay-for-diploma system, neither the students nor unfortunately many professors care much about such things as academic honor. But I don’t think that any of the authors in questions are students, nor do they appear to have Russian-sounding names…

        1. Russian or not, it’s a freaking Springer journal, for heaven’s sake. If they’re going to say their policy is “situational ethics, which vary by the author’s home culture (maybe),” then why bother reviewing and editing it at all? Why not just say “That’s what they submitted – it must be acceptable where they came from”?

      3. It’s not just foreign students. Talk to anyone who teaches high school students or undergraduates at a large public university in the US and you’ll hear plenty of stories, most of them in the it-would-be-funny-if-it-wasn’t-so-depressing category. (The scariest part: many of these students go on to med school.) The problem isn’t cultural differences, of course – it’s a failure to set and enforce clear rules. Usually the worst punishment is failing a single assignment or exam; when I was in college (not too terribly long ago) we were told to expect failing the course at a minimum, and quite possibly facing suspension or expulsion. Most schools would rather not go through the red tape and face possible lawsuits by the parents.

      4. It can be amusing to trace back the chain of one non-native speaker copying verbatim what another non-native speaker has copied verbatim from an original source. I did this recently, while helping a foreign graduate student. The first plagiarizer had left out just a few letters when copying from the source (with a citation but no quote marks), so his sentence made no sense, and therefore the graduate student’s plagiarized sentence (copied verbatim from the first plagiarizer with a citation but no quote marks) made no sense. The original did make sense. But non-native students often are trying to generate pages full of words without understanding what the words mean.

        Yes, you can explain and show and explain and show and explain and show, and in the next draft you’ll see another sentence copied verbatim with no quote marks. They take the easiest path and apologize if they get caught.

    3. I don’t buy into the “cultural differences” argument in relation to scientific writing. It is after all part of the scientific process and this has – since the Enlightenment – been based on the knowledge and science produced by our scientific predecessors. And along with this comes the need to acknowledge and attribute prior work so that a transparent scientific process is established (of course I’m talking about Utopia here…) . If we remove or somehow dilute this praxis, the basis of science itself comes under threat. This is something all science students at all levels should be taught.

      What I am surprised about in this whole discussion, is the fact that plagiarism is also a copyright infringement. Why not strike offenders down with a DMCA takedown notice? Why doesn’t the publisher of the original article or whoever is the copyright holder send a ceased and desist letter? This does not even require a lawyer. I know of so many cases in Germany (and the USA) where private persons (bloggers) have been harassed by rightsholders for quoting from e.g. newspaper articles which are really minor cases compared this example. I am generally not a friend of litigation, but these examples of plagiarism – and the journal editors’ responsed – really require some robust action.

  11. Definitely a decent plagiarism check is a must in the current climate of endlessly proliferating magazines with little to nothing in a way of review and editing. In this particular case the paper in question should be analyzed in full with a view of possible cannibalization of other publications. I do not believe that having lifted so much material from one review this bunch just stopped. Pathetic, really, 6 authors didn’t have enough grey matter between them to come up with one little review in their own words…

    1. I agree. There probably are more instances of plagiarism in the paper. Will the editor check for that? Probably not. So many administrators try not to see misconduct and refuse to find any misconduct beyond what they are forced to acknowledge when a third party (or the offended party) conducts an investigation. In-house investigations are almost always whitewashes.

  12. Hmm, disappointing all round. This may be a naive question but do the publishers ever wade in on this? Presumably you assigned the copyright of your material to publisher A, which is now being violated by publisher B, so isn’t there some recourse that A should seek against B?

    Might be an example of value-add that publishers can offer their authors – protection from plagiarism =)

  13. Yes, it is true, different cultures have different views on plagiarism. In some cultures it is deemed a high form of respect to plagiarize ones mentor. That is not, however, true in this country where plagiarism in any publication (or grant application) that cites government funding is considered research misconduct and the offender is at risk of losing future funding. It is important for universities and mentors in the US to recognize these cultural differences and educate accordingly.

  14. An “undeliberate mistake” my ass! Multiple instances of a verbatim copy of multiple sentences — that doesn’t happen by accident. The journal editor is a total wimp to consider an erratum and an apology as sufficient.

    It’s generous of you to find a silver lining in the cloud, but I doubt most authors are as sanguine about this kind of violation. An acquaintance of mine, Louis Longchamps, was thrilled when I told him that I had read on Retraction Watch (http://www.retractionwatch.com/2011/12/28/plagiarism-kills-weed-paper/#more-5678) about the retraction of a paper that plagiarized one of his papers. He sounded like he had been — and was still — pretty angry about the incident.

    One of my friends, an anesthesiologist at a university hospital, says all the manuscripts coming out of his unit have to pass a plagiarism review (I forget the name, but it’s one of the computerized plagiarism services) before submission to a journal. He’s a bit worried about the potential for being flagged over unavoidable unoriginality in the description of standard research designs, but I doubt that this will be a problem. “Double blind” and “paired t-test” probably won’t ruffle the software’s feathers. It’s those complete sentences and entire paragraphs lifted intact from another paper that trigger warnings.

  15. My suspicion for why publishers stay clear of the ‘plagiarism’ accusation is that they’d rather avoid libel suits in some jurisdiction or other. I’m surprised how frequently ‘duplicate publication’ is acknowledged, errata published, yet that dreaded word ‘plagiarism’ ain’t mentioned…

  16. Given that so many formerly independent journals are now owned by big publishers like Springer, the failure to use software to check for plagiarism is an indication of not caring all that much about it. After all, it’s not Springer’s IP/profits that are at risk. At least not as long as there’s no organized pressure on the publishes and their individual editors to do something.

  17. I had the ‘honor’ of being selected to be an anonymous reviewer for a paper when I had the exact same thing happen: “This paragraph sounds good! Wait… it sounds familiar… Wait… these are MY words… These are MY words!!!” How serendipitous that I was asked to review a paper where my own work was plagiarized. I then went through the paper and googled other sentences that sounded “really good” and found four other papers that were plagiarized. I stopped there – there were probably more. I told the editor that had assigned me the paper how very upset I was, and he has contacted the author’s institution. (My fear was that a simple rejection would just send the damn thing to another journal… there has to be a more significant consequence to stop these fradulent actions.) I have not yet spent the time to check this author’s previous publications.

    I suggest that when a scientist finds their work has been plagiarized you:

    1) Contact the editor of the ‘offending’ journal AND the editor of the journal that published your piece.
    2) Insist that an apology or erratum is insufficient.
    3) Contact the office of research integrity at the offending authors’ institutions.
    4) Do not correspond with the offending author directly.

    I was angry when this happened to me – and it hadn’t been published yet! And I don’t think that anger is misplaced or inappropriate.

    1. I think LNV makes a very good comment. Marya, you should consider also contacting the Editor (and/or Publisher) of the journal that published YOUR article. They may not be so happy with a mere “erratum” either.

    2. The major problem is when the journal disagrees with you on the plagiarism. Once I got plagiarized both on images and narrative, and the journal (german) said that the narrative was over 80% original, thus it was “acceptable” and said nothing about the images, one of them merely turned upside down.

      1. The question isn’t whether Journal B (the second journal) thinks it is plagiarism. The question is whether Journal A (the first journal, where you published your original work) thinks it is. They hold the copyright, while you hold (back) your extreme anger!

        Perhaps Journal A doesn’t want to pursue it, because they have a behind-the-scenes handshake with the editors of Journal B. Or perhaps the same publisher owns Journals A and B, and doesn’t want the trouble. In that case, point your indignation at the editor of Journal A, and make it clear how unhappy you are. I think the chances of this happening are slim because journals want to protect their ‘property’, i.e. your words.

  18. Yes, there are cultural differences. Here in Brazil I am not at all proud to say that plagiarism is not considered a serious offense as people got used to being robbed by governants everyday. Actually a clan-custody rather tribal feeling makes stealing from a friend a bad behavior (not a serious crime), while stealing from people you do not know a common survival instinct which should be tolerated.

    So much for cultural differences, Science should be international.

  19. Marya, I agree with your comment – ” In fact, in some cultures it is considered impolite to mention the source”.
    For other members in the discussion : Yes, there is a strong cultural aspect to plagiarism (especially from Asian countries) which many western educated academics/authors/editors conveniently ignore. The western concept of plagiarism (“stealing words”) is not adopted by all. On the contrary sometimes citing directly from texts/teachers are considered a honor to the original author/teacher !

    Based on my personnel experience, plagiarism is a major issue with lot of international students studying in US/Canadian Universities. There are several articles published on this subject ( please google the subject). Unfortunately majority of students don’t realize it and are appalled when confronted with the fact that plagiarism constitutes misconduct. I sincerely hope that proper training (a compulsory preparatory course) can help them.
    Marya, it was nice of you to bring the cultural aspect of plagiarism – a fact ignored most of the times!!

    1. Citing directly from texts / teachers is fine – using their words without citing them is not. That’s the whole point!

    2. Nice for a general discourse, but where do you see Asians on the list of authors? And looking on MDs and PhDs beside their names I don’t think we are dealing with students here.

      1. Right. My intention was aimed at general discussion on cultural aspects of plagiarism raised by Marya. The issue of International students was used as a case study for above subject. My apologies if I was not clear..

    3. Dear ClimateChange

      “On the contrary sometimes citing directly from texts/teachers are considered a honor to the original author/teacher !” — This is WAY different from what was described in the post, which was lack of citation.

  20. I think that the problem of plagiarism comes from the fact that students in science are absolutely not trained to put their ideas on paper or to develop an independent mind and critical thinking.

    In our universities, PhD students in science are trained as super specialized technicians but not as scientists capable of reflection.

    You would be surprised to know how many PhD students (and this is true in many countries) are unable to describe or define what the scientific method is and what epistemology is…..

    We do not teach them how to write or think, we ask them to know how to do a PCR or to design vectors for molecular cloning (..etc) and that is it. Many university professors have resigned from their responsibility to train real scientists. In science, university professors are asked to publish at all costs. Forming new scientists takes time and patience and does bring any direct personal benefits in term scientific achievements. So, the only way to get into this highly competitive environment when you do not have the appropriate formation is to copy others! I do not agree with plagiarism but to focus only on people who practise it, does not resolve the problem…

    Concerning the ‘cultural aspect of plagiarism’ or to a larger extend the ‘cultural aspect of fraud in science’. We have to admit (look at the ranking of retractions by countries in this blog) that the countries where we find most of the plagiarism and scientific frauds are in order 1) United States, 2) Germany and 3) UK… Three big Western Countries where ethic is supposed to be the highest in the world… (this is a sarcastic statement…)

    Someone would probably argue that these countries are related to the ‘affiliated institutions’, and not to the nationality of the authors of the retracted articles. But, according to me, the affiliated institutions should be as responsible as the authors. I think that big scientific Western institutions close their eyes on many things….because it is in their interests of closing their eyes….but when something goes wrong or arises publically it is easy to say that the problem come from people with different cultures….

    1. Lilly,

      While I agree with you on many issues here especially the way scientists are trained these days (more than often times only to become a highly skilled technician), I do not agree with you that higher number of retractions coming from western countries or institutions in western countries actually reflect any reality. Well it is oversimplification of the problem and probably a good analogy to this oversimplification can be seen in statements like people have less cancer incidences in African countries than in United States.

      And let me make it more clear that I am from a third world country too currently living in US, so I have seen both side of world too. Well we see more retractions from US because more scientific institutions are here which conduct more research than any 3rd world country and most importantly we have strict intra-institutional and federal regulatory bodies to keep an eye and do un-biased or honest investigation system in place once whistle has been blowed. I do not find any regulatory body such as ORI in India and even if they have one there, it is impossible to get convicted someone through those agencies because most of them are so corrupt…..they can dilute any such investigation with their typical deeply corrupt bureaucratic system….

      a very small example is right here:

      A controversy erupted in National Centre for Cell Science (NCCS), Pune in 2006 when an anonymous mail alleged that the authors (H. Rangaswami and Colleagues from the group of Dr. Gopal Kundu) may have misrepresented data (especially through Western blots) in a paper published in Journal of Biological Chemistry. The allegation was that they had rehashed the same set of data which they had published earlier. An internal committee of the NCCS advised the authors to take back their paper, however an independent committee led by G. Padmanabhan, a former director of Indian Institute of Science, Bangalore, concluded that there was no manipulation in the data. This led to some heated debate between Indian Scientists with several viewpoints being presented. However, most important turning point in this case was that JBC withdrew the paper amid allegations of data manipulation. The authors still maintain that the two papers used different set of data though similar experiments. In November 2010, after an internal investigation by its ethics committee, the Indian Academy of Sciences banned Gopal Kundu from participating in their activities for three years.

      Well so people who have more time to go through all that manipulations will understand that there are clear evidence of data manipulations, but this scientist was never reprimanded for his actions, rather he is still doing science, enjoying all the facilities and training new students….can we expect such covering-up thing happening in US? no…..we have more retractions here cause we are honest and still value honesty and a legal and ethical system in place which works…..for the betterment of science….

      1. Dear Vigilante,

        I am coming from a Western country, and as a scientist I have worked in different countries in America and Europe. I guess it is true that in Western countries, there are many institutional structures that have been set-up to fight against scientific fraud, but honestly I do not think that Westerners are ‘more honest’ in their behaviour.

        When I have worked in a specific country in Europe (I do not want to target this country here) I was shocked by the GENERALISED lack of ethic toward scientific work and data, and this lack of ethic came from ‘pure Westerners’….

        So, may be, ‘not being honest’ is a universal human being phenotype…:)

      2. Lilly:

        “the concept of mentorship does not exist any more in our universities. PhD students and Postdoc are lab rats who have to produce results at any cost”

        I totally agree with your words….science policy makers should critically analyze these issues and change the overall structure/system…from changing parameters in grant system to publication and getting a job based on their publications, everything has to be changed…..its not an easy goal that can be be achieved in months, but unless taken seriously this whole plagiarism issue or misconduct in scientific research that too especially in biological sciences will keep continuing….

    2. There is much in what you say. University professors in the United States sometimes either don’t teach their students correct procedures or actively model incorrect procedures. Professors who don’t notice plagiarism even when it is plagiarism of a previous product of their own lab, professors who are not sufficiently well-read to notice plagiarism of the recent literature, professors who do each other favors by handing around the output of staffers to be used in a pinch for the professors’ reports on progress, professors who list an undergraduate who participated minimally or not at all as first author on a poster or paper in order to get credit for promoting undergraduate research, departments that punish people who report plagiarism instead of punishing the plagiarist — these things are wrong and the students who see how to get ahead in the system without putting in effort are encouraged to copy improper practices. One of the instances of plagiarism that I reported involved an American graduate student whose American father was a professor at an American university. There was no “cultural difference” as an excuse. It was outright plagiarism. But the department didn’t want to hear about it.

      True, institutions close their eyes to things when it is to their benefit. The foreign student whose government-sponsored tuition money keeps the institution afloat is unlikely to be put through a rigorous educational process. This is bad news for the foreign student, who is not getting what his/her country is paying for, unless his/her country is paying for a degree rather than an education. So many students these days, both foreign and domestic, are paying for a degree rather than an education, and the universities are willing to oblige because they need the money. So I agree that both the person and the institution are to blame.

      1. Just want to add here that unfortunately, the concept of mentorship does not exist any more in our universities. PhD students and Postdoc are lab rats who have to produce results at any cost, regardless of the way to obtain these results. The end justifies the means…and education is not done.

      2. To get away a little bit from pure plagiarism – what about self-plagiarism or its more palatable variety, tautology? I have been reviewing a review manuscript, that was not particularly original, had scores of self-references (to other reviews by the same authors) rather than citations of original work, was sloppy overall, and was to be published in the same year when the group had already 5 other reviews out on the very same topic! Of course the stuff was largely redundant. It is hard to get enough original ideas for one review, so that it reads a bit better than a simple recap of the literature, and here they were going onto the 6th in one year… No, it wasn’t cut and paste, a bit smarter than that, but the essentially the same stuff if not the same wording nonetheless. Not as bad as stealing other people work, but a part of the same culture when academic publishing takes a life of its own. As long as it’s published the goal is achieved, for there is no other. Oh yeah, and naturally despite my objections, the review was accepted and is floating somewheres in the interwebs…

    1. @Pymoladdict: I agree with your point. Professional review writers may repeat their opinions/paragraphs in their publicaitons. As you have experienced, editors tend to ignore reviewers comments. That is sad. Unfortunately, it may be difficult to stop any of these even with ORI or similar institutions.

      @Vigilante: I read the case you mentioned in India. Very interesting one. How can the former director of a big institute comes up and says there is no misconduct. This is unbelievable…

      1. You know, there is one nice editorial practice that can help in dealing with these and other cases, when editors ignore the opinion of one or even all of the reviewers, and in informing readers of the concerns that otherwise are swept under the rug. It is practiced by EMBO J and I believe some other publications, who put online the reviews together with the authors’ response. It is not fail-safe, but it can discourage some of the editors from ignoring the strong criticisms, and informs the readers what (hopefully) experts in the field thought about technical level, validity of the data, etc. I love this practice, it makes reviewing efforts a bit less futile and ephemeral…

  21. One of my colleagues had a chapter of his dissertation plagiarised by an article (in an ethics journal of all places!) The on-line version of that article was taken down, and the print journal published an apology. In that case, over half of the article came straight out of the dissertation. The author (who was already in trouble) lost his job.

    I would think in this case something similar should happen. The electronic article should at a minimum be taken down and replaced with one that properly quotes & cites your article at every location it was plagiarised in the original text, and an explanation of why this is different from the print version and an apology should appear in the acknowledgements of the new version, or as some other note, but ON THE PDF. Further, an apology *from the Journal* and a note that this occurred should appear in the next printed issue.

  22. Dear Mayra, I can tell you, for sure, the feeling of being plagiarized: YOU FEEL ROBBED (outraged). Someone comes into your home and takes everything away, but to be considered a brief note is left at te front door: THANK YOU, my silly friend!!. This is how you feel, I am suffering from this now and the editor´s journal is missed since January 24th. In this case a student took a whole research project (designed in 2003 further refined, 2006, presented at an International Congress, but needed to re-check because some experiemtal pitfall committed by the student) and published and signed along with some unknown person in the field……………………..And no one knows any, nor Editor´s team neither Ethical Committe of the University……everybody in love!!!!. Everybody happy and you………………..you ar trying to climb up to the surface and struggle with the hypocrisy of those who get their mouth full of ethics rule, but never apply them. These plagiarists are fully aware of the lack of compromise of the “Institutions” to comply with exemplary sentence against this misconducted praxis, and they take advantage of this and cynical laugh in your face. In the same way they dare to do plagiarism, they must know the action will be strong enough to deter them, even to dream on this, if any.

  23. Plagiarism is such a nasty act and plagiarists need to be exposed publically and punished. I feel your pain Marya and wish you well in your quest for fairness and a satisfactory solution.
    However, plagiarists, when exposed will fight back and their employers will adopt a “deny, deny, deny” position and try to sweep in all under the carpet. I had 100 lines of text and equations copied verbatim from one of my research articles into a US patent. Despite being a case of obvious plagiarism, this was also copyright infringement for the creation of a for-profit document. Following formal complaints, the so-called inventor’s institution investigated itself and the President of that university deemed the matter to be “acceptable plagiarism” and both documents still remain in the public domain. What a farce! I wish Marya more success – and I suspect she will thanks to the support and exposure via Retraction Watch.

  24. Inform the publisher of your intent to retain the services of an Intellectual Property Attorney. That usually speeds things up, considerably.

  25. PLAGIARISM IS A FORM OF THEFT and should be dealt with as such!

    When someone from the other side of the world steals credits (money) from your bank account (credit or debit card) in order to benefit himself, it is universally considered as theft.

    Likewise, plagiarising portions of someone’s work (i.e. using it without acknowledging the original source) is, de facto, stealing someone’s “credits”, as the perpetrator claims the credit with the clear intention to benefit from it.

    Just like thefts differ in terms of the size of the damage done – stealing an apple (one off event done from an individual) or stealing several billion dollars in multiple transactions done from organized group – plagiarism can vary as well.

    Consider the following examples:

    (i) Student has plagiarised a paragraph from his colleague (not published work) for the coming minor assessment. This is one off event (theft) in order to benefit (pass the assessment) and is presented to one person only (the supervisor);

    (ii) Speaker at a conference has plagiarised someone’s classification. This is one off event presented to many (20 – 200 people) in order to benefit (generate business);

    (iii) Faculty member has plagiarised the work of a non-academic author and has published it as his own. This is infinite theft (the paper is permanently available online), potentially presented to billions of people (present and future populations), and this is committed intentionally in order to generate more income for the perpetrator in future (as you are well aware the number of publications and citations is No. 1 criteria for getting grants).

    If we all are so outraged with the theft of millions/billions of dollars (although that none of these cases is infinite, i.e. perpetuate forever), WHY we are so passive and lenient towards plagiarism – the perpetual theft of credit for one’s work?!?

    Besides, plagiarism can have devastating emotional, psychological and economic effect not only on the victim – the plagiarised person — but on his/her family as well (just ask the victims). So, WHY we are so passive and lenient towards plagiarism?

    In a word, because there is a big silence about cases of plagiarism, because:

    1. There is a perception that this is something minor, because it did not happen to me and I do not hear much about it. This is so, because

    2. The victims of plagiarism do not speak out (like many victims of rape, and this is so for similar reasons), because

    3. There is strong alliance of forces opposing the publicity of cases of plagiarism.
    Apart from the perpetrators it includes their institutions and associations, the editors, the publishers, all of which consider any publicity as damaging for their reputation. Therefore, all of these parties act in a coherent and synergistic way in order to keep everything secret, to slap the victim for raising the issue, and ultimately to cover up the whole story.

    4. Last, but not least, because the perpetrators (together with editors, publishers, affiliated institutions) rely on our passiveness. There is a proverb that “Every wonder is for 3 days”. In other words, they think “Yes, there will be a little bit of embarrassment, a little bit of noise, but soon all of this will be forgotten and what remains will be my paper, which ultimately matters (to build up my reputation, career and income)”.

    Well, this is precisely what we should PREVENT by all means – keeping silence and forgetting quickly the cases of plagiarism. In this light I regard Retraction Watch as a very positive step into the right direction – zero tolerance to any plagiarism and especially in peer reviewed publications.

    So, Marya, while I do appreciate bringing up your case to Retraction Watch, I do not agree with your attitude of lenient treatment of the perpetrators. Such tolerance can only encourage plagiarism, as the perpetrators see that they can get away with it. You should fight for justice and I’ll support you in any possible way, just like, I hope, other bloggers will do the same and not only for you, but also for me and ANY BODY else in the same situation.

    I have worked on some suggestions for simple actions to limit/prevent plagiarism and, should Retraction Watch is interested, this can be a topic of a new post which can evolve with the unfolding discussions.

    1. This is very well put.

      I think that it may not be the niche of this blog, but there could be a post somewhere where people could list papers with similar content for the peruse of other readers. This is not an accusation of plagiarism however may very well help investigate and expose several cases.

      I have seen many investigations, and all I can say for sure is that plagiarism is considered a ‘minor fraud’ and on the other hand as such is among the easiest to yield a public retraction. Data fabrication is seen as the ‘mother of frauds’ and there is even stronger opposition against bringing it to public, including death threats. This has to end, if science is to remain a reliable area in the modern, high-throughput times.

  26. It’s cases like these that turn Copyright Transfer Agreements into toilet paper. The journal technically owns the copyright of the papers it has published. By allowing plagiarism to occur, the offending journal has technically violated this copyright, and the original publishers should pursue legal action.
    And I don’t care where you are from, every reputable journal forces the authors to sign a declaration that all of their work is original. At best, signing such a declaration for plagiarised work demonstrates incredible incompetence. And the penalties for breaking the agreement ? Apparently there are none.
    I take back my original statement. At least toilet paper has a use.

  27. Dear Rollo, Rafa and Pimoladdict,

    Thank you so much for pointing out the Copyright irregularities (let’s put it in a milder way) stemming from the cases of plagiarism.

    I have few examples as brilliant illustration about this, where identical figures appear (with different titles and without any attribution) in several publications, whereas each one of them claims the copyrights on the work (including the figures)!

    This is a copyright absurd – just as if several parties claim to have the original Mona Lisa!

    Since all publications involve the same authors, however, without any attribution to the earlier (original) work (figures), this is also illustration for self-plagiarism featured in the editorial titled “Recycling Is Not Always Good: The Dangers of Self-Plagiarism” and mentioned in Retraction Watch here:
    http://www.retractionwatch.com/2012/02/01/no-small-matter-acs-nano-journal-growing-alarmed-by-self-plagiarism/#more-6155

    For example, one of the implications is that, if I want to use the figures in my presentation at a conference and I get permission to use these figures from one publisher, technically I will infringe copyright of the other publishers, so, do I have to pay to all of them for using the same figures?!

    I have contacted some of the parties involved in order to sort out this complete copyright mess and this is what I got in return:

    — one of them (big and well-known publisher) did not answer at all, in spite of my several emails to three different recipients within the publisher;
    — another publisher answered that “there is no evidence to support the claim of copyright infringement”?!?
    — the original publisher was very reluctant to do anything and answered politely that this matter is non of my business ??
    — so, I did not bother to contact two other smaller publishers about this.
    — the institution of one of the authors, which specifically mentions as Research Misconduct: “Portraying one’s own work as original or novel without acknowledgement of prior publication” found absolutely no wrong doing and even accused me for harassing the author??

    And please note that we all, as tax payers, finance the whole venture: past, present, and future, as the main role of the publications is to gain more public grants down the track.

    One wonders is this a Greek comedy or tragedy?

    The story is rather big for a “Replay”-section, and I have provided the evidence to Ivan, so let’s hope that it will be featured as a separate post on Retraction Watch.

  28. I edit papers for international publication and am astonished by the incredibly mild response of the Journal and the Editor-in-Chief. Three words struck me as particularly distasteful: “unfortunate”, “undeliberate” and “mistake”. All three are clearly lies. The plagiarism was clearly “fortunate” from the authors’ perspective. Obviously, copying sentences complete with plagiarized references is not “undeliberate.” And this was clearly not a “mistake.” So, if the Editor-in-Chief lies, why not allow the authors to steal? What is the difference?

    I constantly combat this in my international writing. The last one I found, the person simply
    found someone else to edit their paper – a major researcher at Cambridge. How can I insist on a high standard, if the editors refuse to uphold these and lie to cover their own “mistakes”?

  29. Dear Mayra, I also know the feeling of being plagiarized. It’s so unfair. After you have been working hard someone comes and stole your work.
    I was victim of plagiarism and I’m going to talk and say it the world.
    In my case, they have made a plagiarism of all my work, even changed the title.
    I saw it today, and I’m so angry…
    My paper was published in March of 2011 by IATED, in a conference INTED 2011
    http://library.iated.org/view/LOPES2011TEA
    and I discovered that almost a year later, February of 2012, Javad Mehrabia and Masoumeh sadat Abtahi from IRAN published my paper in ELSIVIER as you can see in
    http://www.sciencedirect.com/science/article/pii/S1877042812025554

    Thanks so much, I think we have to denunciate this kind of situations.

  30. Colleagues, I have found today that our paper was plagiarized entirely. The problem, though, that the editor of the journal, where it came out, does not have even contact e-mail.

    I really do not want to spend my life calling to India.
    Do you know if there is any plagiarism database where could I report such an accident?
    I probably try to contact the institution of the author of this plagiarism,
    Any suggestions are welcome!

    for interested in details , see the info below “I have found the copy our paper published in Adv Exp Med Biol in entirely different journal (Journal of Cancer Research and Therapeutics) authored by somebody else (http://www.cancerjournal.net/article.asp?issn=0973-1482;year=2012;volume=8;issue=1;spage=3;epage=10;aulast=Tiwari). See citation of of our paper below.
    Microarrays for cancer diagnosis and classification.
    Perez-Diez A, Morgun A, Shulzhenko N.
    Adv Exp Med Biol. 2007;593:74-85. Review.
    PMID:17265718

    1. @andrey: on a lighter note, the author has acknowledged your help. Seriously, this is a big problem. Here is the email address of the Editor-in-Chief nagrajhuilgol(at)gmail(dot)com He is a radiation oncologist and he has contacted me earlier requesting to be on the editorial board. You may also want to check with one of the editorial board members based in the USA such as the one from NCI Radiation Research Program (Guess it is grant management section). Some of the retraction watch readers know this name well enough!! You can also contact the Executive Editor at [email protected] I am not sure whether you will be successful in getting any response from them but this should be pursued. This has become rampant over there – as I had experienced this in conferences in India where invited speakers outrightly use someoneelses’ data ; pictures from my own papers were shown by some prominent and award winning speakers in front of me during a conference. What a shame!!

      1. thanks a lot for tips! The e-mail to both editors has been sent and I will keep you posted here about results of this communication

        1. sure…let me know. moreover, the author of the review is a prolific review writer – i am sure he/she is doing the same thing for all the reviews.

          1. you are right . I have identified at least one another review in the same journal by this author with the same type of approach copy/citation/acknowledgements to real authors.
            I know one of TRUE authors of that one . So I have sent him information. Let’s see what happens . The comments of people here are not very encouraging -:)

          2. From other cases I have seen, the most effective and safest approach is for the plagiarized author directly contact the publisher, and making it clear he/she is outraged and that some action is demanded. Usually editors will try to soothe things up and cover, and if de complainer is not the main author, someone always tries to turn the situation against him.

    1. The best one can do is to expose the thief, and still this is very dangerous. Unfortunately there is no true policemen in science, and political influence is the ruling power. Eventually thieves achieve some power from stealing work from others and thus will remain untouched while chasing their opposers.

  31. In reply to Andrey, July 19, 2013 at 8:14 pm

    Andrey, I have been plagiarized and I know very well how does it feel like and what is ahead for you.

    No contact details of the editor is just the beginning. From my experience, after you get contact details, they start playing the “deaf & blind monkey” (i.e. ignoring the person who points out misconduct). This is the 1st stage of how editors/institutions handle these issues.
    2nd stage (if you persist) is playing the “surprised nanny” (i.e. the editor/institution tells you that they are surprised to hear such things and that they take all allegations very seriously, and they’ll ask you to provide as much evidence as you can, which in reality means “show us all your cards, so that we know how to trick you”).
    3rd stage is a letter telling you that after an “in-dept, detailed investigation” the investigating body found no evidence for any misconduct (even if the evidence provided by you contradicts to the institution’s own Framework to deal with misconduct).
    4th stage is that, should you persist, you’ll be threatened to be sued.
    5th stage is that they’ll try to ruin your carrier and life using all possible means.
    6th stage is that they might hire a hitman.

    Should you see my other comments @ Retraction Watch, you may learn a lot.

  32. I am quite surprised. I not only read the original article, but many of the replies. It seems to be a very common bevahior. I am an international reviewer since 2010 for a prestigious journal. I have to admit I didn’t suppose this problem were so high. I have not yet detected plagiarism (but I do have found changes of the source’s data!) I will be more carefull from now and on, specially because I myself have been plagiariazed (that’s why I started to search for info of scientific plagiarism).

    My case is a little weird. I did a research in a certain field (I don’t want to give details because it’s in process with the journal’s editors) for my degree thesis. Back then I asked my thesis supervisor to publish my results, but he insisted it was not publishable, that it was needed more results from another thesist. Since I was very unsecure, I thought it was worthless and I was not good at it. I changed drastically my field for the masters degree. Now I searched for similar papers to try to publish my results (in the case no one did a similar research these years). What a big suprise was to find my research published by MY THESIS SUPERVISOR, along with other 4 people!

    I am very angry and dissapointed. I searched the paper’s title in google and gave 10,200 results! It’s good to know I wasn’t that bad at that field, but he hadn’t to do that, I think and think and don’t understad why to do that, he knows my intention was to publish the results toguether. Anyway, I have written to the journal’s editors. I really hope they show they represent the most important journal in the field and do something, but after reading the replies… I’m waiting for the 6th stage and the hitman. If if happens, have it for sure I will make noise somewhere.

    By the way, the programs to detect plagiarism (which, as you said, we as reviewers don’t have access to) are VERY limited. They can only detect plagiarism with the words. It would have been useless in my case, even if had published a paper instead of simply the thesis, because you can easily change the words and keep the equations. They actually even changed a couple of terms to one main equation, so no program could ever detect plagiarism; it’s needed a person to realize (that’s why I sent to the editors the comparission step by step and showing them the little make-ups). Another case is, as one said, the figures.

    I only can finish this by saying that I wil do my best as reviewer to stop it when I catch it.

    1. Sue the guys, it is the most effective way out. Editors will ignore you as it means bigger trouble for them taking any action. Authors will threaten and retaliate, and many colleagues will shun you from raising your voice. They will all become more docile if you go to court — academics are horribly afraid of the court.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.