An article purporting to find that black children are at substantially increased risk for autism after early exposure to the measles-mumps-rubella vaccine has been shelved.
Although we don’t know if the events are related, the move comes amid claims that a CDC whistleblower has accused health officials of suppressing information about the link.
Not surprisingly, the prospect that the CDC has been sitting on evidence of an autism-vaccine connection for more than a decade has inflamed the community of activists wrongly convinced that such a link exists.
The paper, “Measles-mumps-rubella vaccination timing and autism among young african american boys: a reanalysis of CDC data,” was written by Brian Hooker, an engineer-turned-biologist and an active member of that community. It was submitted in April, accepted on August 5, and published on August 8.
Translational Neurodegeneration, which published the article earlier this month, has now removed it and posted the following notice:
This article has been removed from the public domain because of serious concerns about the validity of its conclusions. The journal and publisher believe that its continued availability may not be in the public interest. Definitive editorial action will be pending further investigation.
BioMed Central, which publishes the journal, declined to comment while the inquiry is ongoing. We’ve made the open access paper available here, and Science-Based Medicine has a thorough overview of the case. Here’s the abstract of the paper:
Background: A significant number of children diagnosed with autism spectrum disorder suffer a loss of previously-acquired skills, suggesting neurodegeneration or a type of progressive encephalopathy with an etiological basis occurring after birth. The purpose of this study is to investigate the effectof the age at which children got their first Measles-Mumps-Rubella (MMR) vaccine on autism incidence. This is a reanalysis of the data set, obtained from the U.S. Centers for Disease Control and Protection (CDC), used for the Destefano et al. 2004 publication on the timing of the first MMR vaccine and autism diagnoses.
Methods: The author embarked on the present study to evaluate whether a relationship exists between child age when the first MMR vaccine was administered among cases diagnosed with autism and controls born between 1986 through 1993 among school children in metropolitan Atlanta. The Pearson’s chi-squared method was used to assess relative risks of receiving an autism diagnosis within the total cohort as well as among different race and gender categories.
Results: When comparing cases and controls receiving their first MMR vaccine before and after 36 months of age, there was a statistically significant increase in autism cases specifically among African American males who received the first MMR prior to 36 months of age. Relative risks for males in general and African American males were 1.69 (p=0.0138) and 3.36 (p=0.0019), respectively. Additionally, African American males showed an odds ratio of 1.73 (p=0.0200) for autism cases in children receiving their first MMR vaccine prior to 24 months of age versus 24 months of age and thereafter.
Conclusions: The present study provides new epidemiologic evidence showing that African American males receiving the MMR vaccine prior to 24 months of age or 36 months of age are more likely to receive an autism diagnosis.
Last week, a video narrated by none other than Andrew Wakefield appeared on YouTube which associates the Hooker study with the infamous Tuskegee syphilis experiments. (Wakefield, by the way, cites his own thoroughly discredited work with pride.) It also includes muddy audio recordings of an alleged CDC whistleblower, vaccine researcher William Thompson, PhD, apparently coming clean about the cover-up. In the video, Hooker says Thompson has “expressed significant remorse” for “lying” about the data.
So far, we haven’t heard from Thompson about the matter. Perhaps he will stand by the tape, perhaps not. We tried to contact him by phone — at a (770) number, not a (404) area code, as Hooker says in the video, it’s worth noting — but couldn’t reach him.
We’ll update as we learn more.
Update, 5:45 pm Eastern, 8/27/14: Thompson has issued a statement that reads, in part:
I regret my co-authors and I omitted statistically significant information in our 2004 article published in the journal Pediatrics. The omitted data suggested that African American males who received the MMR vaccine before 36 months were at increased risk for autism. Decisions were made regarding which findings to report after the data was collected, and I believe that the final study protocol was not followed.
I want to be absolutely clear that I believe vaccines have saved and continue to save countless lives. I would never suggest that any parent avoid vaccinating children of any race. Vaccines prevent serious diseases, and the risks associated with their administration are vastly outweighed by their individual and societal benefits.
My concern has been the decision to omit relevant findings in a particular study for a particular sub-group for a particular vaccine. There have always been recognized risks for vaccination and I believe it is the responsibility of the CDC to properly convey the risks associated with the receipt of those vaccines.
I have had many discussions with Dr. Brian Hooker over the last 10 months regarding studies the CDC has carried out regarding vaccines and neurodevelopmental outcomes including autism spectrum disorders. I share his belief that CDC decision-making analyses should be transparent. I was not, however, aware that he was recording any of our conversations, nor was I given any choice regarding whether my name would be made public or my voice would be put on the Internet.
How did this paper pass peer review? Can we see the peer review reports?
Very good question!!! Looking at the statistical analysis of the data, I can’t believe that the reviewers accepted that the results supported the conclusions.
The editorial board for the journal in question. http://www.translationalneurodegeneration.com/about/edboard
https://drive.google.com/file/d/18QL4g_XZ9k1E6EakSDYEVjQAXzt-ay8a24zy8oO7pU580yxoq2ZsKqirc8poo1pln0ODGywyj0mIAWqa/edit?usp=sharing
I’m no statistician, but I’m am routinely asked to evaluate papers and can’t get past the study design and data treatment without wondering if there was ANY review. Academia is an interesting animal in that it consumes it’s own output. Read into that as one will.
I’ve read a article recently that you may be interested in….and is how i believe inaccurate information gets published. Here is the link
http://ottawacitizen.com/technology/science/respected-medical-journal-turns-to-dark-side
Excellent question indeed. I can’t believe any reviewers would have been ignorant of the problems of post-hoc subgroup analysis.
Hooker appears not even to have understood the original DeStefano et al. paper, as he claims that they were reporting RRs (an inapplicable measure, given the data set) rather than ORs. Indeed, it’s not even clear that he understands the difference in the first place, given that the abstract claims “an odds ratio of 1.73 (p=0.0200),” yet this figure is described as an RR in the text.
“Brian Hooker, an engineer-turned-biologist”
He trained as a biochemical engineer, so he was always a biologist. Written like that makes it look like he used to design bridges or something.
And he was never an epidemiologist.
In fact, his professional training was as a chemical engineer, not a “biochemical engineer.”
I just looked at that link, and his papers are related to plant chemicals and processes. So his biology education is mostly in botany. Which makes sense both universities listed are very big in agricultural research, which includes environmental impacts.
Still, this does not make him an epidemiologist nor fully qualified on human biology.
Interested in the fact that Dr. Hooker had a plant background, I ran a quick search on some data bases. I will be honest, there is one policy that Springer has of duplicating papers as books, as occurs with one of Hooker’s papers.
1) Journal article: Ziyu Dai, Brian S. Hooker, Ryan D. Quesenberry, Jianwei Gao. Expression of Trichoderma reesei exo-cellobiohydrolase I in transgenic tobacco leaves and calli. Applied Biochemistry and Biotechnology Spring 1999, Volume 79, Issue 1-3, pp 689-699
http://link.springer.com/article/10.1385/ABAB%3A79%3A1-3%3A689
9 citations
2) Ziyu Dai, Brian S. Hooker, Ryan D. Quesenberry, Jianwei Gao. Expression of Trichoderma reesei Exo-Cellobiohydrolase l in Transgenic Tobacco Leaves and Calli. Twentieth Symposium on Biotechnology for Fuels and Chemicals. Applied Biochemistry and Biotechnology 1999, pp 689-699
http://link.springer.com/chapter/10.1007/978-1-4612-1604-9_63
No citations.
I guess what I am trying to say is imagine all scientists had all their articles republished as book chapters, creating double the literature. I would be interested in knowing whay this is correct, or not problematic.
I wouldn’t dismiss Dr Hooker too speedily.
His collaboration with Andrew Wakefield is, of course, disastrous for him. But the question remains as to whether CDC whittled down a study by deploying a retrospective subgroup analysis so as to minimise a – possibly artifactual – statistical association between MMR and autism in African American boys.
This one has some way to run, methinks.
Which only occurs for vaccination between 24 and 31 months and does not reach statistical significance once multiple comparisons are taken into account.
It still was knowingly omitted, based on Thompson’s letter. Had they included the information in the first place, it’d be a different story.
No, “it” wasn’t, because “it” is an output from Hooker’s “analysis,” which didn’t employ conditional logistic regression.
I would. Dismiss him speedily, that is. He is an activist, not a statistician, or even an autism researcher. I’ve been seeing this “CDC whistle blower” story everywhere, without it being made clear that this is a tortured statistical re-analysis with a desire to find a link. Not a scientific way to approach the data. It was published in Translational Neurodegeneration, an open-access journal I am not familiar with. Hooker, the author, is on the board of “Focus Autism”, an advocacy group he founded after his son’s autism diagnosis. Who also paid the journal fees. And is responsible for spamming the CNN user-submitted “iReport” page.
TN does not currently have any APCs.
As someone appears to have found this simple observation unsatisfactory, I suppose I should document it:
Thanks for pointing that out, Narad. I stand by my dismissal of the paper’s content, however, it seems I was wrong to lump this journal in with other predatory pay-to-publish journals. http://www.sciencebasedmedicine.org/bad-science-journals/
Any “evidence” that relies on personal credentials, or lack thereof, is an ad hominem attack. The real evidence is what the data says, not who says it, or whether we like that person, or what side that person is on, or even if that person has a PhD or MD. Data is data; logic is logic. Who presents the data and the logic is largely irrelevant insofar as the validity of the overall argument goes.
Artifacts can work in both directions.
As my understanding there is some socio-economic bias in likelihood of seeking a diagnosis.
Another question is whether the children were ill when vaccinated in the first place. Often the only time children are brought to the doctor is when they are ill, for example, running a fever, ear ache, etc. Therefore, in order to meet vaccination targets, the child is often vaccinated at the same appointment, despite vaccination guidelines to the contrary. It may be that the vaccination + the presenting illness affect the child in some way as yet unreported.
The ACIP, the AAP and the CDC did not have recommendations in place to delay any vaccinations, for any child with a mild illness and a low grade fever. If such recommendations were in place, doctors and nurses would have delayed vaccinations for all children who fit that category you mentioned…not just African-American boys.
I’m just going to leave this here:
From the paper:
*Competing interests*
Dr. Hooker has been involved in vaccine/biologic litigation
*Acknowledgements*
Funding for this research was provided for by a grant from Focus Autism, Inc
From the Focus Autism website: The Focus Autism Foundation is dedicated to providing education, investigation, advocacy and science to the public that exposes the cause or causes of the autism epidemic and the rise of chronic illnesses in general with a specific focus on the role of vaccinations.
How is that different from vaccine company sponsored studies that show vaccines do not cause autism, or people who have clear financial interests in vaccine companies who publish with grants from these corporations?. I see the same conflict of interest. Don’t you? Maybe we are dismissing this too fast. It will not be the first time this has happened.
All of the studies I’ve seen were sponsored by unbiased government health agencies, not companies.
And public health officials have absolutely no connection of any kind to vaccine corporations, and hold no stock in those companies either…RRRRIGHT.
I’m definitely not a sympathizer with the vaccines -> autism crowd, but I’d be aware of potential conflicts of interest between corporations that manufacture vaccines and government proponents of such. The only question is exactly where the relationship falls on the spectrum from ‘potential conflicts of interest’ to ‘very corrupt and nefarious’.
Former head of CDC, Dr. Julie Gerberding, left to work for Merck, president of Vaccine Division.
Enoughsaid?
No, not enoughsaid. What are you trying to say?
Citation?
Which studies are they? Every study I have seen that refutes the MMR-autism link was academic or government sponsored.
William W. Thompson, through his attorneys, has issued a statement.
http://www.morganverkamp.com/august-27-2014-press-release-statement-of-william-w-thompson-ph-d-regarding-the-2004-article-examining-the-possibility-of-a-relationship-between-mmr-vaccine-and-autism/
“I have had many discussions with Dr. Brian Hooker over the last 10 months regarding studies the CDC has carried out regarding vaccines and neurodevelopmental outcomes including autism spectrum disorders. I share his belief that CDC decision-making analyses should be transparent. I was not, however, aware that he was recording any of our conversations, nor was I given any choice regarding whether my name would be made public or my voice would be put on the Internet.”
“William W. Thompson, through his attorneys…”: Lawyering-up in a matter related to science is generally not a good sign. Also, recording conversations without the other party’s consent is a criminal offense in California, where Hooker is from (his contact info in the paper is Simpson U. in Redding, CA). I predict we haven’t heard the last of this…
It is an offense but the police will need to receive a complaint from a party with standing – in this case William Thompson. Which he might not do.
Perhaps what is interesting is that Hooker felt the need to make the recording, in as much he believed Thompson feared repercussions from speaking out himself to the point he might even deny conversations.
Lawyering up? He has issued a statement refuting misrepresentation of his views by anti-vaccine advocates, which, if left unchallenged, could have jeapordised his career. As it is his judgment still looks poor, but nothing like as poor as Hooker’s quote-mining expedition made him look.
Amongst all the remarkable amount of blah blah blah that surrounds this story, one comment stands out to me: Dr. Thompson, referring to the 2004 Pediatrics article, saying ” I believe that the final study protocol was not followed.” I’m very curious to know what he meant by that.
Brian Hooker is irrelevant to the crux of the matter. Whether or not he has an anti-vaccine agenda, or if he does not have the proper training to properly interpret the data, or if he illegally recorded his conversations with William Thompson, or if he has any other nefarious characteristics you want to ascribe to him doesn’t matter. Kill the messenger if you want, but the important question is whether or not the orginal research actually indicated an increase in the risk for autism. In his statement yesterday, Thompson essentially said that it did:
“I regret that my coauthors and I omitted statistically significant information in our 2004 article published in the journal Pediatrics. The omitted data suggested that African American males who received the MMR vaccine before age 36 months were at increased risk for autism.”
The thing to do now is not to hoist Hooker’s head on a stake, but to have an open and honest evaluation of all the data to ascertain if the risk is real.
I’m hardly the best trained statistician in this blog’s readership, so I’ll defer to more qualified individuals.
But, that having been said, even I’m deeply suspicious that Hooker chose to use extremely simplified contingency statistics (it’s a little obtuse as to whether it’s chi-square or Fisher’s exact in his Methods). They’d be far more prone to false positives, since they’re less capable of isolating out confounding factors than the multiple logistic regression employed by the Distefano, 2004 reference.
And yeah – given if we ignore his clear CoI’s, it’s difficult to see why a truly unbiased investigator would choose to undertake a reanalysis of data with a far less robust statistical analysis.
Again, Hooker is irrelevant. It doesn’t matter one whit what methods Hooker used to reanalyze the data, or why he chose those methods. Screw Hooker, his methods, and his conclusions. But Thompson is not irrelevant. Just yesterday, he (Thompson, not Hooker) said that he and his fellow researchers “omitted statistically significant information” that “suggested….increased risk for autism”. Is Thompson telling the truth? And if he is, what are the implications?
I agree that Hooker’s biases are irrelevant here. It is Thompson who now either needs to retract the study or acknowledge and correct all its errors.
Post-hoc subgroup analysis is generally considered problematic in this kind of situation. It’s doubly problematic when the person doing the analysis has an ideological commitment to the outcome. That has a bearing on the degree of scrutiny any such paper should receive, and the detail that should be provided about the exact methods which are used.
What we have here is basically a “trust me, I ran the math” statement form someone who we cannot be expected to trust. At the very least it should have been made explicit precisely which tests were being used and why. There is, after all, no known biological reason why this one subgroup should have this different outcome when such a vast body of research from around the world – around 21 million subjects analysed to date, IIRC – has failed to document it in any whole population, the principal source for the claim under test is a paper known to have been fraudulent, and subsequent research has pinpointed genetic factors as the most likely determinant.
Given everything else we know about autism and the fact that African-American children are more likely to be economically disadvantaged and thus not see their doctor regularly, the more parsimonious explanation is as offered elsewhere in this thread:
1. Child sees the doc, they get a backlog of shots.
2. Either: parents see the hysteria and identifies the signs that were in fact always there; Or: the doc also notes possible signs of autism and flags for follow-up.
3. Diagnosis.
This is much more likely than this one subgroup having a susceptibility that does not exist in the whole population or in other subgroups analysed elsewhere, because both of the eventualities at 2 are known to happen whereas autism triggered by vaccines is not, and is in fact thought extremely unlikely to happen based on a large amount of data.
“There is, after all, no known biological reason why this one subgroup should have this different outcome when such a vast body of research from around the world – around 21 million subjects analysed to date, IIRC – has failed to document it in any whole population”
Except that development runs by the clock and specific toxins at key stages can generate a particular phenotype. In the case of Autism there are five known causative agents including Thalidomide and Valproate, which only have effects in very narrow developmental windows; and again not in all cases.
Is this from here? The thalidomide angle is . “Known causative agent” is a bit of a stretch.
Some recent thalidomide references
Maternal administration of thalidomide or valproic acid causes abnormal serotonergic neurons in the offspring: implication for pathogenesis of autism.
Int J Dev Neurosci. 2005;23:287–97
Hallene KL et al.
Prenatal exposure to thalidomide, altered vasculogenesis, and CNS malformations.
Neuroscience. 2006;142:267-83
Rout UK, Clausen P.
Common increase of GATA-3 level in PC-12 cells by three teratogens causing autism spectrum disorders.
Neurosci Res. 2009;64:162–9
Ema M, et al.,
Fetal malformations and early embryonic gene expression in cynomologus monkeys maternally exposed to thalidomide.
Reprod Toxicol. 2010;29:49-56
Dufour-Rainfray D, Vourc’h P, Tourlet S, Guilloteau D, Chalon S & Andres CR.
Fetal exposure to teratogens: evidence of genes involved in autism.
Neurosci Biobehav Rev. 2011;35:1254-65
A review is here:-
Current Developmental Disorders Reports
March 2014, Volume 1, Issue 1, pp 8-19
Environmental Factors in the Onset of Autism Spectrum Disorder
Antonio M. Persico & Sara Merelli
http://link.springer.com/article/10.1007/s40474-013-0002-2
Thanks. However, this appears to all be in vitro, rodent, and modeling work. My point was that the only actual human signal seems to come from Strömland et al.; all the rest follows from this, as far as I can tell. Thus, “known causative agent” struck me as a reach.
In fact, if we take your Hallene et al. entry, the conclusion states that “the most prominent teratogenic effect of THAL is phocomelia, but increasing evidence suggests that when taken at later stages, THAL causes a spectrum of neurological effects including autism and seizure disorders” (emphasis added).
One thing that fell out of looking at these references that I find interesting is that lenalidomide turns out to have been proposed as a treatment.
There is no good animal model of ASD. The people looking at effects of Thalidomide on people were not exactly looking for ASD-like symptoms. The data is what it is, as is the dating.
I will simply note that I was not the one to invoke animal models in the first place, and I would reiterate that the N = 100 convenience sample of Strömland et al. seems to be the entirety of the signal.
Thank you for your references. I am not medically trained, but I find it interesting that infants must be given 3- and 4-fold the vaccine serum dose compared to a 5-year-old to provoke the same immune response. Personally I chose to delay vaccination for my second child until 3 years old, after my first child (infant, then) had behavior changes, high fever and other problems with early vaccinations. I think that the anti-vaccine and the CDC and other pro-vaccine advocates would both benefit from removing emotions from this issue and examining the data: perhaps together they could conclude that we need to continue vaccinating but that we are: administering too many combined vaccines-so we cannot determine the source of adverse reactions-; administering many vaccines too early in life; administering vaccines at too high a dosage; administering vaccines at the non-optimal age; not measuring effective immunity after vaccination; and administering vaccines without sufficient study.
The issue is not vaccines in the US, Careful Parent. It’s the delivery system. Elsewhere in this comments area it was noted that vaccines are being administered in spite of the child being ill. That is a blatant violation of the doctor’s obligations to protect their patients as most vaccines clearly warn against such a thing. If a child is healthy there is no reason not to vaccinate within the first two years of life.
There are biological reasons why a vaccine can and triggers a lot of side effects, ASD included. There are several papers about proteoms overlapping; following these overlapping motifs we could even anticipate the side effects.
http://www.ncbi.nlm.nih.gov/pubmed/?term=Kanduc+D
https://www.bioscience.org/2012/v4s/af/341/fulltext.php?bframe=2.htm
http://www.ncbi.nlm.nih.gov/pubmed/18582510
Exactly my point. Hooker, like Wakefield his new buddy, had made up their mind, with lucrative litigation in mind (Wakefield was retained, Hooker a plaintiff), before they carried out their “research”.
The question of interest is whether CDC, likewise, is driven to prove a prior conviction. It’s worth noticing that a lot of their vaccine papers are not authored by lonesome grunts down the end of some corridor someplace. They are quite senior management of the agency that runs the vaccine program.
As someone with no dog in the race as to whether vaccines may or may not do, or cause, anything, that’s what interests me: the process, not the outcome.
Statistically it is a mess. They claim to use Pearsons Chi-square but then the tables have Fishers Exact test in the title. They should produce similar results but it is sloppy. What is a problem is that it is a matched design, so neither statistic is correct, and they should be using the Conditional Logistic Regression that they mention. It isn’t difficult, so you wonder why it wasn’t used, and that always makes you suspicious that it didn’t give the results that were wanted.
Anyway the feeling with these papers is always that even if there is a relationship found, it is due to something else going on that causes correlation between autism diagnosis and vaccination. The parents who don’t vaccinate their children are probably quite different from this that do, and that factor may result in autism or a diagnosis of autism.
And this invites the question: if Hooker got the data, so can anyone else, so when will the reality-based community be doing the proper statistical analysis?
Obtaining the data is not straightforward. From the CDC website
http://www.cdc.gov/ncbddd/developmentaldisabilities/maddsp-data-sets.html
“Data Sets
Measles mumps rubella / autism study
MADDSP 1991-1996
This data set has been designated as a “restricted access” data set because concerns about confidentiality of study subjects. The Developmental Disabilities Branch (DDB) will only make this data set available under restricted conditions.
To request access to the data, interested researchers must complete a scientific proposal and send it to CDC. The proposal should clearly state which data sets are being requested, how the researcher plans to analyze them, and which hypothesis(es) are being tested. ”
So it’s not like any of us can just download the data and have a poke around. I don’t know what level of stringency the CDC uses for “a scientific proposal” but judging by the Hooker paper the standards aren’t too high. Nonetheless there is a fair amount of work to do to submit a proposal fulfilling their requirements.
All that said, it is unclear that these data can be used to adequately study any link between vaccination and autism, as the sample size is small enough to yield the ambiguous outcomes exploited in this paper. Decent epidemiologists will need to devise a better study plan, involving considerably more cases than available in this data set. Further analysis of this data set will just repeatedly reveal this quirk in this small subset of the dataset, a subset corresponding to a subpopulation already suffering enough in the history of this country. In essence we’ve already “peeked” at this data, so its value is diminished for assessing this question.
Many commenters, as well as the Science-Based Medicine thorough overview linked above, have reasonably deconstructed the weak nature of the Hooker effort. From the end of the conclusions section: “However, consideration should be made in the current United States vaccination schedule for genetic subpopulations that may be associated with vaccine adverse events.” Thankfully there are no additional co-authors whose reputations have to get dragged through the mud because of specious suggestions such as this.
“Additional research is required to better understand the relationship between MMR exposure and autism in African American males.” This is the only honest sentence in the Hooker paper. The evidence in the Hooker paper does nothing to help better understand the relationship between MMR exposure and autism in African American males. This paper will only bring further risk to this group, in as much as people not well trained in assessment of statistical evidence fail to understand that nothing in this paper addresses the specious suggestion that somehow vaccination is yielding the adverse event of autism, and choose not to vaccinate their children.
I am not looking forward to the CDCs future MMR publications showing how many excess children have died due to lack of vaccination just because people like to jump on these conspiracy-based anti-vaccine bandwagons. Shame on BS Hooker.
It now has an expression of concern. Somebody seems to have forgotten to actually restore it, though.
How come no one is reporting that I believe BioMed Central’s take-down of Dr. Hooker’s article is a violation of the publisher’s own policies on article removal? That states:
The two notices have read:
and
You can see what I believe is the contradiction between BMC’s notices and BMC’s policies for yourself in screenshots posted here:
http://www.autisminvestigated.com/biomed-central-brian-hooker/
The article was taken down under the pretense that it might not be objective, yet it is the ongoing removal of the article and subsequent investigation of which that is clearly not objective. The whistleblower’s statement admitting the relationship found in Dr. Hooker’s reanalysis was also found but omitted from the original study’s reported results should clear the air of any lingering concerns about the objectivity of Dr. Hooker’s paper.
I think the issues of (1) the validity of the paper, (2) what BMC is investigating, and (3) BMC’s apparently whimsical attitude toward their own policy are separate matters.
it’s been 4 months since William thompson came out with that statement. Anyone know of any developments?
Sigh. The Hooker paper has been republished by the Journal of American Physicians and Surgeons.
Press release:
https://www.apnews.com/8a3c81fd61037a24b54a6004d3533926?fbclid=IwAR1OuWdpC7s6GRY5Y9CONG8AlATy2CeHYxx39Mx_AgoC-hRBmm37pR8r_Pc
Journal URL
https://www.jpands.org/jpands2304.htm