From: Linda Rosa ([email protected])
Date: Sun Sep 05 1999 - 07:40:08 MST
In the current issue of Skeptic is an article with the following highly
interesting and pertinent observations:
"...In all, publication companies in the United States print approximately
16,000 different peer-reviewed journals.
"Because of this mind-boggling number a hierarchy of journals has developed
over time in each specialty area of medicine. The most prestigious and
selective medical journal is The New England Journal of Medicine (NEJM),
established in 1812 by the Massachusetts Medical Society. Only one out of
10 articles submitted survives the NEJM peer review process. NEJM editors
send articles submitted to anonymous reviewers--experts in the fields
covered by the articles to be considered for publication. These highly
responsible referees decide whether to reject, accept with recommended
revision, or (rarely) accept without revision all articles that clear the
editors' primary review.
"A close second to the NEJM is the Journal of the American Medical
Association (JAMA). The American Medical Association has published JAMA
since 1883. JAMA also accepts about one in 10 articles submitted. For the
fields of general medicine, these two journals have attained the first tier
status in the hierarchy of medical journal reporting.
"Journal editors today require that all authors subject their data to
sophisticated statistical evaluation. One of the referees evaluating the
article must be a statistician who must agree with the analysis of the
data. First tiered medical journals require that authors follow set
formats. Editors insist on pertinent references supporting views the
authors present. Authors must disclose their sources of financial
support...
"Lower tiered journals tend to accept articles which meet with ever lower
standards of quality review. The lowest tiered journals may accept and
publish nearly all articles submitted. Obviously, one needs to scrutinize
and be more wary of conclusions tabulated in the lower tiered journals.
Unpublished internet announcements have the smallest amount of peer review
and accordingly should be read with the greatest skepticism. They rarely
have reviews of materials and methods or statistical evaluations of the
data. The quality of internet study conclusions is highly problematic."
--Harry K. Ziel, MD, "How to Evaluate New Medical Discoveries," Skeptic,
Fall 1999, 7(2):40-41.
-------------------------------------
As someone who has published in JAMA, I can attest to the rigor and
painstaking that the "first tier" requires of its authors. We went through
_three_ passes of editorial/peer review, had some lengthy phone calls
between the authors, had to call in some outside help on the statistical
analysis, and had to word-smith practically every one of our 6,000 words
(most especially the conclusions). At the last, it still took a vote of
the 15-member editorial board (I'm told it was unanimous) to allow
publication. It was ten months of this to see it finally in print.
(Even at that, two small errors--neither having any bearing on the analysis
or conclusions--crept in. One was about what distance from the body TTPs
wave their hands [we said 5-10 cm--it's 5-15 cm]. The other one was about
the probability of getting 8 out of 10 right by chance [we said p=0.044,
technically correct, but we should have included the changes of getting 9
or 10 out of ten right for p=0.055].)
Its grueling nature aside, for a non-academic to experience such rigeur
first-hand gives one a true appreciation for the requirements of scientific
conclusions and the integrity of the process and people involved, at least
at the uppermost levels. One can also appreciate why there was so much
fuss made about an eleven-year-old getting results published there! Last
but not least, we all learned much about science, scientists and
notoriety.
Dr. Barrett--Steve--has earned our eternal gratitude, as well as respect,
for what he did in the project as co-author. We'd be honored to work with
him again, if I ever again have the opportunity. By the way, in case
anyone would like to know, beneath his gruff, no-nonsense quackbuster
exterior is a...gruff, no-nonsense scientist. And he doesn't just hate vitamin
supplements; he also hates sentences with commas in them.
Linda and I have since been reviewers for a non-first tier medical journal
and can
appreciate the clout that someone has when in that role. It sobers one to
realize that, for the sake of scientific knowledge, this power has to be
exercised responsibly.
As Dr. Ziel has said, as you go down the tiers, you meet ever lower
standards of review. I concur with Dr. Uthman that readers have some
responsibility as well. Even so, some efficiencies can be had. Generally,
the readers of NEJM and JAMA can presume trustworthiness from publication
there and have to spend very little time checking sources or reasoning. At
the same time, readers of nursing journals generally should trust nothing
of what is found there; they are at the very bottom of the peer-review
quality heap (alongside Social Text that was duped by physicist Alan Sokal).
As a relatively new publication, ScRAM still has to prove its worth. The
fact that it published plagiarized work in its first issue is not an
auspicious start. That the plagiarer remains on its editorial board is not
a good thing. That ScRAM's publisher, Prometheus, is having that plagiarer
edit a book is a bad thing. If at length ScRAM actually publishes Long's
study, it may prove an irretrievable embarrassment for the publication,
indicating that its review process is suspect. I have not yet read the
finished article, but no amount of editing can make a silk's purse out of
the sow's ear that was Long's "experiment" in the first place.
-- Larry Sarner & Linda Rosa
------------------------------------------------------------------------
MyPoints-Free Rewards When You're Online.
Start with up to 150 Points for joining!
http://clickhere.egroups.com/click/805
eGroups.com home: http://www.egroups.com/group/beliefs
http://www.egroups.com - Simplifying group communications
This archive was generated by hypermail 2.1.7 : Tue Mar 18 2003 - 12:49:47 MST