Posted by: FLPatriot | September 9, 2009

Is peer review important?

I am always fascinated by the need of the Darwinian community to have the approval of their peers in order to justify their views. Many times in debates they fall back on the need for peer reviewed articles to measure the legitimacy of an idea. If you take this stand point, would you accept an article if it were published in a peer reviewed journal that allows articles on creation science or articles pointing to the legitimacy of Intelligent Design?

Does the publishing of articles that you do not agree with remove the authority of peer review from a journal? Who has to give their stamp of approval on a journal to give it the gold seal of legitimacy? Does it matter if an article is printed in a scientific journal if it is published in a book?


Responses

  1. Here you are just displaying your ignorance of what the peer review process is. It’s not a popularity contest and it’s not a self esteem building exercise.

    This is an excellent, easy to understand explanation of peer review.

  2. So your answer to the questions are what again?

  3. It occurs to me that you may not click on that, so:

    When a researcher, or team of researchers, finishes a stage of work, they usually write
    a paper presenting their methods, findings and conclusions. They then send the paper
    to a scientific journal to be considered for publication.
    If the journal’s editor thinks it is suitable for their journal they send the paper
    to other scientists who research and publish in the same field asking them to:
    • Comment on its validity – are the research results credible;
    are the design and methodology appropriate?
    • Judge the significance – is it an important finding?
    • Determine its originality – are the results new?
    Does the paper refer properly to work done by others?
    • Give an opinion as to whether the paper should be published,
    improved or rejected (usually to be submitted elsewhere).
    This process is called peer review. The scientists (peers)
    assessing the papers are called referees or reviewers.

    Peer review is an essential dividing line
    for judging what is scientific and what is
    speculation and opinion. Most scientists
    make a careful distinction between their
    peer-reviewed findings and their more
    general opinions.

  4. You said, “I am always fascinated by the need of the Darwinian community to have the approval of their peers in order to justify their views.”

    Are you also fascinated by the need of medical researchers to have the approval of their peers in order to justify their views? How would you like to visit a doctor who wanted to perform a procedure on you that had not been vetted by other practitioners? What would you think about ingesting a chemical concocted in a lab and never tested on anything?

    Do you believe the scientific method has any validity at all? Or do you merely disapprove of the scientific method when the findings contradict your religious beliefs?

  5. You also asked, “If you take this stand point, would you accept an article if it were published in a peer reviewed journal that allows articles on creation science or articles pointing to the legitimacy of Intelligent Design?”

    Let me ask you another question. Where are the peer-reviewed journals on creation science or intelligent design? Is there any actual science to back up the beliefs?

    I would think that you should be excited that the “evolutionists” are publishing their findings in journals. They are laying out the evidence for you to criticize and attack. There’s your opening. Go for it. Start reading those journals and publishing your arguments. If they have any merit, people will notice.

    In the marketplace of ideas, the best ones will win out.

    Certainly, you will win me over if you have something worthwhile to offer. My criteria is the theory that best explains the evidence. I would hope that that is your criteria, too. If not, I would appreciate an explanation of what criteria you use to determine whether an idea is right or wrong.

  6. (Thaddeus, use the Google… They’re out there. I’m including a link to one article below…)

    Mike, when you say “I am always fascinated by the need of the Darwinian community to have the approval of their peers in order to justify their views.” I get a little confused as to just what you mean. Let me try to explain by asking, more generally: How do you know how credible, intelligent or reliable _anything_ is? Forget science altogether for a moment…

    It should come as no surprise that even outside the realm of science, we put a premium on the insight from smart people who have spent a great deal more time studying and considering the issue at hand. You do it with your accountant, you do it with your mechanic and you probably do it with your pastor as well. While most of us realize that they don’t know everything either, if we asked 100 lawyers a legal question and 99 agree – it’s probably good advice – or at least good enough to imply that you should seriously re-consider why you might think it’s not. That doesn’t mean that people are only looking for the approval of their peers – the vast majority of them just recognize that they didn’t go to pre-law, then law school, then graduate, pass the bar, etc.

    We use peer review in virtually everything – even your church has large meetings and conferences to discuss complex things because it’s worthwhile to make sure that we’ve thought something. In the sciences especially, we’re very careful about this. With regard to peer journals, contesting views and disputes over data or interpretation are published in peer-reviewed journals all the time – in fact – that’s their purpose. It’s all about vetting the information and ideas in front of a lot of really well-informed, very smart people. When something has been run through the gauntlets of peer reviews, we can gain confidence in the same way that you might be more convinced after getting a second medical opion that confirmed the first.

    Now – I think that your question is more specifically that by definition ID, and more specifically creationism posits larger positions/topics that are more radically outside the scientific mainstream. Considering them often requires multi-domain acceptance of things that science generally does not accept – that geologic features, for example, were created by a single, global, worldwide flood. This seemingly creates a situation where in order to be taken seriously one needs to be published in a peer-reviewed journal, but in order to get published they need to be taken seriously. In other words – it’s appears to be no-win scenario. I understand your frustration there – it’s tough trying to make that case…

    Let me point something out however – there are at least some real scientists in both camps (ID and creationism) and it is possible to publish evidence for and disputes or against something in a scientific way that deals with specifics without reference to the bible or God. If you want to build a big case – you do it one step at a time. But where are those? Have any of them even been submitted? For example, why would it not be possible – if they’re so sure – to publish on something like this: http://creation.com/stickleback-evolution ? I think that mostly – that’s what many of us are referring to – but I don’t like to speak for others.

    When you ask, “Would you accept an article if it were published in a peer reviewed journal that allows articles on creation science or articles pointing to the legitimacy of Intelligent Design?” Yes and no. I would certainly be “more willing to look at something” like that for the simple fact that peer review always helps if for no other reason than that it helps clarrify the larger picture and see if there is any agreement, coherence – or just as importantly, is there ever any critical disagreement on factual (ie, non-faith based) basises. It lets me know that people are actually considering _something_. Believe it or not, things are “considered” before they make it into peer reviewed journals – its just got to work its way through the process of rigorous questioning – but I think that many people look to scientists to ultimately answer questions of science is quite different than just looking to our peers for approval – it’s really looking to them for expertise and insight.

  7. Brian, I knew I could count on your for a well thought out comment, and a long one🙂 .

    I do not say that getting information checked out by people that specialize in a field is not needed. I do not oppose a peer review system. I only commented that fascinates me that some will point to the current peer review process and say that unless your information is accepted by this process, it does not count.

    I think the current peer review process is broken and corrupt. Much like most things in this world it is being driven by money and not by the need to further knowledge.

    It is my opinion that if a journal where to print any article that showed the possibility of intelligent design it would be boycotted. If this is true, what are the chances a journal would take the chance of this happening and in the desire to save themselves will reject any article of this sort. This scenario reduces the credibility of peer review, which is dissapointing.

    I have read several accounts by scientists of how the peer review process is used to exclude any cutting edge advancements that goes against the norm. Yes, I know advancements in science are happening but scientists are being forced to make those advancements with the commonly accepted processes.

    But the main question of my post is this, Would you accept a peer reviewed journal as authoritative if it published articles supporting intelligent design?

  8. You asked, “Would you accept a peer reviewed journal as authoritative if it published articles supporting intelligent design?”

    It would depend on the type of article it allowed. Imagine an article saying, “No one has an explanation for X, therefore God must be the explanation.” This should not be in a science journal because “god” is not a scientific concept. It can’t be measured, observed or inferred from the measurable/observable.

    Imagine a different type of article that brings up some ‘standard’ ID arguments that have already been shot down. This article should not be admitted in a scientific journal because it isn’t original. (See Personal Failure’s comments above.)

    I would not consider a journal publishing these types of articles authoritative.

    However, if an article was published that didn’t refer to the concept of ‘god’ as an explanation, and was discussing something original, and was posing a problem for the theory of evolution then it should be published somewhere. If it met the standards of a particular journal in terms of quality and depth of research, why shouldn’t it be published?

    But, then, that’s just science and not specifically ‘intelligent design’. Even if the author has a particular religious perspective, that doesn’t negate the work or its relevance. And there are plenty of scientists who believe in God who do get their work published because it meets the standards for publication.

    So, to take this a little further, what do you mean by an “article supporting intelligent design”? I am not aware of an actual working theory of intelligent design. Not in the sense that the theory is expressed in a way that makes it scientific — i.e., predictive, testable, etc.

    As I understand it, the best that intelligent design has going for it are some skeptical scientists like Behe who present some evidence that has been challenging for evolution to specifically account for. (It should also be noted that there are people who do take Behe’s arguments into account and do respond with counter-critiques that incorporate evidence with plausible theories of how ‘irreducible complexity’ is not irreducible. For example, read “Only a Theory: Evolution and the Battle for America’s Soul” by Kenneth R. Miller. This should defeat the notion that Behe’s ideas are not being considered by the scientific community.)

  9. BTW, I decided to take Brian’s suggestion. I used google to look up “peer reviewed intelligent design”. One of the first articles to come up in the results can be found here:

    http://www.talkorigins.org/indexcc/CI/CI001_4.html

    This article addresses your question very directly. It is a very short read. Please consider this article when responding to my question posed above: “…what do you mean by an ‘article supporting intelligent design’?”

  10. Thaddeus, I have to say that I kind of disagree that the talkorigins article that you cited answers the really question that he asked – or at least the one that I think that he is trying to ask… I understand your point there, but also the nature of his question and frustration.

    The larger question – I think – is what are you willing to consider reading and honestly thinking about at all (likewise – what aren’t you willing to honestly consider) – and why? For example, Thaddeus – would you read Behe’s Darwin’s Black Box with an open mind? (That’s kind of rhetorical as I know from another conversation that you are waiting to get it). I know that you’ve already read things which you think debunk it, but if you get one of the special updated editions, he responds to some of those… So — imagine that you read it and t makes sense, but disagrees with a lot of scientists then why wouldn’t you believe it? Well… I think that you wouldn’t necessarily “believe it” – wouldn’t you at least be demanding a good debate and perhaps suspicious of the review process and demanding that it be peer reviewed in an honest and fair way?

    For many people – I am going to have to say no. Many people have to operate through a filter – for reasons I’ve already explained. While to some this may seem “weak” – I actually believe that it is to their credit, that they simply admit when they are out of their league in terms of the background and are unwilling to put in the time it would take to seriously consider the question with the rigor it deserves and thus defer to the experts – or at least those dedicated and interested enough to take the time to weigh the options….

    Which brings up a related point… Mike, have you read Malcom Gladwell’s book The Tipping Point? (I would highly recommend literally every single one of his books – especially Outliers which I thought was just great – it has nothing to do with Science or ID or Creationism btw). If not – Let me say that I think that it explains exactly what ID and creationism need to do if it is to be taken seriously: You need some mavens. You can read a bare minimum about mavens here http://en.wikipedia.org/wiki/The_Tipping_Point though it certainly doesn’t do the book justice. Here’s the gist – mavens are the types of people that for, sometimes profoundly weird reasons, are incredibly interested in something and willing to spend hour upon hour researching, thinking, questioning, improving, writing, etc. about a topic that most of us don’t even think about… Until we have a question – then they’re the ones that we turn to for advice or help in understanding the stuff that we filter. The thing about mavens is that they can actually serve as kind of a peer review in the market of idea and ultimately lend some credibility to one or more of your arguments, thereby allowing good ideas to bubble progressively to the top. If your maven can say “you know… there are a few things in there that really are worth discussing” then other people are more likely to consider it.

    Now… To answer Mike’s question from my personal perspective, here is my response: I’m an anomaly. It’s not just about science for me – I like to think, read, research and consider. If I see someone holding a view very much outside my own I can see that there are only a few possibilities: Maybe I am right and this person is wrong. If that’s the case, why would I not want to help my brother understand? However, maybe they are right and I am wrong. If that’s the case, why would I not want to understand myself? More likely, however, neither of us really understand as well as we think we do and there’s a great opportunity for both of us to learn… If we’re really interested in understanding there’s really nothing to be afraid of. Indeed, there are some questions for which even people like me may eventually have to defer to the experts – but at least in the end I’ll have a much better idea as to who they are – and why.

  11. Haha – I’m not sure that my answer answers your question either🙂 Let’s see if this works: If by “authoritative” you mean thoroughly vetted – it’s as good as those vetting it. Even things that make it through scientific peer review are not scientific “law” – they could still be wrong… We just have much higher confidence in them.

    Thus, if an ID article passes muster and is thoroughly vetted in a big science journal, that’s how I would consider it. If it is vetted by philosophers or mathematicians – that’s how I would consider it. However – even if it is vetted by a smart friend – that’s how I would consider it. It doesn’t mean that I totally discount it – it just provides some measure of reliability and confidence.

  12. Thaddes,

    You say, ”

    It would depend on the type of article it allowed. Imagine an article saying, “No one has an explanation for X, therefore God must be the explanation.”

    In naturalism, they use this method in which you described only the conclusion is different, here is an example…

    There is a dispute over gravity, many scientists have now claimed there is not enough visible matter in the Universe to hold everything together…

    Their answer to the problem: “dark matter” or invisible matter is the cause and exists in 95 percent of the Universe. The majority agreed on the assumption. So in other words it’s supposedly alright to claim something invisible as the cause as long as it’s considered “natural” and to get the backing of the peers.

    One gentleman argued to me that dark matter will someday be detected. Well how does he know this?

    From a creationist point of view, while we believe God is the cause of original information, we would like to explore the idea perhaps we do not know as much about visible matter as we think. The assumption about dark matter is faith based in a godless way.

    There is a misconception you guys love to accuse creationists with and that is since we believe that God created all things, we don’t have any desire to learn how things work which is needed in science. Some creationists are interested in how cars work, others how computers work, and so on. Creationists are not against science, it drives science.

  13. Dark matter is postulated to fill in a void, as you point out. The difference between that and “god” is that dark matter forms the basis of new investigations in physics. Dark matter is postulated to make up the difference in the gravity necessary to hold the universe together. But, it isn’t accepted as fact because it hasn’t been verified.

    Someday dark matter might be detected. That would validate the scientific process. But, if it isn’t detected, then the dark matter hypothesis is just that. An unproven hypothesis.

    The “god” conjecture cannot be proven at all. So it should be a concept that science ignores.

  14. Thaddeus,

    If what you say cannot be proven at all with the means we have today, indirectly or directly, then how can you say it’s “conjecture?”

    Here is another example of using what you call, “conjecture” in evolution. Remember those sickle-shaped claws on the feet of Velociraptor that terrified visitors in the Jurassic Park movies? Phil Manning (U of Manchester) says the claws were not used for prey but to climb trees…

    They should avoid jumping to conclusions (conjecture) about extinct animals we cannot observe. There are no velociraptors around to see how they used those claws.

  15. >”If what you say cannot be proven at all with the means we have today, indirectly or directly, then how can you say it’s ‘conjecture?'”

    Conjecture means speculation. If something cannot be proven, then you are left with speculation.

    Good point, though, about the claws of the velociraptor. I’m sure there is a fair amount of conjecture involved in paleontology.

  16. Michael,

    First, let me say that I don’t think that anyone has intended to make – a blanket statement that creationists have no desire to learn how things work. As you rightly point out many hold jobs where that is required and just to be a human being in modern society you have to learn all kinds of stuff. Many are, in fact, very smart people. It’s actually something considerably more specific that these people are referring to so your ability to present it otherwise seems to imply a misunderstanding that I hope this will help clarify.

    Let me explain what I see as the difference between the two and you tell me where you think I am mistaken. You state (I think) that given two articles which read:

    A) “No one has an explanation for X, therefore dark matter must be the explanation.”

    B) “No one has an explanation for X, therefore God must be the explanation.”

    The two are essentially the same(in your opinion) in that they both require faith in something which is “invisible”. By “invisible” I think you mean scientifically observable – after all – quite a lot of virtually indisputable science deals with what people would historically call “invisible”. I can see why you would say that – in that simple a form – it would appear hard to say that they aren’t – but let me try to explain. First, one posits a rational cause – not just another variable. Consider the phrasing of three statements about something we had some historical trouble with before the scientific method:

    A) “No one has an explanation for where maggots come from, but we observe them to originate, seemingly without cause on meat, so spontaneous generation, by which I mean when a thing comes into existence with no parentage is involved, from rotting meat is my hypothesis.”

    B) “No one has an explanation for where maggots come from, but we observe them to originate seemingly without cause from meat, so God must be creating them – for some reason – from rotting meat.”

    C) “No one has an explanation for where maggots come from, but we observe them to originate, seemingly without cause from meat, so God creates them from meat is my hypothesis.”

    Version (B) is an idea defended by Augustine of Hippo (St. Augustine) as he claims that it is supported by the literal text of Genesis. A few examples – Man was formed from the dust of the ground (the meat) and the seas were told to bring forth life (continuous generation post creation). other church fathers explained that it was in fact evidence for immaculate conception. So, you see – the Bible backs up my statement (not hypothesis) meaning – no need to look further. Do you see the problem with that? I don’t want to make assumptions – but I think that you do.

    Version (A) is what it would be today closest to a modern hypothesis – but it sounds like possibly Version (C) might be one too – right? Both require some kind of faith in an speculative explanation of the unexplained – in fact – both seem to more or less say the same thing – right? That seems to be the assertion… But correct me if I am misrepresenting.

    So – is the statement that they are equally good or that Version C might actually have benefit. If so – what is it?

    In other words: What would it benefit to say the same thing in Version (A) only replacing the words “spontaneous generation” with “A Generative Act of God”? What is gained? It does not specify a deity. It could be the generative act of Osiris or Yahweh or Allah or Jesus or Mithra or Eloheim or Jehova, Zeus, etc?… Presumably it is merely an recognition of the Divine – but if the Divine really cared whether it was attributed — would that be enough? And might it actually be disrespectful of the Divine to try to have to prove its ability – or to try to disprove it?

    But is there any drawback – or are the two equal? In other words – Does it just exclude God needlessly?

    If there is nothing to be gained, it might be logical that it could bias your experimental results. Version (A) includes falsifiable statement and specific postulations and expectations. The first major experiment to challenge this hypothesis was done by placing meat in a jar covered with a fine cloth. No maggots appeared on the meat – but they appeared on the cloth. Version (A) drives us forward by creating new questions – whereas not all – but at least _some_ proponents of Version (B) might argue, for example, that this is just more proof of God’s inerrant wisdom and creational power: He wouldn’t create a fly on the meat – because then it would be trapped in the jar and unable to fulfill its destiny in his creation. Therefore, he simply supernaturally intervened moving the necessary invisible bits onto the cloth and creation took place there. Prove he didn’t. You can’t.

    The same goes for things like dark matter. It’s very a very wrong portrayal to imply that science just said “we can’t account for that – let’s just call it dark matter and the problem is solved”. Actually, Dark Matter is a detailed and involved hypothesis – there are some variants even — it lays out specific ideas and observations in very direct ways. There have been numerous experiments and attempts to observe, test and explain – and even competing hypotheses. That’s how science works. To date, this seems to fit with everything else better than anything else – so that’s where there is much “agreement”.

    As to your friend who said “dark matter will be detected some day” I agree, that seems a bit like a matter of faith in his phrasing… But we’ve already “detected” the something – it’s just a matter finding a way of more directly observing what that something is and explaining what and how it works. I would rather just claim, “I have faith that the only way that we come to answers is if people keep asking questions – and the scientific method is the best way I know”.

  17. ID cannot BE peer reviewed because it offers nothing for review. In scientific circles (as was stated earlier), peer review looks at the methodology, the data, and the controls of the research. There is no data or control when it comes to ID…just opinion. To ‘peer review’ ID would be like ‘peer reviewing’ a piece of art.

    Science uses peer review because it is one of the best tools to sift the good information from the bad. It seems like you want to lower the standards of science just so you can claim ID as science…which it ain’t.

  18. Mike Burns: Your comment has nothing to do with the post. If you read the post you could try and answer the questions that where put forth.

    Besides I think the leaked climate emails show the legitimacy of the current “scientific” peer review process. If this is the best tool to “sift” the information we are all in more trouble than I thought.

  19. You asked if peer review was important. It is. Scientists are human and can and do exhibit biases (intentional and unintentional) or be blind to an error they introduced. By asking unaffiliated others to review that which is presented, we have the best chance of mitigating those biases and errors .

    I am still trying to get some clear info on the ‘leaked climate e-mails’. Initial press might give the impression that a vast conspiracy [unfathomably vast] has been uncovered. The reality is something much less titillating … though I am trying to determine HOW much less.

    At worst; a small group of scientists willfully misrepresented data to the public and have been outed. If so their careers have hit a dead-end. This one incident (and the data they were reviewing) is but a drop in the ocean of climate change evidence and, on the whole, affects nothing.

    At best (or another worst); Scientists often perform ‘side-tests’ or ancillary analysis to see if they might explain some phenomenon or result by willfully manipulating data. These could be wholly separate from their core task. Scientists often do such manipulation when trying to determine cause and effect (I do it in my line of work all the time debugging computer systems). If such e-mails, taken out of context, were merely discussing such ancillary analysis, then FOX news (and others) would have a conniption fit thinking they just unearthed ‘the great lie’ of climate change.

    Like I said; I don’t yet know all the specifics and hope to find out more soon.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: