Reproducibility

.

If a type of measuring device has a design flaw, then it is likely that this artefact will be apparent in all models.

The Process of Replicating Research
For most of the physical sciences, reproducibility is a simple process and it is easy to replicate methods and equipment.
An astronomer measuring the spectrum of a star notes down the instruments and methodology used, and an independent researcher should be able to achieve exactly the same results, Even in biochemistry, where naturally variable living organisms are used, good research [2] shows remarkably little variation.
However, the social sciences, ecology and environmental science are a much more difficult case. Organisms can show a huge amount of variation, making it difficult to replicate research exactly and so reproducibility is a process of attempting to make the experiment as reproducible as possible, ensuring that the researcher can defend their position.
In addition, these sciences have to make much more use of statistics [3] to dampen down experimental noise caused by physiological and psychological differences between the subjects. This is one of the reasons why most social sciences accept a 95% probability level [4], which is a contrast to the 99% confidence required by most physical sciences.

Reproducibility and Generalization -A Cautious Approach
Observing due caution with the process of generalizing [5] results helps to strengthen the case for experimental reproducibility.
In any study, there is a far smaller chance of finding confounding [6] evidence if the claims are narrowly defined than if they are sweeping generalizations.
For example, a psychologist who found that aggression in children under the age of five increased if they watched violent TV, could generalize that all children under five would display the same condition.
Extending this to all children means that the experiment is prone to replication issues -A researcher finding that aggression did not increase in nine year old children would invalidate the entire premise by questioning the reproducibility.

Reproducibility is not Essential
It is important to understand that creating replicable research is not essential for validity, although it does help. Sometimes, due to sheer impracticality, temporal difficulties, or expense, this is not possible.
The Framingham Heart Study, an experiment testing three generations of nurses for cardiac issues, has been going on for over 60 years and nobody is seriously expected to replicate it.
Instead, results from other studies around the world are used to build up a database of statistical [3] evidence supporting the findings.

Reproducibility -An Impossible Ideal?
Many scientists argue that reproducibility is not an important factor for many sciences observing natural phenomena, such as astronomy, geology and, notoriously, evolution.
The rise of the Intelligent Design movement has seen evolutionary science under attack, because creationists claim that evolution is not reproducible and, therefore, it is not valid. This has opened up an intense debate [7] about the role of replication study as, for example, a geologist cannot very well recreate conditions found on the primordial earth and observe rocks metamorphosing.
However, creationists misunderstand the idea of reproducibility and assume that it applies to an entire theory. In fact, this is incorrect and it is a manipulation of scientific practices; replicating research only applies to a specific experiment or observation.

Reproducibility and Specificity -A Geological Example
If I go into the Greek Mountains and observe trilobite fossils lying above ammonite fossils, I assume that trilobites came later than ammonites.
However, a more talented geologist than I later travels to exactly the same place, and points out that the rocks there are deformed and twisted 180 degrees, so my observations were the wrong way around. My field study was reproducible in that another researcher could come and try to replicate my observations.
Looking at the process from the other angle, imagine that an astronomer discovers a planet circling around a distant star. Nobody is suggesting that he builds a gaseous cloud and waits a few billion years for matter to accrete and an identical solar system to form, because that would be absurd.
Performing a replication study would involve other astronomers observing the star to try to find the planets, showing that there really are planets and that the original astronomer had no equipment malfunction.

Reproducibility and Archaeology -The Absurdity of Creationism
When Arthur Evans discovered Knossos, on Crete, and proposed that there was an ancient, advanced Minoan civilization, nobody suggested that he should recreate such a civilization and see if they built an identical city. Absurd as it may seem, this is the type of assumption that proponents of Creationism make.
Looking at this process in reverse, if a team of builders builds an exact replica of Knossos, it does not prove that such a civilization existed, although it would be a useful exercise in looking at some of the techniques used by ancient builders, allowing archaeologists to refine their ideas. To suggest otherwise really is a deliberate misunderstanding and warping of the scientific method [8].
Ultimately, if Creationists use the argument that evolution is wrong because it is not reproducible, then they destroy their own argument. If evolutionary processes cannot be subjected to replicable research, neither can Intelligent Design, so their argument founders on its own presumptions. Surely, proponents of ID need to recreate the six days of Genesis before their ideas can be accepted by science!