Observational Studies:
The quickest and cheapest way to test a hypothesis is to go out in the field and observe whatever organisms we are interested in. An observational study allows us to easily collect information from a broad area (see replication) and because we are not influencing our subjects at all, the results are highly realistic. But observational studies have drawbacks as well.
Let’s suppose we want to test the hypothesis that increased soil moisture causes cottonwoods to produce biggerleaves.
We know that there will be more moisture in the soil near a river so we can measure the leaf sizes on trees close to the Oldman River, and on those further away (we might also want to measure soil moisture at each site to verify our assumption that moisture declines further from the river). We are taking advantage of natural variation in the amount of soil moisture, to see if this has any relationship with leaf size. Let’s further suppose we find bigger leaves on trees near the river. This is consistent with the hypothesis being tested, but we must always ask if there are other possible explanations for our results. It could be that our site further from the river has different soil than the near-river site, or perhaps the trees are not the same age at the two sites. We have shown that there is an association (or correlation) between soil moisture and leaf size, but we can’t say with any confidence that soil moisture causes bigger leaves because there are many other possible explanations. Scientists summarize this problem with observational studies by noting that: Correlation does not imply causation.
So our study is only a weak test of our hypothesis. It can falsify the hypothesis, but can only provide limited support for it. We could make our study stronger by comparing the soil at our two sites to make sure it was similar, and by including only trees of the same size and age, but we could never rule out the possibility that something was different between the two sites other than soil moisture. This potential for alternative explanations, or confounding factors, is the main disadvantage of observational studies.
Experiments:
Experiments reduce the problem of confounding variables in two ways. First, we do not depend on natural variation. We manipulate the factor we are interested in. And further, we deliberately control everything else that might influene the outcome of our experiment. An experiment designed to test the same prediction used above would manipulate the amount of soil moisture, while keeping everything else constant. We would grow some cottonwoods in moist soil, and a similar number in dry soil (these two levels of soil moisture are called our treatment groups). Everything else that might effect leaf size — such as tree size and age, soil nutrients, and temperature — would be the same in both treatments. Now we can have more confidence (although never certainty) that if the leaves are bigger in the moist-soil treatment, it is because moisture is causing the difference. Formally this is known as making a strong inference that moisture is the cause.
Field vs. Lab
Every biological experiment is a balancing act. On one hand, we want to have as much control as possible over all of the factors that might influence the outcome, so we can be confident that the variable we are manipulating is the only thing influencing the results. On the other hand, we want conditions to be as close as possible to nature. Our goal is to understand what is going on in the real world, and the further experimental conditions stray from natural conditions, the less confident we can be that our experiment captures what is going on in nature. One important decision the researcher must make is whether the experiment should take place in the lab or in the field.
Field:
Field experiments typically don’t provide as much control over confounding variables as a lab experiment, but provide a greater degree of realism.
Lab:
Lab experiments generally provide the greatest degree of control over confounding variables, but may lack realism. They may also present logistical problems, and are often limited to a smaller scale than field observations.
Neither type of experiment is best in all situations. A researcher must weigh the costs and benefits for her particular research objectives. Ideally, it is best to perform both types of study, since the strengths of one complement the weakness of the other.
Literature Studies:
A third type of study should also be noted in passing. Literature studies combine the results of many previous experiments and look for large trends and generalizations that were not apparent to the original authors. Their advantage is their potential for a huge bank of data covering a broad range of organisms, but comparing different types of data collected with different methods can be difficult. Literature studies are beyond the scope of these pages. Students interested in learning more should consider taking Biology 3600 (Evolutionary Ecology).
Anecdotal Observations:
It is natural for humans (and many other animals) to learn from personal experience, and to draw generalizations, based on this experience. However, scientists are sceptical of these anecdotal observations, for several reasons. First, they tend to be based on a very small sample, often a single case. Second, the sample may be biased, since it was not collected through a formally designed study. And third, there is no statistical analysis to see if the results can be explained by chance alone. Anecdotal observations are not without value, but to the scientist they provide only weak evidence for any conclusions. They should provide the inspiration for a more rigorous test, not an alternative to that test.
Some examples of anecdotal observations from daily life:
After a hectic shift, full of emergency calls, an ambulance driver looks up at the sky and notices there is a full moon. He concludes that accidents and illness are more common during a full moon.
A mother notices that a day after she eats garlic, her baby is colicky. She concludes that the garlic, expressed in her breastmilk, has given the infant indigestion.
A gardener washes his tomato seeds in dilute bleach before planting. His harvest is the best in years, and he concludes that the bleach has killed off pathogens and improved the yield of his plants.
Feeling a cold coming on, a woman takes large doses of vitamin C. Within a few days she feels better, and she concludes the vitamins have cured her cold.