Skip to main content
NC3Rs | 20 Years: Pioneering Better Science
Guidance

Key elements of a well-designed experiment

A well-designed and correctly analysed experiment not only increases the scientific validity of your results but can also reduce the number of animals you use.

Ensure your experiment is unbiased

When two or more treatment groups are compared, the animals in each group should be in identical environments (or variations in environments should be the same across all groups) and be similar in every way apart from the applied treatments. Bias can be minimised by:

  • Randomly allocating animals to treatment groups (the random sequence used must be generated by a valid process such as tossing a coin or using a computer generated random sequence).
  • Ensuring that all subsequent treatments (including allocation to housing) are applied in a random order.
  • Ensuring that when you measure experimental outcomes you are unaware of the treatment received (i.e. you are blinded) until the final statistical analysis.

The Experimental Design Assistant (EDA) can help you design rigorous experiments, including guidance for implementing blinding and generating a randomised allocation sequence based on your experiment.

Make sure your experiment is adequately powered

An adequately powered experiment is one in which you have a high chance of detecting an effect if it is truly there. High power is achieved by:

Determine your sample size 

Determine your sample size using a formal method such as a power calculation [1]. Power is increased with increasing sample size, but an unnecessarily large experiment will waste animals and scientific resources. The number of animals you use should be the minimum number that is consistent with the aims of the experiment. Underpowered experiments, with sample sizes too small to detect a meaningful biological difference will waste animals and resources [3, 4].

If you are comparing two groups you will need the following information for your power calculation:

  • The type of statistical test you are going to use (e.g. a t-test or a chi-squared test)
  • The significance level (5% is often used)
  • The power (usually 80-90%)
  • The sidedness of the test (a 2-sided test is usually the most appropriate)
  • The effect size of biological interest (i.e. what difference between the two groups would you consider interesting and worth taking forward into future research?)
  • An estimate of the standard deviation (when comparing means), the best source for this is a previous experiment.

Power calculators can be found on the EDA website, statpages.info and in the freely-available software G*Power.

Control variation

Control variation by randomly allocating animals to treatment groups, while balancing differences in the animals (such as baseline weight or age) across groups. Introducing systematic variation into studies (e.g. testing the same intervention on several different strains, or performing tests across multiple sites) leads to results that are more generalisable [2].

Minimise measurement error

Minimise measurement error by using careful technique, good equipment, and by implementing blinding so you are not aware of the animal’s treatment allocation.

Consider the range of applicability of your experiment

Your results will be more widely applicable if you have included both sexes, different strains, different environments, and other factors [2,5]. A response to a drug may depend on prior treatment of the animal, the effects of other drugs or the route of administration. These effects can be studied efficiently using factorial experimental designs.

Factorial experimental designs allow you to investigate the effect of a treatment on both males and females without doing two separate experiments, or using twice as many animals. To do this, make half of your subjects in each experimental group male and half female [6]. An adequately powered factorial experiment will show whether or not the two sexes respond in the same way, which is not possible if the sexes are studied individually in two separate experiments.

Simplify your experiment

Overly complex experiments make mistakes more likely, and increase the chances of  statistical analyses becoming unduly complicated.

Before you start a major experiment use a small pilot study to ensure the experiment is feasible and enable you to make any changes to improve the experimental design or increase the efficiency of the main experiment.

Plan your experiments, including statistical analysis methods, before starting and do not change while they are in progress without good reason. If you do need to make changes make sure you document the changes and why you needed to deviate from your plans.

Indicate the uncertainty in your results

For each of your experiments generate descriptive statistics. These results can be used when planning future experiments. Descriptive statistics include mean and standard deviation or median and interquartile range. Indicate the range of uncertainty in your results, or a measure of variation such as standard deviations or confidence intervals.

See the ARRIVE guidelines for guidance when describing uncertainty in experimental results.

 

References and further reading

  1. Dell RB et al. (2002). Sample size determination, ILAR J 43: 207-13. doi: 10.1093/ilar.43.4.207
  2. Voelkl B et al. (2018). Reproducibility of preclinical animal research improves with heterogeneity of study samples. PLoS Biol. 16(2): e2003693. doi: 10.1371/journal.pbio.2003693
  3. Lazic SE et al. (2018). What exactly is ‘N’ in cell culture and animal experiments? PLoS Biol 16(4): e2005282. doi: 10.1371/journal.pbio.2005282
  4. Button KS et al. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 14: 365–76. doi: 10.1038/nrn3475
  5. Karp NA et al. (2017). Prevalence of sexual dimorphism in mammalian phenotypic traits. Nat Comms 8: 15475 doi: 10.1038/ncomms15475
  6. McCarthy (2015). Incorporating Sex as a Variable in Preclinical Neuropsychiatric Research. Schizophr Bull 41: 1016-20. doi: 10.1093/schbul/sbv077
  7. Festing MF and Altman DG (2002). Guidelines for the design and statistical analysis of experiments using laboratory animals, ILAR J 43: 244-58. doi: 10.1093/ilar.43.4.244
  8. Festing MF et al. (2016). The design of animal experiments: reducing the use of animals in research through better experimental design. 2nd edition. Sage Publishing.