Skip to main content
SpringerLink
Log in

The Power and the Pitfalls of Large-scale, Unreplicated Natural Experiments

  • Published:
Ecosystems Aims and scope Submit manuscript

Abstract

Large-scale, unreplicated natural experiments (LUNEs) have a unique power to test hypotheses at ecologically realistic scales and have delivered insights of great power into cosmology, evolution and geology. Yet, LUNEs are relatively rare in the field of ecology and continue to meet resistance due to their lack of replication. However, in the vast majority of cases, large-scale experiments cannot be replicated for practical and ethical reasons. Here, we make the case that LUNEs have had a disproportionately positive effect on conservation policy and are a crucial next step in the extrapolation of our understanding of ecological processes from small-scale experiments to relevant scales, particularly in the context of the current “replication crisis” affecting many sciences. Greater inclusion of LUNEs in mainstream ecology will help humanity to solve global problems as human transformation of the planet accelerates in coming decades.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

REFERENCES

  • Allison GW, Lubchenco J, Carr MH. 1998. Marine reserves are necessary but not sufficient for marine conservation. Ecol Appl 8:79–92.

    Article  Google Scholar 

  • Atwood TB, Connolly RM, Ritchie EG, Lovelock CE, Heithaus MR, Hays GC, Fourqurean JW, Macreadie PI. 2015. Predators help protect carbon stocks in blue carbon ecosystems. Nat Clim Chang 5:1038–45.

    Article  Google Scholar 

  • Bell JD, Westoby M. 1986. Variation in seagrass height and density over a wide spatial scale: effects on common fish and decapods. J Exp Mar Biol Ecol 104:275–95.

    Article  Google Scholar 

  • Bence JR, Stewart-Oaten A, Schroeter SC. 1996. Estimating the size of an effect from a before-after-control impact paired series design. In: Russell J, Schmitt CWO, Eds. Detecting ecological impacts: concepts and applications in coastal habitats. San Diego: Academic Press.

    Google Scholar 

  • Bennett S, Bellwood DR. 2011. Latitudinal variation in macroalgal consumption by fishes on the Great Barrier Reef. Mar Ecol Prog Ser 426:241–52.

    Article  Google Scholar 

  • Brown GP, Phillips BL, Shine R. 2011. The ecological impact of invasive cane toads on tropical snakes: field data do not support laboratory-based predictions. Ecology 92:422–31.

    Article  PubMed  Google Scholar 

  • Carpenter RC. 1990. Mass mortality of Diadema antillarum. I. Long-term effects on sea urchin population-dynamics and coral reef algal communities. Mar Biol 104:67–77.

    Article  Google Scholar 

  • Carpenter SR, Chisholm SW, Krebs CJ, Schindler DW, Wright RF. 1995. Ecosystem experiments. Science 269:324–7.

    Article  CAS  PubMed  Google Scholar 

  • Carpenter SR. 1989. Replication and treatment strength in whole-lake experiments. Ecology 70:453–63.

    Article  Google Scholar 

  • Chave J. 2013. The problem of pattern and scale in ecology: what have we learned in 20 years? Ecol Lett 16:4–16.

    Article  PubMed  Google Scholar 

  • Chown SL, Gaston KJ, Robinson D. 2004. Macrophysiology: large-scale patterns in physiological traits and their ecological implications. Funct Ecol 18:159–67.

    Article  Google Scholar 

  • Clark JS, Gelfand AE. 2006. A future for models and data in environmental science. Trends Ecol Evol 21:375–80.

    Article  PubMed  Google Scholar 

  • Cottenie K, De Meester L. 2003. Comment to Oksanen (2001): reconciling Oksanen (2001) and Hurlbert (1984). Oikos 100:394–6.

    Article  Google Scholar 

  • Darimont CT, Fox CH, Bryan HM, Reimchen TE. 2015. The unique ecology of human predators. Science 349:858–60.

    Article  CAS  PubMed  Google Scholar 

  • Edgar G, Stuart-Smith R, Willis T, Kininmonth S, Baker S, Banks S, Barrett N, Becerro M, Bernard A, Berkhout J, Buxton C, Campbell S, Cooper A, Davey M, Edgar S, Försterra G, Galván D, Irigoyen A, Kushner D, Moura R, Parnell P, Shears N, Soler G, Strain E, Thomson R. 2014. Global conservation outcomes depend on marine protected areas with five key features. Nature 506:216–20.

    Article  CAS  PubMed  Google Scholar 

  • Ehrlich PR, Ehrlich AH. 2013. Can a collapse of global civilization be avoided? Proc Biol Sci 280:1–9.

    Google Scholar 

  • Estes JA, Palmisano JF. 1974. Sea otters: their role in structuring nearshore communities. Science 185:1058–60.

    Article  CAS  PubMed  Google Scholar 

  • Estes JA, Terborgh J, Brashares JS, Power ME, Berger J, Bond WJ, Carpenter SR, Essington TE, Holt RD, Jackson JBC, Marquis RJ, Oksanen L, Oksanen T, Paine RT, Pikitch EK, Ripple WJ, Sandin SA, Scheffer M, Schoener TW, Shurin JB, Sinclair ARE, Soulé ME, Virtanen R, Wardle DA. 2011. Trophic downgrading of planet Earth. Science 333:301–6.

    Article  CAS  PubMed  Google Scholar 

  • Ewers R, Didham R, Fahrig L, Ferraz G, Hector A, Holt R, Kapos V, Reynolds G, Sinun W, Snaddon J, Turner E. 2011. A large-scale forest fragmentation experiment: the stability of altered forest ecosystems project. Philos Trans Royal Soc B: Biol Sci 366:3292–302.

    Article  Google Scholar 

  • Fisher RA. 1926. The arrangement of field experiments. J Minist Agri 33:503–13.

    Google Scholar 

  • Folt CL, Nislow KH, Power ME. 1998. Implications of temporal and spatial scale for Atlantic salmon (Salmo salar) research. Can J Fish Aquat Sci 55:9–21.

    Article  Google Scholar 

  • Fraser DF, Gilliam JF. 1987. Feeding under predation hazard: response of the guppy and Hart’s rivulus from sites with contrasting predation hazard. Behav Ecol Sociobiol 21:203–9.

    Article  Google Scholar 

  • Gabric AJ, Cropp R, McTainsh G, Butler H, Johnston BM, O’Loingsigh T, Van Tran D. 2015. Tasman Sea biological response to dust storm events during the austral spring of 2009. Marine and Freshwater Research 9999.

  • Gadgil M, Bossert WH. 1970. Life historical consequences of natural selection. Am Naturalist 104:1–24.

    Article  Google Scholar 

  • Grossman J, Mackenzie FJ. 2005. The randomized controlled trial: gold standard, or merely standard? Perspect Biol Med 48:516–34.

    Article  PubMed  Google Scholar 

  • Grubbs RD, Carlson JK, Romine JG, Curtis TH, McElroy WD, McCandless CT, Musick CFC& JA. 2016. Critical assessment and ramifications of a purported marine trophic cascade. Sci Rep:20970.

  • Hale M, Rivkin RB. 2007. Interpreting the results of oceanic mesoscale enrichment experiments: caveats and lessons from limnology and coastal ecology. Limnol Oceanogr 52:912–16.

    Article  Google Scholar 

  • Hargrove WW, Pickering J. 1992. Pseudoreplication: a sine qua non for regional ecology. Landsc Ecol 6:251–8.

    Article  Google Scholar 

  • Hildrew AG, Woodward G, Winterbottom JH, Orton S. 2004. Strong density dependence in a predatory insect: large-scale experiments in a stream. J Anim Ecol 73:448–58.

    Article  Google Scholar 

  • Hillerislambers J, Ettinger AK, Ford KR, Haak DC, Horwith M, Miner BE, Rogers HS, Sheldon KS, Tewksbury JJ, Waters SM, Yang S. 2013. Accidental experiments: ecological and evolutionary insights and opportunities derived from global change. Oikos 122:1649–61.

    Article  Google Scholar 

  • Hixon MA, Anderson TW, Buch KL, Johnson DW, Mcleod JB, Stallings CD. 2012. Density dependence and population regulation in marine fish: a large-scale, long-term field manipulation. Ecol Monogr 82:467–89.

    Article  Google Scholar 

  • Högberg P, Nordgren A, Buchmann N, Taylor AF, Ekblad A, Högberg MN, Nyberg G, Ottosson-Löfvenius M, Read DJ. 2001. Large-scale forest girdling shows that current photosynthesis drives soil respiration. Nature 411:789–92.

    Article  PubMed  Google Scholar 

  • Hüffmeiera J, Mazeia J, Schultze T. 2016. Reconceptualizing replication as a sequence of different studies: a replication typology. J Exp Soc Psychol. doi:10.1016/j.jesp.2015.09.009.

    Google Scholar 

  • Hurlbert SH. 1984. Pseudoreplication and the design of ecological field experiments. Ecol Monogr 54:187.

    Article  Google Scholar 

  • Hurlbert SH. 2004. On misinterpretations of pseudoreplication and related matters: a reply to Oksanen. Oikos 104:591–7.

    Article  Google Scholar 

  • Ioannidis JPA. 2005. Why most published research findings are false. PLoS Med 2:0696–701.

    Google Scholar 

  • Johnson DH. 2002. The importance of replication in wildlife research. J Wildl Manag 66:919–32.

    Article  Google Scholar 

  • Johnson DH. 2006. The many faces of replication. In: Crop science. Vol. 46. pp 2486–91.

  • Kates RW, Clark WC. 1996. Environmental Surprise: expecting the Unexpected? Environ: Sci Policy Sustain Dev 38:6–34.

    Article  Google Scholar 

  • Kessler JD. 2011. A persistent oxygen anomaly reveals the fate of spilled methane in the deep gulf of Mexico. Science 331:312.

    Article  CAS  PubMed  Google Scholar 

  • Knowlton N. 1992. Thresholds and multiple stable states in coral reef community dynamics. Am Zool 32:674–82. http://www.jstor.org/stable/pdfplus/3883648.pdf.

  • Kreyling J, Jentsch A, Beier C. 2014. Beyond realism in climate change experiments: gradient approaches identify thresholds and tipping points. Ecol Lett 17:125–8. http://doi.wiley.com/10.1111/ele.12193.

  • Lennon JT. 2011. Replication, lies and lesser-known truths regarding experimental design in environmental microbiology. Environ Microbiol 13:1383–6.

    Article  PubMed  Google Scholar 

  • Lester SE, Halpern BS. 2008. Biological responses in marine no-take reserves versus partially protected areas. Mar Ecol Prog Ser 367:49–56.

    Article  Google Scholar 

  • Leuzinger S, Luo Y, Beier C, Dieleman W, Vicca S, Körner C. 2011. Do global change experiments overestimate impacts on terrestrial ecosystems? Trends Ecol Evol 26:236–41.

    Article  PubMed  Google Scholar 

  • Lezama F, Baeza S, Altesor A, Cesa A, Chaneton EJ, Paruelo JM. 2014. Variation of grazing-induced vegetation changes across a large-scale productivity gradient. J Veg Sci 25:8–21.

    Article  Google Scholar 

  • Likens GE, Bormann FH, Johnson NM, Fisher DW, Pierce RS. 1970. Effects of forest cutting and herbicide treatment on nutrient budgets in the hubbard brook watershed-ecosystem. Ecol Monogr 40:23–47.

    Article  Google Scholar 

  • Madin EMP, Gaines SD, Warner RR. 2010. Field evidence for pervasive indirect effects of fishing on prey foraging behavior. Ecology 91:3563–71.

    Article  PubMed  Google Scholar 

  • McArdle BH. 1996. Levels of evidence in studies of competition, predation, and disease. N. Z. J Ecol 20:7–15.

    Google Scholar 

  • McNutt M. 2014. Reproducibility. Science (New York) 343:229. http://www.sciencemag.org/content/343/6168/229.short.

  • Miao S, Carstenn S. 2006. A new direction for large-scale experimental design and analysis. Front Ecol Environ 4:227.

    Article  Google Scholar 

  • Micheli F. 1999. Eutrophication, fisheries, and consumer-resource dynamics in marine pelagic ecosystems. Science (New York, NY) 285:1396–8

  • Morris SC. 1989. Burgess Shale faunas and the Cambrian explosion. Science 246:339–46.

    Article  CAS  PubMed  Google Scholar 

  • Morrisette PM. 1989. The Evolution of Policy Responses to Stratospheric Ozone Depletion. Nat Res J 29:793–820.

    Google Scholar 

  • Moss R, Watson A, Parr R. 1996. Experimental prevention of a population cycle in Red Grouse. Ecology 77:1512–30.

    Article  Google Scholar 

  • Myers RA, Baum JK, Shepherd TD, Powers SP, Peterson CH. 2007. Cascading effects of the loss of apex predatory sharks from a coastal ocean. Science 315:1846–50.

    Article  CAS  PubMed  Google Scholar 

  • Naeem S, Thompson LJ, Lawler SP, Lawton JH, Woodfin RM. 1994. Declining biodiversity can alter the performance of ecosystems. Nature 368:734–7.

    Article  Google Scholar 

  • Nosek BA, Alter G, Banks GC, Borsboom D, Bowman SD, Breckler SJ, Buck S, Chambers CD, Chin G, Christensen G, Contestabile M, Dafoe A, Eich E, Freese J, Glennerster R, Goroff D, Green DP, Hesse B, Humphreys M, Ishiyama J, Karlan D, Kraut A, Lupia A, Mabry P, Madon TA, Malhotra N, Mayo-Wilson E, McNutt M, Miguel E, Paluck EL, Simonsohn U, Soderberg C, Spellman BA, Turitto J, VandenBos G, Vazire S, Wagenmakers EJ, Wilson R, Yarkoni T. 2015. Promoting an open research culture. Science 348:1422–5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • O’Dowd DJ, Green PT, Lake PS. 2003. Invasional ‘meltdown’ on an oceanic island. Ecol Lett 6:812–17.

    Article  Google Scholar 

  • Pace ML, Cole JJ, Carpenter SR. 1998. Trophic cascades and compensation: differential responses of microzooplankton in whole-lake experiments. Ecology 79:138–52.

    Article  Google Scholar 

  • Pauly D, Christensen V, Dalsgaard J, Froese R, Torres F. 1998. Fishing down marine food webs. Science (New York, NY) 279:860–3. http://www.ncbi.nlm.nih.gov/pubmed/9452385.

  • Power ME, Dietrich WE, Sullivan KO. 1998. Experiment, observation, and inference in river and watershed investigations. In: Bernardo J, Resetarits WJ, Eds. Experimental ecology: issues and perspectives. Oxford: Oxford University Press. p 113–32.

    Google Scholar 

  • Prevedello J, Dickman C, Vieira M, Vieira E. 2013. Population responses of small mammals to food supply and predators: a global meta-analysis. J Anim Ecol 82:927–36.

    Article  PubMed  Google Scholar 

  • Raffaelli D, Moller H. 1999. Manipulative field experiments in animal ecology: do they promise more than they can deliver? Adv Ecol Res 30:299–338.

    Article  Google Scholar 

  • Ritsema J, Van Heijst HJ. 2000. Seismic imaging of structural heterogeneity in Earth’s mantle: evidence for large-scale mantle flow. Science progress 83.

  • Rogers H, HilleRisLambers J, Miller R, Tewksbury J. 2012. ‘Natural experiment’ demonstrates top-down control of spiders by birds on a landscape level. PLoS One 7:1–8.

    Google Scholar 

  • Rose GA, Leggett WC. 1990. The importance of scale to predator-prey spatial correlations: an example of Atlantic fishes. Ecology 71:33–43.

    Article  Google Scholar 

  • Rowan R, Knowlton N, Baker A, Jara J. 1997. Landscape ecology of algal symbionts creates variation in episodes of coral bleaching. Nature 388:265–9.

    Article  CAS  PubMed  Google Scholar 

  • Ruppert JLW, Travers MJ, Smith LL, Fortin MJ, Meekan MG. 2013. Caught in the middle: combined impacts of shark removal and coral loss on the fish communities of coral reefs. PLoS One 8:e74648.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Sarnelle O. 1997. Daphnia effects on microzooplankton: comparisons of enclosure and whole-lake responses. Ecology 78:913–28.

    Google Scholar 

  • Schindler DW, Armstrong FAJ, Holmgren SK, Brunskill GJ. 1971. Eutrophication of lake 227, experimental lakes area, northwestern Ontario, by addition of phosphate and nitrate. J Fish Res Board Can 28:1763–82.

    Article  CAS  Google Scholar 

  • Schindler DW, Fee EJ, Ruszczynski T. 1978. Phosphorus input and its consequences for phytoplankton standing crop and production in the experimental lakes area and in similar lakes. J Fish Res Board Can 35:190–6.

    Article  CAS  Google Scholar 

  • Schindler DW, Hecky RE, Findlay DL, Stainton MP, Parker BR, Paterson MJ, Beaty KG, Lyng M, Kasian SEM. 2008. Eutrophication of lakes cannot be controlled by reducing nitrogen input: results of a 37-year whole-ecosystem experiment. Proc Natl Acad Sci U. S. A. 105:11254–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Schindler DW. 1998. Replication versus realism: the need for ecosystem-scale experiments. Ecosystems 1:323–34.

    Article  Google Scholar 

  • Schindler DW. 2012. The dilemma of controlling cultural eutrophication of lakes. Proc Royal Soc B: Biol Sci 279:4322–33.

    Article  CAS  Google Scholar 

  • Shurin JB, Borer ET, Seabloom EW, Anderson K, Blanchette CA, Broitman B, Cooper SD, Halpern BS. 2002. A cross-ecosystem comparison of the strength of trophic cascades. Ecol Lett 5:785–91.

    Article  Google Scholar 

  • Smith SA, Bell G, Bermingham E. 2004. Cross-cordillera exchange mediated by the panama canal increased the species richness of local freshwater fish assemblages. Proc Biol Sci 271:1889–96.

    Article  PubMed  PubMed Central  Google Scholar 

  • Stewart-Oaten A, Bence JR, Osenberg CW. 1992. Assessing effects of unreplicated perturbations: no simple solutions. Ecology 73:1396–404.

    Article  Google Scholar 

  • Stewart-Oaten A, Murdoch WW, Parker KR. 1986. Environmental impact assessment: ‘pseudoreplication’ in time? Ecology 67:929–40.

    Article  Google Scholar 

  • Stroebe W, Strack F. 2014. The alleged crisis and the illusion of exact replication. Perspect Psychol Sci 9:59–71.

    Article  PubMed  Google Scholar 

  • Vanni MJ, Luecke C, Kitchell JF, Allen Y, Temte J, Magnuson JJ. 1990. Effects on lower trophic levels of massive fish mortality. Nature 344:333–5.

    Article  Google Scholar 

  • Walsh SM, Hamilton SL, Ruttenberg BI, Donovan MK, Sandin SA. 2012. Fishing top predators indirectly affects condition and reproduction in a reef-fish community. J Fish Biol 80:519–37.

    Article  CAS  PubMed  Google Scholar 

  • Wardle DA, Barker GM. 1997. Competition and herbivory in establishing grassland communities: implications for plant biomass, species diversity and soil microbial activity. Oikos 80:470–80.

    Article  Google Scholar 

  • Weimin X, Peet RK. 2011. The complexity of catastrophic wind impacts on temperate forest. In: Lupo A, Ed. Recent hurricane research - climate, dynamics, and societal impacts. InTech. pp 503–34.

  • Worm B, Paine RT. 2016. Humans as a hyperkeystone species. Trends in Ecology and Evolution2.

  • Zoback MD, Gorelick SM. 2012. Earthquake triggering and large-scale geologic storage of carbon dioxide. Proc Natl Acad Sci 109:10164–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

ACKNOWLEDGMENTS

We would like to thank the University of Western Australia, Perth, which partly funded this research via an International Postgraduate Research Scholarship to SB.

Author information

Author notes

    Authors and Affiliations

    1. School of Animal Biology and the Oceans Institute, University of Western Australia, 35 Stirling Highway, Crawley, Perth, Western Australia, 6009, Australia

      Shanta C. Barley & Jessica J. Meeuwig

    Authors
    1. Shanta C. Barley
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Jessica J. Meeuwig
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Shanta C. Barley.

    Additional information

    Author contributions

    JM/SB conceived of the review, JM/SB contributed to structuring the manuscript, SB wrote the paper and JM/SB edited the paper.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Barley, S.C., Meeuwig, J.J. The Power and the Pitfalls of Large-scale, Unreplicated Natural Experiments. Ecosystems 20, 331–339 (2017). https://doi.org/10.1007/s10021-016-0028-5

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s10021-016-0028-5

    Keywords

    Search

    Navigation