Evaluating the predictive abilities of community occupancy models using AUC while accounting for imperfect detection

Ecological Applications
By: , and 



The ability to accurately predict patterns of species' occurrences is fundamental to the successful management of animal communities. To determine optimal management strategies, it is essential to understand species-habitat relationships and how species habitat use is related to natural or human-induced environmental changes. Using five years of monitoring data in the Chesapeake and Ohio Canal National Historical Park, Maryland, USA, we developed four multi-species hierarchical models for estimating amphibian wetland use that account for imperfect detection during sampling. The models were designed to determine which factors (wetland habitat characteristics, annual trend effects, spring/summer precipitation, and previous wetland occupancy) were most important for predicting future habitat use. We used the models to make predictions of species occurrences in sampled and unsampled wetlands and evaluated model projections using additional data. Using a Bayesian approach, we calculated a posterior distribution of receiver operating characteristic area under the curve (ROC AUC) values, which allowed us to explicitly quantify the uncertainty in the quality of our predictions and to account for false negatives in the evaluation dataset. We found that wetland hydroperiod (the length of time that a wetland holds water) as well as the occurrence state in the prior year were generally the most important factors in determining occupancy. The model with only habitat covariates predicted species occurrences well; however, knowledge of wetland use in the previous year significantly improved predictive ability at the community level and for two of 12 species/species complexes. Our results demonstrate the utility of multi-species models for understanding which factors affect species habitat use of an entire community (of species) and provide an improved methodology using AUC that is helpful for quantifying the uncertainty in model predictions while explicitly accounting for detection biases.
Publication type Article
Publication Subtype Journal Article
Title Evaluating the predictive abilities of community occupancy models using AUC while accounting for imperfect detection
Series title Ecological Applications
DOI 10.1890/11-1936.1
Volume 22
Issue 7
Year Published 2012
Language English
Publisher Ecological Society of America
Publisher location Ithaca, NY
Contributing office(s) Patuxent Wildlife Research Center
Description 11 p.
Larger Work Type Article
Larger Work Subtype Journal Article
Larger Work Title Ecological Applications
First page 1962
Last page 1972
Google Analytic Metrics Metrics page
Additional publication details