Distinguishing between human impacts and natural variation in abundance remains difficult because most species exhibit complex patterns of variation in space and time. When ecological monitoring data are available, a before‐after‐control‐impact (BACI) analysis can control natural spatial and temporal variation to better identify an impact and estimate its magnitude. However, populations with limited distributions and confounding spatial‐temporal dynamics can violate core assumptions of BACI‐type designs. In this study, we assessed how such properties affect the potential to identify impacts. Specifically, we quantified the conditions under which BACI analyses correctly (or incorrectly) identified simulated anthropogenic impacts in a spatially and temporally replicated data set of fish, macroalgal, and invertebrate species found on nearshore subtidal reefs in southern California, USA. We found BACI failed to assess very localized impacts, and had low power but high precision when assessing region‐wide impacts. Power was highest for severe impacts of moderate spatial scale, and impacts were most easily detected in species with stable, widely distributed populations. Serial autocorrelation in the data greatly inflated false impact detection rates, and could be partly controlled for statistically, while spatial synchrony in dynamics had no consistent effect on power or false detection rates. Unfortunately, species that offer high power to detect real impacts were also more likely to detect impacts where none had occurred. However, considering power and false detection rates together can identify promising indicator species, and collectively analyzing data for similar species improved the net ability to assess impacts. These insights set expectations for the sizes and severities of impacts that BACI analyses can detect in real systems, point to the importance of serial autocorrelation (but not of spatial synchrony), and indicate how to choose the species, and groups of species, that can best identify impacts.