Hydrology and water quality (suspended sediment, total nitrogen, ammonia, total Kjeldahl nitrogen, nitrate plus nitrite, and total phosphorus (TP)) were monitored in two small agricultural drainages in northwestern Mississippi to document changes in water quality that coincided with the implementation of BMPs in upstream drainages. Using an event-based dataset and bootstrapping techniques, we tested for difference and equivalence in median event concentration and differences in concentration-discharge (C-Q) relationships between an early and late period at each site, where most of the major BMP implementation occurred during the early period. Results for one site were inconclusive. None of the constituents had statistically different or equivalent event concentrations between the periods, indicating a lack of evidence to tell whether water quality had changed or stayed the same, and only TP had a significantly higher C-Q slope during the late period. At the other site, more than half the constituents had a significantly different median, slope, or intercept between periods, indicating a 35% or more decrease in event concentration following a period of intense BMP implementation. These mixed results could be due to variety of differences between the sites including BMP implementation, production practices, and crops. We also used the monitoring data to generate synthetic data and perform a simulation-based power analysis to explore the ability to detect change under 25 scenarios of sampled event counts and hypothetical percent changes. The simulation-based power analysis indicated that high natural variability in event concentration and flow hindered our ability to detect change. Based on our monitoring, data analysis, and power analysis, we provide recommendations for future monitoring.