Atmospheric nitrogen (N) deposition has been shown to decrease plant species richness along regional deposition gradients in Europe and in experimental manipulations. However, the general response of species richness to N deposition across different vegetation types, soil conditions, and climates remains largely unknown even though responses may be contingent on these environmental factors. We assessed the effect of N deposition on herbaceous richness for 15,136 forest, woodland, shrubland, and grassland sites across the continental United States, to address how edaphic and climatic conditions altered vulnerability to this stressor. In our dataset, with N deposition ranging from 1 to 19 kg N⋅ha−1⋅y−1, we found a unimodal relationship; richness increased at low deposition levels and decreased above 8.7 and 13.4 kg N⋅ha−1⋅y−1 in open and closed-canopy vegetation, respectively. N deposition exceeded critical loads for loss of plant species richness in 24% of 15,136 sites examined nationwide. There were negative relationships between species richness and N deposition in 36% of 44 community gradients. Vulnerability to N deposition was consistently higher in more acidic soils whereas the moderating roles of temperature and precipitation varied across scales. We demonstrate here that negative relationships between N deposition and species richness are common, albeit not universal, and that fine-scale processes can moderate vegetation responses to N deposition. Our results highlight the importance of contingent factors when estimating ecosystem vulnerability to N deposition and suggest that N deposition is affecting species richness in forested and nonforested systems across much of the continental United States.