In North America, grassland bird abundances have declined, likely as a result of loss and degradation of prairie habitat. Given the expense and limited opportunity to procure new grasslands, managers are increasingly focusing on ways to improve existing habitat for grassland birds, using techniques such as tree removal. To examine the potential for tree removal to benefit grassland birds, we conducted 446 point counts on 35 grassland habitat patches in the highly fragmented landscape of west-central Minnesota during 2009–2011. We modeled density of four grassland bird species in relation to habitat composition at multiple scales, focusing on covariates that described grass, woody vegetation (trees and large shrubs), or combinations of grass and woody vegetation. The best-supported models for all four grassland bird species incorporated variables measured at multiple scales, including local features such as grass height, litter depth, and local tree abundance, as well as landscape-level measures of grass and tree cover. Savannah Sparrows (Passerculus sandwichensis), Sedge Wrens (Cistothorus platensis), and Bobolinks (Dolichonyx oryzivorus) responded consistently and negatively to woody vegetation, but response to litter depth, grass height, and grassland extent were mixed among species. Our results suggest that reducing shrub and tree cover is more likely to increase the density of grassland birds than are attempts to improve grass quality or quantity. In particular, tree removal is more likely to increase density of Savannah Sparrows and Sedge Wrens than any reasonable changes in grass quality or quantity. Yet tree removal may not result in increased abundance of grassland birds if habitat composition is not considered at multiple scales. Managers will need to either manage at large scales (80–300 ha) or focus their efforts on removing trees in landscapes that contain some grasslands but few nearby wooded areas.