In light of global climate change, ecological studies increasingly address effects of temperature on organisms and ecosystems. To measure air temperature at biologically relevant scales in the field, ecologists often use small, portable temperature sensors. Sensors must be shielded from solar radiation to provide accurate temperature measurements, but our review of 18 years of ecological literature indicates that shielding practices vary across studies (when reported at all), and that ecologists often invent and construct ad hoc radiation shields without testing their efficacy. We performed two field experiments to examine the accuracy of temperature observations from three commonly used portable data loggers (HOBO Pro, HOBO Pendant, and iButton hygrochron) housed in manufactured Gill shields or ad hoc, custom‐fabricated shields constructed from everyday materials such as plastic cups. We installed this sensor array (five replicates of 11 sensor‐shield combinations) at weather stations located in open and forested sites. HOBO Pro sensors with Gill shields were the most accurate devices, with a mean absolute error of 0.2°C relative to weather stations at each site. Error in ad hoc shield treatments ranged from 0.8 to 3.0°C, with the largest errors at the open site. We then deployed one replicate of each sensor‐shield combination at five sites that varied in the amount of urban impervious surface cover, which presents a further shielding challenge. Bias in sensors paired with ad hoc shields increased by up to 0.7°C for every 10% increase in impervious surface. Our results indicate that, due to variable shielding practices, the ecological literature likely includes highly biased temperature data that cannot be compared directly across studies. If left unaddressed, these errors will hinder efforts to predict biological responses to climate change. We call for greater standardization in how temperature data are recorded in the field, handled in analyses, and reported in publications.