National Hydrography Dataset (NHD) stream permanence classifications (SPC; perennial, intermittent, and ephemeral) are widely used for data visualization and applied science, and have implications for resource policy and management. NHD SPC were assigned using a combination of topographic field surveys and interviews with local residents. However, previous studies indicate that non‐NHD, in situ streamflow observations (NNO) frequently disagree with NHD SPC. We hypothesized that differences in annual climate conditions between map creation years and the years NNO were collected contributed to disagreement between NNO and NHD SPC. We compared NHD SPC to 10,055 NNO (classified as “wet” or “dry”) collected in the Pacific Northwest between 1977 and 2015. Annual climate conditions were described with the Palmer Drought Severity Index (PDSI). Stream order was added as a covariate to account for different effects along the stream network. NHD SPC agreed with 80.5% of NNO. “Dry” NNO were five times more likely to disagree with NHD than “wet” NNO (p < 0.0001). Disagreement was greatest on first‐order streams. When NHD SPC were collected during a wetter period than NNO the probability of disagreement increased by a factor of 1.17 (p < 0.0001) per unit difference in PDSI. The influence of climate on disagreements between NNO and NHD SPC provides support for the continued development of dynamic models representing SPC as opposed to static NHD classifications.