Regional seismograms from earthquakes in Northern California show a systematic difference in arrival times across Southern California where long period (30–50 seconds) SH waves arrive up to 15 seconds earlier at stations near the coast compared with sites towards the east at similar epicentral distances. We attribute this time difference to heterogeneity of the velocity structure at the crust-mantle interface beneath the California margin. To model these observations, we propose a fast seismic layer, with thickness growing westward from the San Andreas along with a thicker and slower continental crust to the east. Synthetics generated from such a model are able to match the observed timing of SH waveforms better than existing 3D models. The presence of a strong upper mantle buttressed against a weaker crust has a major influence in how the boundary between the Pacific plate and North American plate deforms and may explain the observed asymmetric strain rate across the boundary.