We develop a methodology to perform finite fault source inversions from strong motion data using Green's functions (GFs) calculated for a three-dimensional (3-D) velocity structure. The 3-D GFs are calculated numerically by inserting body forces at each of the strong motion sites and then recording the resulting strains along the target fault surface. Using reciprocity, these GFs can be recombined to represent the ground motion at each site for any (heterogeneous) slip distribution on the fault. The reciprocal formulation significantly reduces the required number of 3-D finite difference computations to at most 3NS, where NS is the number of strong motion sites used in the inversion. Using controlled numerical resolution tests, we have examined the relative importance of accurate GFs for finite fault source inversions which rely on near-source ground motions. These experiments use both 1-D and 3-D GFs in inversions for hypothetical rupture models in order (1) to analyze the ability of the 3-D methodology to resolve trade-offs between complex source phenomena and 3-D path effects, (2) to address the sensitivity of the inversion results to uncertainties in the 3-D velocity structure, and (3) to test the adequacy of the 1-D GF method when propagation effects are known to be three-dimensional. We find that given "data" from a prescribed 3-D Earth structure, the use of well-calibrated 3-D GFs in the inversion provides very good resolution of the assumed slip distribution, thus adequately separating source and 3-D propagation effects. In contrast, using a set of inexact 3-D GFs or a set of hybrid 1-D GFs allows only partial recovery of the slip distribution. These findings suggest that in regions of complex geology the use of well-calibrated 3-D GFs has the potential for increased resolution of the rupture process relative to 1-D GFs. However, realizing this full potential requires that the 3-D velocity model and associated GFs should be carefully validated against the true 3-D Earth structure before performing the inverse problem with actual data.