Reconstruction of missing data in satellite and in situ data sets with DINEOF (Data Interpolating Empirical Orthogonal Functions)

DINEOF (Data Interpolating Empirical Orthogonal Functions), a method to reconstruct missing data in geophysical data sets, is presented. Based on a truncated Empirical Orthogonal Functions (EOF) basis, DINEOF uses an iterative procedure to calculate the values at the missing locations. A clear advantage of DINEOF is that no aprioriate knowledge about the statistics of the data set being reconstructed is needed (such as covariance or correlation length): the EOF basis is used internally to infer necessary information about the data, so no estimation of those parameters is needed. This characteristic is specially interesting for heterogeneous data distributions for which is difficult to derive this information. Also obtained are estimations of the error covariance of the reconstructed field, and outliers, i.e. data that present anomalous values with respect to the surrounding information in the original data, for which the residuals are larger than the statistically expected misfit calculated during the analysis.
When very few data is available, the estimated covariance between two successive images used in the EOF calculation might not sufficiently robust. As a consequence, spikes appear in the temporal EOFs, which result in unrealistic discontinuities in the reconstruction. A temporal filter has been applied to the covariance matrix used to determined the EOFs, which effectively enhance temporal continuity. This has been applied to a SST data set of the Black Sea and the reconstruction error is estimated by cross-validation.
On-going work includes the development of a merging capability within DINEOF that will allow to blend data from different platforms (satellite and in situ data).