Data Interpolating Empirical Orthogonal Functions (DINEOF): a tool for geophysical data analyses

in Mediterranean Marine Science (in press)

An overview of the technique called DINEOF (Data Interpolating Empirical Orthog- onal Functions) is presented. DINEOF reconstructs missing information in geophys- ical data sets, such as satellite imagery ... [more ▼]

An overview of the technique called DINEOF (Data Interpolating Empirical Orthog- onal Functions) is presented. DINEOF reconstructs missing information in geophys- ical data sets, such as satellite imagery or time series. A summary of the technique is given, with its main characteristics, recent developments and future research di- rections. DINEOF has been applied to a large variety of oceanographic variables in various domains of different sizes. This technique can be applied to a single variable (monovariate approach), or to several variables together (multivariate approach), with no complexity increase in the application of the technique. Error fields can be computed to establish the accuracy of the reconstruction. Examples are given to illustrate the capabilities of the technique. DINEOF is freely offered to download, and help is provided to users in the form of a wiki and through a discussion email list. [less ▲]

Generation of analysis and consistent error fields using the Data Interpolating Variational Analysis (Diva)

in Ocean Modelling (2012), 52-53

The Data Interpolating Variational Analysis (Diva) is a method designed to interpolate irregularly-spaced, noisy data onto any desired location, in most cases on regular grids. It is the combination of a ... [more ▼]

The Data Interpolating Variational Analysis (Diva) is a method designed to interpolate irregularly-spaced, noisy data onto any desired location, in most cases on regular grids. It is the combination of a particular methodology, based on the minimisation of a cost function, and a numerically efficient method, based on a finite-element solver. The cost function penalises the misfit between the observations and the reconstructed field, as well as the regularity or smoothness of the field. The intrinsic advantages of the method are its natural way to take into account topographic and dynamic constraints (coasts, advection, . . . ) and its capacity to handle large data sets, frequently encountered in oceanography. The method provides gridded fields in two dimensions, usually in horizontal layers. Three-dimension fields are obtained by stacking horizontal layers. In the present work, we summarize the background of the method and describe the possible methods to compute the error field associated to the analysis. In particular, we present new developments leading to a more consistent error estimation, by determining numerically the real covariance function in Diva, which is never formulated explicitly, contrarily to Optimal Interpolation. The real covariance function is obtained by two concurrent executions of Diva, the first providing the covariance for the second. With this improvement, the error field is now perfectly consistent with the inherent background covariance in all cases. A two-dimension application using salinity measurements in the Mediterranean Sea is presented. Applied on these measurements, Optimal Interpolation and Diva provided very similar gridded fields (correlation: 98.6%, RMS of the difference: 0.02). The method using the real covariance produces an error field similar to the one of OI, except in the coastal areas. [less ▲]

Viewing through the clouds in satellite images

Poster (2012, February 24)

Study of the ecology, population structure and dynamic of the macroalgae Codium elisabethae in Faial (Azores) with underwater visible imagery.

Doctoral thesis (2011)

Codium elisabethae O.C. Schmidt is a dark green globose macroalgae isolating an internal sea water volume in a lumen. Codium elisabethae is endemic to the Macaronesian region and is very similar to Codium ... [more ▼]

Codium elisabethae O.C. Schmidt is a dark green globose macroalgae isolating an internal sea water volume in a lumen. Codium elisabethae is endemic to the Macaronesian region and is very similar to Codium bursa C. Agardh whose distribution range spans the West-European, North-Western African and Mediterranean coasts and which was proposed as a potential indicator of coastal environmental changes based on the study of its ecology, revealing its long lifespan. Until recently, relatively little was known on Codium elisabethae as compared to the more widespread Codium bursa. To fill this gap, the present research aimed at producing an accurate description of the ecology and population dynamics of Codium elisabethae occupying the rocky shores of the Monte da Guia Special Area of Conservation (SAC)/Natura 2000 network (Faial, Azores). To achieve this, two reference sites were selected for long term monitoring: a sheltered no-go reserve exhibiting a dense Codium elisabethae population (Caldeirinhas), and a location experiencing more exposed conditions holding a sparser population (Ponta Furada). First, environmental conditions experimented by benthic organisms were extensively quantified and interpretated in regard to topographical particularities of each site. The study of reproduction dynamics showed a persistent summer fertility and an important vegetative reproduction. Important nutrient concentration ratio was found between the Codium elisabethae lumen water and surrounding sea water (mean ratios: nitrates: 5.7; ammonium: 3.4; phosphates: 3.1). In situ counting’s and size measurements revealed much higher densities of young recruits in the site of the Caldeirinhas (order of 20 ind/m²) than in the one of Ponta Furada (order of 1/m), for both summer and winter. Secondly, underwater visible imagery was exploited as an efficient and non-invasive alternative to classical in situ population estimation. Between August 2003 and November 2005, fifteen seabed photo coverages were collected by scuba-divers. Subsequent image processing consisted in mosaicing, interactive identification, and automatic change detection methods. This allowed quantifying the seasonal fluctuations of population structures (density, percentage cover and biomass) and of population dynamics (growth, recruitment, mortality and primary production). Chi-square tests of image-derived estimates and in situ measurements confirmed the validity of a centimeter precision for the estimation of population structure of individuals above 4 cm diameter. Important variability of population structure and density was observed within the sites at small spatial scales. Significant differences of population structure and dynamics parameters are demonstrated between two close-by but contrasting coastal habitats. Population density showed a sharp reduction in autumn 2003 and did not recover fully in spring and summer 2004. During the following year, population of the protected site maintained density and biomass, while at the exposed site population density dropped. In contrast with conclusions from earlier studies on the Azorian Codium elisabethae and on the Mediterranean Codium bursa, the present study revealed higher biomass (34 - 730 g dry wt.m-²), growth rates (up to 2.5 cm/month in summer) and primary production (0.53 – 11.5 g dry wt.m-².day-1), and demonstrated the seasonal fluctuations of these parameters for the studied Azorian Codium elisabethae population. The lifespan of Codium elisabethae was estimated to reach at least 7 years in the SAC of Monte da Guia based on an integration of average seasonal growth rates measured by imagery on extended population samples. This study demonstrates the high potential of registered underwater photomosaics time-series for long term surveys of macroalgae populations. This work provides also a strong framework to further developments and applicability to other species, which should be helpful to strengthen our current understanding of benthic ecosystem processes. [less ▲]

Advanced Data Interpolating Variational Analysis. Application to climatological data.

Poster (2011, March 21)

High-resolution Climatology of the northeast Atlantic using Data-Interpolating Variational Analysis (DIVA)

in Journal of Geophysical Research (2010), 115(C08005), 20

Numerous climatologies are available at different resolutions and cover various parts of the global ocean. Most of them have a resolution too low to represent suitably regional processes and the methods ... [more ▼]

Numerous climatologies are available at different resolutions and cover various parts of the global ocean. Most of them have a resolution too low to represent suitably regional processes and the methods for their construction are not able to take into account the influence of physical effects (topographic constraints, boundary conditions, advection, etc.). A high-resolution atlas for temperature and salinity is developed for the northeast Atlantic Ocean on 33 depth levels. The originality of this climatology is twofold: (1) For the data set, data are collected on all major databases and aggregated to lead to an original data collection without duplicates, richer than the World Ocean Database 2005, for the same region of interest. (2) For the method, climatological fields are constructed using the variational method Data-Interpolating Variational Analysis. The formulation of the latter allows the consideration of coastlines and bottom topography, and has a numerical cost almost independent on the number of observations. Moreover, only a few parameters, determined in an objective way, are necessary to perform an analysis. The results show overall good agreement with the widely used World Ocean Atlas, but also reveal significant improvements in coastal areas. Error maps are generated according to different theories and emphasize the importance of data coverage for the creation of such climatological fields. Automatic outlier detection is performed, and its effects on the analysis are examined. The method presented here is very general and not dependent on the region, hence it may be applied for creating other regional atlas in different zones of the global ocean. [less ▲]

High-resolution measurements and modelling of the Cape Ghir upwelling filament during the CAIBEX cruise

Conference (2010, April 26)

A web interface for gridding and visualizing oceanographic data sets

Conference (2010, March)

Spatial interpolation of observations on a regular grid is a common task in many oceanographic disciplines (and geosciences in general). Diva (Data-Interpolating Variational Analysis) is an analysis tool ... [more ▼]

Spatial interpolation of observations on a regular grid is a common task in many oceanographic disciplines (and geosciences in general). Diva (Data-Interpolating Variational Analysis) is an analysis tool for gridding oceanographic in situ data. Diva takes the error in the observations and the typical spatial scale of the underlying field into account. Barriers due to the coastline and the topography in general are also used to propagate the information of a given observation spatially. Diva is a command-line driven application. To make Diva easier to use, a web interface has been developed. The user can directly upload his/her data in ASCII format and enter several parameters for the analysis. The analyzed field, location of the observations, and the error mask are then directly visualized in the browser. While this interface allows the user to create his/her own gridded field, a web interface is also developed to visualize pre-computed gridded oceanographic data sets. Those data sets are typically four-dimensional (longitude, latitude, depth and time). The system allows to visualize horizontal sections at a given depth and time to study the horizontal distribution of a given variable. It is also possible to display the results on an arbitrary vertical section. To study the evolution of the variable in time, the horizontal and vertical sections can also be animated. The user can customize the plot by changing the color-map, the range of the color-bar, the type of the plot (linearly interpolated color, simple contours, filled contours) and download the current view as a simple image or as Keyhole Markup Language (KML) file for visualization in applications such as Google Earth. The system is build using a client and server architecture. The server is written in Python using the Web Server Gateway Interface. The server implements version 1.1.1 and 1.3.0 of the Web Map Service (WMS) protocol of the Open Geospatial Consortium. On the server, all oceanographic data sets are stored as NetCDF files organized in folders and sub-folders allowing for a hierarchical presentation of the available variables. The client is build as a web application using the OpenLayers Javascript library. The web interface is accessible at http://gher-diva.phys.ulg.ac.be/. It is currently used for climatologies created in the frame of the SeaDataNet project and will be used for the EMODNET project (chemical lot). Thrid-party data centers can also integrate the web interface of Diva to show an interpolated field of in situ data as an additional WMS layer. A demonstration near-real time cloud-free sea surface temperature (SST) product of the Mediterranean Sea is presented. The reconstruction of the data set missing information (due to clouds, for example) is realised using DINEOF (Data Interpolating Empirical Orthogonal Functions). DINEOF is an EOF-based technique that does no need a priori information about the data set (such as signal to noise ratio, or correlation length) and that has shown to be faster and equally reliable than other widely used techniques for reconstructing missing data, such as optimal interpolation. Here we present a daily reconstruction of the Western Mediterranean SST. Cloudy data are downloaded from the Ifremer Medspiration ftp site. After extracting the data from the study zone, they are added to a data set containing the last 6 months of SST. A first DINEOF reconstruction is performed to identify outliers, i.e. pixels for which the analysis-observation difference (the residuals) are larger than the statistically expected misfit calculated during the analysis. Proximity to a cloud edge and deviation respect to a local median also penalize a pixel in the outlier classification. These outliers are removed from the original data set, and a second DINEOF reconstruction is performed, along with the calculation of error maps. Plots are realised, and the reconstruction of the latest 10 days is shown at http://gher-diva.phys.ulg.ac.be/DINEOF/dineof.html, together with the original data, the error maps and identified outliers. The whole procedure takes less than two hours and has been running automatically for more than 5 months. This product is intended as a demonstration of the capabilities of DINEOF as a near-real time technique to reconstruct missing data in satellite data sets. This procedure can be easily applied to other variables and other geographical zones. [less ▲]

DIVA: new features

Scientific conference (2009, October 23)

Evolution of Western Mediterranean Sea Surface Temperature between 1985 and 2005

Conference (2009, April 20)