Document Actions
Verification
Verification by the Ecological Turing Test
Testing the veracity of a virtual ecosystem by comparing selected emergent properties with observations, while paying attention to the uncertainties in each.
Goal
The aim of virtual ecology is to create simulations that are consistent with all available observations. It may be necessary to improve a model or the scenario to achieving that goal. When it has been achieved the virtual ecosystem provides the best available description of the ecosystem under the specified conditions. It is more comprehensive than any set of observations, and serves better for testing conjectures about the causes of observed phenomena.
Errors
Both observation and simulation contain significant errors, which must be assessed before using the former to test the latter.
(1) In observations
Apart from the usual measurement errors in the instruments, the principal source of uncertainty lies in sampling errors. The existence of mesoscale patchiness poses severe problems for observations.
(2) In Virtual Ecosystem.
Sampling errors are also an important cause of uncertainty in the emergent properties of virtual ecosystems. They arise from the use of a limited number of particles to represent the plankton populations. This source of error can be estimated from the inter-instance variance of ensembles of instances of the virtual ecosystem.
More fundamental is the lack of biodiversity represented in the model. It is not feasible to represent all 10,000 of so species of plankton that may be active in the ocean basin. Considerable skill is needed to select a small number of functional groups and representative species to achieve an acceptable simulation of biodiversity and therefore of the emergent environment.
Verification
The Ecological Turing Test is designed to answer the question: Could the Virtual Ecosystem and the Observations both be describing the same ecosystem? Or are the differences between them too large for that? The comparison applies only to emergent properties that can be observed, and vice versa. That severely the scope for testing. It is further limited by the errors in observation and simulation.
Improving the Virtual Ecosystem
When the Virtual Ecosystem fails the Turing test, the aim must be to identify and correct the source of the errors. The corrections must be consistent with the metamodel, and with the commitment to primitive equation modelling.
Tuning
There is no role in Virtual Ecology for tuning model parameters to improve the fit to observations.
Case study
Goal
The test target is the date of transition to summer oligotrophy, when the phytoplankton biomass in the mixed layer attains its annual maximum value, Pmax (see "Global Stability of a Virtual Ecosystem"). This will be compared in two data sets: (1) the ecosystem in a virtual mesocosm held at a fi xed location (27°W 41°N) near the Azores, (2) satellite ocean colour measurements at that site.
Method
The virtual ecosystem is created with a scenario based on weather data with a spatial resolution of one degree of latitude and longitude (approximately 100 km x 100 km). The satelite data have a spatial resolution of approximately 10 km. So the satellite data have one hundred samples in the area of the prediction. There is considerable scatter in the estimate of the date of Pmax. That scatter is attributed to mesoscale patchiness in the ecosystem, which is not represented in the one-dimensional simulation.
Results
The figure shows the comparison of the 100 samples of ocean colour and two versions of the virtual ecosystem based on different optical sub-models. Despite the uncertainties in the observations, there is a significant difference between the dates of Pmax in the observations and prediction. So the Ecological Turing test was successful in this case. It showed that there is a need to improve the specification of the virtual ecosystem
Remedy
The ten-day difference between Pmax in observation and simulation was eliminated by two improvements to the specification:
- using a model with three species of phytoplankton (rather than the one used for the test),
- upgrading the parametrization used to compute the infl uence of clouds on solar radiation.
After making these changes there was no significant difference between the observation and prediction.
Conclusion
Having made those two upgrades, the ocean colour data had no further potential for verifying the virtual ecosystem. At least, not for this particular target. In practice it is difficult to find other targets that give such a clear result. 1
