From descriptive to predictive distribution models: a working example with Iberian amphibians and reptiles
© Arntzen; licensee BioMed Central Ltd. 2006
Received: 17 January 2006
Accepted: 04 May 2006
Published: 04 May 2006
Aim of the study was to identify the conditions under which spatial-environmental models can be used for the improved understanding of species distributions, under the explicit criterion of model predictive performance. I constructed distribution models for 17 amphibian and 21 reptile species in Portugal from atlas data and 13 selected ecological variables with stepwise logistic regression and a geographic information system. Models constructed for Portugal were extrapolated over Spain and tested against range maps and atlas data.
Descriptive model precision ranged from 'fair' to 'very good' for 12 species showing a range border inside Portugal ('edge species', kappa (k) 0.35–0.89, average 0.57) and was at best 'moderate' for 26 species with a countrywide Portuguese distribution ('non-edge species', k = 0.03–0.54, average 0.29). The accuracy of the prediction for Spain was significantly related to the precision of the descriptive model for the group of edge species and not for the countrywide species. In the latter group data were consistently better captured with the single variable search-effort than by the panel of environmental data.
Atlas data in presence-absence format are often inadequate to model the distribution of species if the considered area does not include part of the range border. Conversely, distribution models for edge-species, especially those displaying high precision, may help in the correct identification of parameters underlying the species range and assist with the informed choice of conservation measures.
Atlases of distribution data have provided a popular and successful way of assembling spatial information on a variety of organisms, including amphibians and reptiles. Most mapping projects are terminated on the production of the atlas while others are continued to improve coverage and to increase spatial and temporal resolution [1–3]. Attempts to analyse and interprete atlases have been infrequent [4, 5]. Moreover, the building of distribution models is not always clearly distinguished from the evaluation of the results. These are serious shortcomings because data plots require interpretation before they can be used for i.e., increased biological understanding, policy making and conservation management [6, 7]. Even for the best of empirical data sets, the analysis is not straightforward because one has to choose from a wide variety of spatial-statistical analytical techniques, each of which carries assumptions that are unlikely to be met in full [8–10]. Inadequate and uneven sampling, grid cell data notation instead of exact localities, identification error, colinearity, spatial autocorrelation and other non-independencies of the data and statistical artefacts will bias parameter estimates and therewith affect model selection and results [11–15]. More recently, the emphasis has been how species-specific traits such as habitat usage, detectability, prevalence and tolerance may affect model performance [16–18]. In the present paper I aim to analyse distribution patterns of selected species through the joint analysis of distributional and environmental data, first, to construct descriptive distribution models in a GIS (Geographical Information System) environment; second, to evaluate the usefulness of the models for biological interpretation by testing their performance outside the area in which they were generated; and third, to test the hypothesis that low vs. high precision of descriptive models would be associated with low vs. high accuracy in predictive settings. I choose to work with the amphibians and reptiles of the Iberian Peninsula. Whereas the Portuguese herpetofauna is well surveyed in terms of density and spread, mostly by a single, qualified researcher (Mr. R. Malkmus), records for Spain are less dense, unevenly distributed and obtained from a wide variety sources. Hence, the data for Portugal and Spain serve for model constructing and model testing, respectively.
This paper presents distribution models for amphibians and reptiles in Portugal build from environmental variables. The species are classified according to whether their Portuguese distribution includes part of the range border or not. The descriptive performance of both groups of models is compared, finding that the edge species perform significantly better. Alternative models using only search effort as predictor variable perform better than environmental models for countrywide species, whereas the opposite is true for edge species. The predictive performance of edge species' models in Spain was also better than that of countrywide species.
The correlation between model precision in Portugal and model accuracy for Spain across all species was significant (rs = 0.44, P < 0.01). Analysed separately, the level of significance decreased in the group of countrywide species to rs = -0.01 (not significant) and increased in the group of edge species to rs = 0.71 (P < 0.01; Fig. 5B).
For a given species, occupied and non-occupied areas of the globe will be separated by a – more or less abstract – generalized limit of distribution, characterized by a zone in which populations are few and far between and subject to periodic extinction [20, 21]. This zone can be wide or narrow, yielding vague or sharp species borders, respectively. In vagile species such as birds, large mammals and flying insects, the range borders will tend to be wide and vague, whereas in species with low dispersal capacity, including most amphibians and reptiles, non-flying insects etc., range borders will tend to be narrow and sharp. In the current study 26 species (68%) were classified as possessing a countrywide Portuguese distribution whereas 12 species (32%) were identified as occurring in one part of the country and not the other. Other researchers, dealing with the same set of organisms, differentiate among species by 'prevalence', 'occupancy', 'extent of occurrence' (along latitudinal axis), 'marginality' and 'tolerance' and recognize four groups [see Table 2 in ]. My classification coincides with their 'extent of occurrence' criterion, along both (latitudinal and longitudinal) axes. More survey data, such as underway for the 'National Atlas of Amphibians and Reptiles of Portugal' (A. Loureiro et al., in prep.), support my argument for a naturally bimodal grouping at the UTM10 grid scale.
Twelve species with range borders inside the research area, i.e., Portugal, were profitably modeled from atlas data. In contrast to this, the modeled atlas data for countrywide species reflected more adequately the single variable search effort than the most informative combination out of 13 environmental variables, in 21 cases out of 26 (all open dots in the up-diagonal corner of Fig. 5A). This result indicates that the absence in a particular grid cell of a countrywide species is more likely due to inadequate sampling than that it would reflect its real absence due to local environmental conditions. The conclusion that false absences override ecological signal in these analyses may seem trivial but ecographical models for countrywide species are currently included in methodological comparisons, including approaches relying on presence and absence data . I suggest that species distributed all over the area of investigation – in this study amphibians and reptiles with a countrywide Portuguese distribution – would better be excluded from such exercises, until more detailed surveying data would show firm evidence for internal species borders. This conclusion is supported by the lack of published ecographical models for countrywide species, whereas edge species are readily documented [22–24].
How to proceed from the notion that non-edge species are not adequately modeled from presence/absence data? If search effort could be controlled for, either statistically or by using a homogenous sampling effort in field studies, then presences could reflect abundance, which would be useful for modeling habitat suitability. Efforts should be made in the preparation of atlases to include information on the abundance of species instead of presenting just occurrence data, to begin with species for which observations are readily made such as widespread and locally abundant species. A proxy measure for the abundance of a species could be the effort in the terrain required to demonstrate its presence. For yet other approaches see e.g. Johnson & Sargeant  and Nielsen et al. .
Among the countrywide species, the terrapin Mauremys leprosa and the snake Macroprotodon cucullatus stand out on account of a low impact of search effort E' on the descriptive model fit and a fair predictive model performance (Fig. 5). Mauremys leprosa is well surveyed as part of the EC-LIFE project  and this work reveals a mixed distribution pattern, in which the range is continuous at UTM10 grid scale in the south and the east of the country, semi-continuous in the mid-west and, finally, with isolated occurrences in the northwest of Portugal. Its distribution appears countrywide, but from some regions it is nevertheless distinctly absent, therewith defying the edge- versus non-edge species dichotomy. The fragmented distribution pattern suggests a contracting range and obviously such a non-equilibrium situation would be difficult to model accurately. Macroprotodon cucullatus was not classified as an edge-species, even though it appears absent from the mid- and north-western parts of Portugal. Judgement is hampered by a low number of recordings from the regions where it does occur. Future survey data will hopefully clarify this issue.
Environmental correlates of edge-species
The environmental variables that appear to strongly influence amphibian and reptile distributions in Portugal, for those species with a range border inside the country, are the climatic variables precipitation (PRET), insolation (INSO) and July temperature (TJUL). Parameters that appear to influence distributions less often are chemical composition of surface water (ACID), the climatic parameters describing frost conditions (FROM) and humidity and the thickness of the vegetation cover as measured by NDVI. Nevertheless, the latter parameters may have large perceived impact (values > 1 in Table 1) on particular species distributions, such as ACID in the case of C. lusitanica.
Species for which model fit is highest are three Iberian endemics with restricted distributions in the north of the country. However, parameter selection is not the same, indicating dissimilar ecological regimes. The generalized habitats can be summarized by high precipitation at low and medium altitudes for C. lusitanica, high precipitation in the case of R. iberica and low summer temperature and low insolation for L. schreiberi. These ecological descriptions at the scale of species ranges are complementary to more small-scale field observations that would characterize each of the three species as inhabiting the immediate vicinity of mountain brooks [28–30]. Although not an Iberian endemic, A. fragilis with a well-defined, clearly circumscribed distribution complements this series. Its generalized habitat is characterized by low summer temperature, low insolation and high precipitation. It is the only species for which the distribution model strongly suggests range subdivision over three mountainous areas separated by the rivers Douro and Mondego. Interestingly, a very similar pattern of isolation and differentiation was revealed for C. lusitanica with the help of molecular genetic markers .
The generalized habitats of five southern species can be summarized as follows: high annual temperature for P. ibericus and H. meridionalis, high annual temperature and low precipitation for T. pygmaeus, high annual temperature in non-mountainous terrain for P. waltl and low precipitation and high insolation in the case of A. cisternasii. The small-scale, local habitats of the five species are not broadly overlapping, either through a differential focus on ephemeral ponds (P. ibericus), permanent ponds (T. pygmaeus) and streams (A. cisternasii) for breeding and pre-metamorphic life, or by a differential habitat preference at the post-metamorphic stage, that is either largely terrestrial (H. meridionalis) or largely aquatic (P. waltl).
Distribution models for ten amphibians and two reptiles by logistic regression analysis of presence-absence data for Portugal. Model conditions are with 13 variables (condition 1) or with nine variables (condition 2; details see text). Environmental data are standardized, except for LITH. Paramers with large effect (values >1) are shown in boldface type. The fit of the descriptive models is expressed by Cohen's kappa and the 'Area Under the Curve in Receiver Operating Characteristic' plots (AUC) and asymptotic standard error (SE). Distribution models derived from the equations are shown as 'probability of occurrence' maps for Portugal in figures 1 and 2 and as range maps for the Iberian Peninsula in figures 3 and 4.
Model equation †
AUC ± SE
a) Chioglossa lusitanica
0.989 ± 0.002
0.987 ± 0.003
b) Pleurodeles waltl
0.745 ± 0.018
c) Triturus marmoratus
0.853 ± 0.013
0.854 ± 0.013
d) T. pygmaeus
0.751 ± 0.018
e) Alytes cisternasii
0.871 ± 0.012
f) A. obstetricans
0.879 ± 0.013
0.879 ± 0.013
g) Hyla arborea
0.742 ± 0.021
0.682 ± 0.023
h) H. meridionalis
0.777 ± 0.02
i) Pelodytes ibericus
0.863 ± 0.016
0.843 ± 0.018
j) Rana iberica
0.936 ± 0.008
0.935 ± 0.008
k) Anguis fragilis
0.876 ± 0.016
l) Lacerta schreiberi
0.899 ± 0.010
0.897 ± 0.010
Among edge-species, a statistically significant relationship was found between precision and accuracy of the models, that is, between model fit over documented and extrapolated parts of the range. At one side of the continuum, distribution models with low precision are unlikely to help much in the identification of the ecological parameters underlying species distributions. The information extracted from poor modelling results is unlikely to be useful and should not be made operational. Improved models may be obtained through a biologically better informed selection of explanatory variables [cf. ]. On the other side of the continuum of low versus high model performance, well-fitting descriptive models do tend to predict distributions rather well. Such models yield biologically meaningful information, potentially leading to an improved understanding of species' ecological requirements – information that subsequently can be made to use in conservation management. At the scale of the amphibians and reptiles of Portugal this condition however applies to a minority of the species.
Distribution data on the Portuguese herpetofauna (18 amphibian and 27 reptile species) organised in Universal Transverse Mercator (UTM) grid cells with a spatial resolution of 10 × 10 km were taken from Godinho et al. . This is basically the work of Malkmus  upgraded with data from the literature, theses, technical reports, single species accounts, etc. Species were a priori divided in two groups of (1) edge species, that show a range border inside continental Portugal and (2) countrywide (or non-edge) species, which do not show a range border inside continental Portugal. This distinction was made to test the hypothesis that the spatial-environmental modelling of atlas data is only potentially meaningful if at least some of the reported absences are definitely real (as in group 1 species). Conversely, presented absences for group 2 species could – ultimately – all be 'false absences' (grid cells for which the species was not recorded despite its presence).
Twelve edge species in group 1 are the salamanders Chioglossa lusitanica and Pleurodeles waltl, the frogs Hyla arborea, H. meridionalis, Pelodytes ibericus and Rana iberica, the toads Alytes cisternasii and A. obstetricans and the lizards Anguis fragilis and Lacerta schreiberi; sufficient data were also available for two fairly cryptic newt species Triturus marmoratus and T. pygmaeus following the reconstruction of the contact zone between them in central Portugal (JWA & G. Themudo, unpublished data). The lizard 'Podarcis bocagei' incorporates a cryptic species and both were excluded due to low record numbers . Twenty-six non-edge species in group 2 are two salamanders, two frogs, three toads, two terrapins, nine lizards and eight snakes (for species names see Appendix 1). Grid cells for which less than 50% of the coverage was for continental Portugal and species with less than 50 grid cell records (n = 7, Appendix 1) were excluded from the analyses . The amphibian and reptile distribution data for Spain used to evaluate the models were species range maps [37–39] and atlas data organized in the UTM10-grid .
Search effort is defined as E=(Nobs/Nmax)2, with Nobs = the number of species actually observed in a grid cell and Nmax = the maximum number of species that could be observed in that grid cell, according to atlas data and range maps, respectively. Search effort was calculated for each relevant grid cell of the Iberian Peninsula. E' was used as a measure of search effort uncoupled from the species under consideration, that is, with data for that species excluded. These functions incorporate the notion that cells with greater surveying effort will have more species reported, but that, once the effort becomes substantial, new species will be added only slowly.
Twenty-one ecologically meaningful environmental parameters were pre-selected for analysis. For 18 variables information was available in digital format for Portugal . A vegetation map (normalised difference vegetation index or NDVI) was obtained courtesy of the Royal Dutch Meteorological Institute (KNMI). An altitude map was taken from the internet http://edcwww.cr.usgs.gov/ and used to produce a relief map by a set of filter operations . Maps of the mean January and July temperatures were digitised from the Portuguese climate atlas . Variables were (de)selected using criteria of i) redundancy at rs >0.8, ii) availability for both Portugal and Spain and iii) promise in terms of amphibian and reptile life history. The following 13 variables were retained in three groups : climatic variables – annual precipitation (PRET, in mm), humidity of the air (HUMI, in %), average temperature over the year (TEMP, in °C) and for July (TJUL, in °C), annual number of frost days (FROD, in days), annual number of frost months (FROM, in months) and insolation (INSO, in hours); topographical variables – altitude (ALTI, in m asl), relief (RELI, in arbitrary scale) and geological/geochemical variables – acidity of surface water (ACID, pH in 14 classes), hardness of the water (HARD, CaCO3 mg/l in 17 classes), lithology (LITH, in the classes, sedimentary, sedimentary and metamorphic and igneous). Finally, the vegetation index (NDVI) may be seen as belonging to the latter group of variables. For detail on these candidate predictor variables see Appendix 2. Data for Spain were digitised from the Atlas Nacional de España . All variables (except LITH) were standardized to an average of zero and standard deviation of unity, to increase the comparability of their effects. The variables were introduced into the GIS analytical software as raster layers with 1 km spatial resolution. Mean values for 10 × 10 km UTM grid cells were obtained by averaging the data (modal values for the categorical variable LITH).
Analysis and modelling
Logistic regression analyses were performed with SPSS 12  with a forward stepwise addition of independent variables and with Bonferroni correction to the initial α = 0.05 . The impact of presences and absences was equilibrated through a weighting variable, using the 'weight cases' option in SPSS. Thus, the outnumbering case, either presences or absences, was down weighted to obtain a balanced dataset with effectively a fifty/fifty distribution of presence and absences. Extrapolation of the models to Spain was performed on the basis of nine environmental variables for which data were available (all but ACID, FROM, HARD and LITH). The strength of agreement among distribution data and distribution models was summarized with Cohen's kappa , using a 50% threshold, and with 'Area Under the Curve' statistics (AUC), determined from so-called Receiver Operated Character plots [48, 49] in SPSS.
The project was partially funded by the 'Fundação para a Ciência e a Tecnologia' project number POCTI/BSE/34110/99. I thank an anonymous reviewer for insightful comments that helped to strengthen some of the arguments.
- Bergmans W, Zuiderwijk A: Atlas van de Nederlandse amfibieën en reptielen en hun bedreiging. 1986, Hoogwoud: Koninklijke Nederlandse Natuurhistorische Vereniging
- Coelen JEM van der , (Ed.): Verspreiding en ecologie van amfibieën en reptielen in Limburg. 1992, Nijmegen: Stichting RAVON
- Araújo MB, Thuiller W, Williams PH, Reginster I: Downscaling European species atlas distributions to a finer resolution: implications for conservation planning. Global Ecol Biogeogr. 2005, 14: 17-30. 10.1111/j.1466-822X.2004.00128.x.View Article
- African Mammals Databank: A databank for the conservation and management of the African mammals. 1999, Brussels and Rome: European Commission Directorate-General for Development Division VIII/A/1, Istituto Ecologia Applicata
- Hofer U, Monney J-C, Dušej G: Die Reptilien der Schweiz: Verbreitung, Lebensräume, Schutz. 2001, Basel: Birkhäuser
- Brito JC, Godinho R, Luís C, Paulo OS, Crespo EG: Management strategies for conservation of the lizard Lacerta schreiberi in Portugal. Biol Conserv. 1999, 89: 311-319. 10.1016/S0006-3207(99)00002-6.View Article
- Gates S, Donald PF: Local extinction of British farmland birds and the prediction of further loss. J Appl Ecol. 2000, 37: 806-820. 10.1046/j.1365-2664.2000.00549.x.View Article
- James FC, McCulloch CE: Multivariate analysis in ecology and systematics: panacea or Pandora's box?. Ann Rev Ecol Syst. 1990, 21: 129-166.View Article
- Manel S, Dias JM, Ormerod SJ: Comparing discriminant analysis, neural networks and logistic regression for predicting species' distributions: a case study with a Himalayan river bird. Ecol Model. 1999, 120: 337-347. 10.1016/S0304-3800(99)00113-1.View Article
- Olden JD, Jackson DA: A comparison of statistical approaches for modelling fish species distributions. Freshwater Biol. 2002, 47: 1976-1995. 10.1046/j.1365-2427.2002.00945.x.View Article
- Legendre P, Legendre L: Numerical ecology. Developments in Environmental Modelling 20. 1998, Amsterdam: Elsevier
- Mac Nally R: Regression and model-building in conservation biology, biogeography and ecology: the distinction between- and reconciliation of – 'predictive' and 'explanatory' models. Biodiv Conserv. 2000, 9: 655-671. 10.1023/A:1008985925162.View Article
- Legendre P, Dale MRT, Fortin M-J, Gurevitch J, Hohn M, Myers D: The consequences of spatial structure for the design and analysis of ecological field surveys. Ecography. 2002, 25: 601-615. 10.1034/j.1600-0587.2002.250508.x.View Article
- Scott JM, Heglund PJ, Morrison M, Haufler JB, Raphael MG, Wall WA, Samson FB, (Eds): Predicting species occurrences: issues of scale and accuracy. 2002, Washington DC: Island Press
- McPherson JM, Jetz W, Rogers DJ: The effects of species' range sizes on the accuracy of distribution models: ecological phenomenon or statistical artefact ?. J Appl Ecol. 2004, 41: 811-823. 10.1111/j.0021-8901.2004.00943.x.View Article
- Brotons L, Thuiller W, Araújo MB, Hirzel AH: Presence-absence versus presence-only modelling methods for predicting bird habitat suitability. Ecography. 2004, 27: 437-448. 10.1111/j.0906-7590.2004.03764.x.View Article
- Segurado P, Araújo MB: An evaluation of methods for modelling species distributions. J Biogeogr. 2004, 31: 1555-1568. 10.1111/j.1365-2699.2004.01076.x.View Article
- Seoane J, Carrascal LM, Alonso CL, Palomino D: Species-specific traits associated to prediction error in bird habitat suitability modelling. Ecol Model. 2005, 185: 299-308. 10.1016/j.ecolmodel.2004.12.012.View Article
- Altman DG: Practical statistics for medical research. 1991, London: Chapman & Hall
- Gorodkov KB: Three-dimensional climatic model of potential range and some of its characteristics. II. Entomol Rev. 1986, 65: 1-18.
- Gaston KJ: The structure and dynamics of geographic ranges. Oxford Series in Ecology an Evolution. 2003, New York: Oxford University Press
- Brito JC, Crespo EG, Paulo OS: Modelling wildlife distributions: logistic multiple regression vs overlap analysis. Ecography. 1999, 22: 251-260. 10.1111/j.1600-0587.1999.tb00500.x.View Article
- Sá-Sousa P: A predictive distribution model for the Iberian wall lizard (Podarcis hispanicus) in Portugal. Herpetol J. 2002, 10: 1-11.
- Teixeira J, Ferrand N, Arntzen JW: Biogeography of the golden-striped salamander, Chioglossa lusitanica: a field survey and spatial modelling approach. Ecography. 2001, 24: 618-624. 10.1034/j.1600-0587.2001.d01-216.x.View Article
- Johnson DH, Sargeant GA: Towards better atlases: improving presence-absence information. Predicting species occurrences: issues of accuracy and scale. Edited by: Scott JM, Heglund PJ, Morrison M, Haufler JB, Raphael M G, Wall WA, Samson FB. 2002, Washington DC: Island Press, 391-398.
- Nielsen SE, Johnson CJ, Heard DC, Boyce MS: Can models of presence-absence be used to scale abundance ? Two case studies considering extremes in life history. Ecography. 2005, 28: 197-208. 10.1111/j.0906-7590.2005.04002.x.View Article
- Araújo PR, Segurado P, Raimundo N: Bases para a conservação das tartarugas de água doce Emys orbicularis e Mauremys leprosa. Estudos de Biologia e Conservação da Natureza, 24. 1997, Lisboa: Instituto da Conservação da Natureza
- Malkmus R: Die Amphibien und Reptilien Portugals, Madeiras und der Azoren. 1995, Magdeburg: Westarp Wissenschaften
- Salvador A, García-París M: Anfibios españoles: identificación, historia natural y distribución. Edited by: Talavera de la Reina: Canseco. 2001
- Salvador A, Pleguezuelos JM: Reptiles españoles: identificación, historia natural y distribución. Edited by: Talavera de la Reina; Canseco. 2002
- Alexandrino J, Arntzen JW, Ferrand N: Nested clade analysis and the genetic evidence for population expansion in the phylogeography of the golden-striped salamander, Chioglossa lusitanica (Amphibia: Urodela). Heredity. 2002, 88: 66-74. 10.1038/sj.hdy.6800010.View ArticlePubMed
- Osborne PE, Suárez-Seoane S: Should data be partitioned spatially before building large-scale distribution models ?. Ecological Modelling. 2002, 157: 249-259. 10.1016/S0304-3800(02)00198-9.View Article
- Burnham P, Anderson DR: Model selection and inference. A practical information-theoretic approach. 1998, New York: SpringerView Article
- Godinho R, Teixeira J, Rebelo R, Segurado P, Loureiro A, Álvares F, Gomes N, Cardoso P, Camilo-Alves C, Brito JC: Atlas of the continental Portuguese herpetofauna: an assemblage of published and new data. Rev Españ Herpetol. 1999, 13: 61-82.
- Sá-Sousa P: Comparative chorology between Podarcis bocagei and P. carbonellae (Sauria: Lacertidae) in Portugal. Rev Españ Herpetol. 2001, 15: 85-97.
- Stockwell DRB, Peterson AT: Effects of sample size on accuracy of species distribution models. Ecol Model. 2002, 148: 1-13. 10.1016/S0304-3800(01)00388-X.View Article
- Barbadillo LJ, Lacomba JI, Pérez-Mellado P, Sancho V, López-Jurado LF: Anfibios y reptiles de la Península Ibérica, Baleares y Canarias. 1999, Barcelona: Planeta
- Sánchez-Herráiz MJ, Barbadillo LJ, Machordom A, Sanchiz B: A new species of pelodytid frog from the Iberian Peninsula. Herpetologica. 2000, 56: 105-118.
- García-París M, Arano B, Herrero P: Molecular characterization of the contact zone between Triturus pygmaeus and T. marmoratus (Caudata : Salamandridae) in Central Spain and their taxonomic assessment. Rev Españ Herpetol. 2001, 15: 115-126.
- Pleguezuelos JM, Márquez R, Lizana M, (Eds): Atlas y libro rojo de los anfibios y reptiles de España. 2002, Madrid: Dirección General de Conservación de la Naturaleza
- DGA: Atlas do ambiente. Direcção Geral do Ambiente, Versão 1.0. 1995, Lisboa: Ministério do Ambiente e Recursos Naturais
- Ilwis: Ilwis 3.0. 2001, Enschede: International Institute for Aerospace Survey and Earth Sciences
- CN: Atlas climatológico de Portugal. 1974, Lisboa: Serviço Climatológico Nacional
- IGN: Atlas nacional de España. Sección II. Grupo 9. Climatología. 1992, Madrid: Instituto Geograficio Nacional
- SPSS: SPSS for Windows. Release 12. 2003, Chicago: SPSS Inc
- Holm S: A simple sequential rejective multiple test procedure. Scand J Stat. 1979, 6: 65-70.
- Cohen J: A coefficient of agreement for nominal scales. Educational Psychology Measurement. 1960, 20: 37-46.View Article
- Fieldings AH, Bell JF: A review of methods for the assessment of prediction errors in conservation presence/absence models. Environm Conserv. 1997, 24: 38-49. 10.1017/S0376892997000088.View Article
- Pearce J, Ferrier S: Evaluating the predictive performance of habitat models developed using logistic regression. Ecol Model. 2000, 133: 225-245. 10.1016/S0304-3800(00)00322-7.View Article
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.