Natural history and desk-based ecology

The recent Intecol meeting in London, celebrating the British Ecological Society’s centenary, was perhaps the most Twitter-active (Twinteractive?) conference I’ve been to, with Twitter-only questions at plenaries and plenty of discussion across multiple parallel sessions. One such discussion I dipped into (#ecologyNH) concerned the extent to which a 21st Century ecologist needs to know natural history, a question I’ve been pondering for a while, and one which surfaced again only yesterday in an exchange triggered by Matt Hill (@InsectEcology) and also drawing in Mark Bertness (@mbertness), Ethan White (@ethanwhite) and others. Now the answer to this of course depends on your particular specialism. If you’re a field ecologist then reliably being able to identify your (perhaps many) study species is clearly critical, and many ecological careers outside of academia require very good identification skills in order to assess habitats, prioritise conservation areas, and so on. But ecology’s a broad field, too broad for any one of us to master all of its subdisciplines, and there are skills other than natural history that are equally useful. In particular, an increasing number of us do a kind of ecology which involves sitting in front of a computer screen and playing with other people’s data. In my case, this is macroecology, trying to understand what determines the distribution and abundance of large groups of species over regional to global scales. Is it really necessary for me to be able to put a face to every species name in my dataset in order to extract the kind of general patterns that interest me?

My view is that the answer to this depends on how we define ‘natural history’. As I’ve posted before, I don’t consider myself much of a natural historian, under the rather narrow definition of being able to key out a large number of species; and I don’t believe this holds me back as an ecologist. But on the other hand, I do think that a ‘feel’ for natural history is important. By this, I mean that understanding in general terms the kinds of organisms you work on, and the sorts of ways in which they interact with each other and with their environment, is likely to enhance your understanding of any dataset, and thus will point you in the direction of interesting questions (and away from silly ones). In the same way, I don’t see why a fisheries minister, for example, should be expected to be able to identify every fish on a fishmonger’s slab in order to make sensible policy decisions; but having some general understanding of fish and fisheries above and beyond numbers on a balance sheet seems important to me.

That’s my general thesis, but if you want some specifics, I believe there are some real practical advantages to be gained from a macroecologist taking the time to learn a bit about the natural history of their system, too. First, we all know how easy it is to introduce errors into a large dataset; being able to relate a species name to a mental image of the kind of organism it represents provides an efficient way to spot obvious errors. This is really just an extension of basic quality control of your data - simple plots to identify outliers and so on. But errors need not be outliers - for instance, if you’re looking at the distribution of body size across a very wide range of species, an obvious mistake, like a 50g cetacean or a 50kg sprat, may not be immediately apparent. One such error was only picked up at the proof stage in this paper, when my coauthor Simon Jennings noticed that one of the figures labelled a 440mm scaldfish which he told me was ‘unrealistically big’, in fact over twice the likely maximum length. He was quite right, as a better knowledge of Irish Sea fish would have told me at the outset; fortunately this time we caught the error on time, and it didn’t affect our conclusions at all.we corrected the figure and did the quick check on all the other species that we should have done at the outset.

Of course, there are more formal ways to check data against known limits, but the point is that a bit of expert knowledge - a basic understanding the range of feasible values for a feature of interest - goes a long way. Having worked on many different taxa, not all of which I have personal experience of, my approach to this is to work with some kind of (preferably colourful) field guide near at hand that I can dip in to to remind myself that points of a graph = organisms in an environment.

Some outliers, of course, remain stubbornly resistant to quality control, and you eventually have to accept that they are real. Here again, a bit of natural history can help you to interpret them and to suggest additional factors that may be important. For instance, I have worked quite a bit on the relationship between the local abundance and regional distribution of species. Such  ‘abundance-occupancy’ relationships (AORs) are typically positive, such that locally common species are also regionally widespread. I put it like this: if you drove through Britain, you’d tend to see the same common birds everywhere on your journey, but the rare ones would vary much more from place to place. However, although AORs are well-established as a macroecological generality, there are often outlying species, for instance species with very high local densities but small distributions. Identifying such points (‘Oh, they’re gannets’) and knowing something about them (‘of course, they nest colonially’) can help to explain these anomalies.

Such simple observations - ‘gannets don’t fit the general AOR’ - can then lead to more general predictions - ‘AORs will be different in species that breed colonially’ - that can influence future research directions. In my experience, observations of natural history will frequently suggest new explanations for known patterns, or will lead you to seek out study systems meeting particular criteria in order to test a hunch. A fascination with natural history may lead you to learn about a new ecosystem -  deep sea hydrothermal vents, say - which you then start to think may be perfect for testing theories of island biogeography or latitudinal diversity gradients.

You might also start to question models that gloss over natural historical details. On a winter walk in the Peak District I made the very obvious observation that the north-facing side of the steep valley was deeply frosted while the other, only a hundred metres or so distant but south-facing, was really quite pleasantly warm. This got me thinking about how the availability of such microclimates would not be captured in most of the (kilometre scale) GIS climate layers people use in species distribution modelling, yet could be crucial in determining where a species occurs. This is unlikely to have been an original thought, and is not one I’ve followed up, but it emphasises how real world observation can colour your interpretation of computational results.

More generally, real world observation - ‘going one-on-one with a limpet’, as Bob Paine puts it in a nice interview on BioDiverse Perspectives - gives you a sense of the set of plausible explanations for the phenomena that emerge from datasets at scales too large for one person to experience. This in turn leads to a healthy scepticism of hypotheses that fall outside that set. To paraphrase an earlier post of mine, simply plucking patterns from data with no feel for context and contingency is unlikely to lead to the understanding that we crave.

That said, however, there are benefits to be had from putting aside one’s personal experience and being guided, from time to time, by the data. I guess I’m influenced here by working on marine systems, where the human perspective is not a good guide to how organisms perceive their environment. We simply can’t sense the fine structure of many marine habitats, or how dispersal can be limited in what looks like a barrier-less environment. Bob Paine admits as much: directly after the limpet quote, he says “How do you do that with a great white shark or blue whale? There’s this barrier to what I would call natural history.” He goes on to talk about the problems with relying on personal experience when working on systems such as terrestrial forests with very slow dynamics. These long-term, large-scale, hard-to-access systems are, I would argue, exactly where the methods of macroecology and other computational branches of our science come to the fore. It is also, dare I say it, where coordinated observational programmes like NEON can make a real contribution.

But let me finish with perhaps the most important justification for spicing up computer-based ecology with a bit of natural history. We’re supposed to be enjoying ourselves, and for most ecologists surely that means getting out into the field, in whatever capacity - for work or for fun - and wherever it may be, from our back gardens to the back of beyond. My personal view is that doing this whenever you can will make you a better ecologist. But even if I’m wrong, it ought to make you a happier ecologist, and that’s important too.

Measuring the intangible: lessons for science from the DRS?

The final Ashes test of this summer has just started, a welcome distraction, no doubt, for some of those academics holed up preparing REF submissions (see Athene Donald’s recent post to get a feel for how time consuming this is, and the comments under it for a very thoughtful discussion of the issues I’m covering here). It also provides the perfect excuse for me to release another convoluted analogy, this time regarding the approaches taken in test match cricket and in academic science to measuring the intangible.

Anyone who follows sport to any extent will know how armchair and professional pundits alike love to stir up controversy, and much of that generated in this current series has revolved around the Decision Review System (DRS) - the use of television technology to review umpires’ decisions. Of course, TV technology is now used in many sports to check what has actually happened, and that is part of its role in cricket - Did a bat cross a line? Did the ball hit bat or leg? Whilst sports fans will still argue over what these images show, at least they are showing something.

Cricket, however, has taken technology a step further. In particular, one of the ways that a batsman can get out in cricket is LBW (leg before wicket), where (in essence) the umpire judges that, had it not hit the batsman’s leg, the ball would have gone on to hit the wicket. LBWs have been a source of bitter disputes since time immemorial, based as they are on supposition about what might have happened rather than anything that actually did. The application of the DRS was meant to resolve this controversy once and for all, using the ball-tracking technology hawk-eye to predict exactly (or so it seems) the hypothetical trajectory of the ball post-pad.

Of course, sports being sports, the technology has simply aggravated matters, as ex-players and commentators loudly question its accuracy. But, as well demonstrating how poorly many people grasp uncertainty (not helped by the illusion of precision presented by the TV representation of the ball-tracking), it struck me that there are parallels here with how we measure the quality of scientific output.

First and foremost, in both situations there is no truth. The ball never passed the pad; quality is a nebulous and subjective idea.

But perhaps more subtely, we also see the bastion of expert judgement (the umpire, the REF panel) chellenged by the promise of a quick techno-fix (hawkeye, various metrics).

What has happened in cricket is that hawk-eye has increasingly been seen as ‘the truth’, with umpires judged on how well they agree with technology. I would argue that citations are the equivalent, at least as far as scientific papers go. Metrics or judgements that don’t correlate with citations are considered worthless. For instance, much of the criticism of journal Impact Factors is that they say little about the citation rates of individual papers. This is certainly true, but it also implicitly assumes that citations are a better measure of worth than the expert judgement of editors, reviewers, and authors (in choosing where to submit). Now this may very well be the case (although I have heard the opposite argued); the point is, it’s an assumption, and we can probably all think of papers that we feel ought to have been better (or less well) cited. As a thought experiment, rank your own papers by how good you think they are; I’ll bet there’s a positive correlation with their actual citation rates, but I’ll also bet it’s <1. (You could also do the same with journal IF, if you dare…)

So, we’re stuck in the situation of trying to measure something that we cannot easily define, or (in the case of predicting future impact) which hasn’t even happened yet, and may never do so. But the important thing is to have some agreement. If everyone agrees to trust hawk-eye, then it becomes truth (for one specific purpose). If everyone agrees to replace expensive, arduous subjective review for the REF with a metric-based approach, that becomes truth too. This is a scary prospect in many ways, but it would at least free up an enormous amount of time currently spent assessing science, to actually doing it (or at least, to chasing the metrics…)

Wild extrapolation and the value of marine protected areas

Last week, the UK National Ecosystem Assessment published a follow-on report on the value of proposed marine protected areas (rMPAs) to sea anglers and divers in the UK. This report gained a fair bit of coverage, likely because the headline numbers it proclaimed are quite astonishing: “The baseline, one-off non-use value of protecting the sites to divers and anglers alone would be worth £730-1,310 million… this is the minimum amount that designation of 127 sites is worth to divers and anglers”. Furthermore, they claim an annual recreational value for England alone of the rMPAs of £1.87-3.39 billion, just for these two user groups (divers and anglers). These numbers are so astonishing, in fact, that my bullshit klaxon went off loud enough to knock me off my chair. See, I’ve been thinking recently about sea angling as an ecosystem service, and so know that there’s estimated to be somewhere around 1-2 million sea anglers in the UK. The number of divers is, I reckoned, likely to be considerably lower (there’s a higher barrier to entry in terms of equipment, qualifications, etc.). So these headline figures imply an annual spend -  purely on their hobby - somewhere in the order of £1000 for every single self-declared sea angler or diver. Which seems rather on the high side, given that one would expect a very long tail of ‘occasional’ dabblers in each activity (e.g. people who spin for a few mackerel on holiday).

So, I bucked down and read the 125 page report, to find that the authors had done some things really nicely. Their valuations are based on online questionnaires featuring a combination of neat choice experiments, willingness to pay (WTP) exercises, and an valuable attempt to characterise the non-monetary value of the sea-angling or diving experience (things like ‘sense of place’, ‘spiritual wellbeing’, etc.). But the headline numbers are highly dubious (worthless, in fact), because they did a few things very very badly indeed. Unfortunately, they did a different bad thing for each of their two major monetary valuation methods, so the numbers emerging from each are equally dodgy, as a modicum of mental arithmetic, common sense, and ground-truthing will show.

First, the annual recreational value models are nicely done, using a choice experiment based on travel distances to hypothetical sites with different features to assess which of those features are most valuable. Mapping these features onto the rMPAs leads to a ranking of these sites in terms of how attractive they are to anglers and divers. One could quibble with details here - perhaps the major quibble would be that there is no ‘control’, i.e. no assessment of the value of sites which are not proposed for protection. But in general, I think this analysis gives a decent estimate of how the survey respondents value the different sites.

They then attempt to get an overall annual value for each site by multiplying its value to individuals by the number of visits it receives in a year. This is where the problem arises: attempting to generalise from these respondents to the entire population of anglers (estimated at 1.1-2 million) or divers (estimated at 200,000). I’m going to concentrate on the anglers because the issue is most extreme here: their models are based on 273 responses, a self-selected group of anglers acknowledged within the report to be especially committed (averaging 3-4 excursions a month) and interested in marine conservation, and representing between 0.01 and 0.02% of the total population, i.e. 1 or 2 responses per 10,000 anglers (they also a self-selected sample of highly experienced divers, representing around 0.5% of all divers, i.e. 5 per 1000). Extrapolating from this sample to the entire UK angling population produces some interesting results.

For example, using this methodology Chesil Beach & Stennis Ledges rMPA on the Dorset Coast has an estimated 1.4-2.7 million visits by sea anglers annually. That translates to 3800-7400 visits every single day of the year. Compare this to a (highly seasonally variable) average of around 3000 visits per month to Chesil Beach Visitors Centre. Or you could look at the Celtic Deep rMPA, a site located some 70km offshore, where they estimate between 145,000 and 263,000 angling visits per year. That’s 400-720 visits a day, which translates to approx 40-70 typical sea angling boats, each full to the gunwales every single day of the year. Of course, this is simply because the tiny sample is uncritically extrapolated. In the case of the Celtic Deep, it is straightforward to calculate that there were actually 36 observed visits, which (when divided by 273 and multiplied by 1.1 or 2 million) gives you 145-263,000 estimated visits. Using this logic, the minimum number of visits a site could receive is (1/273)*1.1 million, or >4000. Diving numbers are similarly unrealistic, with estimates of 123-205,000 visits a year (340-560 per day) by divers to Whitsand & Looe Bay, or 26-44,000 a year (70-120 per day) at Offshore Brighton, which is around 40km offshore.

This kind of wild, uncritical extrapolation is staggering, akin to using the opinions of a focus group of LibDem party activists to predict a landslide in the next election. It’s a textbook example of the utility of a bit of simple guesstimation (e.g. a million visits a year means 10,000 visits/d for 100 days, or ballpark 2500/d over the whole year), allied to some common sense (have you ever experienced those kinds of numbers when you’ve visited the UK coast?)

So, we can discount the big annual recreational value figures. What about the WTP exercise? WTP has its fans and its critics. My view is that it’s a useful way of ranking scenarios according to preference, but I don’t give a lot of credence to the ££ generated, simply because by increasing the number of scenarios you can quickly get people to commit more cash than they intended. But regardless of that, the authors of this report appear to have made a very strange decision in aggregating the WTP estimates arising from their questionnaire. They worded the questions very carefully, presenting each respondent with a single site, outlining its features, and asking how much they would be WTP as a one-off fee for its protection - being sure to think of this amount as a real sum of money, in the context of their household budget. These numbers are then used to give an average WTP for all the rMPAs, which seems reasonable, and a useful way to rank the sites.

But They then simply multiply these site level averages by the whole UK angling (or diving) population to get a total WTP for the whole set of rMPAs.

Think about what they’ve done there.

They’ve asked people how much they would be willing to pay to protect a single site, and have then assumed that the same person will pay a similar amount for every site in the network. So if you agreed that you’d be prepared to pay a one-off sum of £10 to protect a site, you could find yourself with a bill for over £1000 to protect the whole network. (This is a slight over simplification, as specific values are site-specific, but it is essentially what they’ve done.) You simply cannot aggregate WTP like this. I mean, I’m not an economist, but if economists think you can do this, they are deluded.

Again, a bit of common sense would have helped here. The authors compare this WTP to an insurance premium, which is a useful analogy. But how many anglers or divers are really, when it comes down to it, prepared (or even able) to shell out a £1000+ insurance premium to prevent damage to the marine environment which may or may not occur in the future?

Anyway, that’s what’s been bugging me these last few days. I could go on (for instance, on a more philosophical level, is replacing strictly regulated commercial fishing with unregulated recreational angling necessarily a good thing for the marine environment? Will diving or - especially - angling actually be allowed in these rMPAs?). And there are some useful things in the report. It confirms that people do value the marine environment, really quite highly, and that different features are valued differently by different groups - a useful starting point for some more focused research, and helpful in placing relative values on different rMPAs. But unfortunately - inevitably - media attention has focused on the ludicrous headline numbers, something the authors have actively encouraged in their framing of the report.

A final positive point to end on: my bullshit klaxon seems to be in fine working order.