Quality of BTO data

BTO has pioneered methodologies for the systematic collection of data on the abundance, distribution, habitat associations, movements, survival, behaviour and breeding performance of birds over large geographical areas. This is achieved using networks of volunteers who continue to make a huge contribution to our understanding of the status and ecology of birds in the United Kingdom.

Maintaining the standards and quality of data gathered in extensive surveys of birds

The data are gathered using carefully designed protocols within various schemes and enquiries, each of which has a different purpose, methodology and set of objectives.  Some of these schemes collect data on an annual basis, for example the Breeding Bird Survey (BBS), the Wetland Bird Survey (WeBS), the Nest Record Scheme and the Ringing Scheme.  Others are of short duration.  For instance, distribution atlases take place at approximately 20 year intervals and gather data over a four or five year period.  The BTO has also traditionally undertaken national surveys of the distribution, abundance and habitat use of selected species, most of which are carried out in a single year, though they may be repeated periodically.  A smaller number of enquiries have aimed to answer specific questions relating to bird behaviour.

The purpose of this document is to explain the approaches used to ensure that, in each of these circumstances, the data are of a high standard, both in terms of sampling and their collection in the field, that they meet the objectives of the scheme concerned in the most effective way, and that there is a clear understanding of the strengths and limitations of each survey.  The intention is not to describe individual surveys, details of which are available on the BTO’s website www.bto.org, but to outline the principles by which the BTO operates its gathering of data through volunteer networks.

A key requirement is that survey design should minimise biases within the data, so as not to undermine the scientific objectives of the scheme. In practice there are three main issues that have to be addressed in designing and operating schemes:

  1. that data should be representative of the relevant geographical area or habitats about which conclusions are drawn;  
  2. that, for long-term surveys, the data must be comparable over time and that temporal comparability is not compromised by changes in observer quality, field methods or distribution of samples;
  3. that potential biases arising from variation in detectability between habitats and species needs to be accounted for, and whether analysis of the data without correcting for detectability can be justified.

Sampling

Many BTO surveys are based on sample counts (e.g. BBS and many single species surveys) Many of these involve a random selection of survey units. Appropriate stratification is used so that the sample data are representative of the region / environment being studied. Stratification is usually by land cover type or some other habitat features, occasionally by human population density. However, in surveys targeted on particular species, the stratification will be selected to reflect the ecology of the species concerned, so that the full range of conditions potentially occupied are sampled.  Random sampling is straightforward where regular units such as 1-km grid squares are used. It is less easy where a particular habitat is being sampled, or an uncommon or patchily distributed species is the subject. In such cases the BTO endeavours to adopt sampling regimes that sample across a wide range of variation and that avoid skewing data towards particular conditions.  In some cases professional workers may be engaged to undertake intensive random sampling where volunteers are selecting their own sites; this can be used to correct for any bias that may be evident in the wider volunteer survey and to ensure adequate coverage of areas at the edge of range, in remote areas or where habitat is sub-optimal.  

Some datasets are constrained by the volunteers who contribute to them and the circumstances in which they operate.  It is important not to infer results that cannot be substantiated.  For instance, it is reasonable to report that the number of Knot ringed in the UK doubled between two years, as long as account is taken of any changes in effort and the element of chance that is involved in catching birds of this species.

Statistical advice and analysis

The BTO employs several staff who are highly experienced ecological modellers and statistical analysts. Statistical advice is sought at the design stage of projects and analytical advice is sought in the interpretation of large-scale survey data. The BTO endeavours to maintain expertise in the latest relevant analytical techniques and, where necessary, will seek specialist expert advice from outside the organization.

Protocols and instructions

Each scheme and project has its own protocol for the collection of data in the field. Where new methods are being used, these are generally trialled before use on a large scale. Extreme care is taken with instructions for schemes and surveys, to maximize the probability that different observers will collect data in the same way and that individual observers will record information consistently (e.g. between years). If observer behaviour were to vary systematically with region or habitat, this could introduce a serious bias into the results; recognizing the need to avoid such potential bias is an important part of the development of protocols and instructions.  Some schemes produce regular newsletters and bulletins that may include guidance and training.

Observer ability

All observers, whether professionals or volunteers, vary to some degree in acuity and ability to identify birds.  The BTO endeavours to ensure that observers have the necessary ability and level of skill, which will vary according to the survey. For example, a breeding bird census of all species in woodland requires a very different level of knowledge and skill to undertaking counts of nests in a heronry or rookery. Nest recording, bird ringing and censuses each requires different types of skill. The BTO aims to match the skill levels of observers to surveys. It does this in three main ways. Firstly, by being entirely clear about the expected skill levels of observers who participate in particular surveys. Secondly, by organizing many surveys through networks of regional organizers who attempt to find observers with appropriate skills (for example, within WeBS, the skills required for counting waterfowl on a local pond are very different to those where flocks of waders need to be counted/estimated). Thirdly, within some surveys different tiers of data-collection are provided, requiring different levels of expertise – e.g. collecting individual records, random walks and surveys of assigned plots.

Training

In order to maximise data quality, it is necessary to provide opportunities for volunteers to develop both their skills and confidence. The BTO runs several training courses each year in scheme methods and bird identification. Mentoring is used in projects organised by the Demography Team.  Some schemes produce regular newsletters and bulletins that may include periodic advice about field methods.  Mandatory and lengthy training, to an extremely high standard, is required to participate as an independent ringer, though this is largely for bird welfare reasons.

Record validation

Individual records are checked both by regional organizers and BTO staff. Unusual records (these could be erroneous identifications, miss-keyed data or exceptionally large counts) are checked with observers and if necessary not accepted. Some schemes, using online recording, have automated checks. For example, exceptionally high counts may be queried on entry.

Minimizing computer input errors

For its long-term and largest surveys, the BTO is increasingly moving away from paper records to online data entry by observers. However, where paper records need to be input into databases, double inputting of data is routinely used.

Publication and openness

The BTO is entirely open about the sampling and field rationales that it uses in its individual schemes. Many of the protocols are available on the website; all are available from the appropriate scheme organizer. The BTO is committed to publishing results from its schemes and surveys in a variety of forms including peer-reviewed scientific journals where methods are subjected to the greatest scrutiny.

Advice on survey development and coverage issues

Ideas for extensive surveys are generally developed by the staff in consultation with other ecologists and conservation professionals. For partnership projects (e.g. BBS and WeBS), specific working groups advise on the development and running of the surveys. For other extensive surveys, two committees – the Regional Network Committee and the Ringing Committee – have a role in advising on practical aspects such as the extent to which volunteers are likely to want to participate, pressures on local organizers, tactics for improving regional coverage etc.

Related content