Measuring Spatial Data Quality

The general treatment of uncertainty in spatial data reflects the continuing conceptual closeness of digital maps to their paper roots in the minds of users with an overriding emphasis on accuracy and error. Thus, adopted wholesale from the mapping sciences has been the testing of geometrical accuracy based on well-defined points having no attribute ambiguity (Bureau of Budget, 1947; American Society of Civil Engineers, 1983; American Society of Photogrammetry and Remote Sensing, 1985). These are usually stated in the general form: “90% of all points tested shall be correct within X mm at map scale.” In some countries, such as Australia, the law—Survey Co-ordination Regulations, 1981—allows plans to be classified from AA through DD according to a range of permissible plotted positional errors (reported by Millsom, 1991). Testing is supposed to be carried out by reference to a survey of higher order and is usually reported as a root mean square error (RMSE) for x and y dimensions (which we have already used in Chapter 4):

where e = the residual errors (observed – expected), n = the number of observations. Vertical accuracy is generally treated in the same way with separate statements in the general form: “90% of all interpolated elevations must fall within one half of a specified number of contour intervals.” Whereas the accuracy testing described thus far is predicated on point samples, an alternative measure of map accuracy using line intersect sampling is given by Skidmore and Turner (1992). The sampling is used to estimate the length of boundaries on a map that coincide with the true boundaries on the ground. The results can be converted into a percentage accuracy statement for the map. An example of map accuracy for U.K. Ordnance Survey topographic maps is given in Table 8.1. Attributes tend to be treated separately from location geometry. Continuous data, such as digital elevation model (DEM), can be tested for horizontal and vertical accuracy as with most point sampling described above either through interpolating contours or interpolating to known points.

Other quality issues for DEM are the nature and quality of source documents (if digitized), the sampling interval and orientation (if on a grid) in relation to the configuration. Where attributes are recorded on nominal scales or in discrete classes, the use of classification error matrices is widely used. Such techniques are of particular importance in testing the classification of RS imagery according to spectral response. A number of indices can be derived to summarize the matrix, such as proportion correctly classified (PCC), the Kappa statistic, and GT index.

Data conversion from analog to digital format is likely to introduce an element of error due to inaccurate placement of the digitizer cursor (Keefer et al., 1988) and due to simplification arising from point sampling of linear features (Amrhein and Griffith, 1991). Trials reported by Maffini et al. (1989) showed that 90% of discrete entities digitized from 1:50,000 scale maps fell within 0.4 mm. An elaborate experiment by Bolstad et al. (1990) determined that 90% of well-defined points digitized from 1:24,000 scale maps fell within 0.5 mm and, thereby, met map accuracy standards. Error due to registration of the map to the digitizer was a large component. Errors were found to be significantly different from normal with differences between operators also statistically significant. Dunn et al. (1990) found that the interaction of scale, quality of source documents and digitizing operator could result in unexpectedly large amount of error. Vector to raster conversion or rasterization necessarily involves a degree of generalization (loss of precision) as a function of cell size (Veregin, 1989b). Rasterization is often an integral part of database creation or used as an adjunct to layer combination techniques, such as map algebra. An example of the effect of vector to raster and raster to vector conversion can be seen in Figure 8.9 where source contours have been converted to raster and back again.

The discrepancies are obvious. Bregt et al. (1991) used a boundary index BI (total length of polygon boundaries divided by map sheet area; cm/cm2) to compare error resulting from central point and dominant unit rasterization for 1:50,000 scale maps for three sizes of cell. They found that BI explained at least 99% of variance (Figure 8.10) and that for the most complex maps tested (BI > 2) the incremental error ranged from 5 to 20% depending upon grid size. Choice of grid size, which is user driven, therefore, is a critical consideration.

Improving data quality inevitably has cost and time implications. “Few natural resources data can be determined with an accuracy ±10% at a price
resource survey agencies can afford. Hence, there is a tendency to study them intensively at a few ‘representative’ sites and extrapolate” (Burrough, 1986b). The reality would appear to be that many mapping products are just not tested for accuracy (Fisher, 1991) and become instead an act of faith. Recourse to higher-order surveys as a means of checking may increase uncertainty. The object “village,” for example, breaks down into buildings and subland uses at higher resolution making delineation more difficult. Testing based on welldefined, unambiguous points leads to other problems. First, there may be no well-defined points to test as, for example, in a flood hazard map. Second, well-defined, unambiguous points are likely to be more accurately mapped in the first place leading to a biased evaluation of accuracy. The use of error matrices has also been criticized as inadequate (e.g., Lunetta et al., 1991) since they do not address the spatial distribution of errors and the relationship between error and class boundaries. Grundblatt (1987), for example, identified that errors increase along boundaries.

Methods of sampling may also introduce bias because, for example, in soils mapping, testing is frequently carried out purposefully in the interior of polygons and not near the edge. Methods of deriving measures of data quality usually result in global measures of accuracy either for an entire coverage or for individual classes within them. These measures can provide limited information on spatial variation in quality since they assume that error is uniformly distributed. In reality, this is unlikely to be the case. Concepts of statistical accuracy and error are not so easily transferred to the notion of uncertainty in the analytical products derived from the base mapping. This requires us either to be able to model error propagation through to the products or have ways of assessing the fitness-for-use per se.