Accuracy Assessment


Collection 1.0

Accuracy analysis is the main way of assessing the quality of mapping performed by MapBiomas. In addition to telling the overall classification accuracy, the analysis also reveals the accuracy and error rate for each classified class. MapBiomas evaluated global and per-class classification accuracy for each year between 2010 and 2017.

Accuracy estimates were based on the evaluation of a sample of pixels, which we call the reference database. The number of pixels in the reference database was pre-determined by statistical sampling techniques. Each year, each pixel in the reference database was carefully evaluated by three technicians trained in visual interpretation of Lansat images. The evaluation of a pixel in a given year was considered valid only when two or three technicians agreed on the class observed in the pixel. Accuracy assessment is then done using metrics that compare the class mapped with the class evaluated by the technicians in the reference database. We report yearly accuracy estimates with c.a. of 5% error for each class in the mapping.

User’s Accuracies: These are the estimates of the fractions of pixels, for each classified class, correctly classified. The user’s accuracy is associated with the error of commission, which is the error of assigning a pixel to class i, when it belongs to some other class.

Producer’s Accuracies: Are the sample fraction of pixels of each land cover/use class correctly assigned to their classes by the classifiers. The producer's accuracy is associated with the omission error, which occurs when we fail to map a class j pixel correctly.

Global Accuracy: It is the estimate of the overall uncertainty.

The matrix also provides estimates of the different types of errors. For example, we show estimates of true class area composition in each mapping class. Thus, in addition to the hit rate of the class mapped as forest, for example, we also estimate the fraction of these areas that could be pasture or other classes of cover and land use, for each year. We understand that this level of transparency informs users and maximizes the potential of use across multiple types of users.

For this, below are the annual accuracy results for the entire mapped area:

The accuracy global for the level 1 of legend MapBiomas.

2010 0,7746 0,2254 0,0855 0,1399
2011 0,7753 0,2247 0,0725 0,1523
2012 0,7720 0,2280 0,0775 0,1505
2013 0,7774 0,2226 0,0851 0,1375
2014 0,7782 0,2218 0,0811 0,1408
2015 0,7751 0,2249 0,0752 0,1497
2016 0,7737 0,2263 0,0758 0,1505
2017 0,7786 0,2214 0,0776 0,1438