Chapter 14 Accuracy Assessment Introduction to Remote Sensing, p James B. Campbell
O tline Outline
Definisi
Source of Classification Error Error Characteristics Measurement of Map Accuracy
Accuracy & Precission Significance
Error Matrix Omission & Commission User & Producer Accuracy
Interpretation of the Error matrix
Percentage Correct Quantitative Assessment of Error matrix
Definition
Accuracy : correctness, mengukur “kecocokan” antara suatu image yg tidak diketahui kualitasnya dengan sebuah standar image Precission : detail, “The distinction is important because one may be able to increase accuracy by decreasing precission”, Meningkatkan detail = menambah ragam kategori . Misal : forest = caniferous, pine, shortleaf pine atau mature shortleaf pineÎ pineÎ akan menambah peluang klasifikasi error Statistical context : high accuracy = low bias, (estimated value is consistenrly close to an accepted reference value)
Definition
Definition
Significance
Accuracy has many practical implication : effect legal standing, operational usefulness, validity for scientific research.
So ce of Classification E Source Error o
Manual Interpretation : Misidentification, Excessive generalization, Error registration, Variation in detail of interpretation etc. Character of landscape : parcel size, variation in parcell size, i parcell identities id i i number b off categories, arrangement of categories, number of parcel per category category, shapes of parcel parcel, radiometric and spectral contrast with surrounding parcel
So ce of Classification E Source Error o Three error types dominate:
Data Acquisition Errors: These include sensor performance, stability of the platform, and conditions of viewing. We can reduce them or compensate for them by making systematic corrections (e (e.g., g by calibrating detector response with onon-board light sources generating known radiances). We can make corrections, often modified by ancillary data such as known atmospheric conditions, during the initial processing p g of the raw data. Data Processing Errors: An example is misregistration of equivalent pixels in the different bands of the Landsat Thematic Mapper. The goal in geometric correction is to hold the mismatch to a displacement of no more than one pixel. Under ideal conditions, and with as many as 25 ground d controll points (GCP) (GC ) spread d around d a scene, we can realize l this h goal. Misregistrations of several pixels significantly compromise accuracy. Scene--dependent Errors: As alluded to in the previous page, one such Scene error relates to how we define and establish the class class, which which, in turn turn, is sensitive to the resolution of the observing system and the reference map or photo. Mixed pixels fall into this category.
So ce of Classification E Source Error o
E o Characteristics Error Cha acte istics
Classification error : assignment pixel to one category that different from true category ( as determined ground observation/groundobservation/ground-truth ). Error Characteristic :
Error are not distributed over the image g at random,, display p ya degree of systematic, ordered occurrence in space. Often erroneously assigned pixels are not spatially isolated but occur grouped in areas of varied size and shape (Campbell 1981) Errors may have specific spatial relationships to the parcels to which they pertain, for example, they may tend to occur at edges or in the interiors of the parcels
E o Characteristics Error Cha acte istics
Tiga macam eror patern dari Landsat, Cogalton (1984). Dark area = error clasification, white area = correct.
Meas ement of Map Acc Measurement Accuracy ac
Compare the “true map”/ reference map, ((asumsi lebih akurat)) with image g to be evaluated.
Jika pembandingan tanpa memperhatikan posisi pixel, klasifikasi total bisa dianggap sama meskipun sebenarnya posisi dengan image reference tidak sesuaiÎ sesuaiÎ site site-spesific accuracy
Meas ement of Map Acc Measurement Accuracy ac
Site & Non Site Specific Error
Meas ement of Map Acc Measurement Accuracy ac
Error Matrix : matrik perbandingan image g image g yang y g akan reference dengan dianalisa berdasarkan kelompok klasifikasi pixel--p pixel p pixel yang y g sama dalam imageimage g -image g tersebut.Î tersebut. Î dari Error Matrik dapat dihitung g % correct,,
% correct = sum agreement pixel between reff & image(jumlah diagoal pada error matrix)/total pixel
Meas ement of Map Acc Measurement Accuracy ac
Error Matrix
Meas ement of Map Acc Measurement Accuracy ac
Compiling Error matrix
Image direpresentasikan dengan pixel2 Hitung jumlah pixel untuk tiap klasifikasi
Yg perlu diperhatikan : Klasifikasi reference dengan image yg akan diklasifikasi harus compatible compatibleÎ p Î turunan klasifikasi harus masih sesuai dengan kategori pada reference
Meas ement of Map Acc Measurement Accuracy ac
Meas ement of Map Acc Measurement Accuracy ac
Omission & Commission Error
Omission : jumlah pixel pada reference image yang tidak sesuai g kategori g kalsifkasi pada p image g yg dievaluasi dengan Commission : jumlah pixel pada image yang dievaluasi yang tidak sesuai dengan keadaan sebanarnya/klasifkasi pada reference
CA (Customer Accuracy) & PA (Produsen Accuracy)
CA : jumlah pixel pada image yg dievaluasi, yang sesuai dengan kondisi reference dibandingkan jumlah total pixel pada image yg d dievaluasi l untukk klasifikasi kl f k tsb. b PA : jumlah pixel pada image yg dievaluasi, yang sesuai dengan kondisi reference dibandingkan jumlah total pixel pada image reference. reference
Meas ement of Map Acc Measurement Accuracy ac
http://rst.gsfc.nasa.gov/Sect13/Sect13_3.html
Interpretation p of the Error matrix
Percentage Correct (PC)
Ukuran yg sering dipakai Beberapa rekomendasi :
PC = 85 % dibutuhkan untuk landland-use data resource management (Anderson et al, 1976) Fitzpatrick--Lins(1978) akurasi dari USGS land Fitzpatrick land--cover map untuk central Atlantic coastal : 85 % (untuk skala 1:24.000), 77% (1:100.000), 73% (1:250.000) Untuk automasiautomasi-interpretasi dari Landuse menggunakan hanya data MSS PC yang didapat = 38%, dan untuk MSS + ancillary data PC = 78 % (Tom et al. 1978)
http://rst.gsfc.nasa.gov/Sect13/Sect13_3.html
I t Interpretation t ti off th the E Error matrix ti
Quantitative Assessment of the Error Matrix kappa (k) = measured of difference between observed agreementt between b t ttwo map and d the th agreementt th thatt might be attained solely by chance matching two map. map.
k = (observed –expected)/(1expected)/(1- expected)
Observed = percentage correct Expected = product row & column,Î column,Î change agreement two categories when two images superimposed (Fig 14 8) 14.8)
I t Interpretation t ti off th the E Error matrix ti
I t Interpretation t ti off th the E Error matrix ti
k = 0.83 0 83 Î accuracy = 83% better than expected from chance assignment of pixel to cattegories. cattegories k = +1,Î +1,Î accuracy = 100%, perfect classification table 14 classification, 14.6 6
Pen t p Penutup
Accuracy dibutuhkan sebagai ukuran informasi yang didapatkan mendekati nilai standar/referensi tertentu/nilai sebenarnya Untuk aplikasi tertentu direkomendasikan menggunakan suatu nilai accuracy tertentu. Selain itu accracy juga berdampak pada nilai legal dari data dan informasi yang dihasilkan. Accuracy didapatkan dengan membandingkan dengan suatu image referensi tertentu, yg dianggap benar, lebih akurat dst Untuk mengukur accuracy digunakan alat bantu error matrix matrix, dengan menghitung percentage correct, omission&comission error, PA & CA dan kappa, semuanya untuk melihat kerelatifan kebenaran klasifkasi yang telah dilakukan.
