23/10/2012
Blue Print untuk uji kompetensi Mengapa diperlukan?
Sari PuspaDewi KomiteBersama UjiKompetensiDokter Indonesia
Uji Kompetensi sebagai sebuah Sistem Ujian Standarisasi Nasional
- Implementasi KBK - 80% isi kurikulum - Syarat kelulusan
Benchmarking Regional
- Implementasi KBK - Fase akademik (prediagnostik/treatment) - Fungsi formatif
Sistem Ujian Institusi
- Implementasi KBK - 100% isi kurikulum - Syarat kelulusan
1
23/10/2012
Sistem Uji Kompetensi • Prinsip KBK disertai evaluasi hasil belajar berbasis kompetensi (perubahan mind-set).
• Uji kompetensi = competence-based assessment sebagai longitudinal assessment, UKDI (ujian standarisasi nasional/ uji kompetensi) bagian integral dan komplementer terhadap sistem ujian kompetensi di institusi.
Usefulness of the assessment • Usefulness = R x V x P x EI • R = Reliability • V = Validity • P = Practical Acceptability & Feasibility (cost) • EI = Educational Impact (Van derVleuten, 2001)
7
2
23/10/2012
Aspek Praktik Baik Asupan
1. Blueprintingmetodeujian 2. Kontributorsoal 3. Pedomanujian 4. Dukungan IT
Proses
Luaran
1. Standarisasi kontributor soal 2. Proses review 3. Try-out item
Standard setting
4. Pengelolaan ujian 5. Pengelolaan bank soal
Mengapa Blue Print? – The blueprint = matrix. – Outlines assessment in 3 major areas: • Kompetensi yang ingindiujikan. • Materi • Instrumen yang digunakan.
– Others: • Description of Purpose. • Examinee. • Number of Items. • Timing • Scoring • Report design & distribution
3
23/10/2012
Mengapa Blue Print?
Memastikanbahwaterdap atkeseimbanganberbagai aspekpenilaian Penerapanberbagaimeto daujianbiasanyadibutuhk anuntukmencapaitujuan penilaianberdasarkan blue print Penggunaanberbagaiinst rumenpenilaianbalanc
e of strengths and weaknesses of each method
Blueprinting vs metode ujian
4
23/10/2012
Sample Blueprint for Medicine for an MCQ and OSCE Examination Competency
MCQ OSCE
Communication skills
10%
Clinical Skills
20%
Application of Scientific basis knowledge
40%
Management of health problems
30%
Management of information
10%
70%
Self awareness & development
10%
Ethics, morals, professionalism
10%
Sample Blueprint for an MCQ Examination Clinical Tasks Data gathering Data interpretation and synthesis Drug management Non-drug management
Child Health 5
Maternal Adult health health 3 13
Mental health 5
Population health 4
6
3
17
6
4
2
2
10
4
0
2
1
10
3
0
5
23/10/2012
Tinjauan I
Tinjauan 2
Tinjauan 3
Tinjauan 4
Tinjauan 5
Tinjauan 6
Tinjauan 7
Komunikasi Efektif
Kognitif
Recall
Reproductive
Pertumbuhan, perkembangan dan penyakit degeneratif
Promosi kesehatan dan pencegahan penyakit
Individu
Keterampilan Dasar Klinik
Prosedural
Reasoning
Head and Neck
Kelainan genetik dan kongenital
Penapisan (Diagnosis)
Keluarga
Aplikasi biomedik, perilaku, klinis & epidemiologi pada kedokteran keluarga
Konatif
Neurology and behaviour
Penyakit infeksi dan imunologi
Manajemen (Terapi)
Masyarakat
Manajemen Masalah Kesehatan Primer
Dermatomusculo skeletal
Penyakit neoplasma
Rehabilitasi
Penelusuran Kritisi dan Manajemen Informasi
Hematoimmunology
Penyakit akibat trauma atau kecelakaan
Aspek hukum dan etik
Profesional, moral danetikapraktikkedokter an
Cardiovascular
Kesadaran, pemeliharaan, danpengembangan personal
Gatrointestinal, hepatobilier, and pancreas Respiratory
Blue Print UKDI
Urogenital Endokrin and Metabolisme
Workflow of Item Bank Administration
Item writer
Local Item Bank Administrator
Regional Item Bank Administrator
Item Bank 1
Try Out
Item Bank 2
Panel expert
National Item Bank Administrator
Item Bank 3
Item Selection process
Question book
Item Bank 4
6
23/10/2012
7
23/10/2012
8
23/10/2012
9
23/10/2012
KomposisiSoal UKDI XIII berdasarkan SKDI 100,0 90,0 80,0 70,0 60,0 50,0
Soal UKDI
40,0 30,0 20,0 10,0 0,0 2
3A
3B
4
Kategori
Distribusi Tingkat Kesulitan Soal UKDI XIV Berdasarkan Level Kompetensi 100% 90% 32%
80%
30%
50%
70% 60% Mudah 50% Sedang 40% 66%
68%
3A & 3B
4
Sulit
30% 50%
20% 10% 0% 2
10
23/10/2012
Distribusi Faktor Kesukaran Soal UKDI XIV berdasarkan Kategori Kompetensi
100% 90% 80%
1
70%
0.9
60% 0.8 50% ≥ 0.2 40% < 0.2
30%
0.2
20%
0.1
10% 0% 2
3A & 3B
4
DistribusiFaktorKesukaranSoal UKDI XIII berdasarkanKategoriKompetensi 100% 90% 80% 70% 60%
1 0.9
50%
0.8 > = 0.2
40%
< 0.2 0.2
30%
0.1
20% 10% 0% 1
2
3A
3 3B
2
44
Kategori Kompetensi
11
23/10/2012
Distribusi Tingkat KesulitanSoal UKDI XIII Berdasarkan Level Kompetensi 100% 90% 80% 70% 60% Mudah
50%
Sedang Sulit
40% 30% 20% 10% 0% 1
2
3
4
Simple Blueprint Competence Categories
CVS
RS
…
GI
…..
MSS
Endo/ Metab
Haem / Oncology
….
History Taking
Physical Exam
Tests and Procedures …..
Management
Communication and Patient Education
12
23/10/2012
Hierarchy The Hospital Setting Competence Categories
CVS
RS
…
TheCommunitySetting …..
GI
CVS
RS
…
GI
History Taking
Physical Exam
x
Tests and Procedures
x
x
x
…..
Competence Categories
CVS
RS
Case
AMI (3B)
TB Lung
NB
Febrile Conv.
(SP)
GI
Repro
Obgyn Diarrhea (contrasepti (pediatric) on)
Communication & Patient Education
Haem/ Oncology
Low Back Pain
Thyroid (SP)
DHF Grade 3-4 (adult) (SP)
Reflex Examination
CPR
Chest Xray
GUS
Head & Neck
Special Sense
IV line
IM injection
Diet Counseling
Breaking bad news (breast malignancy)
Suturing X-ray fracture
Phsyciatry
MildHearing Urinary Schizoprenia Retentio Moderate Dissorders (SP) n (3A, Head Injury (SP)
(SP)
depkes)
HT Thyroid
Physical Exam
Management
Endo/ Metab
HT fever
History Taking
Tests & Procedures
MSS
(3A, depkes)
Alloanamnesis
PE Thyroid
GCS
IV line
Assessment of hearing
Catheteri zation
Prescribe
On malnutrition
(peds)
13
23/10/2012
Waktu Ujian vs Reliability
ReliabilitasSoal UKDI
39
14
23/10/2012
Dukungan IT •
Website ujian: jadwal, profil ujian, (pendaftaran), selfassessment/quiz, (pengumuman hasil).
•
Item Bank: penyimpanan, pemilihan soal.
•
Pelaksanaan ujian: CBT, OSCE
•
Item Analysis
•
Standard setting: OSCE
Epilogue 41
Blueprint merupakanpijakanawalmeningkatkan usefulness dari assessment. Perlupenetapankeputusanberdasarkan value masingmasingprofesi, dan review terusmenerus. Perlukomitmendalamupayacontinuing quality improvementberdasarkanlesson learnt implementasimenujupraktik yang lebihbaik.
15
23/10/2012
Terimakasih
[email protected]
16
