24/08/2012
PEMBUATAN SOAL & INSTRUMEN EVALUASI KETERAMPILAN MEDIK DENGAN OSCE
OSCE
1
Professional authenticity
Simple model of competence
Does Shows how
Knows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7. OSCE
2
1
24/08/2012
Testing formats Behaviour~ attitude/skills
Professional practice
Does
Shows how Cognition~ knowledge
Knows how Knows
OSCEs EMQs, SEQs MCQs
OSCE
3
Develop a Plan
Make decisions about your OSCE - Summative? - Level of training being assessed? - Clinical objectives or mix of patient problems to be assessed? - Length and number of stations? - What competencis are being ass? (history taking, communication skills, problem solving etc)
OSCE
4
2
24/08/2012
First steps in Developing an OSCE case 1. Describe the purpose of the case (i.e. define the competencies you are testing) 2. Develop the case n instruction to the test takers 3. Create the scoring instruments (checklist, follow up questions and answer keys
OSCE
5
Step 1: Purpose
State the purpose of the station “Demonstrate ability to perform an appropriate physical examination for abdominal pain secondary to appendicitis and be able to identify positive findings.” Statement of purpose is your guide for what to include / exclude as you develop the case
OSCE
6
3
24/08/2012
Step 2: Instructions Write the case and instructions to the student Include patient’s name, age, setting (e.g., emergency, clinic, hospital ward), relevant background information (if any); specify the task and time limit
Example: Marie Beckett, 16 years old, has come to the Emergency Room with a 16-hour history of abdominal pain. In the next five minutes, conduct a focused and relevant physical examination. As you proceed, explain what you are doing and describe your findings. OSCE
7
Step 3: Scoring Instruments
Checklists are useful when assessing
Thoroughness or key elements Student (beginner) levels of ability Procedural tasks Limited time available for training markers and/or for marking
Rating scales are useful when assessing
Behaviours
Emphasis
is on “how well it is done” rather than on “done / not done”
Higher levels of expertise Emphasis
is on judgment rather than thoroughness To weight or not to weight … -- ‘keep it simple’ OSCE
8
4
24/08/2012
How is it done ?
A Clinical competence to be tested is broken down into specific skills A Specific skill is tested at a time It is organized in the form of several stations Each station tests a particular skill For each skill, a checklist is prepared Checklist contains essential steps and precautions to be observed
OSCE
9
How is it done ?
Each step has its own score proportional to the importance of the step/precaution Each component is tested at one fixed station by the same examiner The students rotate through several such locations Time allocated for each station: 4-7 minutes, 10 – 20 minutes Stations 8-20 (UNJANI : 10 sts)
OSCE
10
5
24/08/2012
How is it done ?
Categories of the station : Procedure or performance station Question or interpretation station
The examiner’s checklists & student’s answer sheets are marked according to predetermined scheme
OSCE
11
MENGEMBANGKAN OSCE Menyusun Kisi-kisi
Menyusun Soal/Skenario Melakukan Seting Station Koordinasi Dengan Observer Persiapan Pasien Simulasi Menyusun Check List Mengatur Pelaksanaan OSCE
12
6
24/08/2012
Menyusun Soal/Skenario • Kompetensi yang diharapkan (SKDI) • Waktu tiap soal (station) • Aspek yang terkait
History taking skills Clinical examination Procedure skills Formulation of investigations Interpretation of investigations OSCE
13
Soal • Topik • Instruksi kandidat • Instruksi penguji • Daftar tilik instrumen penilaian • Instruksi pasien simulasi • Daftar peralatan • Penulis • Referensi OSCE
14
7
24/08/2012
MENYUSUN DAFTAR TILIK Tiap item hanya menyangkut satu issue Gambaran rekaman perilaku mahasiswa ditulis dalam bentuk butir (item) Tiap item dapat diobservasi dan diberi nilai Daftar tilik untuk 10` cukup 10-20 item
Item yang PENTING perlu diberi tanda, dengan pembobotan yang lebih besar Hasil: NILAI YANG DIPEROLEH X 100
NILAI / STATION = TOTAL NILAI OSCE
15
Dokumen yang diperlukan penilai/pelaksana OSCE
Daftar mahasiswa Daftar observer/penilai Daftar pasien simulasi/sukarelawan Daftar peralatan Instruksi untuk mahasiswa Instruksi untuk penilai Daftar seluruh “stasiun” Lembar jawaban Rencana lokasi “stasiun” dan arah rotasi OSCE
16
8
24/08/2012
Pass Mark ?
Clinical procedural skills : 100 % (A) Physical examination : 100 % (A) History taking : 80 % (A) Interpretations : 80 % (A) Investigation : 80 % (A) Management : 80% (A)
17
OSCE
CONTOH UJIAN OSCE Heteroanamnese dari ibu yang mempunyai anak sakit
Melakukan prosedur aseptik
Melakukan pemeriksaan hepar
Melakukan pemeriksaan kadar gula darah
Pemeriksaan reflex Evaluasi hasil laboratorium
Membaca hasil EKG
Anamneses penyakit kronis OSCE
19
9
24/08/2012
Station 1
Station 2
Station 8
Station 7
Station 3
Station 4
Rest station
Interstation
Station 6
Station 5
I believe that teaching without testing is cooking without tasting (Ian Lang) OSCE
20
BLOOD PRESSURE MEASUREMENT Level of Student: Cardiovascular Block
STATION INFORMATION
CANDIDATE’S INSTRUCTION
28 Des 07
OSCE
21
10
24/08/2012
Blood Pressure Measurement
Rating
28 Des 07
22
OSCE
STATION INFORMATION
28 Des 07
Domain Discipline
History taking Surgery
The student to be assessed
The 2nd year student
Problem + Diagnosis
Acute Urinary retention
Purpose of Station
Demonstrate ability to perform an appropriate history taking for acute urinary retention secondary to benign prostatic hyperplasia and be able to identify positive findings
Scoring Guidelines
Checklist
OSCE
23
11
24/08/2012
Candidate’s Instructions
28 Des 07
Patient name: Mr Abdullah Age: 65 years Where: in Emergency Room Time allowed: in the next ten minutes Task: history taking Related Information: patient was complaining about inability to void. OSCE
24
Instruction to the Students: You are a medical student doing your surgical practice emergency. You are about to see Mr Abdullah, who has a voiding problem. In The first 6 minutes take a focused history of this patient’s problem. At 6 minutes, you will hear a knock at the door. The examiner will then verbally ask you some questions regarding this patient. At the end of 8 minutes, the examiner will provide you with feedback on your performance
28 Des 07
OSCE
25
12
24/08/2012
Checklist and rating Scale for Voiding Problem History Taking Case No
Item
Item weight
1
Greets and introduces self to patient in a professional manner
1
2
Asks how patient would like to be addressed
1
3
Defines objective (student explains what he/she is going to do)
1
4
Asks how long patient has had inability to void
1
5
Asks how patient first notice of his complain
1
6
Asks about the changing of his voiding pattern
1
7
Asks if the patient feeling a lower urinary tract symptomps (hesistancy, urgency, intermittency, weak stream, dysuria, post voiding dribbling)
4
8
Asks if patient has ever had any urinary symptom in his life
1
9
Asks if patient had suffered from hematuria
2
10
Asks what treatment patient has used so far for his hematuria
1
11
Ask if patient has had similar inability to void before
1
12
Asks if anyone in family has inability to void
1
28 Des 07
Rating
√ if performed
26
OSCE
1=fail, 2= borderline, 3=meet expecations, 4= exceeds expectations Initiation of interview
1 2 3 4
Listening skills
1 2 3 4
Questioning skills
1
Organization of questions
1 2 3 4
Attitudes
1 2 3 4
Non verbal communication
1 2 3 4
Closing
1 2 3 4
Global Rating - overall judgment of studen’s
1 2 3 4
2 3 4
performance
28 Des 07
OSCE
27
13
24/08/2012
Assigment
28 Des 07
Develop one OSCE case that will use a standardized patient. Identify he level of students to be assessed and purpose of the case(i.e.define the competencies that are being assessed) Prepare the case and the instructions to the student Develop the checklist/rating form, possible follow up questions qnd answer keys
OSCE
28
14
