THE ANALYSIS OF CONTENT VALIDITY OF TEACHER MADE ENGLISH TEST OF SECOND YEAR STUDENTS ( AT 2nd SEMESTER) OF SMKN 5 MATARAM IN ACADEMIC YEAR 2015/2016
THESIS JOURNAL Submitted as Partial Fulfillment of requirements for the Degree of Sarjana Pendidikan In English Department
By Nurhidayati E1D 012 057
ENGLIAH EDUCATION PROGRAM TEACHER TRAINING AND EDUCATION FACULTY UNIVERSITY OF MATARAM 2016
i
ANALISIS VALIDITAS ISI DARI SOAL BAHASA INGGRIS YANG DIBUAT OLEH GURU UNTUK KELAS DUA SEMESTER GENAP DI SMKN 5 MATARAM TAHUN AJARAN 2015/2016 Oleh: Nurhidayati, Priyono, Muhammad Amin Program Studi Pendidikan Bahasa Inggris Jurusan Pendidikan Bahasa dan Seni, FKIP Universitas Mataram Email:
[email protected]
ABSTRAK Tujuan dari penelitian ini adalah untuk mencari tahu kualitas soal yang dibuat oleh guru. Secara spesifik penelitian ini menganalisis mengenai validitas isi dari test bahasa Inggris yang dibuat oleh guru untuk siswa kelas dua semester genap SMKN 5 Mataram tahun ajaran 2015/2016. Populasi dari penelitian ini adalah soal Ujian Tengah Semester (UTS/MID) bahasa Inggris yang dibuat oleh guru untuk kelas dua semester genap, sementara data yang digunakan dalam penelitian ini adalah soal UTS bahasa Inggris yang dibuat oleh guru, silabus dan RPP. Penelitian ini menganalisis keselarasan antara indikator yang terdapat dalam RPP dan butir soal yang ada. Hasil penelitian menunjukan bahwa validitas isi dari soal UTS bahasa Inggris yang dibuat oleh guru untuk siswa semester genap kelas dua SMKN 5 Mataram merujuk pada persentase 36, 84% dalam hal indikator ataupun materi yang telah ditestkan. Hasil tersebut menunujukan bahwa validitas isi dari soal tersebut dikategorikan sebagai soal dengan validitas isi yang lemah (>20% - 40%). Kata Kunci: Analisi, Tes Keberhasilan, Validitas Isi, dan Soal Bahasa Inggris Buatan Guru.
ii
THE ANALYSIS OF CONTENT VALIDITY OF TEACHER MADE ENGLISH TEST OF SECOND YEAR STUDENTS (AT 2nd SEMESTER) OF SMKN 5 MATARAM IN ACADEMIC YEAR 2015/2016 By: Nurhidayati, Priyono, Muhammad Amin English Education Program Language and Art Department, Faculty of Teacher Training and Education Mataram University Email:
[email protected]
ABSTRACT The main aim of this study was to find out the quality of teacher-made test. Specifically it analyzed the content validity of teacher-made English test at second year students of 2nd semester of SMKN 5 Mataram in academic year 2015/2016. The population of this study was the teacher-made MID term English test used at second year students of 2nd semester, while the data used were the teacher-made MID term English test of second year students of 2nd semester, syllabus and lesson plans. In doing the analysis, it measured the appropriateness between the indicators stated on the lesson plans and the test item. The result of analysis showed that the content validity of teacher-made MID term English test administered to the 2nd semester of second year students of SMKN 5 Mataram represented on 36, 84% of the indicators/material tested. This means the content validity of the test has low content validity test (>20% - 40%).
Keywords: Analysis, Achievement Test, Content Validity, and Teacher Made English Test.
iii
A. INTRODUCTION According to Arikunto (2015) that such a test should be valid (the test items should cover all materials that have been taught during the teaching and learning process). If the test is not valid, then that test should not be administered to the students. Therefore, teachers should make sure that the test is reflected the criterion of a good test. According to Arifudin (2007) there are some criterions in creating a good test. Those are validity, difficulty, reliability, applicability, relevance, replicability, interpretability, economy, availability and ecceptability (Arifudin, 2007: 185) The real problems of instructional evaluation were found in SMKN 5 Mataram wherein this study would be conducted. The results of observation showed that less than 50% of second year students could not pass the KKM (Minimum Achievement Criteria). The students are helped by adding some related assignments or just by looking at the students good behaviour as the consideration to pass the KKM. The teacher assumes that there is no problem with the test. The teacher firmly believes that the test that had been constructed was already good. It was constructed based on the students competences in English with an assumption that the teacher knows the students well, included how far the students’ understanding toward the materials have been given during the learning process. One thing that is very important to be highlighted here is that the test was not created based on the indicators or the basic competences reflected in the syllabus, but rather it was based on the teacher’s consideration. When the teacher is asked about the students failure in the test, the teacher argues that it only occurs because of the students’ limitation in knowledge and they do not study for the test, while when the students are asked about their failure in the test, they state the reasons in a various way. For example; “it is bacause the content of the test was not appropriate with materials which were taught before”, “I did not understand the instructions”. Moreover, they do not know what to read and there is no enough time to finish the test. Therefore, it is suggested for the teacher that before administering the test, they should be sure that the test is a valid test.
1
However, in fact there is no table of specification was used by the English teacher of SMKN 5 Mataram while constructing the test. The teacher argues that the table of specification has never been used in SMKN 5 Mataram especially in constructing a test. In another way, the teacher admits that the test was made as easy as possible for the students. The teacher’s reason for that action is because it is such a useless thing to create a high content validity of test, if the students themselves do not understand and can not answer the test. Hence, the content of the test which was made by the teacher is determined by what is easy to test rather than what is important to test. As that quite complicated problem, therefore, teacher-made test was chosen as the topic of this study in order to know the content validity of teacher-made English test which is as a media for measuring the students’ achievement. This topic would be analyzed through a study entitled “The Analysis of Content Validity of Teacher - Made English Test of Second Year Students (At 2nd Semester) of SMKN 5 Mataram in Academic Year 2015/2016.
B. METHODOLOGY Research Design Research design in this study refered to descriptive analysis done by document analysis. In this study, it would analyze the content validity of teacher-made English test which was used in MID term test by second year students (at 2 nd semester) of SMKN 5 Mataram. While the data analysis would be the English test made by teacher, syllabus, and lesson plan. Source of Data There are some sources of data in this study included primary and secondary data: 1.
The teacher-made English test which was used in the MID term test at the 2nd semester.
2.
The syllabus and lesson plan for second year students especially at 2nd semester which were used by English teacher.
3.
The list of English learning program of instructional process at second semester of second year students in academic year 2015/2016 and teacher’s
2
information related to the document would also be needed to clarify and ensure the correctness of data. Data Collection The documents used in this research were (1) The Teacher-Made test which was used in MID term test by the second year students (at 2 nd semester) of SMKN 5 Mataram. (2) The syllabus and lesson plan which were used by the teacher. The list of English learning program of instructional process at second semester of second year students in academic year 2015/2016 and teacher’s information related to the data would also be needed as the secondary data. Data Analysis Procedure The data in this study would be analyzed in some steps. Those were: Determining Research Population and Sample The population of this research was the English made test constructed by the teacher of SMKN 5 Mataram in academic year 2015/2016 used in the MID term test at the 2nd semester. The sample of this research would be the English test which was used by the teacher of second year students (at the 2nd semester) of SMKN 5 Mataram in academic year 2015/2016 This test was made by a teacher who is especially teach English in second year students at the 2nd semester in SMKN 5 Mataram. This English test was used to assess 9 majors of second year students which consisted of 2 majors from Desain Komunikasi Visual (DKV A & B), 5 majors from kria (kayu, kulit, keramik, logam and tekstil), 1 major from musik, and 1 major from Teknik Sepeda Motor (TSM). Collecting Data The data that would be collected were the English MID term test constructed by the teacher, syllabus and lesson plan used in second year students at the 2 nd semester of SMKN 5 Mataram. The list of learning program in the second semester would also be used in this study as the additional data to clarify the scoop of materials or basic competencies that have been taught to the students. Analyzing Data 1.
Content Validity 3
Djiwandono (1996 in Fitriani 2009:26) states that the way to analyze the content validity of teacher-made English test is by measuring the appropriateness of target competences or indicators with the test used. The analysis procedures of content validity would be done through analyzing the table of specification. The procedures would be as follows: a. Providing the table analysis of specification of the English subject administered to the second year students of SMKN 5 Mataram. b. Determining indicators based on basic competences which are stated in the lesson plans administered to second year students of SMKN 5 Mataram. c. Matching the items of the test with indicators for each basic competency. 2.
Find out the number of all indicators which were covered by the test items in the form of percentage by using the following formula:
Where: n = indicator tested N = Sum of indicators 3.
The next step was interpreting the percentage into the criterion of
content validity as shown in following table: Percentage
Interpretation
>80% - 100%
Very high content validity
>60% - 80%
High content validity
>40% - 60%
Medium content validity
>20% - 40%
Low content validity
>0% - 20%
Very low content validity
Table 3.2. The level of content validity (Marlina, 2011 retrieved from http://www.marlinawordpress.com)
C. FINDING AND DISCUSSION
4
Based on the list of English learning program at 2nd semester of second year students of SMKN 5 Mataram in academic year 2015/2016, there were 3 topics that had been taught for the MID term test and from these 3 topics there were 26 indicators that should be taught to the students. The topics were Conditional Sentence, Passive Voice, and Procedural Text. Since SMKN 5 Mataram has been using Kurikulum 2013 (K-13) as the curriculum, then the indicators which were related to the basic competencies of spirituality and sociality were not tested. It is because these two competencies are evaluated indirectly during the teaching and learning process. Conversely the indicators which were related to basic competency no. 3 (knowledge) and 4 (skills)
were tested in the MID term test. Hence the number of indicators
analyzed in this study became 19 indicators which were only for basic competencies no. 3 and no. 4. Related to the test construction, the teacher-made MID term English test of second year students of 2nd semester of SMKN 5 Mataram consisted of 13 items. 10 items were in the form of multiple choice with 4 options and the rests were in the form of essay. The content validity of teacher-made MID term English test administered to second year students of 2nd semester of SMKN 5 Mataram was analyzed by comparing the items of the test with indicators of each basic competency which was reflected on the lesson plans made by the teacher. Then the result of analysis showed whether the items of the test represented the whole materials or indicators in the lesson plans or just a part of them. The results are presented in the following table No
Basic
Indicator
Material
Competency
No. Of
Percenta
Item
ge
Tested
(%)
Procedural Text In Manual Form (teks prosedur berbentuk manual dan kiat-kiat (tips)) 1
3.6.
Menganalisis
fungsi
sosial,
struktur teks, dan
1. Siswa
mampu
menjelaskan
ciri-
ciri teks prosedur.
unsur kebahasaan
Struktur: menyebutkan bahan/bagian dari benda
5
2 (essay)
5, 26%
dari teks prosedur
yang
berbentuk manual
dipaparkan
dan kiat-kiat (tips), sesuai
dengan
konteks
2. Siswa
mampu
menjelaskan fungsi
secara
10
5, 26%
8, 9
10, 53%
lengkap, serta daftar langkah
teks prosedur.
penggunaanny.
yang 3. Siswa
mampu
mengidentifikasi
dilakukan Unsur Kebahasaan:
penggunaan imperative
verb
dalam
teks
simple present tense imperative,
prosedur. 4. Siswa
mampu
Nomor yang
mengidentifikasi
menyatakan
penggunaan simple
urutan
present tense dalam teks prosedur. 5. Siswa
mampu
kata keterangan
mengidentifikasi penggunaan nomor yang
menyatakan
urutan dalam teks prosedur. 6. Siswa
mampu
mengidentifikasi penggunaan
kata
keterangan
dalam
teks prosedur. 2
4.9. makna
Menangkap teks
prosedur, lisan dan
7. Siswa
mampu
menjelaskan kandungan
6
tulis,
3
berbentuk
informasi
yang
manual dan kiat-
disajikan
dalam
kiat (tips).
teks prosedur.
4.10. Menyunting teks
prosedur
8. Siswa
mampu
mengedit
dan
berbentuk manual
merevisi kesalahan
dan kiat-kiat (tips),
dalam
dengan
prosedur
memperhatikan
benar.
fungsi
teks dengan
sosial,
struktur teks, dan unsur
kebahasaan
yang
benar
dan
sesuai konteks. 4
4.11. Menyusun / 9. Siswa membuat
mampu
teks
membuat
teks
prosedur berbentuk
prosedur
manual
manual dan kiat-
dan kiat-kiat (tips)
kiat (tips) dengan
dengan benar.
memperhatikan fungsi
sosial,
struktur teks dan unsur
kebahasaan
yang
benar
dan
sesuai konteks.
Passive Voice (Tindakan/Kegiatan/Kejadian Tanpa Perlu Menyebutkan Pelakunya) 5
3.2. fungsi
Menganalisis sosial,
struktur teks, dan
10.
Siswa
mampu Struktur Teks:
menjelaskan fungsi kalimat pasif.
7
Insects are
unsur
kebahasaan
untuk menyatakan
considered 11.
Siswa
mampu
dangerous
tentang tindakan /
mengidentifikasi
kegiatan / kejadian
ciri-ciri
tanpa
pasif dalam konteks
caused by
penggunaannya.
earthquake
perlu
menyebutkan pelakunya
dalam
teks ilmiah, dengan memperhatikan fungsi
kalimat
struktur teks, dan unsur
kebahasaan
yang
benar
dan
sesuai konteks.
Tsunami is
affecting the 12.
Siswa
mampu
seabed.
Unsur ciri-ciri setiap jenis kebahasaan: kalimat pasif yang Kata kerja berhubungan be (is/ am/ dengan waktu are/ was/ kejadian (tenses) were) dan dalam konteks verb 3 form. penggunaannya. Tata bahasa,
4.11.
Menyusun
13. Siswa
mampu
ejaan, tanda
kalimat
baca, tulisan
teks lisan dan tulis
membuat
untuk menyatakan
pasif dalam bentuk
tangan dan
dan
pertanyaan secara
cetak yang
tentang tindakan /
tepat
jelas dan
kegiatan / kejadian
memperhatikan
tanpa menyebutkan
fungsi
pelakunya
struktur teks dan
menanyakan
dalam
dengan
sosial,
teks ilmiah, dengan
unsur
memperhatikan
kebahasaannya.
fungsi
10, 53%
animals.
mengidentifikasi
sosial,
5, 6
sosial,
struktur teks, dan
8
rapi.
5, 6
10, 53%
unsur
kebahasaan
yang
benar
dan
sesuai konteks.
14. Siswa
mampu
membuat
3 (essay)
5, 26%
kalimat
pasif dalam bentuk pernyataan secara tepat
dengan
memperhatikan fungsi
sosial,
struktur teks dan unsur kebahasaannya.
Conditional Sentences in Manual Form (Pengandaian jika terjadi suatu keadaan/kejadian/peristiwa di waktu yang akan datang) 6
3.8.
Menganalisis
fungsi
sosial,
15. Siswa
mampu Struktur Teks:
mengidentifikasi
struktur teks, dan
ciri-ciri
unsur
conditional
kebahasaan
struktur
untuk menyatakan
sentence
dan
konteks.
menanyakan
- If teenagers eat too much fast food, they
dalam
can easily become
tentang
overweight.
pengandaian terjadi
jika suatu
16. Siswa
mampu
- If you
mengidentifikasik
excercise
keadaan / kejadian
an
regularly,
/ peristiwa di
sentence
waktu yang akan
konteks
datang,
sesuai
dengan
konteks
penggunaannya.
conditional dalam
you will get the benefit physically
17. Siswa
mampu
and mentally
menjelaskan fungsi conditional sentence.
9
Unsur Kebahasaan: - If Clauses
7
5, 26%
18. Siswa
mampu
menganalisis
dalam simple present
penggunaan
- Main Clause
struktur
dengan
conditional sentence
modals can/ dengan
will
benar dan sesuai konteks.
7
4.12.
Menyusun
19. Siswa
mampu
teks lisan dan tulis
menulis
untuk menyatakan
menggunakan
dan
conditional
menanyakan
tentang
teks
sentence
pengandaian
jika
terjadi
suatu
dengan
sesuai konteks
penggunaannya
kejadian / kejadian
baik dalam bentuk
/
pertanyaan maupun
peristiwa
di
waktu yang akan datang,
pernyataan.
dengan
memperhatikan fungsi
sosial,
struktur teks, dan unsur
kebahasaan
yang
benar
dan
sesuai konteks. Total
7 indicators tested
8 items
Actually, there were 5 items (26, 31% items) which did not match with materials for second year students at the 2nd semester, but rather those items were tested in the first semester of second year students. Those unrelated questions are about Formal Invitation (item no. 1 to 4 (21, 05% items)) (Teks Khusus, Lisan Dan Tulis, Berbentuk Undangan Resmi Sederhana) and Asking and Giving Opinion
10
(item no. 1 essay (5, 26% items)) (Teks Lisan Dan Tulis Untuk Menyatakan Pendapat Dan Pikiran Serta Responnya). From the whole 13 items tested in the MID term test, only 8 items covered 7 indicators, while the 5 items were unrelated materials. Hence,
from the 19
indicators, there were totally 12 indicators were not tested in the MID term test. Therefore, it could be concluded that only apart of the test items covered the indicators. It meant that the content validity of teacher-made MID term test of second year students of 2nd semester of SMKN 5 Mataram did not cover the whole indicators stated on the lesson plans. Moreover, it can be concluded that the test made by the teacher was not proportional as stated on the table. For example, item no. 5 and 6 which were about Passive Voice indirectly assessed two indicators (Siswa mampu mengidentifikasi ciri-ciri kalimat pasif dalam konteks penggunaannya (10, 53% items) and Siswa mampu mengidentifikasi ciri-ciri setiap jenis kalimat pasif yang berhubungan dengan waktu kejadian (tenses) dalam konteks penggunaannya (10, 53% items)). Moreover, the indicator which was about Siswa mampu menjelaskan kandungan informasi yang disajikan dalam teks prosedur was covered by two items (item no. 8 and 9 (10, 52% items)). Based on the percentage result, it was found that the level of content validity of teacher-made MID term English test of second year students of 2nd semester of SMKN 5 Mataram was 36, 84%. It meant that it’s level was low content validity (>20% - 40%).
D. CONCLUSION AND SUGGESTION Based on the data analysis and the results obtained, there are several points as the conclusion of the research, those are as follows: Conclusion 1. The teacher-made MID term English test administered to the 2nd semester of second year students of SMKN 5 Mataram covered only 7 indicators
11
represented on 36, 84% of the indicators/material tested, while the rests indicators were not tested. 2. The representativeness of indicators covered by the test items is
36,84%.
This means the content validity of teacher-made MID term English test can be categorized as a low content validity test (>20% - 40%) 3. There were totally 13 numbers of test item in the MID term test. Ten (10) numbers were in the form of multiple choice and three (3) numbers were in essay form. From the 19 indicators have been taught, there were five (5) items (31, 58% items) were tested by using multiple-choice test form and there were only two (2) items (10, 52% items) were tested by using essay. 4. From the whole numbers of the test item, only 8 items of test covered the 7 indicators, while the 5 items (26, 31% items) did not match with materials for second year students of 2nd semester. The unrelated materials are Formal Invitation (item no. 1 to 4 (21, 05% items)) and Asking and Giving Opinion (item no. 1 essay (5, 26% items)). They were tested in the first semester of second year students. 5. There were 21, 05% items (4 items) which assessed material about Procedural Text, 15, 79% items (3 items) assessed material about Passive Voice, and 5, 26% items (1 item) assessed material about Conditional Sentence. 6. The teacher-made MID term English test was not proportional. There was certain item accumulated on some indicators such as item no. 5 and 6 (10, 52% items) which were about Passive Voice which assessed two indicators. On the same item, certain indicator was covered by two items (item no. 8 and 9 (10, 52% items)). Suggestion Having made some conclusions, this study would also present some suggestions as follows: 1. School should give training on how to construct a good test especially how to make a high content validity test by inviting the expert who is a master in that field.
12
2. Teacher should make a good test to measure the students’ achievement in one period of study and the test should have a high vontent validity in terms to test all materials that have been taught. 3. No matter what the reasons the teacher made the test for and no matter how bad the students’ results of test are, the teacher should consider that the students must be assessed with a high content validity of test. It is because the achievement test is made for assessing the students’ ability/achievement toward the materials have been taught during the learning process. 4. In constructing a test, it is better for the teacher to use table of specification in order to improve the content validity of the test. in another way, it also gives the teacher description about the test specifically and this table will be the avidence that the test has content validity. 5. For the next researchers who are interested in analyzing the quality of test, it is suggested that they would analyze the teacher-made final test which has more scope of materials that should be analyzed.
REFERENCES Arifudin. 2007. Language Assessment: A Competency Based-Approach. Mataram: Arga Puji Press Arikunto, Suharsimi. 2015. Dasar-Dasar Evaluasi Pendidikan. Jakarta: Bumi Aksara Arikunto, Suharsimi. 2010. Prosedur Penelitian: Suatu Pendekatan Praktik. Jakarta: Rineka Cipta. Bachman, Lyle F. 1990. Fundamental Considerations in Language Testing. New York: Oxford University Press Essays, UK. (23, March 2015). The Influence Of Tests On Teaching Backwash Education Essay. Retrieved on 2016, April 23 from https://www.ukessays.com/essays/education/the-influence-of-tests-onteaching-backwash-education-essay.php. Fera, Fardila. 2013. An Analysis of Content Validity of Teacher-Made English Test of Second Year Students of First Semester At MAN 1 Mataram Academic
13
Year 2012/2013. An Unpublished Thesis of S1 Program of FKIP Unram. Mataram: FKIP Unram. Fitriani. 2009. Multiple-Choice Test Analysis: An Analysis on Teacher-Made Test in Second Year Students of MTs. Nurul Harmain Narmada, Academic Year 2008/2009. An Unpublished Thesis of S1 Program of FKIP Unram. Mataram: FKIP Unram. Hadi, Irwan. 2012. An Analysis of Conent Validity on Teacher Made English Test of School Examination At SMAN 1 Kediri West Lombok Academic Year 2011/2012. An Unpublished Thesis of S1 Program of FKIP Unram. Mataram: FKIP Unram. Heaton, John B. 1990. Writing English Language Test (Longman Handbook for Language Teachers). USA: Longman Inc. New York Hughes,Arthur. 2003. Testing For Language Teacher. 2nd ed. London: Cambridge University Press. Kartini. 2011. An Analysis of Content Validity of Teacher Made English Test At First Semester of First Year at SMPN 4 Lingsar A. Y. 2010/2011. An Unpublished Thesis of S1 Program of FKIP Unram. Mataram: FKIP Unram. Palah, Nurul. 2014. An Analysis of Reliability-Validity of Teacher-Made Test At The Second Grade Students of SMKN 1 Kuripan in Academic Year 2012/2013. An Unpublished Thesis of S1 Program of FKIP Unram. Mataram: FKIP Unram. Rahman, Kasyfur. 2009. A ComparativeStudy on The Reliability of Multiple Choice Items And Cloe Test in Reading Assessment. An Unpublished Thesis of S1 Program of FKIP Unram. Mataram: FKIP Unram. Sugiyono. 2013. Metode Penelitian Pendidikan: Pndekatan Kuantitative, Kualitative, dan R&D. Bandung: Alfabeta.
14
