ENGLISH TESTING AND LEARNING AT THE VOCATIONAL SCHOOL: STUDENT S LIVED EXPERIENCE

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

ENGLISH TESTING AND LEARNING AT THE VOCATIONAL SCHOOL: STUDENT’S LIVED EXPERIENCE

A THESIS

Presented in Partial Fulfillment of the Requirements for the Master’s Degree in English Language Studies

by Neny Mariani Student Number: 056332027

THE GRADUATE PROGRAM IN ENGLISH STUDIES SANATA DHARMA UNIVERSITY YOGYAKARTA 2009


i


ii


STATEMENT OF ORIGINALITY

This is to certify the originality of the written study. No single part of this thesis but materials cited as references, either explicitly or implicitly, shows similarities or resemblance with any other work, and therefore to the best of my knowledge, no copyright violation is indicated. Hereby I declare that this thesis represents my own work. I assume full responsibility for the content of the work and bear all consequences it may result in. Yogyakarta,

21st December 2009

Neny Mariani

iii


LEMBAR PERNYATAAN PERSETUJUAN PUBLIKASI KARYA ILMIAH UNTUK KEPENTINGAN AKADEMIS Yang bertanda tangan di bawah ini, saya mahasiswa Universitas Sanata Dharma Yogyakarta dengan Nama

: Neny Mariani

Nomor mahasiswa: 056332027 menyatakan bahwa demi pengembangan ilmu pengetahuan memberikan kepada Perpustakaan Universitas Sanata Dharma Yogyakarta karya ilmiah yang berjudul: “English Testing and Learning at the Vocational School: Student’s Lived Experience” yang saya tulis beserta perangkat terkait yang tersedia. Dengan demikian kepada Perpustakaan Universitas Sanata Dharma Yogyakarta saya berikan hak dan wewenang untuk menyimpan, mengalihkan dalam bentuk media lain, mengolahnya dalam bentuk pangkalan data, mendistribusikan secara terbatas, dan mempublikasikannya di internet atau media lain untuk kepentingan akademis tanpa perlu meminta ijin dari saya maupun memberikan royalti kepada saya selama tetap mencantumkan nama saya sebagai penulis. Demikian pernyataan ini saya buat dengan sesungguhnya.

Dibuat di

: Yogyakarta

Pada tanggal: 21 Desember 2009

Yang menyatakan,

Neny Mariani

iv


ACKNOWLEDGMENTS

First of all, I wish to offer my biggest adoration to the Almighty, who has granted me an opportunity to pursue further education and kept sending so many wonderful people I learn from and work with. I am so grateful for all the blessings, inspiration and never-ending guide from nothing to something, prior to, during, and upon completion of this master’s thesis. I guess there will never be bigger celebration to mark the accomplishment of this project than expressing the words of glory, "Praise be to Allah, the Cherisher, Sustainer of the worlds!". Also, I do not want to miss this opportunity to express a word of honor to my parents, from whom I received my first education in my earliest years, for raising me up and introducing the Most Gracious and Merciful. Although the content of this thesis is essentially and eventually my own responsibility, this work by all means results from a great number of contributors. Therefore I wish to take this opportunity to acknowledge and appreciate all kinds of contribution which I have received directly and indirectly from those whose names I will not be able to print out here one by one. First of all, I would extend my particular gratitude to my adviser, Pak Bis (Dr J Bismoko), for his great dedication and allowing me to get an easy access for correction and consultation. It would almost be impossible to do this research project without the existence of such a prominent figure behind the preparation, implementation and presentation of the work. I have been largely indebted to all the lecturers and everyone in the graduate v


program of this university. I would like to thank the Graduate Program Director, Prof. Dr. Augustinus Supratiknya for legalizing this report. I thank the Board of Examiners, Pak Muk (FX. Mukarto, Ph.D), Pak Dwi (Dr. B.B. Dwijatmoko, M.A), Pak Alip (Dr. Fr. B. Alip, M.Pd, M.A.) for helping me finalize this work with the thorough examination. I also thank the rest of the lecturers, Pak Barli (Drs. Barli Bram M.S.), Bu Novi (Dr. Novita Dewi, M.S., M.A. (Hons)), and Pak Pomo (Prof. Dr. Soepomo Poedjosoedarmo) for their "shower of sparks" and "drops of dews". For the financial and non financial support as well as attention, my gratitude goes to LPMP Yogyakarta, including Bu Anna (Dra. Anna Maria Hendraswari, M.A.), Pemda Kabupaten Kulon Progo (including Pak Ridwan) and Dinas Pendidikan Kabupaten Kulon Progo (including Pak Yatiman ). Next, I would express my sincere gratitude to my research participants, my fellow English teachers and students, for every word they have contributed, their honesty, and the time and effort they have spent to reply to my enquiries. For me, they have been sunshine in the rainy day and water in the desert. I would like to express my thankfulness to everybody in the school where I spend wonderful years, SMK Negeri 1 Pengasih, which includes the school principal Drs. H. Rumawal, my colleagues and students, for the encouragement, challenge, spiritual, technical and financial support. I thank all authors and researchers whose work is very valuable and contributive to my project. I have made every effort to trace all the resources the best I could, and I hope I do not miss a thing. To all my fellow students and my acquaintances, I would say a big thank-you for the share of knowledge and feelings. All the years when we were involved in such tough vi


and fierce discussions, arguing for and against, sharing laughter and tears have been so precious to me. I will always miss the time when the feeling of friendship was that strong. Upon completion of the work, I would like to dedicate this thesis to the Motherland and to the world. I realize this work will be more meaningful it gains attention from wider audience. I look forward to and welcome any response coming from all directions. Yogyakarta, 21st December 2009

Neny Mariani

vii


TABLE OF CONTENTS TITLE PAGE APPROVAL PAGE DEFENSE APPROVAL PAGE STATEMENT OF ORIGINALITY STATEMENT OF LICENCE AGREEMENT ACKNOWLEDGEMENTS TABLE OF CONTENTS LIST OF FIGURES LIST OF TABLES LIST OF ABBREVIATIONS ABSTRACT INTISARI CHAPTER I A. B. C. D. E. F.

INTRODUCTION

1

BACKGROUND PROBLEM IDENTIFICATION PROBLEM LIMITATION PROBLEM STATEMENT RESEARCH GOALS RESEARCH BENEFIT

CHAPTER II

i ii iii iv v viii xi xii xiii xiv xvi

1 5 10 11 12 12

LITERATURE REVIEW

14

A. THEORETICAL REVIEW 1. Assessment 2. Language testing, English testing and backwash effects 3. Learning and learning autonomy 4. Language learning and language learning autonomy 5. Vocational school 6. Experience and attitude 7. Qualitative research and methodologies B. THEORETICAL FRAMEWORK

14 14 18 29 34 37 39 40 46

CHAPTER III

55

A. B. C. D. E.

METHODOLOGY

RESEARCH DESIGN DATA SOURCES TYPES OF DATA DATA COLLECTION TECHNIQUES DATA PROCESSING PROCEDURE 1. DATA COLLECTING 2. DATA ANALYZING viii

55 60 62 63 66 66 67


F. TRUSTWORTHINESS OF RESEARCH RESULTS CHAPTER IV

.

ANALYSIS RESULTS

A. DATA AND SOURCE SELECTION B. INSTRUMENT VALIDATION AND ACCOUNTABILITY C. DATA GATHERING, RECORDING AND PRESENTATION D. DATA ANALYSIS ACTUALIZATION E. DISCUSSION: DESCRIPTION AND INTERPRETATION 1. Student participant portraits in English testing and learning 2. Thematic description and illustrative quotations 3. Participants’ meaning of testing 129

CHAPTER V

CONCLUSIONS, IMPLICATIONS, AND RECOMMENDATIONS

A. CONCLUSIONS B. IMPLICATIONS C. RECOMMENDATIONS

68 73 73 78 80 85 106 106 115

144 144 148 149

BIBLIOGRAPHY

151

APPENDICES

158

Appendix 1 Interview probes 1.1 Questions to students 1.2 Questions to teachers Appendix 2 Coding blue print 2.1 Participant: students 2.2 Participant: teachers 2.3 Observation on test implementation 2.4 Inductive codes Appendix 3 Observation data of events and behavior from official meetings and environs 3.1 Official meeting 1 ( external) 3.2 Official meeting 2 ( external) 3.3 Official meeting 3 ( external) 3.4 Official meeting 4 (internal) Appendix 4 Observation data of events and behavior from test sites 4.1 Classroom test 1 4.2 Classroom test 2 4.3 Classroom test 3 4.4 Classroom test 4 4.5 Classroom test 5 4.6 High-stakes test 1 4.7 High-stakes test 2 ix

158 158 158 159 159 160 161 161 162 162 163 165 165 166 166 166 167 167 167 168 169


4.8 High-stakes test 3 Appendix 5 Observation data of physical and non physical environs 5.1 Situation prior to classroom test 1 5.2 Situation prior to classroom test 2 5.3 Situation prior to high-stakes test 1 5.4 Situation prior to high-stakes test 2 5.5 Situation after high-stakes test 1 5.6 Situation after high-stakes test 2 5.7 Situation after high-stakes test 3 Appendix 6 Researcher’s diary Appendix 7 Interview data 7.1 Student Atik 1 7.2 Student Atik 2 7.3 Student Siti 1 7.4 Student Siti 2 7.5 Student Endang 7.6 Teacher Iin 1 7.7 Teacher Iin 2 7.8 Teacher Nanik 1 7.9 Teacher Nanik 2 Appendix 8. News-stories and information 8.1 Radio news 8.2 Television news 8.3 Printed news 8.3.1 Printed news 1 8.3.2 Printed news 2 Appendix 9 Documents 9.1 Official document 1 9.2 Official document 2 9.3 Student Endang’s answer sheet 9.4 Student Atik’s work 9.5 Student Atik’s school report 9.6 Classroom test sheet 9.7 High-stakes test booklet 9.8 Test-result report 9.8.1 Score report from high-stakes test 1 9.8.2 Score report from high-stakes test 2

x

170 171 171 172 172 173 174 174 174 175 183 183 191 201 207 214 221 226 238 245 249 249 250 257 257 258 259 259 260 261 262 263 266 267 269 269 270


LIST OF FIGURES 1. Figure 1.1 Tripartite partnership model 2. Figure 1.2 First, essential stage in learning enhancement partnership 3. Figure 1.3 Learning and testing relationship 4. Figure 1.4 Testing and stakeholders 5. Figure 2.1 Communication in testing environment stage 1, 2, 3 6. Figure 2.2 Bachman’s categorization of the use of tests 7. Figure 2.3 Brown’s language test type categorization 8. Figure 2.4 McNamara’s language test type categorization 9. Figure 2.5 Test type categorization 10. Figure 2.6 Testing in the language program toward independent learning 11. Figure 2.7 Positioning English testing at the Vocational School 12. Figure 2.8 English testing aspects categorization 13. Figure 3.1 From research questions to research results 14. Figure 3.2 Types of data 15. Figure 3.3 Data processing procedure 16. Figure 4.1 Student’s lived experience in English testing and learning at the vocational school seen from various angles 17. Figure 4.2 Data processing stages 18. Figure 4.3 High-stakes test and thematic description, gleaned from secondary data sources 19. Figure 4.4 Different interests within testing

xi

6 7 8 9 21 25 27 27 29 47 50 52 59 62 66 79 87 118 140


LIST OF TABLES Table 2.1 Theoretical concept map Table 3.1 Research specification Table 4.1 Student participant data Table 4.2 Teacher participant data Table 4.3 Observation blueprint Table 4.4 Sample data observation 1 Table 4.5 Sample data observation 2 Table 4.6 Probing interview questions Table 4.7 A priori coding inventory for student participants Table 4.8 Sample corpus of data Table 4.9 Sample tabulating table Table 4.10 Semi-ordered list of responses to the open-ended question “Bagaimana perasaanmu sebelum, pada saat, dan setelah mengikuti ulangan harian?” Table 4.11 Category of responses to the open-ended question “Bagaimana perasaanmu sebelum, pada saat, dan setelah mengikuti ulangan harian?” Table 4.12 Emerging themes and categories from student participants’ narratives and behavior Table 4.13 Components, categories and indicators constituting environment around English testing implementation Table 4.14 Emergent categories from primary and secondary data

xii

54 58 76 76 81 82 83 84 85 88 96

101

103 104 116 117


LIST OF ABBREVIATIONS CARLA: Center for Advanced Research on Language Acquisition Depdiknas: Departemen Pendidikan Nasional EFL: English as a Foreign Language ESL: English as a Second Language ESP: English for Specific Purpose FHI: Family Health International LOTE: Language Other Than English MTs: Madrasah Tsanawiyah RRI: Radio Republik Indonesia SLA: Second Language Acquisition SMK: Sekolah Menengah Kejuruan SMP: Sekolah Menengah Pertama TOEIC: Test of English for International Communication TVRI: Televisi Republik Indonesia UWE: University of the West England

xiii


ABSTRACT Neny Mariani, 2009. English Testing and Learning at the Vocational School: Student’s Lived Experience. Yogyakarta: Sanata Dharma University, the Graduate Program in English Studies

In the world of English education, in both theory and practice, testing which is known as a means to measure learners’ abilities, continually poses big problems. In the national level, testing has also become the main issue of a continual debate and never-ending polemic. Despite all the shortcomings it may bring with it, testing is widely used since it serves enormous practical functions. In this situation, there is a need to promote learner-centered learning in that learning and learner’s factors should be taken into consideration in the use and design of English tests. This enhancement will accelerate learning autonomy; improve quality and sustainability of learning process and achievement. This research aimed at describing, interpreting and explaining how the vocational school students experience English testing, revealing what testing means to them and what in their perspective constitutes an ideal English test. As progressive qualitative research, it was not meant to explain, predict, and control the student and teacher properties and behavior. Instead, it was to reveal, describe and understand better how the vocational school students experience English testing. In order not to marginalize the participants as a result of the inquiry, this research adopted the advocacy/participatory approach by starting with the suppression and oppression issues and assumed that as the researcher I would proceed collaboratively. Data were taken from an Indonesian vocational school with three students (primary) and two English teachers (secondary) as data acquisition sources. Data collection instruments included observation, interview and document reviews. Verbal data, the participants’ narratives, were analyzed with content analysis comprising descriptive and interpretative. To improve trustworthiness of the research, data and methodological triangulation and thick description were implemented over the course of the study. As a researcher, I took an active participation and intervened by designing an on-going evaluation in order to advocate becoming more empowered, autonomous, and self-fulfilling. The research reveals that despite the similar concerns among the student participants in testing, learning and achievement, there were unique emotional experience and learning experience. It also reveals emergent themes around the time of English testing which, in addition to the prefigured themes test backwash, qualities, test content, learning and autonomy, includes status of test, fairness, time allocation, achievement, competitiveness, preparedness. In addition, my interpretation of what testing meant to the students as well as teachers shows that there was a shift of concerns among teacher participants in different types within xiv


different contexts of testing. In the context of classroom testing, the teacher participants had different concerns from the student participants, validity and reliability, while the students were concerned about learning and achievement. In the context of high-stakes testing, however, teachers and students shared similar concerns, learning and achievement. That classroom testing potentially created a point of clash and hurt student learning as its implication was due to the different concerns in testing between the teachers and students. That high-stakes testing created a gentleman’s agreement was due to the similar concerns. These research results will lead to better understanding for the participants, students and teachers, researcher as well as the audience. The participant improved awareness of testing and learning and the researcher and audience improved emphatic understanding of the participant experience, would consequently help facilitate learner’s learning need and in turn lead to healthy learning process and better results such as in particular to design a more progressive test and make a wiser decision on why, when, how to use an educative test as assessment.

xv


INTISARI Neny Mariani, 2009. English Testing and Learning at the Vocational School: Student’s Lived Experience. Yogyakarta: Universitas Sanata Dharma, Kajian Bahasa Inggris

Di dalam pengajaran bahasa Inggris, baik dalam teori maupun praktek, tes yang dikenal sebagai alat ukur kemampuan siswa masih menghadapi berbagai masalah. Di kancah nasional, tes juga menjadi perdebatan dan polemik yang tak kunjung selesai. Dengan segala kekurangannya, tes masih digunakan secara luas karena masih banyak fungsinya. Dalam situasi seperti ini, diperlukan upaya peningkatan pembelajaran yang berpusat pada pemelajar dalam hal faktor-faktor kegiatan belajar dan siswa perlu diperhitungkan dalam penyusunan dan pemakaian tes. Penekanan ini akan membawa dampak percepatan pembentukan kemandirian belajar, meningkatkan kualitas dan keberlangsungan proses serta keberhasilan proses dan prestasi belajar. Penelitian ini mencoba menggambarkan, memahami, dan menjelaskan bagaimana pengalaman siswa SMK dalam tes Bahasa Inggris dan mengungkap apa makna tes bagi mereka serta apa pandangannya yang merupakan tes ideal itu. Karena bersifat kualitatif, penelitian ini tidak dimaksudkan untuk menerangkan, memprakirakan dan mengendalikan unsur-unsur yang ada pada siswa dan guru, termasuk perilakunya. Namun, penelitian ini akan membuka tabir, menggambarkan dan memahami secara lebih baik bagaimana pengalaman siswa SMK terhadap tes Bahasa Inggris. Untuk tidak mengesampingkan peserta sebagai akibat dari penelitian ini, pendekatan advocacy/participatory digunakan dengan dimulai adanya isu penekanan dan penindasan, dan anggapan bahwa peneliti akan melakukan tugasnya dengan kompromi. Data diperoleh dari sebuah SMK di Indonesia dengan sumber data dari tiga siswa (data primer) dan dua guru Bahasa Inggris (data sekunder). Alat pengumpulan data berupa observasi, wawancara, dan telaah dokumen. Data verbal yang berupa ucapan peserta dianalisis dengan analisis isi yang meliputi analisis deskriptif and interpretatif. Untuk meningkatkan keterpercayaan hasil penelitian, triangulasi data dan metodologi serta thick description dilakukan selama proses dan pelaporan hasil penelitian. Sebagai peneliti, saya terlibat aktif mengambil bagian dan melakukan intervensi dengan cara melakukan refleksi kegiatan yang terus menerus dalam membantu menyuarakan agar lebih berdaya, mandiri dan dapat memenuhi kebutuhan belajarnya sendiri. Penelitian menunjukkan bahwa meskipun di antara sesama siswa terdapat persamaan dalam kepentingan yaitu belajar dan prestasi, mereka memiliki perbedaan dalam pengalaman emosional dan pengalaman belajar. Penelitian ini juga menunjukkan munculnya tema-tema baru seputar pelaksanaan tes yaitu xvi


selain dampak, kualitas, isi tes, belajar dan kemandirian, juga mencakup status tes, keadilan, alokasi waktu, prestasi, kebersaingan, dan kesiapan. Sebagai tambahan, interpretasi saya mengenai apa makna tes bagi siswa dan guru SMK menunjukkan adanya pergeseran kepentingan guru bila terdapat perbedaan jenis dan konteks pengujian. Dalam konteks tes berbasis kelas, guru mempunyai perbedaan kepentingan dengan siswa, yaitu validitas dan reliabilitas; sementara itu siswa mempunyai kepentingan belajar dan prestasi. Namun dalam konteks tes skala besar, guru dan siswa mempunyai kepentingan yang serupa, yaitu belajar dan prestasi. Bahwa tes berbasis kelas berpotensi menimbulkan konflik dan menghambat proses belajar siswa sebagai implikasinya itu terkait adanya perbedaan kepentingan dalam tes antara guru dan siswa tersebut. Tes berskala besar menimbulkan adanya saling pengertian itu karena kesamaan kepentingan. Hasil penelitian ini akan memberikan pemahaman yang lebih baik bagi peserta, siswa dan guru, peneliti dan khalayak. Meningkatnya kesadaran diri peserta terhadap pengujian dan pemelajaran, dan rasa empati dari peneliti dan khalayak akan membawa ke arah proses belajar yang sehat dan hasil yang lebih baik, dan membantu mengakomodasi kepentingan dan kebutuhan belajar siswa, misalnya dengan merancang tes yang bersifat progresif serta melakukan kebijakan tentang mengapa, kapan, bagaimana menggunakan tes sebagai alat penilaian yang mendidik.

xvii


CHAPTER I

INTRODUCTION

To identify and expose the main research problems elaborately, in this introductory chapter I will examine concepts and issues in English education, specifically English testing in the global as well as local context. I will begin with the background of the study and consecutively discuss the problem identification, problem limitation, problem statement, research goals, and research benefits.

A. BACKGROUND Over the last few decades, as writers say, language test theory has put a great deal of emphasis on three major areas. First is the nature of language; that is when we talk about skills and language components. Next is the nature of language use; that is when we talk about authenticity. Third is the nature of measurement; that is when we talk about validity and reliability. This can be seen from the work of, take for example, Bachman (1995) and McNamara (1996). It is not surprising then that in practice test developers and specialists are reportedly deeply concerned with anything related to defining language competence, understanding the factors which influence performance on language tests and finding valid and reliable ways of measuring language competence and development, and recently greatly worried about whether test materials and tasks represent actual use of language for communication. In conclusion, we are excessively concerned with the overt function of the test, as a means of measurement and assessment, ignoring the covert but influential function, that is, as a learning experience. 1


It is clear for us that most test theories as proposed by Bachman (1995), McNamara (1996) and Brown (2005) imply that they place test-takers as the objects of the test. Test-takers are considered as “something we can do measurement upon”. It is not clear whether it will give any benefits or harms to them. The definitions imply that test is designed for the sake of other parties, while the primary stakeholders, which are in fact the test-takers themselves, are totally forgotten. This is what I call oppression, suppression and colonization. Up to this point, it is obvious that so far we have been busy dealing with constructing a test for its own sake and for its own right, rather than catering for student learning needs or facilitating learning in the stage of evaluation within the teaching and learning process. Whether the test construction has harmful or positive effects on learning remains out of discussion and little or even almost nothing about learning has been taken into consideration in the design and use of language tests in most EFL/ESL courses. The lack of discussion and the absence of such consideration in the test design and use is greatly regrettable. In relation to this, Hughes’ (2003:1) assertion, showing the inadequacy in the test design and use, is worth quoting, “[…] a great deal of language testing is of very poor quality. Too often language tests have harmful effects on teaching and learning […]”. In this problematic situation, we then need a review on the most current educational and learning issues. Life-long education has been widely noted since early times and Delors et al. (1998:20) reaffirm it by iterating that the concept of education pursued throughout life recently emerges as one of the keys to the twenty first century. Education, an ongoing process of improving knowledge, skills and attitude or behavior, with the four pillars on which education throughout life is


based, learning to know, learning to do, learning to be and learning to live together, is said to have a fundamental role to play in personal and social development in that it is an exceptional means of bringing about personal development and building relationship among individuals, groups and nations. Next, let us refer to the postmodernism theory, with its progressivism philosophy, which says education should be directed to make learners empowered, autonomous, and eventually become self-fulfilling individually and socially in their own world. Finney (2002:73) puts it this way, The purpose of education from the point of view of the process model is to enable the individual to progress toward self-fulfillment. It is concerned with the development of understanding, not just the passive reception of ‘knowledge’ or the acquisition of specific skills. The goals of education are not defined in terms of particular ends or products, but in terms of the processes and procedures by which the individual develops understanding and awareness and creates possibilities for future learning. (Italicized words are my emphasis) Here we should note that the key word to education is “self-fulfillment”, and consequently we can say we need learner-centeredness, an approach to education that aims at developing in each student a sense of responsibility for his or her own learning (The University’s Strategic Plan, 1995 as quoted in Hall, 2000:2). In other words it means the development of learner autonomy should be the heart of any educational program, which certainly includes language teaching and learning. To ensure the enhancement of learner-centered learning and the development of learner autonomy as stated above, therefore, I felt it urgent to do research on a particular topic so that learning, which is the most essential of all in education, would be appropriately ensured, adequately addressed and consistently become the main consideration in the whole process of designing and implementing educational course all the way from assessing learner’s learning needs to the stage of program


evaluation, including testing. Test, which is in a general sense the main instrument in learner assessment, is indispensable part of the ESL/EFL/ESP teaching and learning process, and it has been my interest to do a research on. The choice of the topic was due to the following reasons. First, as you observe, test as the main instrument of assessment remains controversial and problematic in both theory and practice in that students’ learning needs and viewpoints in testing has been paid little attention. In addition, we can refer to that of two prominent advocates of ESP Hutchinson and Waters (1994:146), who realize and notice there is a shortage of sound theoretical or empirical basis for ESP testing and lack of discussion or guidance on ESP testing: “Far more research is therefore needed before we will really know what the requirements of a good ESP test are”. Second, my personal and professional background as an EFL teacher had driven me conduct numerous kinds of tests for vocational school students in the local, regional as well as national level. Thus, in Trochim’s words (2006:1), I had been living and grown with the phenomenon, or in other words, it is my “direct experience of the phenomenon”. Therefore in relation to the learning enhancement and the learner autonomy development as I have already described, then I concentrated on the topic, English testing. In relation to the topic, I believed, the focus of further research on testing should be on learning factors and an understanding of the impacts of language testing to learning, instead of language or test property analysis. The statement of Chalhoub-Deville (2003:380) is worth quoting to confirm the idea, Given current indications in language testing, the nature of the components underlying the L2 construct as well as the manner in which these components interact are issues that researchers need to address in the twenty-first century. 4


Indonesian vocational schools, whose mission is to provide a middle-class workforce and to improve their competitiveness, are greatly concerned with their EFL or more specifically ESP teaching and learning. A policy shift from a supplydriven system to a demand-driven system guided by labor market signal, curriculum reform, developing syllabus design, improvement on learning resources, teacher training, the adoption of the international standardized test, periodical debate contest are clear illustrations of the struggle. Nevertheless, the major common problem remains. Having spent their years of schooling, a number of students are still considered ‘unsuccessful’ English learners, not only emerging with a very low proficiency but also indicating low motivation and having a high dependence on teachers in their learning. Thus, despite tremendous progress in curriculum reforms and policies, two significant things, which are implications of the curriculum reforms itself, remain untouched: understanding of student’s learning needs and experience in testing. In such a situation, to do research on English testing and learning is highly relevant since the results will surely be contributive to learning enhancement

and

sustainability and eventually help overcome the persistent problems in education.

B. PROBLEM IDENTIFICATION The problematic situation, which takes place in both theory and practice, implies a kind of oppression or colonization upon learners, and then it raises questions as “If a test is a measurement, why measure?, “Is measuring the ultimate goal in all education?”, ”Where is in language testing the room for learning facilitation which builds learner’s autonomy?”, “Is a traditional paper and pencil


test totally inauthentic in language testing?”, “Does something inauthentic have no meaning and bring bad impacts to learning?”, “If a test is an instrument for us to use, why is it not designed as user-friendly?”. With all these problematic questions, it is clear that an educational reform is urgent to do, and further research on English testing is the right way to implement the reform. To get out of the problematic situation in the world of education, it is a good idea to refer to what Delors et al. (1998: 27) state, The main parties contributing to the success of educational reforms are, first of all, the local community, including parents, school heads and teachers; secondly, the public authorities; and thirdly, the international community. Many past failures have been due to insufficient involvement of one or more of these partners. (bold-faced is my emphasis) From here, it is obvious that the involvement of all parties in the reform is obligatory; local community’s contribution takes precedence over the others, and that the absence of any party in the involvement will lead to another failure. Figure 1.1 presents a clear illustration of this. First Party Local Community

Educational Reforms

Second Party Public Authority

Third Party International Community

Figure 1.1 Tripartite partnership model in education reform (visualizing the idea of Delors et al., 1998 ) 6


Furthermore, for the success of educational reforms, Delors et al. (1998:27) also necessitate a dialog among the parties as they point out, Local community participation in assessing needs by means a dialogue with the public authorities and group concerned in society is a first, essential stage in broadening access to education and improving its quality. (my emphasis)

This is in line with learner-centered teaching orientation which begins with the premise that the learner is the central reference point in decision making (Tudor, 1997). Thus, if we want to enhance learner-centered learning and to know the requirements of a good test, learning-needs assessment is significant to do, and having a dialog with test stakeholders to ensure sufficient involvement of pu blic authority and local community is highly relevant. This can be illustrated in Figure 1.2.

Local Community: Students, parents, teachers, school heads

Educational Reform: Enhancing learner-centered learning Public Authority: Government, researchers Figure 1.2 First, essential stage in learning enhancement partnership (developed from the idea of Delors et al., 1998)

One of the logical consequential applications of this theory is that it is urgent to do research which cares about the learners’ concern by having a dialog with students and teachers as the local community party. Again, Tudor (1997) confirms this idea, stating that needs analysis and goal-setting can be more effective if


students inform them with their knowledge and experience; and learning is more successful if methodology and study mode take into account student preferences and characteristics. In this research, therefore I as the researcher representing the public authority party will facilitate what they want to say in order to implement the reform effectively. To begin with, let us study how learning and testing is interrelated. Here, Figure 1.3 shows that in the context of educational program, testing can be seen as a learning environment, which constitutes one of factors enhancing learning.

Factors enhancing learning

Learning environment

Testing

Figure 1.3 Learning and testing relationship There must be a great number of factors enhancing learning, one of which is learning environment. There must be a lot of things which constitute learning environment. And testing is just a part of it. By all this I mean that if language testing is highly problematic, it is in itself a bad learning environment which in turn may hurt learning. Further, as we are all aware of, language testing as an important phenomenon in language learning has a number of stakes-holders or parties who are concerned in it, ranging from local community such as the student test takers, public authority such as the government to international community, such as authors and test specialists. This is illustrated in Figure 1.4. 8


Government

Test specialists Schools LANGUAGE TESTING Society Educators, Teachers

Parents

Students

Figure 1.4 Language testing and stakeholders Since students and teachers, the most concerned stakeholders, deserve benefits from testing, instead of colonization and oppression, time has come to include their participation in the reform. It is necessary to do a research project which enables to amplify and hear their voices as the starting point to shift the focus from dealing with the nature of language, the nature of language use, the nature of measurement, to understanding of the nature of learning. In order to investigate the problems thoroughly, it is necessary to employ a research paradigm which puts its interests in the participants’ narratives and enables to go deeper by digging out their experience to the research topic; a research with critical theory as the theoretical perspectives, which are concerned with empowering human beings to transcend the constraints placed on them by race, class and gender (Fay, 1987 as quoted in Creswell, 2003:10). And qualitative research with advocacy


approach is proved to be the right answer. Unlike quantitative research, which has interests in making generalization, qualitative research, as literature indicates, does not mean to make any generalization, for human beings cannot be generalized. A number of methods are available to choose, ranging from, observation, open-ended interviews, semi-structured interviews, structured interviews and documentary analysis. More detailed description on the progressive qualitative research methods will be available in the next chapter.

C. PROBLEM LIMITATION To anticipate any probable problem which most likely arose from the research due to various constraints such as the limited time, cost and my own personal capacity, it was necessary to set up boundaries by using delimitation and limitation as parameters for my study (cf. Creswell, 2003:147). I had to narrow down the coverage (scope) and methodology of this study and identify any potential weakness of the study. From the numerous problems in language testing and learning already identified, I wanted to focus my study more specifically on English testing and learning. I limited the scope of English test here only to that conducted at the vocational school consisting of classroom test and high-stakes tests with limited aspects of goal, process and material. The stakeholders were limited to students and teachers, with the reason that they represented local community to include in the dialog and while they were the key factor in the reform, so far their involvement was very low and inadequate as already indicated. To put it in another way, the significant role of their lived experience and attitudes toward English testing had


regretfully ignored in both research and practice. The scope for teachers was limited to those who teach English at the vocational school, while the scope for students here is only to those who major in business studies at the vocational school. As to methodology, I limited to those which are applied in the qualitative research. I chose phenomenology as the strategy of enquiry with narrative designs. Data collecting instruments such as interview, observation and document review would be applicable. The implication of this is that the research results would be interpretable within the limit and the findings could be subject to other interpretations.

D. PROBLEM STATEMENT Testing has been one of the most problematic areas in English education. English testing has been used and designed for its own right and its own sake. Learner’s learning factors have received less attention than they should. It is obvious up to this point that in order to achieve the purpose of education as suggested by the process model of curriculum in the progressivism philosophy, learning enhancement, as already discussed, is urgent to implement. To ensure the learning enhancement, one of the best ways is doing research in the problematic areas. In order to be aware of and understand the students’ lived experience in English testing at the vocational school, this research, therefore, aimed to answer theoretically and empirically the following question, what is the vocational school students’ lived experience in English testing and learning?. Therefore, throughout the research we discussed and tried to seek the answer to how vocational school students of Business Studies experience their English language testing and learning.


E. RESEARCH GOALS This research aimed to describe, and interpret, how vocational school students experience English testing and what in their perspective constitutes a good English test and to reveal the answer to the research question theoretically and experientially. As qualitative research, it was not meant to explain, predict, and control the participant properties and behavior (cf. Creswell, 2003:31). Instead, it was to reveal, describe and understand better how the vocational school students experience English testing and to help them understand their own perspective, in order to advocate becoming more empowered, self-fulfilling, and autonomous, which means they can do what they are supposed to do and they can do it on their own when they learn (cf. Finney, 2002:73). The goal of this research was therefore to describe and interpret the students’ lived experience in English testing around the time they learn English.

F. RESEARCH BENEFITS This research, in general, will surely be beneficial to the evolving body of knowledge in English education, science and technology. First, it will contribute further insights for responding to problems regarding the use and development of English test not for its own sake but in relation to what ESL is concerned with: learning; and second, it will help educational policy makers and test specialists in relation to course design and more specifically to the use and design of English tests. The research, which comprises the process and the report, is expected to bring benefits to the participants, researcher, and audience. The research process will develop the participants’ awareness and the researcher’s empathic understanding


of what English tests mean to the students. In another way of saying then, a positive change is expected to occur in that the results will enlighten marginalized students. For the teacher and student participants, the results help them to understand their own perspectives. Meanwhile the research report which consists of the description and interpretation of what is the participants’ lived experience in English testing will bring benefits to the audience. Specifically for teachers of English, it will help tackle persistent problems in English teaching delivery and more particularly in testing. For broader audience, it will lead to their empathic understanding, emancipation, empowerment, autonomy and self actualization so that learning in general will be more significant and sustainable.

13


CHAPTER II

LITERATURE REVIEW

The main purposes of this chapter are; first, to ensure the status of the research topic, and second, to identify and clarify the research question theoretically, how I will use it to account for the research problems and intended solutions. It consists of two parts; theoretical review, in which I will try to show the complexity of the research problem theoretically, and theoretical framework, which will be a review of the answer to the research question and why so.

A. THEORETICAL REVIEW Here I will examine central issues related with research subject and research methodologies comprising constructs, concepts and their sub-categorizations. Theories upon assessment, language testing, English testing, backwash effects, learning, learning autonomy, language learning, language learning autonomy, experience, attitude, vocational school and qualitative research methodologies will therefore be reviewed. Each will include the what, why and how. 1. Assessment To present the clearer definition of testing, let me begin with the context where it takes place, i.e. assessment. The existence of assessment in an educational program is beyond question. Rayment (2006:1) presents a number of definitions of assessment, and I would bring three of them. According to him, the first definition of assessment is 14


The process whereby evidence is obtained through the outcome of specific questioning such as tests and surveys, and used to determine results based on the findings of such methods. These may also define the status or value of an event, thing or person’s abilities based on performance or importance. Within an educational context, it also means “a method of inquiry to determine the extent of learning” and “the process of evaluating students.” In comparison, I will refer to Indonesian Directorate for Vocational School affairs, Direktorat Pembinaan SMK, and Department of

National

Education, Depdiknas. This is so as the two institutions deal with all vocational and educational matters in Indonesia, where this research takes place. Assessment, which is translated into Indonesian as “penilaian”, according to Direktorat Pembinaan SMK (2008:49), is serangkaian kegiatan untuk memperoleh, menganalisis, dan menafsirkan data tentang proses dan hasil belajar peserta didik yang dilakukan secara sistematis dan berkesinambungan, sehingga menjadi informasi yang bermakna dalam pengambilan keputusan which means a series of activities to obtain, analyze and interpret data on the process and achievement of student learning which is systematically and continually implemented so as to become meaningful information in decision making. Similarly, Depdiknas (2003:12) defines it as: [...] penerapan berbagai cara dan penggunaan beragam alat penilaian untuk memperoleh informasi tentang sejauh mana hasil belajar siswa atau ketercapaian kompetensi (rangkaian kemampuan) siswa. Penilaian menjawab pertanyaan tentang sebaik apa hasil atau prestasi belajar seorang siswa which is equivalent to: the application of a variety of methods and employment of various means to obtain information on to what extent students get their learning results and achieve their competencies or a series of abilities. Assessment answers how well the student achievement is. 15


By these definitions, I can tell that assessment is actually a process or activity employing various means with the purpose of obtaining information on student’s performance. This is the fundamental meaning and purpose of assessment. Hughes (2003) and Harmer (2007:379) make a difference between summative and formative assessment. Summative assessment is defined as the kind of measurement that takes place to round things off or make a one-off measurement such as the end-of-year tests or the big public exams which many students enter for. Formative assessment, on the other hand, is said to relate to the kind of feedback teachers give students as a course is progressing and which, as a result, may help them to improve their performance. Indonesian government through PP Tahun 2005 (2005 Government Regulation) No. 19 Article 63-70, pp 19-22 presents the national standard educational assessment and includes three types of assessment to be conducted by three different parties: (individual) educator/teacher, educational institution, and the government (official). Each serves different functions. That which is conducted by the individual educator aims to monitor the process, progress of and improve learning achievement. The second type of assessment is to evaluate the extent to which the graduate competency standard can be achieved. The third, which is conducted by the government aims to nationally evaluate the extent to which the graduate competency standard for particular subjects – one of which is English - can be achieved. Added to the basic purpose of monitoring school performance as such, as Maughan (2009:1) notices, there are now aims of raising standards by improving the effectiveness of individual learning and of encouraging the development of


higher order skills. With the new aims, as she explains, assessment can now be designed to impact on the actual learning that takes place in schools rather than purely to measure the outcomes of learning. Similarly, McNamara (1996) recommends that assessment needs to be integrated with the goals of the curriculum and have a constructive relationship with teaching and learning. Wiggins (1998:1) makes a similar point that the ideal of assessment is “an educative assessment system that is designed to improve, not just audit, student performance” and that “elements of educative assessment are first, authentic tasks and second, feedback, which is more than blaming and praising”. From here, we notice that the change in emphasis is going on. Assessment may be developed to produce a positive backwash on the teaching and learning and in the long run to promote learning autonomy. The National Forum on Assessment (1995) as quoted by North Central Regional Educational Laboratory (2009:1) strengthens this idea by providing the more detailed description of quality assessment, High-quality assessment must rest on strong educational foundations. These foundations include organizing schools to meet the learning needs of their students, understanding how students learn, establishing high standards for student learning, and providing equitable and adequate opportunity to learn. I fully support this and wish to emphasize the main key points “meet the learning needs”, “understanding how students learn”. This is also in line with the idea proposed by Hutchinson and Waters (1994) available in the next discussion about the starting point of language learning and teaching.

17


2.

Language testing, English testing and backwash effects As already discussed, one instrument of assessment most widely

recognized is test. It is then necessary to search for what it means. Bachman (1995:20), basing on Carroll (1968), states that “a test is a measurement instrument designed to elicit a specific sample of an individual’s behavior”. He continues to explain, “What distinguishes a test from other types of measurement is that it is designed to obtain a specific sample behavior” (1995:20 – 21). Meanwhile, language test is defined by McNamara (1996) as a procedure for gathering evidence of general or specific language abilities from performance on tasks designed to provide a basis for predictions about an individual’s use of those abilities in real world contexts. So based on what those scholars state, I can say that English test is a measurement instrument for gathering evidence of general or specific English abilities of an individual. The nature of testing, as McNamara (1996:4) further explains, has changed quite radically over the years to become less impositional, more humanistic, conceived not so much to catch people out on what they do not know, but as a more neutral assessment of what they do. More than this, above all, I wish to amplify that tests must be part of learning experiences for all involved (Coombe and Hubley, 2008:2) and testing should always be seen as part of the learning process (Riddell, 2003). Similarly, Brown (2005:252) stresses the importance of looking at tests within the context of real, living language programs. He argues that testing is not an isolated entity floating free of any language teaching reality and tests “should never be treated as though they are somehow divorced from the language teaching and learning process […],


“and should be done in as pleasant an atmosphere as is possible in the circumstance”. As one part of learning process and experiences, then – let me stop for one-moment reflection – testing, then should first support learning facilitation with a deep understanding of how people learn, and second, lead to learning autonomy. This is most compatible with the recent philosophy of learning and matches the ideal assessment criteria previously discussed. Wiggins (1998:21) confirms, “[t]esting that is deliberately designed to teach and improve, not just measure, is the essence of educative assessment.” Hughes (2003:2) adds that we cannot expect testing only to follow teaching and rather, we should demand of it that is supportive of good teaching and where necessary, exerts a corrective influence on bad teaching. I intend to show that in any testing event, the test must have and bring a certain degree of authenticity in language use. Here, then I shall lead the discussion on the position of tests and testing in English language learning, referring to what Egbert et al. (1999:3-7) write for optimal language learning environment. First, as they say, learners have opportunities to interact and negotiate meaning. Here I will argue that in English testing, students and educators are interacting and negotiating meaning. Students try to understand and do the instructions, decipher and answer the questions. Second and third, learners interact in the target language with an authentic audience and are involved in authentic tasks. Language testing is in fact students’ and teachers’ real life experience in the form of two way


communication, from which they can learn something. Bachman (1995:2) reminds us: Unlike tests of other abilities or areas of knowledge, where we frequently use language in the process of measuring something else, in language tests, we use language to measure language ability. In line with the statement, three possibilities will take place in the case of English testing: A. In an English test, we use English in communicating the instruction or rubrics and the content B. In an English test, we use languages other than English (LOTE) in communicating the instruction or rubrics and use English in the content C. In an English test, we use English and languages other than English in communicating the instruction or rubrics and the content. In the context of English education, A and C, I will argue, are basically authentic, no matter what English ability or knowledge we want to measure. This is so because both give opportunity to experience using English to communicate. In the context of English education, where learners learn to communicate, testing is by all means authentic practice where learners communicate with their educators, teachers or test makers, and test administrator whether or not it totally employs English as the medium of communication. Therefore, in my point of view, regardless of how low the quality of an English test is said, in this context I would say that all English tests have a particular degree of authenticity. The fact that there is a communication process in English testing is beyond question. Biagi (1992:149) describes communication as the act of sending ideas and attitudes from one person to another and indicates elements of


communication:

sender,

message,

channel/medium

and

receiver,

and

acknowledging scholars who describe the process of mass communication, she explains: “A sender puts a message on a channel, which is the medium that delivers the message to the receivers”. Referring to Biagi’s then, the phenomenon of communication in English testing environment can be visualized as Figure 2.1.

Communicator

Medium

Communicant

Tester

Testing

Testee

Communicant Tester

Medium Test Work

Communicator Testee

Communicator

Medium

Communicant

Tester

Test Result

Testee

Figure 2.1 Communication in testing environment, stage 1, 2, 3 (Adapted from Biagi’s Elements of Mass Communication, 1992)

In English testing, there may be sorts of communication such as interpersonal communication, when it involves two people (e.g. teacher and student). There is also a possibility what she terms mass communication to take place. This is when, as she says, communication runs from one person or group of persons through a transmitting device (a medium) to large audience (e.g. between teacher and students). Figure 2.1 illustrates communication process taking place in English testing.


I would like to strengthen my argument that English testing in English education is by all means authentic in itself, by adopting the ethnographic framework of communication proposed by Hymes (1974) as quoted in Wardhaugh (1992:245-246). The context of interaction in English testing is characterized by the following variables: Setting and Scene

: taking a test/ an examination, formal situation, at school

Participant

: teacher/test maker and student/test-taker

End

: evaluation, sense of accomplishment

Act of Sequence/Topic

: language form, meaning and use

Key

: serious

Instrumentalities/Modes

: written and spoken

Norm of interaction and interpretation: rule-bounded Genre

: various

Fourth, learners are exposed to and encouraged to produce varied and creative language. In English testing, students are exposed to varied and creative language, in written as well as spoken forms. In a particular test, students are also encouraged to produce varied and creative language. Fifth, learners have enough time and feedback. In English testing, students are given time allocation to do the test, and later they will get the feedback in the form of scores as well as other information. 22


Sixth, learners are guided to attend mindfully to the learning process. This is still questionable whether in any English testing, students are guided to attend mindfully to the learning process so that learning becomes and remains sustainable. Seventh, learners work in an atmosphere with an ideal stress/anxiety level. English testing never takes place in isolation. It takes place in the context of teaching, and like another test, it is an influential factor, since it brings psychological situation, such as anxiety, distress, enjoyment, feeling of success and failure. Tests, for example, are identified by Bailey in Ellis (2004:480) as one of the sources which may cause anxiety. Ellis (2004:522) believes that anxiety arising out of tests and a fear of negative evaluation is likely to have a debilitating effect on L2 learning, nevertheless, he believes that it can also have a facilitating effect. Further, he says “[h]ow anxiety affects learning will depend on its strength and the situational context.” Eighth, learner autonomy is supported.

It is still a question to answer

whether every English testing supports learner autonomy. It is one major task of this research to find out through the narrative stories of the students as one main stakesholders how the actual use of English test supports learner autonomy. In the previous discussion, however, we come to a conclusion that English testing should. Up to here, we see then that English testing has opportunities to bring conditions for optimal language learning. If we are more concerned with learning, therefore we should make good use of the opportunities and be aware of its backwash effects since the use and the design of tests is said to bring certain implication to learning, whether positive (beneficial) or negative (harmful). The term “backwash” or “washback” refers to “the effect of testing on


teaching and learning” (Hughes & Ur, 2003:1), “the impact of tests on the teaching programme […]” (McNamara, 1996:23), “the effect that testing practices have on teaching practices” (Wharton, 2005:4).

The existence of test backwash is

acknowledged by researchers and writers. McNamara (1996), speaking about test backwash, concludes that the use of multiple choice standardized test for achievement purpose has a negative effect on classrooms as teachers teach to the test and there is also often a mismatch between the test and the curriculum. Standardized tests, he goes on to say, are seen too often having a negative, restricting influence on progressive teaching. Caine (2005) investigated the backwash effects of the existing English tests and the proposed new speaking test on teaching and learning in Japan. He noticed the harmful backwash effects of the existing test under study since the examinationdriven “hidden syllabus” prevents the teachers from implementing, in practice, communicative methodology. And, confirming earlier findings by other researchers, he observed the backwash effects fall only at the level of content (what is actually taught) instead of the level of methodology (how teachers teach). Wharton (2005:4), referring to Prodomou 1995, furthermore states backwash is determined not merely by test writers but also teachers and learners: An important part of the description of a test, then, is the description of any backwash effect that it hopes to have or is believed to have. That is not to say that backwash is entirely determined by test writers. It also depends on the ways in which teachers and learners interpret test requirements, and work with them in the classroom.(my emphasis) What I have to underline from this statement is that backwash also depends on the ways in which teachers and learners interpret test requirements and work with them in the classroom. I can say it in another way that students’ and teachers’ attitude


toward the test and their experience in it plays a vital role in deciding whether a test will bring either positive or negative impacts. In relation to this, Taylor (2000:2) in Caine (2005:17) points out, Some of the stakeholders listed above (e.g. examiners and materials writers) are likely to have more interest in the ‘front end’ of a test, i.e. the test assessment criteria or test format. Others may see their stake as being primarily concerned with the test score. Some stakeholders, such as learners and teachers, will naturally have an interest in all aspects of the test. (my emphasis) In an educational language program, the fundamental use of testing, is said to provide information for making decisions, that is, for evaluation (Bachman, 1995:54, 78). In Bachman’s term, tests with the fundamental use means they are used for “purely descriptive purposes”, that is only when the results of tests are used as a basis for making a decision that evaluation is involved (1995:22). In addition to the fundamental use, Bachman (1995:21) states that, in line with Wiggins’ (1998) view, tests are often used for pedagogical purposes, either as a means of motivating students to study or as a means of review material taught, in which case no evaluative decision is made on the basis of the test result. For pedagogical purpose, it is clearly the positive which is expected to come from testing.

The use of test

Figure 2.2 Bachman’s categorization of the use of tests According to Brown (2005:1), the first and most basic distinction in language testing involves two very different functions. In his dichotomy, he uses a specific term “family” to refer to each different group of tests. The first helps administrators


and teachers to make program level decisions. This includes placement, graduation, and proficiency. The second helps teachers make classroom decisions. It includes achievement and diagnostic tests. In relation to the test purpose, Brown (2005:8-12) and Wharton (2008:2-3) also reports that most books on language testing distinguish four main categories of test: placement, diagnostic, achievement and proficiency tests. This categorization, however, as you observe, functions to serve what Bachman’s (1995) terms as descriptive purpose only. So far, there has been no discussion about pedagogical purpose. In relation to what they mean, Wharton (2008:2-3) defines the four types of test as follows. Placement test is that which is designed to place students in an appropriate course or class for their language level. The second type, diagnostic test, as she continues to explain, is usually used within a class to identify areas where a student is having particular difficulty. The third type, achievement test is that which is related to a syllabus or set of objectives. The last type, which is not related to a particular syllabus, is what we mean by proficiency test. Such a test is intended to assess language ability independently of the courses of study that individual students may have followed. Further, Brown (2005:1-2) also acknowledges the popular classification of language tests into two categories, according to which Bachman (1995) terms as “frame of reference”. The first category is norm-referenced test which is designed to measure global language abilities, such as overall English language proficiency, academic listening ability and reading comprehension. Next, there is criterionreferenced test, which is usually produced to measure well-defined and fairly


specific instructional objectives.

Types of test

Figure 2.3 Brown’s language test categorization In addition to the categorizations, he also differentiates between discrete point test, which measures the small bits and pieces of a language as in a multiple choice test and integrative test, which is typically designed to use several skills at one time. Meanwhile, McNamara (2004:5-6) puts types of language test into the categorizations as illustrated in Figure 2.4.

paper and pencil method performance Types of test achievement purpose

proficiency

Figure 2.4 McNamara’s language test type categorization Paper and pencil tests include those which test reading ability and language components. Performance tests most commonly measure speaking and writing abilities. Achievement tests relate to the past teaching, while proficiency tests look to the future situation. Next, I think there will be a good idea if we turn our head for a moment to ESP. ESP, as Hutchinson and Waters (1994:146-149) say, recognizes three basic


types of assessment. There are tests which are used to place learners in the ESP course most suited to their needs, and normally com at the beginning of the course. Such tests are namely placement tests. There are also achievement tests, which test how well the learner is keeping up with the syllabus and administered any time through the course. The others are proficiency tests, which assess whether or not the learner can cope with the demands of a particular situation; for example, study at a university or reading technical manuals. Yet, although it is impossible to create a single test that can fulfill the functions of all types of language test (Brown, 2005:1, 12), he goes on to remind us that “in many educational and social contexts, a given test may be fulfilling a number of purposes”. With this reason, I would say that one single test or testing may serve a descriptive as well as pedagogical purpose. A good language test, according to Coughlin (2006:1), should have a positive effect on learning, and teaching should result in improved learning habits. In conclusion then, as Figure 2.5 shows, English test, as a major means of assessment, has two purposes: the descriptive (fundamental use) and pedagogical purpose. The functions fall into two categories, help make program level decision and classroom decision. Since one type of test can fulfill multifunction, I suppose, that pedagogical purpose should be the “living spirit” or “driving force” which should penetrate any English testing in an educational context.

28 proficiency


program-level decision placement/ promotion/graduation descriptive achievement Purpose

classroom decision pedagogical

diagnostic objective

paper and pencil Method TEST

essay performance norm-referenced

Frame of reference criterion-referenced

discrete point Content integrative

Figure 2.5 English test type categorization (developed from various sources) The most central issues of testing, in addition to its practicality or efficiency, are its validity, and reliability. These notions have been widely known for more than one decade. Bachman (1995:160-161) terms validity as something which is concerned with ’How much of an individual’s test performance is due to the language abilities we want to measure?’, and explains reliability is concerned with ‘How much of an idividual’s test performance is due to measurement error, or to factors other than the language ability we want to measure? 3. Learning and learning autonomy In various places throughout this study, I have indicated my concerns about the humanistic orientation of learning. This is ideal for this research as it is highly relevant to my research philosophical assumption about knowledge claims which are constructivist and advocacy/participatory. Nevertheless, for comparison, I will


also examine consecutively other major orientations to learning which have been long recognized such as the behaviorist, the mentalism, the cognitive, the affective factor, the social/situational orientation. In the humanistic orientation, the basic concern is for human growth with the primary goal self-actualization, which means learning contributes to psychological health (Sahakian 1984 in Merriam and Caffarella 1991:133, as quoted by Smith, 1996:2). The humanistic curriculum, of which deepest goal or purpose is to develop the whole person within a human society, as Dubin and Olshtain (1994:75) note, fosters sharing of control, negotiation and joint responsibility by co-participants; it stresses thinking, feelings and action; it attempts to relate subject matter to learners’ basic needs and lives; and it advances the self as a legitimate object of learning. They go on to say that this orientation puts high value on people accepting responsibility for their own learning, making decisions for themselves, choosing and initiating activities, expressing feelings and opinions about needs, abilities and preferences. The humanistic movement in education has led its way to the emergence of the new concept of learner-centeredness. Learner-centeredness, a teaching approach that is centered on students, is a “philosophy which guides teachers’ practices on how to teach, how to interact with learners, and how to design courses, learning activities and assessment strategies, as well as how to define learning goals” (McCown, Driscoll and Roop, 1996, as quoted in Tepsuriwong and Srisunakrua, 2009:2,5). Pulist (2009:1), further explains that learner-centered learning is “an active and dynamic process through which learners develop deep understanding taking responsibilities of their own learning”. This paradigm, as he highlights,


enables learners and teacher to recognize what knowledge learner brings to the learning process and keeps to empower learners within the learning process preparing them to be lifelong learners. Furthermore he adds that learning can be defined as individual “discovery”, and quoting Muller (1998), it empowers the learner to take control of their learning as they take control of their destiny. Next, we will observe the behaviorism learning theory. In this orientation, learning is considered as a habit formation as advocates of this view state that “learning is a mechanical process of habit formation and proceeds by means of the frequent reinforcement of a stimulus-response sequence” (Hutchinson and Waters, 1994:40). Advocates of this theory and orientation, (Hartley 1998 in Smith 2005:1) believe in the importance of activity, repetition, generalization, discrimination, positive reinforcement, clear behavioral objectives in learning. In contrast with the behaviorist, the mentalist believes in thinking as rulegoverned activity and views that “Learning consists of acquiring rules, instead of forming habits”. The cognitive code, sharing some similarities with the mentalist considers learners as thinking beings and views that learning is “a process in which the learner actively tries to make sense of data”. Learning is said to take place “when the learner manages to impose some sort of meaningful interpretation or pattern on the data, which means that we actually learn by thinking about and trying to make sense of what we see, feel, and hear” (Hutchinson and Waters, 1994:43). The key principles of this view, as Hartley (1998) in Smith (1999:1-2) identifies, are the need for well-organized and clearly structured instruction, the importance of prior knowledge, the perceptual features of the task, individual differences and cognitive feedback.


The affective factor, as Hutchinson and Waters (1994:47) note, considers learners as emotional beings and believes that learning is “an emotional experience, and the feelings that the learning process evokes will have a crucial bearing on the success or failure of the learning”. Meanwhile, in social /situational learning theory, it is not so much that learners acquire structures or models to understand the world, but they participate in frameworks that have structure. Learning involves participation in a community of practice (Hartley, 1998). Up to this point, the orientations to learning have brought me to a conclusion that the behaviorism focuses on the psycho-motoric activity, the mentalist and cognitive code focus on mental activity, the affective factor focuses on the emotional activity while the social/situational focuses on the social activity. It seems to us that the theories contradict with each others; however, as we refer to Delors et al.’s (1998), in the current context of learning, we may say that all the theories and orientations are considered as complementary. The behaviorism focuses on learning to do, the mentalism and cognitivism focus on learning to know, the affectivism focuses on learning to be and the social situational learning theorist supports learning to live together. The interplay of the theories and orientations, being adjusted with the students’ learning needs in a balanced portion, will be highly relevant to the current context of education. From the above illustration, I feel sure; the humanistic orientation excels any other orientation, in that it caters all kinds of activity. Psychomotoric, mental, emotional and social domains have equal room for development. It is obvious that recently concerns about learning have largely prioritized the development of autonomy. There have been several terms used to address the


similar thing: learning autonomy, learner autonomy, learner control and selfdirection. Little (1991:4) preferring to use learning autonomy points out, Essentially, autonomy is a capacity for detachment, critical reflection, decision making and independent action. It presupposes, but also entails, that the learner will develop a particular kind of psychological relation to the process and content of his learning. The capacity for autonomy will be displayed in the way he or she transfers what has been learned to wider contexts. Meanwhile, according to Benson (2001:1), autonomy in learning is described as “[it] is about people taking more control over their learning in classrooms and outside them”. In addition, Benson (2001:1)) also describes it as “a capacity to take charge of, or take responsibility for, or control over your own learning, which means autonomy involves abilities and attitudes that people possess, and can develop to various degrees.”. Healey (2007:1) describes autonomy by referring to learner’s attitudes as self-direction and the instructional framework (degree of learner’s independence in setting goals, path to the goal, pace of learning, and measurement of success). Shrader (2003:1) uses the term learner autonomy which refers to self-directed learning or a shift of responsibility for learning from teacher to students. There is a question whether learner autonomy is teachable. In response to this, Conttia (2007:3), Broady and Kenning, (1996:9) and Benson (2003:290) agree that learner autonomy cannot be taught; instead, it has the possibility to be fostered and promoted. In this case, Healey (2007:1) argues that we can create environments that enable students to become self-directed. From this point, it follows that when we consider English test as one of the existing phenomena in language learning environment, there is a great possibility to have students self-directed. It is then necessary to identify self-directed or autonomous learners.


Autonomous learners, according to Dickinson (1995), have the characteristics as having the capacity for being active and independent in the learning process, ability to identify goals, formulate their own goals and change goals to suit their own learning needs and interests, and the ability to use learning strategies and to monitor their own learning. Similarly, Nunan (2000:1) explains that autonomous learners can (1) self-determine the overall direction of their learning, (2) become actively involved in the management of the learning process, and (3) exercise freedom of choice in relation to learning resources and activities. 4. Language learning and language learning autonomy Now, let us move our discussion from the wider scope of learning and learning autonomy to the more specific, language learning and language learning autonomy. There are several views about language learning, which are in fact in line with a particular orientation of learning previously discussed. Language learning, (Krashen 1981 as quoted by Wilson, 2000:2), refers to the “conscious knowledge of second languages, knowing the rules, being aware of them, and being able to talk about them”. This definition indicates the existence of a tendency toward the mentalism for it put stresses on mental activities. Meanwhile, according to Feez and Joyce (2002:12, 24-25), language learning is “a social activity and is the outcome of a joint collaboration between the teacher and the student and between the student and other students in the group”. Similarly, Hutchinson and Waters (1994:72) views that learning is not just a mental process; it is a process of negotiation between individuals and society. This illustrates language learning as a social activity, thus it is closely related to the social/situational orientation. When we further compare


these two views, Krashen’s definition implies that in language learning, there is no need for interaction. In Feez and Joyce’s, it clearly shows that interaction is highly important. The statement of two prominent advocates of ESP, Hutchinson and Waters (1994), again, is valuable to refer to. They explain that learning theory (behaviorism, mentalism, cognitive code, the affective factor) should be selected in accordance with Sweet’s elegant principle of ‘whether or not the learning of the language will be facilitated thereby’. I agree with this statement for it not only implies the existence of learning facilitation as the main concern and an approval to the humanistic orientation of learning but also matches the fundamentals on which the concept of learning throughout life is based. Furthermore, Hutchinson and Waters (1994:39) remind us that the key to successful language learning and teaching lies not in the analysis of the nature of language but in understanding the structure and processes of the mind, therefore the starting point of all language teaching should be an understanding how people learn. Language learning, they argue, is conditioned by the way in which the mind observes, organizes and stores information. So far, nevertheless, according to them, learning factors have been much neglected in that learning factors are incorporated only after the language base has been analyzed. In addition, to consider learning factors, it is a must to look to the environment, the situation in which language is used, and see that it plays a significant role. Researchers have shown that language learning takes place within the context of communication (Hatch 1978 as quoted by Hutchinson and Waters 1994). Learning is said to be facilitated when it is considered as a positive, relaxed


and enjoyable. Learners only learn what they really need or want to learn. Riddell (2003) states that learning has to include authentic practice, and has to continue outside the classroom. Next, to talk more about the humanistic view of language teaching, which I am deeply concerned with, I would like to refer to Dubin and Olshtain’s (1994). According to Dubin and Olshtain (1994:102), it emphasizes the development of learner autonomy as the major objective of the syllabus. Therefore, they continue to say, the success of the course and level of achievement on the part of the students is not only measured by how much the students know at the end of the course or by what they can do with language, but rather by how independent they have become as language learners. Nunan (2000:3) underscores that second language learning will proceed more effectively if learners are allowed to develop and exercise their autonomy. The statements of the scholars are obviously still relevant nowadays, since it shows a great concern about the learner’s learning independence or autonomy. In relation to autonomy in language learning, Nunan (2000:4), referring to Little (1997:99), reminds us, [It] is not merely a matter of control over learning activities and resources. It is also a matter of a particular orientation towards language learning in which, “for the truly autonomous learner, each occasion of language use is an occasion of language learning and vice versa Harmer (2007:394) shares the same idea that the promotion of learning autonomy should be encouraged in language learning. One major reason for this is that, as he argues, “to compensate for the limits of classroom time and to boost the chances for successful language learning and acquisition, students need to be encouraged to develop their own learning strategies so that as far as possible, they


become autonomous learners.” Furthermore, talking about autonomy in language learning, Benson (2001:1) describes that it is about people taking more control over the purposes for which they learn languages and the ways in which they learn them. Thanasoulas (2008:4), citing Benson and Voller’s (1997:53) put it this way: Within this approach, learning concerns issues of power and ideology and is seen as a process of interaction with social context which can bring about social change. […] Certainly, learner autonomy assumes a more social and political character within critical theory. As learners become aware of the social context in which their learning is embedded and the constraints the latter implies, they gradually become independent, dispel myths, disabuse themselves of preconceived ideas, and can be thought of as ‘authors of their own worlds’. In my conclusion, then, the starting point of language learning and teaching is understanding how people learn. This is the key to success. The ‘end’ point is that how independent students have become as language learners. This is the measurement of success. 5. Vocational School Vocational school is a school which delivers vocational education. The American Heritage Dictionary of the English language (2008) defines it as “a school, especially one on a secondary level, that offers instruction and practical introductory experience in skilled trades such as mechanics, carpentry, plumbing, and construction”. The Indonesian vocational school, SMK, (PP 74 th 2008:4) is defined as salah satu bentuk satuan pendidikan formal yang menyelenggarakan pendidikan kejuruan pada jenjang Pendidikan Menengah sebagai lanjutan dari SMP, MTs atau bentuk lain yang sederajat atau lanjutan dari hasil belajar yang diakui sama atau setara SMP atau MTs which means:


one type of formal education institution which deliver vocational training in the secondary level of where further education is pursued after junior high school, Islamic middle school or another type with the same level, or as a continuation from any learning program certified as junior high school or Islamic middle school. Being in a secondary education level, the vocational school is faced with challenges it should respond to immediately. Delors et al. (1998:24) write: “[…] secondary schools cut rather a sorry figure in educational thinking. They are the target of considerable criticism and they provoke a considerable amount of frustration. […] There is also the distress felt by school-leavers who face a shortage of opportunities […]. Mass unemployment in many countries only adds to the malaise.” The educational purposes of the vocational school are to promote intelligence, knowledge, personality, noble behavior and skills to live an autonomous life and pursue further education relevant to their respective vocational specialization (Depdiknas, 2006). This is in line with what Delors et al. (1998:18) state, realizing that education must be contributive to economic and social development: The education system is all too often blamed for unemployment. […] valid responses to the problems of mismatch between supply and demand on the labour market can come from a more flexible system that allows greater curricular diversity and builds bridges between different types of education, or between working life and further training. Such flexibility would also help to reduce school failure and the tremendous wastage of human potential resulting from it it. According to Depdiknas (2006), the vocational school, aiming to ensure the availability of smart, competitive (Indonesian) human resources, cover 35 different areas of study programs one of which is Business and Management. Some of characteristics of its educational components are (1) Curriculum has undergone a process of validation in cooperation with stakeholders (business/industrial worlds) with enrichment of particular subjects and adjustment to the development of science and technology, (2) Assessment includes cognitive, psychomotoric and affective


aspects by means of written and oral tests, practice, observation and competency examinations. The vocational school is said to experience its booming period in Indonesia. It has reportedly been gaining a steady increase of popularity during the last few years. The Jakarta Post (2005) writes “[…] vocational schools, aiming to supply the professional market, find themselves favored by both students and businesses”, showing the fact that the 667,000 new enrollments in 1997 jumped to 760,000 in 2003. Further, the report says that currently technical schools, business and management schools and tourism schools represent the three most favored study subjects. And quoting the Central Statistics Agency in 2003, this paper reports that the number of vocational school graduates working as professional workers outnumbered that of regular high school graduates. 6.

Experience and attitude As cited from Wharton’s (2005) in the previous discussion, students’ and

teachers’ attitude and experience toward the test play a vital role in the description of any backwash effect. Hence, I need to highlight the keywords “attitude” and “experience”. In this section, therefore it is necessary for me to trace the essential meaning of “attitude” and “experience”. According to The American Heritage of the English Language (2000) attitude is defined as “a state of mind or a feeling, a disposition”. Meanwhile lesson4living defines an attitude as a point of view about a situation which is made up what you think, what you do, and what you feel. (www. lesson4living.com) So, in my conclusion, there are three components related with opinion, doing and feeling. Further, the American Heritage of the English Language (2000) defines


opinion as “a belief or conclusion held with confidence but not substantiated by positive knowledge or proof”; “active participation in events or activities leading to the accumulation of knowledge or skill”; and “an event or a series of events participated in or lived through”. The meaning of word do (Hornby 1987:254) is “perform, carry out (an action); busy oneself with”. Feeling is defined as “an affective state of consciousness, such as that resulting from emotions, sentiments, or desires”. It is also described as “an awareness or impression, a general impression conveyed by a person, place or thing”. Experience, according to Hornby (2000), is the things that happened to you that influence the way you think and behave. The American Heritage of the English Language (2000) further defines it as “the apprehension of an object, thought, or emotion through the senses of mind”. Here we see the definition the dictionary provides suggests that experience is closely related with the attitude. In other words, one who experiences a thing or the one who has an experience about something will have all the components of their attitude comprising apprehension of the object, thought and emotion through the senses of their mind. Hastings (2004:135) quoting Nias (1989) confirms that an examination of experiences would be incomplete if it did not incorporate discussion about emotions, both positive and negative. Lived experience refers to that which is meaningful, influential and has prominent existence. 7.

Qualitative research and methodologies There has been an escalating trend in the academic world in the employment

of qualitative studies. Hull (1997:1) observes that educational research has undergone a slow sea change as qualitative studies have gradually come into their


own. Holliday (2002:7) confirms this by showing the fact that qualitative research is increasing in use in a wide range of academic and professional areas. Qualitative research is defined as “a loosely defined category of research designs or models, all of which elicit … visual, tactile, olfactory, and gustatory data in the form of descriptive narratives […]” (Preissle, 1993, as quoted by Ratcliff, 2006:3). It aims to understand human experience, to reveal both the processes by which people construct meaning about their worlds and to report what those meanings are (Hull, 1997:3); to help us to understand the world in which we live and why things are the way they are (Hancock, 1998:2). It is concerned with the social aspects of our world and developing explanations of social phenomena and is concerned with the opinions, experiences and feelings of individuals and finding the answers to questions which begin with why, how, in what way, instead of how much, how many, how often, to what extent (Hancock, 1998:2). Holliday (2002:7) further adds that qualitative research represents a broad view that to understand human affairs it is insufficient to rely on quantitative survey and statistics, and necessary instead to delve deep into the subjective qualities that govern behavior. Hull (1997:3) furthermore explains: Qualitative data are obviously not amenable to quantification and thus won’t satisfy those who want findings based on numerical data and reports of statistical significance. Nor are findings from qualitative research generalizable from one setting to another without comparable research elsewhere. There are eight general characteristics of qualitative research which are recommended by Creswell (2003:181-183), quoting Rossman and Rallis (1998). I will cite and present them here in revised order with additional reference from other scholars.


First, qualitative research occurs in the natural setting. It means that in contrast to the use of conventional laboratory and artificial setting, it is the real-life situation which is encouraged to utilize. Hull (1997:1) further explains that it is its “naturalism”, its intimacy with real people in real situation and its concern for understanding human beings as they act in the course of their daily lives which constitutes the great strength of qualitative research. Second, it views social phenomena holistically. It means that qualitative studies appear as broad, panoramic views rather than micro-analyses. Ary et al. (2002:25) makes clear that qualitative researchers seek to understand a phenomenon by focusing on the total picture rather than breaking it down into variables. Third, qualitative research is emergent rather than tightly prefigured. It means it allows several aspects to emerge. Another scholar, Hull (1997:1) points out that although researchers formulate research questions to guide their enquiry, they expect their questions to change or sharpen as the study progresses. FHI (2008:3) explains that in qualitative research, some aspects of the study are flexible such as the addition, exclusion, or wording of particular interview questions. Fourth, it adopts and uses one or more strategies of inquiry as a guide for the procedures in the qualitative study. In relation to this, I wish to refer to what Trochim (2006:1) writes. In conducting qualitative research, as he explains, there is “approach” (a term he uses to mean Creswell’s methodology or strategy of enquiry): “a general way of thinking which describes either explicitly or implicitly the purpose of the research, the role of the researcher(s), the stages of research, and method of data analysis”. While Trochim (2006) presents four major approaches: ethnography, phenomenology, field research and grounded theory, Creswell


(2003:183)

recommends that researchers

choose among five possibilities,

including narrative, phenomenology, ethnography, case study and grounded theory. Fifth, it employs multiple methods (techniques) of data collection that are interactive and humanistic. It means that the methods of data collection are growing and they increasingly involve active participation by participants and sensitivity to the participants in the study. In qualitative research, as Holliday (2002:21) says, one does not begin by choosing a method. Methods, as he furthermore says, can be sufficiently flexible to grow naturally from the research question, and in turn from the nature of the social setting in which the research is carried out. Sixth, it is fundamentally interpretive, meaning, based on the interpretation of the researcher. This means that the researcher makes an interpretation of the data, which is subject to other interpretations. Seventh, it uses complex reasoning that is multi-faceted, iterative and simultaneous. This means that both inductive and deductive processes are at work and the thinking process is also iterative, with a cycling back and forth from data collection and analysis to problem reformulation and back. There are also simultaneous activities of collecting, analyzing, and writing up data. Eighth, it systematically reflects on who the qualitative researcher is in the inquiry and is sensitive to his or her personal biography and how it shapes the study. Next, let us observe how qualitative research is classified. According to Holliday (2002:19), qualitative research comes in two different paradigms, naturalistic and progressive, which represent different theoretical perspectives (philosophy of learning). Naturalistic, which comes from post-positivism and realism, views that reality is relatively straightforward (still quite plain to see) and


researchers must not interfere with real setting. Progressivism - coming from critical theory, constructivism, postmodernism and feminism - views that reality and science are socially constructed and researchers are part of research settings. In naturalism, the aim is to see the research setting as though the researcher were not there. Within progressivism, however, it is recognized that the presence and influence of the researcher is unavoidable and indeed a resource, which must be capitalized upon. (Holliday, 2002:145); and “[p]rogressivist research no longer accepts the need for the writer to remain an invisible presence, and allows her to state her position and argument in the first person ‘I’”(2002:23). In relation to the approach and methodology qualitative study employs, Ary et al. (2002:25-28) acknowledge eight approaches: ethnography, case studies, document analysis, naturalistic observation, focused interviews, phenomenologic studies, grounded theory and historical studies; meanwhile Creswell (2003:183) recommends five possibilities in employing the methodologies or strategies of enquiry, three of which I will cite here as there is a great possibility for me to employ the methods. First is phenomenology, second, grounded theory, and third, narrative design. Let us trace what they mean from the literature. According to Trochim (2006:1), phenomenology is a school of thought that emphasizes a focus on people’s subjective experiences and interpretations of the world: the phenomenologist wants to understand how the world appears to others. Meanwhile, Hancock (2002:4), who defines that phenomenology literally means the study of phenomena, provides further explanation: It is a way of describing something that exists as part of the world in which we live. Phenomena may be events, situations, experiences and concepts.


[...]. Phenomenological research begins with the acknowledgement that there is a gap in our understanding and that clarification or illumination will be of benefit. Phenomenological research will not necessarily provide definitive explanations but it does raise awareness and increases insight. I move my discussion to another strategy of enquiry, grounded theory by starting with what it means. Drawing from the two authors, Glaser and Strauss 1967 and Strauss and Corbin 1990, Borgatti (2007:1) makes clear that, Grounded theory refers to theory that is developed inductively from a corpus of data. The basic idea of the grounded theory approach is to read (re-read) a textual data base (such as a corpus of field notes) and “discover” or label variables (called categories, concepts and properties) and their interrelationships […]. Of course, the data do not have to be literally textual… they could be observation of behavior, such as interactions and events in a restaurant. Often they are in the form of field notes, which are like diary entries The main feature of grounded theory, as Hancock (2002:9) points out by acknowledging Glaser and Strauss, is the development of new theory through the collection and analysis of data about a phenomenon. He further states that grounded theory “goes beyond phenomenology because the explanations that emerge are genuinely new knowledge and are used to develop new theories about a phenomenon.” It goes beyond question then that grounded theory is useful to explore processes, activities and events (Creswell, 2003:183). Let me move on to the third strategy, narrative design. It is said it claims that narratives constitute a fundamental form of human understanding, through which individuals make sense of themselves and of their lives. Clandinin and Connely, 2000 in Creswell (2003:15) make clear that it is […] a form of inquiry in which the researcher studies the lives of individuals and asks one or more individuals to provide stories about their lives. This information is then retold or restoried by the researcher into a narrative chronology In the end, the narrative combines views from the participant’s life with those of the researcher’s life in a collaborative narrative.


This means that the words of the research participants are of great value in a research using this kind of strategy or approach.

B. THEORETICAL FRAMEWORK Finally, to position myself as the researcher in relationship to the research, it is urgent to present the following summary as my conceptual framework which will be central to my research and help to provide answers to the research question. In general it includes my agreement and disagreement with the recent discussions and current issues (cf. Holliday 2002:52-53). The major points emerging from the discussion presented in the theoretical review include the following. Vocational school is a secondary educational institution which prepares for middle-class professionals with high competitiveness in local as well as global context. To carry out the mission, vocational school delivers numerous subjects, including English, in the educational program. The success in implementing the English educational program at the vocational school is not merely measured by how much the students know at the end of the course or by what they can do with English, but rather by how independent they have become as English learners (cf. Dubin and Olshtain, 1994:102). This is the essence of learner-centered learning, progressivism theory, humanistic education, and process model of curriculum. In an educational language program, objectives should result from a marriage between target situation analysis and learning needs analysis. From the objectives, we move on to teaching and learning or to testing. Figure 2.6 (developed from Hutchinson and Waters’, 1994, and Brown’s, 2005), which shows where in the


language program testing is, makes it clear.

Target situation

Learning needs

analysis

analysis

Objectives

Testing

I

Independent learning

Teaching and learning

Figure 2.6 Testing in the language program toward independent learning (Developed from Hutchinson and Waters, 1994; and Brown, 2005)

English testing is practically used to measure student ability and serves as the major means in the process of evaluating students within an educational context such as vocational school. English testing must also be seen as a part of learning process at the vocational school (cf. Riddell, 2003). As illustrated, it is a part of the curriculum, which contributes to the success of the educational goals. In line with the critical theory and the current progressivism philosophy, where education should be


directed to learning enhancement, empowerment and autonomy, English testing should be held responsible for all of these by catering and accommodating student’s learning needs. It should include authentic practice and be done in pleasant atmosphere. Since my research concern was learner’s learning, I adopted their framework for analyzing learning needs (in this respect, in the context of testing). By learning needs I mean, as they say, what students need to do in order to learn; and more specifically in this case what they need to do in the test in order to learn. The key related points to my research questions adopting their framework for analyzing needs are therefore as follows, (1) the identity of student/teacher, covering their age, interest, socio-cultural background, learning/teaching styles they are used to; (2) the reason for giving/taking the test and what (they think) students will achieve; (3) how students learn, covering learning background, concept of teaching and learning, interesting learning, boring technique and (4) how they experience English testing and their learning, covering how they find the test, how they have been treated and their attitude toward English testing in terms of goal, process and material. Language learning takes place within the context of communication (Hatch 1978). In English testing, where we use English to measure English ability, teachers and students are actually engaged in a real-life two-way communication, therefore testing is to some extent is an authentic task and a living phenomenon in English learning. English testing involves elements of communication; thus English language learning must take place within the context of English testing. As any other real-life phenomenon, English testing brings psychological effects such as feeling of success, anxiety.


A statistically good test does not necessarily mean a humanistically good test. English testing is said to be ideal if it not only measures performance but also functions as an educative assessment as well, and is used for pedagogical purposes. It should be designed not only to audit, measure (Wiggins, 1998) but also be supportive to good teaching and exert a corrective influence on bad teaching and improve performance. The ideal testing will drive independent learning, while the conventional testing will result in bad teaching and learning. Testing is one form of assessment which has the influence on teaching and learning (backwash effects). Backwash effects can be either harmful or beneficial. English testing may bring either beneficial or harmful backwash effects to learning and teaching. First, there is a possibility that English testing does not facilitate learning. There is a possibility that English testing, its components and implementation as well as environment are not contributive to learning. Second, there is a possibility that the test stakes holders experience psychological effects ranging from enjoyment to a state of shock. It is therefore the positive effect which is expected to bring on learning and teaching (beneficial backwash). Backwash is not entirely determined by test writers; it also depends on the ways in which teachers and learners interpret test requirements, and work with them in the classroom (Prodomou, 1995, as quoted by Wharton, 2005:4). Backwash effects may be determined by teachers’ and students’ experience and attitude toward the test. Thus they have their own ideal to testing, which is based on all aspects, which may be different from other parties. That is the reason why it is necessary to explore their lived experience as a means of dialog with the most concerned stakes holders for the educational reforms.


Vocational school employs two categories of English tests: classroom tests, and, high-stakes tests. English testing at the vocational school (as illustrated in Figure 2.7) serves two basic purposes. First, it is implemented for instructional purposes. Testing of such a kind is classroom test such as speaking competency test and semester-end achievement test. Second, it is for administrative purposes. This kind of testing includes high stakes test such as semester-end, regional test, national exam and TOEIC test. Different types of test serve different functions. No single test type can fulfill all functions (Brown 2005), therefore instead of being contradictory with each other, we should see them as complementary.

Vocational school English education (English) classroom test (English) high-stakes test

Figure 2.7 Positioning English testing at the vocational school With no intention to discharge any classification or categorizations available, here in this research, I would prefer to adopt the following categorizations of test types, drawn from Brown’s categorization of test purpose and based on Harmer’s assessment dichotomy. For this categorization, I would argue that in presenting the description and explanation, I have tried to see the test from the point of students.


1. Classroom test It is an instrument in the formative assessment. A test of this type enables students to get their learning feedback on daily, weekly, or monthly basis. It may be in the form of integrative or speaking, reading, listening, writing competency test or discrete-point tests which help students to monitor their learning progress on their vocabulary and grammar mastery. 2. High-stakes test It is an instrument in the summative assessment. It does not only enable students to obtain their learning feedback on a longer period basis but also provides them with non-instructional benefits. A test of this type may be in the form of promotion, graduation or proficiency test which enables to inform not only student test-takers but a large number of people their language ability and linguistic component at high speed. In addition to test type categorization above, I would like to present a diagram of English testing aspect categorization (Figure 2.8) showing a summary we can gather from the literature review.

51 descriptive


Goal pedagogical

Material Content Level of difficulty

validity

ENGLISH TESTS

Qualities

reliability

practicality

Implementation

Backwash

beneficial

harmful

Figure 2.8 English testing aspect categorization (adapted from various sources) Since the existence of English testing is beyond question, it can surely be seen as a special phenomenon which offers a big appeal to explore. Here, in this research, in order to advocate my research participants, I would gather any evidences of anything that is close to what English testing can do for the student test-takers, instead of what English testing can do to them. English testing for learners is therefore seen as a form of socialization which takes place within a specific context where language use occurs and is an authentic language activity where they are experiencing a meaningful verbal interaction.


A research which is concerned with opinion, experiences and feelings of individual is qualitative research. Therefore exploring the phenomenon of English testing at the vocational school through the lived experiences of the students and teachers by means of qualitative research is highly feasible and eligible. Since learners and teachers will naturally have an interest in all aspects of the tests, this research will portray the phenomenon of English testing at the vocational school holistically, with the main focus on the lived experience of vocational school students as English learners. In contrast to empirical, observational experience, lived experience is affected by personal life history, interests, awareness, intention, belief, and action. More clearly for example, the way one responds to English testing and the way one put meanings on it are related to all the features. In conclusion then, this confirms and supports what Brown (2005:252) highlights: the importance of looking at tests within the context of real, living language programs. We observe, as he states, that tests can and should be integral parts of the larger curriculum in a language program. We positively agree that tests should never be treated as though they are somehow divorced from the language teaching and learning processes that are going on in the same context. Consequently, I would utilize the concept as mapped out in Table 2.1 as a guide during the implementation of my research.

53


Table 2.1 Theoretical Concept Map Construct English testing

Conceptual Definition

Sub Category/ Indicator

Category

An educational instrument which is basically used to measure student’s performance and learning outcome is seen in a different light, that is something which is authentic in itself, an authentic real-life learning experience in communicating and using, and learning a language

Functions/Purpose Test type Qualities

Content Implementation Backwash

Learning

Developing potentials for selfactualization

Learning autonomy

A capacity for detachment, critical reflection, decision making and independent action

Descriptive Pedagogical Classroom High-stakes Validity Reliability Practicality Materials Level of difficulty Atmosphere Feedback Beneficial Harmful

Self determination

Ability to identify, formulate and change their own goals

Active involvement and freedom

Capacity for being active and independent Ability to use learning strategies and to monitor their own learning

Experience

Event, activity, which has given process of gaining knowledge and how one is treated; actualization of the attitude in response to the world realities or phenomena;

Lived Experience

Meaningful, influential event; specific experience, affected by personal life history and interests A point of view about a situation

Attitude

What they think

What they feel

What they do/react

Vocational School

Qualitative progressive research

An educational institution in the middle level offering instruction and practical experience in skilled trade A study which occurs on a natural setting with a researcher as a source, focusing on an interest in meanings, perspectives and understanding human behaviors

54

before during after before during after before during after

-

-

-

-


CHAPTER III

METHODOLOGY

The previous chapter has shown and provided a number of arguments to answer the research question theoretically, and it is now the turn for this chapter to provide description of how to get to the empirical answers to the research problems by presenting relevant methods and strategies employed in my study. In general, as Holliday (2002:57) suggests, it will mainly deal with the broader questions of approach, presenting my justification of what and why a particular design, strategies/approaches, and methods are used. It covers the research design, data sources, types of data, data collection techniques, data processing procedures, trustworthiness of research results.

A. RESEARCH DESIGN This study would try to answer simple research questions “What is the students’ lived experience in English testing and learning?“ To answer such a question, the qualitative research, which is concerned with the opinions, experiences and feelings (Hancock, 1998:6) will be the right paradigm. As a qualitative research, a focus on natural settings, an interest in meanings, perspectives and understandings, an emphasis on process, and a concern with inductive analysis (Woods, 2006:2) are the main features in this study. My other reasons to utilize this qualitative approach are as follows. First, there was no aim to test a theory or explanation. Instead, it was to obtain insights by investigating how learner autonomy could be promoted through the participants’


experience in taking or conducting English tests. Second, the study is exploratory, meaning learning facilitation concept in English testing needs to be deeply understood since little research has been done on it. I would explore how the participants experience testing in vocational school. Third, my personal experience background is largely engaged in language and a literary form of writing and therefore numerical mastery is beyond my capacity. Last, but most important, my social and academic environment, my audience, is supportive to the approach. For this qualitative progressive study, as the researcher I would then utilize advocacy/participatory/emancipatory assumptions. Concerning to this choice, I would like to refer to Creswell (2003). Conceptualizing Crotty’s model, Creswell (2003:5), addresses three questions which are central to the design of research, one of which is what knowledge claims are being made by the researcher. In relation to research utilized advocacy/participatory knowledge claims such as this study, Creswell (2003:9-10) explains: Another group of researchers claims knowledge through an advocacy/participatory approach. […]. These researchers believe that inquiry needs to be intertwined with politics and a political agenda. Thus, the research should contain an action agenda for reform that may change the lives of the participants, the institutions in which individuals work or live, and the researcher’s life. Hence, the choice of this research design was due to the fact that this is the only research with a paradigm which would enable me to help build learning autonomy in the recent context of emancipation and liberation by examining an issue related to suppression and oppression of individuals. In addition, it was due to the fact that English education deals with the most complicated human behavior. Being interpretive in nature, the research is concerned with the unique context of the study itself and its primary goal is to understand holistically my


participants’ lived experience in English testing as a particular phenomenon in English teaching and learning process. With this in mind, therefore, I expect to find from their stories the description of the experience and reveal the meaning by making my personal interpretation by the end of the study. In this context and condition, this study therefore attempted to utilize two possible research approaches, which are applicable to progressive qualitative, namely phenomenology and narrative, despite the fact that there are numerous approaches available in qualitative progressive. Chenail (1995:8-9) reminds us, “Try to keep your method simple because in qualitative research, the complexity is in the data. If you get too complex in your method, the reaction between a complex method and complex data will be disastrous”. Since this study would focus on the two approaches, phenomenology and narrative, it is known as a phenomenological study with narrative design. Phenomenology is noted as one major type of qualitative research. As reviewed in the previous chapter, it is a form of qualitative research in which the researcher attempts to understand how one or more individuals experience a phenomenon (Chenail, 1995:8), and in this context, the phenomenon is English testing. Narrative design studies the lives of individuals and asks them to provide their stories (Clandinin and Connely, 2000 as quoted by Creswell, 2003:15). In the context of my study, the individuals are students and teachers. As the researcher I would ask them to tell their stories about their experience in English testing. Table 3.1 provides the complete visual description of this research design. Yet, as the interaction between participants and researcher would be dynamic and progressive, the research design was emergent and subject to change, adjustment and


modification. Table 3.1 Research specification

Research Paradigm

PROGRESSIVE QUALITATIVE

Philosophical assumption about knowledge claims

ADVOCACY/PARTICIPATORY CONSTRUCTIVISM

Theoretical Perspectives (Philosophy of learning)

CRITICAL THEORY CONSTRUCTIVISM HUMANISM

Strategies of Enquiry/Approach

PHENOMENOLOGY, NARRATIVE DESIGN

Methods Technique, procedures

INTERVIEWS, DOCUMENT REVIEWS, OBSERVATION (IMAGE DATA ANALYSIS, FIELD NOTES), RESEARCHER’S NOTES

The basis of qualitative research is that one does not predetermine or delimit the directions the investigation might take, therefore it is important to detail the specific stages that a research will follow in addressing the research questions (cf. Trochim, 2006). There are numerous variations in the way qualitative research is done (Thomas and Nelson, 1996:3). Simply as a guide, I would utilize the procedures I outline here, although later in the actualization, changes, modification could happen as the nature of progressive qualitative is emergent. To set up research questions which would guide the research process, I began with identifying research problems which needed addressing in order to develop the existing evidence base, searching the existing literature base and critical appraisal of the literature (UWE, 2007:1). This was explicitly presented in Chapter 1.


Next, to establish theoretical framework and concept map, I employed a theoretical base to examine the problem. The function is to position me as the researcher in relationship to the research, resulting from my agreement and disagreement with current discussion and issues (Holliday, 2002:52-53). As this study was concerned in English testing field, the main theoretical base logically came from (English) language learning and learning autonomy. This was presented in detail in Chapter 2.

Setting up research questions

Establishing theoretical framework and concept map

Producing a priori coding inventory

Data collecting: observation, interview, documentary analysis

Reading through and assigning a priori codes to the relevant data

Assembling the coded data, match it with the reconstructed questions

Rereading the corpus of data and put it into categories of responses referring to the theoretical concept map, employing inductive codes

Showing relationship among categories, connecting categories to find major and minor categories.

Interpretation of results and making claims with evidence Ensuring research questions are answered

Figure 3.1 From research questions to research results 59


The complete procedures from the research questions to research results are shown in the Figure 3.1 and the factual report on the actualization will be presented in the next chapters.

B. DATA SOURCES Data sources would be of three kinds: the participants, setting and the environs, and myself as a researcher. To have a more clearer description about the sources, here, then I would like to establish boundaries of my study. The first data sources to mention are the participants, and more specifically students. As we are aware of, students are the primary stakes holder of education whose rights need protection. To hear their voices and to have emphatic understanding on them means to include them in the education reforms. The participants, as the research title suggests, would be vocational school students. In addition to student participants, I would also include teachers, in this respect, English language teachers at the same vocational school. The data elicited from the teacher participants would be used as support data. To optimize data quality, purposive or non-probability sampling, which decreases the generalizability of findings, was utilized. Sampling is known as the method for selecting people, events or objects for study in research (UWE, 2006:2). In qualitative research, the selection of participants is, as writers say, purposive, which in essence means that a sample is selected from which one can learn the most. Qualitative research studies do not attempt to make inferences from their participants to some larger population; rather, they are selected because they have certain characteristics. There are pragmatic reasons concerning


the location and the availability of participants (cf. Thomas and Nelson, 1996). With regard to this, Trochim (2006:1) states, a non probability sampling, which is in contrast to probability sampling, does not involve random selection. He goes on to say that it does not necessarily mean that non-probability samples are not representative of the population; but it does mean that non-probability samples cannot depend upon the rationale of probability theory. Therefore this study will not be generalizable to all areas, which means, as Nunan (1994) says, the insights and outcomes generated cannot be applied to contexts or situations beyond those in which the data were collected, and the results will be subject to other interpretations. With these considerations, I was determined to access data from three students and three teachers at the commencement of the research project. This study highlights the importance of research ethics since this was conducted on people. FHI (2000:8) reminds us, […] the well-being of research participants must be our top priority. The research question is always of secondary importance. This means that if a choice must be made between doing harm to a participant and doing harm to research, it is the research that is sacrificed. My research is therefore concerned with the participants’ lives, safety, and human rights. In relation to these, no efforts would be done to control their behaviors and the names would be under anonymity protection. The second data sources to mention are the setting and environs. As literature says, to conduct a qualitative study, the setting should be accessible: the researcher must have access to the field setting and be able to observe and interview the participants at the appropriate time and location (Thomas and Nelson, 1996:5). The third source is me as the researcher. Progressive qualitative research


acknowledges that the researcher is not only considered as the instrument for collecting and analyzing the data, but also a resource as well (cf. Holliday, 2002:145). With this reason, then, for my progressive qualitative study, I would consider myself as one of the reseach instruments and resources.

C. TYPES OF DATA All description of what I would see and hear, what participants said and wrote, what participants used and produced (c.f. Holliday, 2002:96) would be collected, resulting in verbal (narrative) and image (visual) data which would be stored in field notes, audio recording, transcript and diary. In addition, data would also be in texts. Types of data

Text

Narrative

Primary Secondary

Student’s official letter, work, public news school report

Visual

Primary Secondary

Student’s accounts

Teacher’s accounts

Primary

Secondary

Student’s Description of behavior events, setting teacher’s behavior

Figure 3.2 Types of data (adapting UWE, 2007:1) Data falls into two categories, primary and secondary. The term primary and secondary here is slightly different from what Chenail (1995:3) proposes. In Chenail’s definition, primary data is original data collected for the new research study; whereas secondary data is that which originally used for a different purpose. Nevertheless, as you see here, I also considered original data collected


for my study as secondary such as teacher’s accounts. I also treated that which originally collected for a different purpose as primary. Therefore primary data for my study includes interview, transcripts of conversations, and observation of behavior, events, activities, appearance, facial expressions, account, talk of the research participants. Secondary data includes radio/TV news, speech and written documents such as official letters, emails, websites, newspapers, magazines, memos, annual reports, school report, test booklets, test answer sheet, ledger, which will be seen as documentary artifacts of culture, as well as the information about it. As Holliday (20002:96) states, decisions about what sort of data to collect depend on what is encountered in the research setting and which data is important emerges during the research process. This denotes flexibility and progressivism in the study design. Three kinds of data collection employed in this research were expected to produce three kinds of data: quotations, descriptions, excerpts of documents; and resulted in narrative, description, charts and diagrams (cf. Patton, 1990 as quoted by Ratcliff, 2006:6).

D. DATA COLLECTION TECHNIQUES Qualitative approaches to data collection, according to Hancock (1998:13), usually involve direct interaction with individuals on a one-to-one basis or in a group setting. As a beneficial result, he goes on to explain that the use of the qualitative methods (which here I prefer to replace with techniques to avoid confusion) such as interview and observation may bring richness of data and deeper insight into the phenomena under study.


The appropriate, applicable techniques as data gathering instruments and data presentation for my study would therefore comprise: 1. In-depth unstructured and semi-structured interviews In-depth interviews are said to be optimal for collecting data on individuals’ personal histories, perspectives and experiences, particularly when sensitive topics are being explored (FHI, 2000:2). The open ended questions are not necessarily worded in exactly the same way with each participants; instead, participants are free to respond in their own words and these responses tend to be more complex than simply “yes” or “no” (FHI, 2000:4). It was to probe the ideas and elicit data on the participants’ experience, practice, views, opinions, beliefs and feelings on goals, process, materials of testing. This individual interview would be imposed on both teachers and students face to face to assure the data originality. The questions were around English testing and personal background. Due to the nature of progressive qualitative the interview questions were pre-planned, yet subject to change, which means the inquiry is expansionist. All would be conducted in Indonesian, carefully recorded, and transcribed. By the use of the native Indonesian I meant to build a special rapport and be more effective at eliciting the participants’ perspectives so that the information gathered would be truthful. 2. Direct observation Participant observation is appropriate for collecting data on naturally occurring behaviors in their usual contexts (FHI, 2000:2). As the data from the interview may be of limited value or is difficult to validate, observation


can serve as a technique for verifying or nullifying information provided in face to face encounters (Hancock 1998:16). In doing the observation, I would prefer to consider myself as a complete participant, which means becoming a full member of the group and not informing the participant that I was studying them (cf. Chenail, 1995). To do so, I could utilize the moments when I was with them. For my study, observation would function as follows. First, it would help select research participants, compose probe questions and to obtain a full description of the setting, the participants and their behaviors, assumptions and beliefs relationships and accounts of events. Second, it was to provide observer impressions and reflection about the events and commentary. The techniques which would enable me to record the data during the observation would consist of written descriptions, photographs, artifact, and documentation. Therefore field notes, drawings and image data would be produced from the observation activities. The observation would be conducted in the implementation of classroom tests comprising the speaking competency tests (achievement tests) and high stakes tests (national school-leaving test, regional and international proficiency test). In addition to all these, observation would also be conducted in various official meetings, which would serve as support data. 3. Documentary review Copies of school-report, test paper and student’s work/answer sheet, score-reports were considered as relevant materials which would increase the trustworthiness of this study. They would be used as artifacts.


E. DATA PROCESSING PROCEDURE In qualitative research, it is difficult to cleanly separate out data collection or generation from data analysis because there is movement back and forth between generation and analysis (UWE, 2007:1). With this reason, I was determined to utilize the term data processing procedure here. By data processing procedure, I mean a series of activities which consists of two main types: collecting and analyzing. It covers the steps all the way from acquiring, inventorying, treating and dealing with the data until it produces outcomes (research results). In line with the research paradigm I chose, the possible data processing is that which is dialectic, not linear; comprising collecting, analyzing, re-collecting and re-analyzing. As Thomas and Nelson (1996:9) say, simultaneous collection and analysis of data, as shown in Figure 3.3. is an important feature of qualitative research since it would enable me as the researchers to focus better on certain questions and in turn to direct the data collection more effectively.

Collecting

Analyzing

Figure 3.3 Data processing procedure 1. Data Collecting To help me in the data collection phase, the strategies I would utilize as I presented previously comprise observation, interview, my diary, and document review. This was to ensure the adequacy and accuracy of the data. 66


2.

Data Analyzing Data analysis is the process of making sense out of one’s data

(Thomas and Nelson, 1996:10); a process whereby researchers systematically search and arrange the data in order to increase their understanding of the data and to enable them to present what they learned to others (Ary et al. 2002:465). It aims to search for patterns, themes and holistic features (Chenail, 1995). According to Ary et al. (2002), it involves two main activities. They suggest reducing and organizing, and synthesizing and summarizing. In analyzing the data, I decided to adopt the approaches proposed by Hancock (2002) and Ary et al. (2002). Analysis of all data from interview recording, transcript, observation, documents, and artifacts would be conducted throughout the data collecting process inductively. For the purposes of classification, summarization and tabulation of the verbal data, content analysis would be employed. By content analysis I mean, refering to Hancock (2002:21), is a procedure for the categorization of verbal or behavioral data, which involves continually revising the data and reviewing the categorization of data until the categories used to summarize and describe the findings are a truthful and accurate reflection of the data. This comprises two levels, descriptive and interpretative. The analysis outcome should emerge from the data.

67


F. TRUSTWORTHINESS OF RESEARCH RESULTS From here, I will move on to the accountability of qualitative research. As Onwuegbuzie and Daniel (2003:6) explain, in order to be taken seriously qualitative researchers must be accountable fully at all phases of their research study, including the data collection, analysis and interpretation stages. They state, “[s]uch accountability can only come to the fore by providing evidence of representation and legitimation. Thus, rigor in research is needed [...]”. In addition, Holliday (2002:ix) explains that an important part of the accountability of qualitative research is setting what one does against what has been done before. Further, Onwuegbuzie and Daniel (2003:6) indicate one of the most common errors in qualitative research studies. Such is failure to provide evidence for judging the credibility (i.e. validity) and dependability (i.e. reliability) of findings. Nevertheless, they acknowledge that the issue about the importance of validity has been the subject of disagreement among qualitative researchers. They point out there are those who contend that validity for qualitative research should be interpreted in the same manner as for quantitative research and those post modernists who question the appropriateness of validity in qualitative research, asserting that the goal of providing evidences of validity is utopian. At this point, they propose redefining the concept of validity: “[…] it is important that qualitative researchers assess the legitimacy of their interpretations. This can be undertaken by re-defining the concept of validity in qualitative research.”. In relation to this, Creswell and Miller (2000) as quoted in Cresswell (2003:195) strongly point out:


[…] reliability and generalizability play a minor role in qualitative inquiry. Validity, on the other hand, is seen as strength of qualitative research, but it is used to suggest determining whether the findings are accurate from the standpoint of the researcher, the participant, or the readers of an account. Nunan (1994:14) explains that validity has to do with the extent to which a piece of research actually investigates what the researcher purports to investigate. According to him, validity comes in two types, internal validity and external validity. Internal validity refers to the “interpretability of research” (Nunan, 1994:15) or the extent to which researchers can demonstrate that they have evidence for the statements and descriptions they make (Grix, 2004). External validity refers to “the extent to which the results can be generalised from samples to population” (Nunan, 1994:15); “a study’s generalisability, that is the relevance of the study statements over and above the case study used” (Grix, 2004). Since the findings of qualitative research would not be generalized from samples to population, this research does not need to ensure what Nunan (1994) terms as external validity. Meanwhile, although many interpretivists reportedly refrain from using the term ”reliability” when pertaining to qualitative data, Onwuegbuzie and Daniel (2003:7) affirm that reliability as a concept in qualitative data analysis is increasingly gaining acceptance. Reliability, according to Nunan (1994:14), refers to the consistency and repricability of research, which includes (1) internal reliability, which refers to the consistency of data collection, analysis, and interpretation and (2) external reliability, which refers to the extent to which independent researchers can reproduce a study and obtain results similar to those obtained in the original study.


To assess the truth value of findings and interpretations in qualitative research, as Onwuegbuzie and Daniel (2003:7) show, a myriad of methods have emerged in the literature such as triangulation, prolonged engagement, persistent observation,

member

checking,

clarifying

researcher

bias,

making

contrasts/comparison and thick description. According to them, utilizing and documenting such techniques should help to reduce methodological errors in qualitative research. In order to legitimize my research results and interpretation, then it was necessary for me to establish strategies for quality assurance of my study. In the report presentation of qualitative research, transparency or openness has been one of the main issues which need addressing. As one of the major instruments and resources in qualitative research, it is necessary for the researcher present herself and her biases, which certainly greatly influenced her interpretation of the findings. Ratcliff (2006:1) reminds us, “[t]hose reading report need to know about the instrument, so you need to describe relevant aspect of self: your biases and assumptions, expectations, relevant history.” Similarly, Chenail (1995:2), referring to Atkinson, Heath and Chenail (1991) points out: […] the process of establishing the trustworthiness of any study comes down to the quality of the relationship built between the researcher and the community of readers and critics who examine the study. […] As with any important relationship, […] openness is an extremely important factor in the value and quality of that interaction. (my emphasis) Taking the above into my considerations, I would report about myself on the next chapter. Validation of this study would therefore be done throughout the steps in the process of research, to improve the chances of getting better, more reliable


data; to minimize the chances of biased findings; to check for their accuracy and validity (trustworthiness, authenticity and credibility) (cf. UWE, 2006:1). However, it is important to note that as quoted by Morrissey and Higgs (2006:168) from Higgs (2001) the customary evaluation criteria of validity, reliability and generalizability and objectivity in quantitative research are not applicable to phenomenological research. First, it was necessary to know the criteria for good data. According to Holliday (2002:97), essential criteria for good, meaningful data are (1) data is interconnected in systems or pathways, (2) the degree to which it provides the potential for thick description by revealing different, deeper aspects of the phenomenon being studied, (3) what the researcher does with the data, the sense that she makes of it and how well she communicates this sense, (4) that the researcher sees the familiar as strange and does not see things as taken for granted. In qualitative research, data must also be auditable through checking that the interpretations are credible, transferable, dependable and confirmable (UWE, 2006:1). I was sure the data I would obtain would meet the criteria. Triangulation, an activity in which I had to observe the phenomenon from various angles and use various sources and instrument, is one of suitable instruments to serve this purpose. Onwuegbuzie and Daniel (2003:7), quoting Eisenhart and Howe 1992; Lincoln and Guba 1985, call for attention, “In qualitative research, information gleaned from observations, interviews and the like must be ‘trustworthy’, otherwise any themes that emerge from these data will not credible”. Referring to Fielding and Fielding and Maxwell, they further write, “Triangulation reduces the possibility of chance association […] thereby


allowing greater confidence in any interpretations.” To cross-check information and conclusions through the use of multiple sources and procedures, there are two types of triangulation available, namely triangulation of data, which is the use of multiple data sources to help understand the phenomenon (Johnson, 2008:12) and methodological triangulation, which is the use of multiple research methods of data collection to approach an object of study from different angles in order to investigate whether the data collected with one instrument confirms the data collected using a different procedure or instrument. I was ready to do these types of triangulation. In summary then, the strategies to check the accuracy of the findings which would consequently promote research validity and trustworthiness include two types of triangulation, thick description, member checking, long engagement and repeated persistent observation at the site and clarification of the researcher’s bias. The strategies to ensure this research reliability are (1) detailed account of the focus of the study, the researcher’s role, data sampling and selection and context (2) triangulation of data collection methods and data analysis methods (3) detailed report of data collection and analysis strategies (4) checking for consistent patterns of theme development.

72


CHAPTER IV

ANALYSIS RESULTS

In this chapter, I will describe what actually happened to what I had designed and planned as shown in Chapter III. This chapter is thus to serve three purposes. First, it is to present the explanation of procedures which applies the principles set out in the discussion of qualitative research presented in the previous chapter to practicalities of the day-to-day realities of the setting and structuring of activities (cf. Holliday, 2002:57). Second, it is to explain how the data acquired from the setting and sources was processed, analyzed, and interpreted. Third, it is also to articulate the outcome of the analysis and interpretation. To serve the first purpose, I will include some reports comprising those of datasource selection, of the instrument validation or accountability, and those of the data gathering, including their recording and presentation. While to serve the second purpose, I will present the actualization of data analysis together with their results. And to serve the third purpose, I will discuss the data extracts drawn from the corpus under each thematic heading and argue for any interrelations connecting with the theoretical framework, research goal and implication. This will be presented in section Discussion: Description and Interpretation.

A. DATA SOURCE SELECTION As I already mentioned in the previous chapter, I included three kinds of data sources; the setting, participants, and myself (researcher). Here is the full description of the actualization.


The primary data setting I chose was SMK Negeri 1 Pengasih, a state vocational school in the region of Kabupaten Kulon Progo, the territory of Yogyakarta, Indonesia. Meanwhile, the environs would be secondary. The reason for the choice is that it met the criteria for sensible research setting. First, it was accessible. I went to SMK Negeri 1 Pengasih almost every day. I felt as if it were my second home. Second, it was most likely to provide highly rich but manageable data, therefore the data was considered as representative. Third, it is sufficiently small and provides a variety of relevant, interconnected data such as documents, pictures and implements. SMK Negeri 1 Pengasih was noted as having an important role in the reform of the national vocational education and training, since the implementation of the national curriculum try-out in 2003. Having as many of 80 teachers and 864 students distributed in 6 study program departments: Accounting, Secretarial Affairs, Marketing, Multimedia, Hotel Accommodation, Clothing Design made this school highly feasible for the study. The instructional and formal language in the school is the national Bahasa Indonesia, while the casual language of the teachers and students is a code-mix between Bahasa Indonesia and the local native Javanese. English is considered an important subject for all students to study, although different study program departments allocate different amount of time for the subject, ranging from 2 to 7 hours a week in one semester. The English classes were heterogeneous as well as homogeneous, with a size of 10 to 36 students each. In ensuring the quality of its English education, this school employed various learning resources and tests. One of the noticeable resources is the English module series of the novice and elementary levels. Students were required to take a speaking


competency test before they could move to another module. There was also an additional written test. Here in this research, this test is viewed as “classroom test”. In addition to the classroom test, it offered another type of test consisting of periodical promotion (regional) test, provincial end-test tryout test (see sample test booklet as shown by Appendix 8.5), proficiency test (TOEIC) (see the official document as shown by Appendix 8.6) and national examination (UAN/UN/Unas). This type of tests, categorized as paper and pencil test, asked questions with multiple answers to choose. The total number ranged from 50 to 200. The second sources which provided the primary as well as secondary data for my research are participants. There were two groups: students, who provided primary data; and teachers, who provided support data. Since there is no rule of size and sample, data was taken from three students majoring in Accounting, Secretarial Affairs and Marketing representing Indonesian vocational school students who provided primary data and two English language teachers, who provided the secondary data. The reasons and considerations related with the source selection are as follows. First - related with the number - why I decided to take on 5 people is that it allowed me to get in-depth data with an easy access and to work with the data. Regarding to the selection, it is because I encountered events which puzzled me and raised big questions to answer (cf. Holliday, 2002:31). I was determined to opt for extreme case selection , which means I selected cases that represented the extremes on some dimension, and opportunistic selection (which therefore can also be termed as non-probability selection), meaning that I selected useful cases as the opportunity arose (cf. Chenail, 1995).


Table 4.1 Student participants’ data No. 1. 2. 3.

Name

Sex

Age

Grade

F F F

17 16 15

3 2 1

Atik Siti Endang

Native Language Javanese Javanese Javanese

The student participants are – let me use pseudonyms to protect their anonymity, Atik, Siti and Endang. The teacher participants are Iin and Nanik. Table 4.1 presents the complete information on the student participants, while Table 4.2 presents data on teacher participants. The scope of inquiry expanded as problems encountered during the research. Table 4.2 Teacher participants’ data No. 1. 2.

Name Iin Nanik

Sex F F

Age 33 33

Experience 7 years 12 years

Native Language Javanese Javanese

These are background stories drawn from the events I encountered. One day as I passed by, a colleague of mine, Bu Titik, a guidance and counseling teacher, called me, telling her problem and asked questions related to Student Atik (SA). She told that SA, a thirdgrader, had an opportunity to pursue higher education without any entrance test for she was considered as a student with achievements. One requirement for enrollment was submitting the schoolreport, which informs everything since the first semester of her schooling. The problem was, there were so many blank spaces where English competency scores should been presented (See Appendix 9.4). Bu Titik told that SA was complaining, and therefore she wanted a clarification why so and asked how to solve the problem. I felt sorry for that, answering that was so as SA had not taken many classroom tests. This is how I chose SA as my participant.


When I was assessing Student Siti’s performance using a speaking competency test, she looked very nervous, although the test room was quiet enough. There were only three people: SS, I and her friend. I was curious for she could respond to my questions and instruction very well. Later, when the interview ended and I informed her score, her reaction changed 180 degrees. She screamed into joy, thanking me and sharing her happiness with her friend. That curiosity drove me to invite her as the next participant. The next event happened when I was doing a correction for my students’ work from a promotion test, consisting of objective test items. To my surprise, Student Endang’s work presented “homogeneous answers” (See Appendix 9.3). With this reason, I decided to include SE in the project. For me, what she did raised a big question and made me curious to explore the answer. In selecting teacher participants, I did not have many things to consider. There were eight EFL teachers available in school, including me. Out of the seven, one was a new-comer, with less than one-year experience. As I was concerned with experience, I did not think I could include her in my project. Two other teachers were busy with their professional tasks and study. I did not think it wise to include any of them. One preferred to transfer my request to another. In the beginning, I was determined to include three teacher participants as I informed in the previous chapter. Later as the project was progressing, I decided to take two since one of them quitted and moved to another school. The third source is me as the researcher. Here I would present myself. I began my career in 1985. I had been teaching at the vocational school where I did this research project for almost twenty years. Since then, my contact to


English teaching and testing had been continually improved. I employed and designed a number of tests. Moreover, in 2003 when competency based curriculum and communicative approach were applauded, I began to apply speaking competency test as the formal classroom test at school, and on some occasions composed test items for the provincial and national examinations. Since the mid of 2005, I had pursued my higher education in a post graduate program majoring in English Studies with the special interest in English Education, which promotes humanistic values.

B. INSTRUMENT VALIDATION AND ACCOUNTABILITY To confirm that my data and analysis were valid and accountable, it was necessary for me to make verification. As I stated earlier, I utilized strategies such as triangulation of data and technique, repeated observation, clarification of researcher bias to ensure internal validity. I also included thick description to ensure probable external validity. And to ensure reliability, I employed a detail account of the focus of the study. Figure 4.1 provides a visual description on how I have dealt with the validity and accountability of my study. As you see, to get to the answer to the research question empirically, data were mainly obtained from participant’s account, in conformity with the description of physical and non physical environment, of events, of participant’s behavior, and my own interpretation.

78


Description of physical and non physical environment

A

Description of events

B

C

Description of participant’s behavior

D

E

F

Participant’s accounts

G

H

I

J

K

L

M

N

Document reviews

O

My (researcher’s) interpretation

Note: Green boxes (D, E, F, J, and N) represent primary data

Figure 4.1 Student’s lived experience in English testing and learning seen from various angles. The verification, corroborating and validating was implemented over the course of study, along the way from the commencement to the end of the research project. In summary, the activities included

(1) two types of

triangulation (2) thick description which includes information about the context of an act, the intention of actor and the process in which action is embedded (Denzin, 1978 as cited by Dey, 1993); (3) member-checking which means finding feedback from participants about the results of the study; (4) long engagement with the participants and repeated persistent observation at the site which is said to improve research credibility (UWE, 2006:1); therefore, being there in the research setting for years was a real benefit for me, since I could do repeated persistent observation; (5) clarification of researcher bias by


presenting my professional and education background.

C. DATA GATHERING, RECORDING AND PRESENTATION Data gathering took place in two periods of research. The first occurred from February 2007 to July 2007, while the second ran from January 2008 through April 2008. To store my data, I employed several techniques. As I had fixed the research setting, data sources and instruments with the research question on my mind, I started to collect data by means of observation and interviews. In conducting the observation I followed what Holliday (2002) suggests. I used an observational protocol or form which is a single page separating descriptive notes and reflective notes, to present my discursive commentary in which I wrote about the data: what it meant, how it connected, and in what way it was significant, and providing a space for coding and demographic information about time, place and date. To access data from observation, I participated in internal as well as external meetings, wandered around the school building, entering the test rooms, observing my own class, taking a part in testing activities. I jotted down what I saw, heard and felt in words, map and pictures. During the observations, I used field notes, recorder, my diary and photograph. To help me focus my attention on the relevant aspects of interests, I utilized a blue print format, as shown in Table 4.3.

80


Table 4.3 Observation Blueprint

No.

Component

1.

Context & Situation

2.

Test type

3.

Test Backwash

4.

Learning

5.

Miscellany

Indicator Physical Non physical Classroom Highstakes Beneficial Harmful Activity

. I observed English class and test implementation which included oral competency tests (classroom tests), periodical regional test, school-year end tests, local and provincial tryout exams, TOEIC proficiency test (high stakes tests) by using photo and sound documentation. I observed events discussing national examination by using my own field notes and recording verbal and non verbal data in writing. In addition, I also accessed data from news stories and opinion on national exam from the radio and TV. These were transcribed and interpreted into English. I also compiled clipped news and opinion on national exam from emails, websites, newspapers which served as background reference to the highstakes test implementation. On some occasions, I encountered interesting events spontaneously in the research setting. Since they were of great value and use as background reference, I recorded them in my diary, using a blank note where I recorded what I saw and heard. Once a while, I documented the events with a snapshot. Official meetings, testing activities, stories and events encountered in the research setting and environs were also my interest to do observation upon. I also kept official


documents, soft copy as well as hard copy, from the research setting and reviewed them. As an illustration, I would like to show in more detail what I did together with some samples of the data presentation. First, I would like to show how the observation data is presented. This was taken from the first observation on a meeting discussing the national examination 2007 (Table 4.4). As you can observe in Table 4.4, I write what I saw and heard on the left column. The data is in words and image (sketch). In addition to the description of events, I also include my own commentaries. Table 4.4 Sample data presentation 1 Observation Data and Comment Thursday, 25th January 2007. Dinas Pendidikan Provinsi DIY’s Auditorium. 08.30 – 11.30 Approximately 110 EFL teachers from the province’s vocational schools had an assembly with local authorities as a part of preparation for the national exam, Unas. Among those who were present were the deputy head of vocational school principal organization, K3SK, the head of vocational school EFL teacher association MGMP, chief of high education, Kabid Dikmen. They were seated with the following configuration A

B

C

D

E

Code

H/Be/NP/assembly

F

Audience The head of MGMP invited the audience to form what he called “Tim Tangguh” or “Tim Sukses” for the coming national examination and to listen to the key note speaker.

H/Be/NP/to form special management team

The Kabid Dikmen added by reminding and emphasizing that the team’s service period was limited from the day of the meeting through the day before the national examination, 25th January 2007 to 16th April 2007.

H/Be/NP/fairness assurance

Meanwhile, the deputy head of K3SK informed anything about Provincial Unas try-out scheduled to take place on March 6,7, and 8. It would use a replicate of the real national examination. There would be two testpackages used, A and B. Student test takers with odd registration numbers would do different test package from those with even numbers. The student answer sheets would be sent to the local office, which would then forward this to the provincial office. He advised all facilities such as tape player and speakers be prepared and checked. It was expected that with the participation of all EFL teachers and Dikmen, the number of successful students would increase.

H/Be/P/test replica H/Be/NP/situation replica H/Be/P H/Be/NP/high expectation H/Be/NP/concern with graduation rate


The second (Table 4.5) was from students’ activity prior to the regional promotion test in 2007. The full versions of the data presentation are available in the appendices. Table 4.6 Sample data presentation 2 8 February 2007 08.37. SMK 1 Pengasih Observation Data

Comment

Code

These students get together at tables reading and discussing learning materials. They look serious. Test time is still one and half hour away. It shows high independency on learning and tells that a test motivates learning. Such a group activity may also motivate an individual to do the same.

H/Be/NP/Au BB

Situation 1 (prior to regional test)

Next, what I did is selecting research participants based on special events encountered in the setting as described in the previous section and interviewing them. I started this activity by preparing interview blueprint and formulating probing questions. I utilized an interview protocol presenting key research questions, probes to follow key questions, space for recording, the interviewer’s comments, and space for reflective notes. The complete probing interview questions are available in Table 4.6. 86


Table 4.6 Probing interview questions • • • • • • • •

•

•

•

Menurut pendapatmu, tes ulangan harian yang kamu tempuh itu bagaimana? [What do you think of the periodical test you took?] Menurut pendapatmu, tes unas/penjajakan/TOEIC/regional itu bagaimana? [What do you think of the promotion/graduation/TOEIC/regional test you took?] Menurut pendapatmu, tes yang ideal itu seperti apa? [What is your ideal English test like?] Sebelum, saat dan sesudah mengikuti ulangan harian, bagaimana perasaanmu? [How did you feel before, during and after the periodical tests?] Sebelum, saat dan sesudah menempuh ulangan harian, apa yang ada di benakmu? [What did you think before, during and after the periodical tests?] Sebelum, saat dan sesudah mengikuti ulangan harian, apa saja yang kamu lakukan? [What did you do before, during and after the periodical tests?] Sebelum, saat dan sesudah mengikuti ulangan harian, apa yang kamu alami? [What did you experience before, during and after the periodical tests?] Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/ regional, bagaimana perasaanmu? [How did you feel before, during and after the promotion/graduation/TOEIC/regional tests?] Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/ regional, apa yang ada di benakmu? [What did you think before, during, and after the promotion/graduation/TOEIC/regional tests?] Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/ regional, apa saja yang kamu lakukan? [What did you do before, during and after the promotion/graduation/TOEIC/regional tests?] Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/ regional, apa yang kamu alami? [What did you experience before, during and after the promotion/graduation/TOEIC/regional tests?]

As the probing questions were available, I started to access data from interviews by conducting initial in-depth unstructured interviews with students with special cases and teachers in native Indonesian, tape recording and carefully transcribing them. The reason to use Indonesian language was to build a closer and more intimate rapport with the participants so that the data were not only indepth but also more accurate and truthful. This produced data as shown in Appendix 7. Later, I stored all these data in a word processor for easy retrieval and


management. The raw data resulting from these activities is presented in (1) observation protocol, (2) full and edited transcripts of interview and news, (3) picture documentation, (4) my diary, and (5) sound documentation (not attached).

D. DATA ANALYSIS ACTUALIZATION As we are aware, data is presented through words and images. The primary data to analyze was that resulting from student participants’ narratives and behavior. As the raw data from observation and interviews was obtained and completed, what I did is trying to make sense out of the mass of data. To do this, I employed both a priori as well as inductive codes. Table 4.7 A-priori coding inventory for student participants

Student

Aspect

Atik (Sat)

Feel ( 1 ) Think(2 ) Do ( 3 ) Exp ( 4 ) Feel ( 1 ) Think(2 ) Do ( 3 ) Exp ( 4 ) Feel ( 1 ) Think(2 ) Do ( 3 ) Exp ( 4 ) Feel ( 1 ) Think(2 ) Do ( 3 ) Exp ( 4 )

Siti (Sit)

Endang (Sen)

SGroup (Sgr)

Student

Aspect

Atik (Sat) Siti (Sit) Endang (Sen) SGroup (Sgr)

Think

Before (1) Classroom Highstakes Test ( A ) Test ( B ) Sat-1-1-A-? Sat-1-1-B-? Sat-2-1-A-? Sat-2-1-B-? Sat-3-1-A-? Sat-3-1-B? Sat -4-1-A-? Sat -4-1-B-? Sit-1-1-A-? Sit-1-1-B-? Sit-2-1-A-? Sit-2-1-B-? Sit-3-1-A-? Sit-3-1-B? Sit -4-1-A-? Sit -4-1-B-? Sen-1-1-A-? Sen-1-1-B-? Sen-2-1-A-? Sen-2-1-B-? Sen-3-1-A-? Sen-3-1-B? Sen -4-1-A-? Sen-4-1-B-? Sgr-1-1-A-? Sgr-1-1-B-? Sgr-2-1-A-? Sgr-2-1-B-? Sgr-3-1-A-? Sgr-3-1-B? Sgr-4-1-A-? Sgr-4-1-B-?

Functions (I) Classroom Highstakes Test Test (A) (B) Sat-I-A-? Sat-I-B-?

During (2) Classroom Highstakes Test ( A ) Test ( B ) Sat-1-2-A-? Sat-1-2-B-? Sat-2-2-A-? Sat-2-2-B-? Sat-3-2-A-? Sat-3-2B? Sat -4-2-A-? Sat -4-2-B-? Sit-1-2-A-? Sit-1-2-B-? Sit-2-2-A-? Sit-2-2-B-? Sit-3-2-A-? Sit-3-2B? Sit -4-2-A-? Sit -4-2-B-? Sen-1-2-A-? Sen-1-2-B-? Sen-2-2-A-? Sen-2-2-B-? Sen-3-2-A-? Sen-3-2-B? Sen -4-2-A-? Sen-4-2-B-? Sgr-1-2-A-? Sgr-1-2-B-? Sgr-2-2-A-? Sgr-2-2-B-? Sgr-3-2-A-? Sgr-3-2B-? Sgr-4-2-A-? Sgr -4-2-B-?

Measurement Qualities ( II ) Classroom Highstakes Test Test (A) (B) Sat-II-A-? Sat-II-B-?

After (3) Classroom Highstakes Test ( A ) Test ( B ) Sat-1-3-A-? Sat-1-3-B-? Sat-2-3-A-? Sat-2-3-B-? Sat-3-3-A-? Sat-3-3-B? Sat -4-3-A-? Sat -4-3-B-? Sit-1-3-A-? Sit-1-3-B-? Sit-2-3-A-? Sit-2-3-B-? Sit-3-3-A-? Sit-3-3-B? Sit -4-3-A-? Sit -4-3-B-? Sen-1-3-A-? Sen-1-3-B-? Sen-2-3-A-? Sen-2-3-B-? Sen-3-3-A-? Sen-3-3-B? Sen -4-3-A-? Sen-4-3-B-? Sgr-1-3-A-? Sgr-1-3-B-? Sgr-2-3-A-? Sgr-2-3-B-? Sgr-3-3-A-? Sgr-3-3-B? Sgr -4-3-A-? Sgr-4-3-B-?

Content and implementation ( III ) Classroom Highstakes Test ( A ) Test (B) Sat-III-A-? Sat-III-B-?

Think

Sit-I-A-?

Sit-I-B-?

Sit-II-A-?

Sit-II-B-?

Sit-III-A-?

Sit-III-B-?

Think

Sen-I-A-?

Sen-I-B-?

Sen-II-A-?

Sen-II-B-?

Sen-III-A-?

Sen-III-B-?

Think

Sgr-I-A-?

Sgr-I-B-?

Sgr-II-A-?

Sgr-II-B-?

Sgr-III-A-?

Sgr-III-B-?


A priori codes are codes that are developed before examining the current data, while inductive codes are codes that are developed by directly examining the data (Johnson, 2008:4). As illustration, Table 4.7 is presented to show the complete a priori codes for student participants. As you notice in the table, each code ends in quotation mark, which means to allow a room for emergent themes. Sat-1-1-A-? therefore means what does student Atik feel before the classroom test?. Appendix 2 presents the full list of the a priori and inductive codes. The raw data was extracted, processed and “scanned” to obtain any information which would potentially answer my research questions, labeling each with the relevant code by referring to the available a priori coding inventory. This resulted in the grand data corpus. To make the data corpus manageable, it was repeatedly read and broken down into chunks, resulting in semi-ordered list of responses. Using inductive codes, the responses were then classified and categorized. Next, I connected the categories to find the thematic organization of data. It was then carefully evaluated so as to produce quality data. The last was producing accounts in the form of texts, tables and diagrams. The activities are included in Data processing stages as visualized in Figure 4.2.

86


PHASE

STAGE

Data Collecting

ACTIVITY AND RESULT

Observation, Interview, Document Review

Transcribing

RAW DATA

1 Data Analyzing

Compressing

DATA CORPUS

Segmenting

SEMI-ORDERED LIST OF RESPONSES

Categorizing

CATEGORIZATION OF RESPONSES

2

3

4 Connecting categories, creating hierarchical category system

THEMATIC ORGANIZATION OF DATA

5 Corroborating evidence

ORGANIZED QUALITY DATA

6 Producing account

TEXTS WITH TABLES AND DIAGRAMS

Figure 4.2 Data processing stages The more detailed process of data analysis run as follows:

87


Stage 1. Reducing the raw data acquired from the setting to find a focus, by assigning a priori codes. This was done for easier processing. This activity resulted in the corpus of data. Table 4.8 presents the sample corpus of data gleaned from the first interview with Student Atik. For clarity and transparency, I include my translation in the presentation of data corpus here as follows. Table 4.8 Sample corpus of data Comment Interviewer: Researcher Interviewee: Student Atik Setting: SMK N 1 Pengasih. 22 March 2007. 13.50 – 14.25. SA has a high sense of learning autonomy. She has critical reflection on what learning situation should be. In her view, learning English should be enjoyable. There should be enjoyment in learning English. She has independent action to fulfill her needs, to make her own decision on what and how she should learn. She knows that testing is contributive to her learning. As she knows the test format and materials, she knows what to do

Narrative

Code

Kalau mengenai waktu, saya tidak kontinyu. Jadi tidak harus setiap hari misalnya belajar dua jam, tidak, tapi tergantung mood karena yang saya tahu, apabila belajar itu dipaksakan, justru kita akan menjadi benci dengan belajar.

LAUT

[As to when I study, I do not do it on a regular basis. In other words, I do not have to study, say, two hours every day. In fact it depends on the mood. As far as I know, if we are compelled to study, we will hate to do it].

Ya, jadi bagaimana belajar itu kan tidak sebenarnya dalam Bahasa Inggris khususnya, sambil nonton TV pun bisa, karena sekarang di TVRI itu ada program pendidikan dan bahasa Inggrisnya banyak.

LAUT

[Well.. the way we study is not (only like that). In fact studying English can also be done while we are watching TV, as TVRI has many educational and English programs now.]

Saya lebih senang belajar sendiri karena untuk belajar dengan teman nanti justru akan e mengobrol.

LAUT

[I prefer studying on my own, as when you do it with a friend, you will be chatting, instead. ]

Kalau reading, saya membaca dalam hati dan kemudian kalau ada kata-kata yang tidak saya mengerti, saya cari di kamus. Dan untuk listening, saya sering nonton film luar negeri dengan volume suara yang keras. In learning reading, I try to comprehend it. When I find new words, I look up the dictionary to find the meanings. In learning listening, I do it by watching foreign films with high level sound volume.]

LAUT


Sedikit banyak tes itu mempengaruhi cara belajar. Bagaimana pun kita akan berusaha belajar lebih intensif.

BB

[Tests are rather influential to the way we learn. We will try to study more intensively.]

Belajar mengenai soal-soal tes TOEIC. Kan e beberapa kali saya pernah menjalani tes TOEIC, jadi sudah tahu apa kita-kira soalnya seperti apa, misalnya soal ya grammar, reading. Jadi belajar mengenai yang kira-kira keluar. [Studying TOEIC-like test materials. I have taken such a test on several occasions, so I know what the materials are like, such as grammar, reading. In short, I study those I expect to encounter in the test]

Menurut saya cukup penting karena untuk mengetahui kemampuan kita dalam Bahasa Inggris. Kita sudah sampai sudah level apa? Selain itu untuk tes TOEIC menurut sepengetahuan saya juga sertifikatnya berguna untuk mencari pekerjaan.

Sat-3-1-B-doing test practice Sat-2-1-B-she knows what the test is like and what will be tested

Sat-I-B-assess one’s ability Sat-I-B-know level Sat-I-B-career development

[In my opinion, TOEIC test is very important. It is to know which level we are in English. In addition, the certificate will be useful to get a job]

Ada, sebetulnya ada. Kan competency test, walaupun itu speaking, tapi kan dalam speaking kita juga harus memperhatikan grammarnya, tidak asal ngomong, harus memperhatikan tatabahasanya.

Sat-III-Agrammar

[It is useful too for the competency test. The competency test requires us to speak, but when we are speaking, we must not merely say a string of words; in fact we must pay attention to the grammar...]

Biasanya, e, membaca modul. Itu kan tes competency test kan biasanya ber.. berupa pertanyaan yang jawabnya seperti mengarang.Walaupun dialog, tapi dialognya harus mengarang. Dan di dalam modul itu kan banyak contoh-contoh dialog. Jadi kita belajar sebagai contoh kita membuat dialog. Selain itu kita juga tanya-tanya sama teman-teman yang sudah tes duluan. [Usually, I er.. read the module. The competency test include a question which requires us to compose sentences. So, although it is in the form of dialog, we must compose it first. And in the module there are model dialogs, So we learn them. In addition, we usually ask friends who have already taken the test]

Sat-3-1-A-reading modules, learning from model dialogs Sat-3-1-A-asking friends who has taken the test


Kalau saya pribadi lebih menyukai soal-soal seperti TOEIC. Jadi listening, dan apa namanya, grammar, reading dan seperti itu. Ya, speaking itu juga penting, ‘tapi e kan untuk membuat dialognya itu agak malas, ‘gitu.

Sat-1-1-Areluctant to prepare for speaking test

[Personally, I prefer TOEIC-like test. It tests listening, and what is it..., grammar, reading and the like.Well, speaking test is also important, but to compose the dialog is not fun. And I feel traumatic. The questions are not relevant at all and make me confused to answer. Since tnen I have become scared. Too many questions make me afraid.]

Dan itu ‘kan seperti dari awal itu seperti tidak wajib. Jadi, katanya, walaupun tidak selesai tetap lulus, gitu Bu. Dan teman-teman juga masih di A5. Kita masih santai: teman-teman juga masih A5. Dan itu juga pengaruh teman. Kalau temannya nggak tinggi ya ’nggak.

Sat-2-1-A-not necessary to take the speaking test Sat-4-1-A-not motivated

[And it seems that taking the speaking competency test is not compulsory. So, although we have not completed all the tasks, we have opportunity to pass. We feel relaxed: other friends also finish not more than five. When our friends do not take more, I am not (encouraged)...]

’Nggak. Ya karena kan belum ada teman yang di atasnya jadi belum ada yang ditanyai, gitu lho Bu. Biasanya tanya hasil competency test. Terus tanya, kemarin ditanya apa sama Bu Guru? Terus gimana gimana. Dan saya juga pernah agak trauma

Sat-1-3-Atraumatic

[No...(not encouraged). You know there is no one above, so no one can be asked... Usually, we ask the result of (friends’) competency test, how it went on.... And I have a traumatic experience too...]

Karena pertanyaan menyimpang jauh dan membuat saya bingung menjawabnya.

Sat-1-2-Aconfused to answer

[As the questions were not relevant and made me confused to answer,]

Pertanyaannya terlalu jauh. Misalnya tentang ... sebentar ... A5 itu apa Bu? [The question went out of the track. It is about... what is A5?]

Sat-2-2-Ateacher’s questions are not relevant


Kalau ndak salah itu malah ditanya – saya dulu pengurus OSIS – sebagai pengurus OSIS gimana menurut kamu kepala sekolah kita? Terus gimana, berani ndak kalau mengajukan usul kepada kepala sekolah, yang tidak ada hubungannya dengan describing people. [I was asked – I was a member of student’s association – what do you think of our headmaster? I was also asked if I dare to make a proposal to him. It was nothing to do with describing people....] Tapi setelah itu saya jadi takut, Bu. Terlalu banyak pertanyaan itu saya takut.

Sat-1-3-A-afraid due to too many questions

[Since then, I have been terrified. Too many questions make me afraid.]

E, misalnya kemampuan Bahasa Inggris itu ya harus didasari dengan teori yang bagus, harus dengan pengetahuan tentang grammar, tentang vocab, kemudian dengan listening. Setelah listening baru speaking. Dan untuk saat ini saya untuk speaking belum cukup mampu

LAUT

[Er..English ability should be based on a good theoretical foundation, with a good knowledge of grammar, vocabulary and listening. After listening is mastered, comes speaking. And for this moment, speaking is still beyond my capability.]

Ya, untuk ke depannya.. untuk kedepannya saya juga ingin listening yang benar-benar bisa listening yang dengar bisa paham misalnya nonton TV.

LAUT

[Well, later, in the future, I want to be able to understand fully when listening, say, when watching TV.]

Selain itu speaking tetapi tidak hanya speaking seperti penjual yang melayani turis. Itu kan speakingnya waton dimengerti ’gitu lho Bu. Saya ingin speaking yang grammarnya juga betul, gitu

LAUT

[I want to be able to speak too, but it is not the kind of what a seller can do for tourists. That kind of speaking is merely understandable, right? I want the speaking ability with appropriate grammar.] In her view, speaking can only be mastered only if she masters its theory ( grammar, vocabulary)

Sejauh ini kalau soal listening itu dengan tadi dengan nonton. Terus dengan speaking, baru mempelajari teorinya. Untuk prakteknya belum.

LAUT


[So far, as I told you, for listening I it by watching TV. For speaking, I am still learning the theory. I haven’t practised it.]

Belum. [Not yet] Reading is the main activity in learning English

Saya kira teman-teman saya belajar dari membaca. [I think my friends learn by reading.] E, kalau menjelang tes, perasaan takut itu pasti, mesti, ada, tapi kalau saya tidak terlalu takut karena ya apa pun itu yang diteskan kan sudah disesuaikan dengan kemampuan anak SMK. Jadi saya yakin tidak akan terlalu sulit bagi anak bagi anak SMK. [Before taking a test, anxiety is naturally there, but I don’t feel so afraid, as the test materials must be within students’ reach. So I am sure they will not too difficult to students.]

Sat-1-1-B-not too afraid Sat-2-1-B-test materials are within student’s reach Sat- 1-1-B-sure that the test will not be difficult

E, ada soal itu yang dengan kertas dibagikan itu. Terus soalnya berbentuk beberapa berbentuk multiple choice, melengkapi kalimat dan ada esainya sedikit dan esainya itu simpel. [Er.. it is a paper and pencil test. It consists of multiple choice, sentence completion, a limited number of simple essay.]

Ya ada rasa takut, tapi lebih takut yang sekarang yang di SMK daripada dulu.

Sat-1-2-B-fear, anxious

[Yes... there was a certain degree of anxiety, but now it turns higher]

Karena dulu itu lebih ya simpel, gitu, belum ada grammar, jadi masih simpel sekali. [The test was simpler, without grammar materials. So it was much simpler...]

Ya ditambah sedikit. Tidak sampai wayangan begitu lho, tidak. [I study more, but not all night long.]

Sat-3-1-B-study more


Ada, sedikit. [A little(nervous)]

Sat-1-2-A/Bnervous

Kalau suasananya tenang, e suasana perasaan cukup tenang itu sudah baik. Tapi yang saya tidak suka kalau ada misalnya pas penjajakan waktunya sudah selesai tapi teman-teman sudah keluar mengganggu konsentrasi. Pada hal kita sebaiknya menghabiskan waktu tes itu. Walaupun kita sudah selesai. Kita menghabiskan waktu itu untuk mengulang terus mengulang. Terganggu adanya siswa yang keluar lebih dahulu.

Sat-1-2-Bdisturbed by other’leaving

[If the situation is quiet and I have good mood, that will be good. But when I was doing a tryout test, I was disturbed by those who had finished the test as they left the room earlier: it really removed my concentration. I wish they had spent the whole allocated time (in the room), although we had finished it earlier. We should spend the rest of the time to make a review to our answers. I felt disturbed by those who were leaving.] Anxiety is due to strict supervision, familiarity with supervisor

Sejauh ini perilaku Bapak ibu guru yang mengawasi tidak ada yang mengganggu, namun perasaan saya sendiri, misalnya kemarin pas pada penjajakan Bahasa Indonesia. Yang mengawasi kan Bu Yuli, guru. Bagaimanapun ada sedikit nervous, gitu. [As far as I know, what the test supervisors did was not disturbing. It’s my own feeling which matters. For example, in the last tryout test of Indonesian, Bu Yuli became my supervisor. I felt a little nervous.]

Pengawas yang bukan guru mata pelajarannya dan justru tidak mengajar, jadi kurang mengenal saya, begitu Bu. [(I expect to have) a supervisor who is not my teacher, someone who is not familiar with me.]

Kalau lebih suka yang agak kendor pengawasannya karena. Sebenarnya misalnya saya nggak nurun nak kerjasama tapi kalau dilihati bapak/Ibu guru bagaimana ya nervous. [I prefer relaxed supervision, because being watched by the supervisor makes me nervous, although it is not because I am cheating.]

Kalau jengkel sih enggak, Bu tapi ada yang di luar dugaan gitu Bu. Misalnya ada komentarnya banyak terlalu banyak mengritik, tapi kok nilainya bagus. Ada yang diam sepertinya saya tidak melakukan banyak

Sat-1-3-Asurprised to see the score Sat-4-2-A-being


kesalahan, tapi ternyata nilainya jelek. [I don’t think I am annoyed; in fact, I find something surprising. There is a teacher who commented and criticized a lot, but awarded a good mark. Another just kept silent as though I had not made any erroneous answers, but gave a bad mark.] Score results in beneficial backwash

Kalau saya lebih ke meningkatkan nilai lebih dari 6.

criticized Sat-4-2-A-getting no comment on her performance

BB

[For me, it will encourage me to promote the score so what I can get is more than 6.]

Mempertahankan. Saya ingin mempertahankan. Justru mempertahankan lebih sulit daripada meningkatkan. [Maintaining. I want to maintain (my achievement). Maintaining ( the good achievement) is indeed harder than struggling for it.]

Ada yang bagus waktu belajar listening. Setelah misalnya mendengarkan 10 soal listening. Setelah itu pembahasan dan waktu pembahasan saya lebih senang kaset itu diputar lagi. Jadi misalnya nomor satu, ibu guru memberi jawaban A tapi setelah itu diulangi lagi mendengarkan apakah betul A itu. Kan ada guru yang memberikan hanya kuncinya saja. Nomer 1 A, 2 B. [I found something good when learning listening. As we heard ten listening test practice items, we had a discussion. I felt happier as the teacher replayed the cassette. Say, the teacher said that the answer to number one was A, later we could check whether it was really A. You know another teacher just gave the key answers: number 1 A, 2 B.] The higher level of difficulty brings sorrow, distress

Tergantung pada tesnya. [It depends on the test....]

Kalau tesnya susah. [.... if I find it is difficult.]

Setelah penjajakan ya mungkin perasaan saya agak tenang, karena saya merasa agak bisa, tapi kalau seperti tadi. Ada rasa sedih ada rasa di sini pusing, di sini rasanya berat sekali. Karena tidak bisa. Karena belum selesai mengerjakan, takut nilainya jelek.

Sat-1-3-B-relax Sat-1-2-B-can do the test Sat-1-3-Bsad,dizzy Sat-1-2-B-cannot do the test

[After the tryout test, I felt a bit relaxed as I thought I

Sat-4-2-B-fail to


could do it, but what happened just now... I felt sad, dizzy as I couldn’t. As I failed to finish it, I am afraid I will get a low mark.]

Menurut saya kalau dibandingkan antara TOEIC dan tes penjajakan menurut saya lebih mengesankan TOEIC Obligation is prefered

[I feel TOEIC is more impressive than the tryout test]

Saya agak keberatan dengan metode seperti itu. Saya lebih suka kalau ya mungkin ada Yang waktunya dibatasi saja, menurut saya mulai dari kelas satu bulan I semua harus lulus A1, bulan II [I don’t think the method OK. I would prefer if we set the time limit; say, for the first month, the firstgraders should pass competency A1, the second month ....]

Ya menurut saya bagus. SMK kan kebanyakan langsung kerja, jadi penting apalagi menghadapi globalisasi katanya banyak orang luar yang masuk sini. Dunia kerja pekerja dan untuk meraih tujuan sekolah mengadakan competency test memang seperti itu. Speaking harus dilatih. Itu tadi waktunya diharuskan. [Well, I think it is good. Most vocational school graduates enter the working world right away, so it is important, especially now as we are welcoming globalization era when many foreigners come. To anticipate it and to achieve the school goal, the competency test should indeed be done that way. Speaking should be trained. But we should set the time limit as I said.]

Misalnya error recognition itu misal sebenarnya yang diarah oleh soal itu misalnya mengenai gerund, tapi kalau di TOEIC itu sepertinya itu tidak terlalu banyak kalimatnya itu lho, Bu. Jadi kalau gerund langsung bisa ditangkap kalau itu yang. Kalau di penjajakan error recognition panjang-panjang. Itu membingungkan [In the error recognition part of the TOEIC test, an item which tests our knowledge of gerund comes with a limited number of sentences, Ma’am. So, we can catch what it means. In the same part of the tryout test, it comes with confusing, long sentences,] .

finish the test Sat-1-3-B-afraid to get a low mark


Stage 2. Managing the corpus of data by careful reading, coding, which includes breaking down the rich data into smaller chunks. The messy, jumbled chunks from the sample corpus recently presented and the rest data corpus from other interviews, together with the a priori codes and interpretation were blended and carefully processed, sorted by the codes answering the similar question probes and in the process stored in a tabulating table as represented by Table 4.9. Table 4.9. Sample tabulating table 1.

Menurut pendapatmu, tes ulangan harian yang kamu tempuh itu bagaimana? [What do you think of the periodical test you took?] I-A II-A III-A “Kalau saya pribadi lebih menyukai soal-soal seperti TOEIC. Jadi listening, dan apa namanya, grammar, reading dan seperti itu. Ya speaking itu juga penting, tapi e kan untuk membuat dialognya itu agak malas, ‘gitu” “ Saya agak keberatan dengan metode seperti itu. Saya lebih suka kalau ya mungkin ada yang waktunya dibatasi saja, menurut saya mulai dari kelas satu bulan I semua harus lulus A1, bulan II ...”(Sat) “Ya menurut saya kalau uji kompetensi yang modul-modul itu bagus, bu karena bisa memotivasi siswa gitu, bu. Wah kalau teman saya sudah berapa gitu levelnya njuk saya kok masih di sini, jadi memotivasi gitu lho bu, agar belajarnya lebih ditambahi lagi, gitu” (Sit) “Sepertinya sudah cocok semua, kok Bu.Bisa mengikuti kegiatan itu.” “Ya bisa berlatih untuk lancar menggunakan bahasa Inggris, bisa semuanya bisa menggunakan apa itu dengan baik bahasa Inggris.” (Sen)

2.

Menurut pendapatmu, tes unas/penjajakan/TOEIC/regional itu bagaimana? [What do you think of the promotion/graduation/TOEIC/regional test you took?] I-B II-B III-B ”Misalnya tes TOEIC atau tes apa itu pendidikan yang lainnya itu. Tes TOEIC itu bisa untuk kita supaya berlatih mandiri untuk belajar Bahasa Inggris supaya tidak tergantung kepada teman dan melatih pendengaran kita supaya kita itu peka terhadap perkataan dengan menggunakan kata-kata Inggris yang tidak secara tulisan, secara lisan.” ”Ya, itu semuanya itu sudah baik, tapi yang yang terakhir itu masih kurang saya pahami, gitu lho Bu. Kosakata-kosakata itu masih banyak yang sama, masih banyak yang kurang dipahami. Itu masih banyak yang menyalahkan gitu lho. Ya, a, b, seperti bahasa Indonesia itu soalnya mengecoh gitu.” ”Ya, saya suka. Untuk melatih pendengaran kita supaya kita itu cepat mudah mempelajari Inggris.” (Sen)


(cont’d) “Menurut saya cukup penting karena untuk mengetahui kemampuan kita dalam bahasa Inggris. Kita sudah sampai sudah level apa?“Selain itu untuk tes TOEIC menurut sepengetahuan saya juga sertifikatnya berguna untuk mencari pekerjaan.” (Sat) ”Menurut saya bagus karena bisa mengukur kemampuan kita dalam berbahasa Inggris. ” ”Saya rasa juga cocok karena sudah pernah dipelajari, gitu, Bu. Jadi siswanya nggak kesulitan dalam mengerjakannya.(Sit)

3.

Menurut pendapatmu, tes yang ideal itu seperti apa? [What is your ideal test like?] ”Materinya itu yang sesuai dengan yang diajarkan, namun juga tidak terlalu sederhana. Jadi ada pengembangannya agar siswa itu lebih kreatif.” ”Disesuaikan dengan waktu. Jadi artinya kalau waktunya dua jam, soalnya itu tidak terlalu banyak, juga tidak terlalu sedikit. Mungkin yang kemarin waktunya kan sangat sisa, itu mungkin waktunya yang terlalu panjang atau soalnya yang terlalu sedikit.” ”[...]selama ini belum ada kan yang tes mungkin ujian nasional yang ditambah writing. Jadi itu inovasi baru. Jadi jangan sampai inovasi baru itu malah menjatuhkan.” (Sat) ”[...] berkaitan dengan kehidupan sehari-hari kita, jadi nggak sulit-sulit, gitu Bu.” “[...] sudah dipelajari, gitu Bu.”(Sit) “Sepertinya sudah cocok semua, kok Bu. Bisa mengikuti kegiatan itu.” “Bisa berlatih untuk lancar menggunakan bahasa Inggris.” (Sen)

4.

Sebelum, saat dan sesudah mengikuti ulangan harian, bagaimana perasaanmu? [How did you feel before, during and after the promotion, graduation/TOEIC/ regional tests?] 1-1-A 1-2-A 1-3-A “Dan saya juga pernah agak trauma.Karena pertanyaan menyimpang jauh dan membuat saya bingung menjawabnya.” “Tapi setelah itu saya jadi takut, Bu. Terlalu banyak pertanyaan itu saya takut.” [Sat] ”[...] nervous, bu. Mungkin karena belum siap, belum terlalu siap gitu. [...] kalau speaking itu kan nanti bagaimana pengucapannya kalau salah terus berfikitr gitu lho, Bu” (Sit) ”Wah, kalau teman saya sudah berapa gitu levelnya, njuk saya kok masih di sini, jadi memotivasi gitu lho, Bu, agar belajarnya ditambahi lagi” (Sit) ”Ya kurang PD gitu, kan masih takut kalau punya kita itu masih ada yang salah gitu. Takut kalau disalahkan teman-teman gitu. Ya, tapi masih biasa kok, belum terganggu” ”Deg-degan, tegang, gitu”. ”Ya soalnya majunya itu harus duluduluan gitu lho, Bu. Tidak, tidak apa urut gitu tidak. Siapa yang cepat dia itu langsung maju, yang bisa langsung njawab, gitu”. ”Ya masih kurang puaslah wong bijinya apa nilainya itu belum bagus, belum semaksimal, belum seperti teman-teman yang lancar berbahasa Inggris” (Sen)


5.

(cont’d) Sebelum, saat dan sesudah menempuh ulangan harian, apa yang ada di benakmu? [What did you think before, during and after the periodical classroom test?] 2-1-A 2-2-A 2-3-A “Pertanyaannya terlalu jauh. Misalnya tentang... sebentar...A5 itu apa, Bu? Kalau ndak salah itu malah ditanya – saya dulu pengurus OSIS – sebagai pengurus OSIS gimana menurut kamu kepala sekolah kita? Terus gimana, berani ndak kalau mengajukan usul kepada kepala sekolah, yang tidak ada hubungannya dengan describing people.” “Dan itu ‘kan seperti dari awal itu seperti tidak wajib. Jadi, katanya, walaupun tidak selesai tetap lulus, gitu Bu.” (Sat)

6.

Sebelum, saat dan sesudah mengikuti ulangan harian, apa saja yang kamu lakukan? [What did you do before, during and after the periodical classroom test? 3-1-A 3-2-A 3-3-A “Biasanya, e membaca modul. Itu kan tes competency test kan biasanya ber...berupa pertanyaan yang jawabnya seperti mengarang. Walaupun dialog, tapi dialognya harus mengarang. Dan di dalam modul itu kan banyak contoh-contoh dialog. Jadi kita belajar sebagai contoh kita membuat dialog. Selain itu juga tanya-tanya sama teman-teman yang sudah tes duluan.” “...biasannya tanya hasil competency test. Terus tanya, kemarin ditanya apa sama Bu Guru? Terus gimana gimana.” ”Ya latihan membuat teks, menghafalkan gitu, tapi masih banyak kosa kata yang itu belum tenses benar gitu, Bu, masih kurang.” Ya kadang-kadang kalau ada kosakata yang belum tahu, tanya teman, tapi apa selebihnya masih belajar sendiri.” (Sat) “Saya kalau bisa ya buat sendiri, Bu. Nanti Cuma tanya temannya tiu bagaimana bentuk-bentuknya itu sudah benar atau belum, ‘gitu, Bu. Nanti kalau belum ya dikoreksi lagi, ‘gitu.”. “ Kalau pengucapannya sya minta teman saya itu mendengarkan, nanti bagaimana, yang salah mananya.” (Sit) “Ya latihan membuat teks, menghafalkan ‘gitu.” “Ya kadang-kadang kalau ada kosakata yang belum tahu tanya teman, tapi apa selebihnya masih belajar sendiri.” “... tapi kalau sudah di sekolah, minta bantuan teman untuk memperbaiki.” (Sen)

7.

Sebelum, saat dan sesudah mengikuti ulangan harian, apa yang kamu alami? [What did you experience before, during and after the periodical classroom tests?] 4-1-A 4-2-A 4-3-A “Kalau jengkel sih enggak, Bu tapi ada yang di luar dugaan gitu Bu. Misalnya ada komentarnya banyak terlalu banyak mengritik, tapi kok nilainya bagus. Ada yang diam sepertinya saya tidak melakukan banyak kesalahan, tapi ternyata nilainya jelek.”(Sat) “Lulus”(Sen)


8.

(cont’d) Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/ regional, bagaimana perasaanmu? [How did you feel before, during and after the promotion/graduation/TOEIC/ regional tests?] 1-1-B 1-2-B 1-3-B “E kalau menjelang tes, perasaan takut itu pasti, mesti ada, tapi kalau saya tidak terlalu takut karena ya apa pun itu yang diteskan kan sudah disesuaikan dengan kemampuan anak SMK.” “Setelah penjajakan ya mungkin perasaan saya agak tenang, karena saya merasa agak bisa, tapi kalau seperti tadi.... Ada rasa sedih, ada rasa di sini pusing, di sini rasanya berat sekali karena tidak bisa. Karena belum selesai mengerjakan, takut nilainya jelek.” (Sat) “Kalau suasananya tenang, e suasana perasaan cukup tenang itu sudah baik. Tapi yang saya tidak suka kalau ada misalnya pas penjajakan waktunya sudah selesai tapi teman-teman sudah keluar, mengganggu konsentrasi. Pada hal kita sebaiknya menghabiskan waktu tes itu. Walaupun kita sudah selesai, kita menghabiskan waktu itu untuk mengulang, terus mengulang. Terganggu adanya siswa yang keluar lebih dahulu.” “E sebelumnya sebelum ujian itu memang saya ya agak nervous, nervous juga ada, tapi ketika saya sudah baca soalnya ya perasaan nervous itu sedikit terkurangi karena saya melihat bahwa soal ujian nasional itu malah lebih mudah dari pada soal-soal ujian nasional ya mungkin tahun-tahun lalu. Jadi lebih sederhana.” (Sat) “Perasaan saya tadi waktu mengerjakan tes agak nervous ya karena menurut saya ya gimana ya soalnya cukup sulit walaupun sudah belajar semalaman tapi sepertinya masih agak sulit, Bu” [laughing] (Sit) ”Ya, tegang, kaget, gitu, lah, Bu untuk mendengar gitu. Belum biasa.” (Sen)

9.

Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/ regional, apa yang ada di benakmu? [What did you think before, during and after the promotion/graduation/ TOEIC/regional tests?] 2-1-B 2-2-B 2-3-B ”... jadi sudah tahu apa kira-kira soalnya seperti apa, misalnya soal ya grammar, reading. (Sat) ”Saya berfikir itu apa itu peraturannya itu sangat ketat, ternyata benar. Itu sangat ketat [ laughing]. Lalu ya suasana yang tenang dan sepi, itu ternyata ya memang seperti itu” ”Paragrafnya itu terlalu panjang, ’gitu.”(Sit) ” Gerund, gitu. Susah gitu.” ”[..] nggak begitu kaget. Soalnya mengerjakan bahasa Inggris itu belum terlalu bisa itu”(Sen)


(cont’d) 10. Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/ regional, apa saja yang kamu lakukan? [What did you do before, during and after the promotion/graduation/TOEIC/ regional tests?] 3-1-B 3-2-B 3-3-B “Belajar mengenai soal-soal tes TOEIC. [...] . Jadi belajar mengenai yang kira-kira keluar.” “Saya e mencari tambahan di luar. Jadi tidak tidak hanya yang disampaikan Bapak/Ibu guru. [...] saya kan mengikuti bimbingan belajar. Jadi bimbingan belajar itu saya cari buku-buku misalnya buku tentang grammar. Kalau malam menjelang tess ya saya memang belajar intensif, tapi tidak sampai terlalu malam, nanti kalau terlalu malam justru merugikan paginya. Terus kalau paginya saya usahakan sampai di sekolah ya buka-buka sedikit, tapi jangan sampai terlalu serius. Mungkin itu malah membuat kacau pikiran. Jadi sedapat mungkin seperempat jam sebelum tes dimulai itu saya malah berhenti belajar. Jadi menenangkan diri. “Saya sendiri, karena memang saya lebih senang belajar sendiri.”” Kalau keluar biasanya memang hanya mendiskusikan jawban soal.” E, berdiskusi dengan temanteman, mencocokkan jawaban. Misalnya nomor sekian jawabannya apa, jawabanku apa gitu.” (Sat) ”Saya mengerjakan soal-soal yang ada di rumah saya yang saya pinjam dari perpustakaan sekolah, Bu.”(Sit) ”Spekulasi. Soalnya saya itu belajar bahasa Inggris itu belum terlalu bisa, gitu lho Bu”. ”Belum, ya saya itu kosakata-kosakata Inggris itu belum terlalu tahu, gitu lho. Terus saya itu menjawabnya sebisanya.” ”Saya tidak mengejar target. Yang penting saya itu bisa tahu tentang Inggris gitu lho.” (Sen)

11. Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/ regional, apa yang kamu alami? [What did you experience before, during and after the promotion/graduation/TOEIC/ regional tests?] 4-1-B 4-2-B 4-3-B ”...saya melihat bahwa soal ujian nasional itu malah lebih mudah dari pada soalsoal ujian nasional ya mungkin tahun-tahun lalu.” ”... soalnya tadi sepertinya cukup sederhana. Dan untuk listeningnya juga sangat jelas walaupun ya saya ada yang lepas sedikit, tapi sangat jelas.” (Sat) ”Ya ada. Ada beberapa.” ”Ada teman-teman tapi tidak bertanya pada saya. Tapi kegiatan tanya jawab ada sama yang lainnya, Bu. (Sit) ”Belum, masih kurang, Bu. Nilainya kemarin aja mengulang itu” (Sen)

100


This activity resulted from the reconstruction of question and response and eventually stored in Semi-ordered list of responses (as represented by Table 4.10). Table 4.10 Semi-ordered list of responses to the open ended question “Sebelum, pada saat, dan setelah mengikuti ulangan harian bagaimana perasaanmu?” [How did you feel before, during and after the periodical classroom tests?] Participant

Coded Responses

Atik

Kalau saya pribadi lebih menyukai soal-soal seperti TOEIC. Jadi listening, dan apa namanya, grammar, reading dan seperti itu. Ya, speaking itu juga penting, ‘tapi e kan untuk membuat dialognya itu agak malas, ‘gitu. Sat 1-1-A-reluctant to prepare for speaking test due to test material Dan teman-teman juga masih di A5. Kita masih santai: teman-teman juga masih A5. Dan itu juga pengaruh teman. Kalau temannya nggak tinggi ya nggak. Sat-1-1-A-not motivated

Karena pertanyaan menyimpang jauh dan membuat saya bingung menjawabnya. Sat-1-2-A-confused to answer due to low level reliability

Dan saya juga pernah agak trauma. Sat-1-3-A-traumatic Tapi setelah itu saya jadi takut, Bu. Terlalu banyak pertanyaan itu saya takut. Sat-1-3-A-afraid due to numerous irrelevant question

Kalau jengkel sih enggak, Bu tapi ada yang di luar dugaan gitu Bu. Misalnya ada komentarnya banyak terlalu banyak mengritik, tapi kok nilainya bagus. Ada yang diam sepertinya saya tidak melakukan banyak kesalahan, tapi ternyata nilainya jelek. Sat-1-3-A- surprised to see the score

Siti

Wah kalau teman saya sudah berapa gitu levelnya, njuk saya kok masih di sini, jadi memotivasi gitu lho, Bu, agar belajarnya ditambahi lagi, gitu.. Sit-1-1-A-challenged to study by other's achievement


(cont’d) Kalau speaking itu kan nanti bagaimana pengucapannya kalau salah, terus berfikir gitu lho, Bu.. Sit-1-2-A-nervous because of being haunted by making mistakes in pronunciation. Terganggu gitu, Bu. Kalau berbicara terus ada orang-orang berbicara kan nanti malah ah mau dengerin dulu, gitu. Sit-1-2-A-disturbed by noises and other's leaving Ya ada, tapi kalau sudah biasanya kalau sudah selesai tesnya itu baru lega gitu lho Bu, rasanya. Sit-1-3-A-content

Endang

Ya, tapi masih biasa kok, belum terganggu. Sen-1-2-A- undisturbed in noisy situation Ya, kurang PD gitu, kan masih takut kalau punya kita itu masih ada yang salah gitu. Sen-1-2-A-not confident with her skill. Deg-degan, tegang, gitu. Ya soalnya majunya itu harus dulu-duluan, gitu lho, Bu. Tidak, tidak apa urut gitu tidak. Siapa yang cepat dia itu langsung maju, yang bisa langsung njawab, gitu. Sen-1-2-A-tensed because of competing with others. Ya masih kurang puaslah wong bijinya apa nilainya itu belum bagus, belum semaksimal, belum seperti teman-teman yang lancar berbahasa Inggris Sen-1-3-A-not satisfied with the score.

Since there are eleven interview questions, consequently there are other ten lists of this kind. Stage 3. Understanding and using content analysis (descriptive and interpretative) and semiotic analysis and putting data into categories using inductive codes. At this stage, I repeatedly read through all the data, refering to the theoretical concept map as a guide, and being prepared to observe different themes to emerge. It results in Categorization of responses (as represented here by Table 4.11).

102


Table 4.11 Categorization of responses to the open-ended question “Sebelum, pada saat dan setelah mengikuti ulangan harian, bagaimana perasaanmu?” [How did you feel before, during and after the periodical classroom tests?] Inductive Categories Anxiety : readiness, bad performance, errors

Mungkin karena belum siap, belum terlalu siap, gitu. Sit-1-2-A-nervous because she is not ready for the test. Kalau speaking itu kan nanti bagaimana pengucapannya kalau salah, terus berfikir gitu lho, Bu.. Sit-1-2-A-nervous because of being haunted by making mistakes in pronunciation. Ya kurang PD ‘gitu, kan masih ada yang salah, ‘gitu. Takut kalau disalahkan teman-teman, ‘gitu Sen 1-2-A-inconfident

Tensity: competition

Reluctance: test content

Confusion: test reliability, situation

Coded Responses

noisy

Deg-degan, tegang, gitu. Ya soalnya majunya itu harus duluduluan, gitu lho, Bu. Tidak, tidak apa urut gitu tidak. Siapa yang cepat dia itu langsung maju, yang bisa langsung njawab, gitu. Sen-1-2-A-tensed because of competing with others. Kalau saya pribadi lebih menyukai soal-soal seperti TOEIC. Jadi listening, dan apa namanya, grammar, reading dan seperti itu. Ya, speaking itu juga penting, ‘tapi e kan untuk membuat dialognya itu agak malas, ‘gitu. Sat 1-1-A-reluctant to prepare for speaking test due to test material Kalau saya pribadi lebih menyukai soal-soal seperti TOEIC. Jadi listening, dan apa namanya, grammar, reading dan seperti itu. Ya, speaking itu juga penting, ‘tapi e kan untuk membuat dialognya itu agak malas, ‘gitu. Sat 1-1-A-reluctant to prepare for speaking test due to test material Karena pertanyaan menyimpang jauh dan membuat saya bingung menjawabnya. Sat-1-2-A-confused to answer due to low level reliability Dan saya juga pernah agak trauma. Sat-1-3-A-traumatic Terganggu gitu, Bu. Kalau mau berfikir tapi suasananya itu berisik gitu rasanya mengganggu sekali. Sit-1-2-A- disturbed

Excitement: achievement

Ya ada, tapi kalau sudah biasanya kalau sudah selesai tesnya itu baru lega gitu lho Bu, rasanya. Sit-1-3-A-content

Dissatisfaction: score, competition

Ya masih kurang puaslah wong bijinya apa nilainya itu belum bagus, belum semaksimal, belum seperti teman-teman yang lancar berbahasa Inggris. Sen-1-3-A-not satisfied with the score.

103


Stage 4. Connecting categories available in the table to find major categories (themes) and minor categories, At this stage, all the minor categories and indicators emerging at the previous stage were gathered and classified. It results in the thematic organization of data as shown by Table 4.12. Table 4.12 Emerging themes and categories from student participants’ narratives and behavior Major Categories Emotional Experience

Learning Experience

Minor Categories reluctance anxiety tensity fear doubt sadness confusion trauma dissatisfaction disappointment relief certainty surprise excitement learning motivation learning process learning feedback learning enforcement learning achievement learning autonomy

Stage 5. Corroborating evidence, which is procedure in which we think critically about the quality of the data (Becker&Geer in Dey 1993:224) by collecting evidence so that the quality is ensured. It employed the instrument validation or accountability as already discussed in the previous section. Being a qualitative study, corroborating and validating was done throughout the data collection, analysis, and write-up process (cf. Johnson, 2008:11). However, for the clarity and openness, I wish to show what happened in this particular stage, where I closely observed the themes and categories emerging from the students’ account and validated them with data from other sources, such as teachers’ account, observation and my own diary.


A student participant’s narrative, take for instance, Kalau saya pribadi lebih menyukai soal-soal seperti TOEIC. Jadi listening, dan apa namanya, grammar, reading dan seperti itu. Ya, speaking itu juga penting, ‘tapi e kan untuk membuat dialognya itu agak malas, ‘gitu. Dan saya juga pernah agak trauma. Karena pertanyaan menyimpang jauh dan membuat saya bingung menjawabnya. Tapi setelah itu saya jadi takut, Bu. Terlalu banyak pertanyaan itu saya takut. (Student Atik) [Personally I prefer something like the TOEIC test. So something like listening or what is it called….? Grammar… reading… and the like….Well, speaking is also important, but er…. It is not fun to construct the dialog. And I am traumatic too as the teacher’s questions in the speaking test were not relevant at all and made me confused to answer. Since then I have become scared. Too many questions made me afraid]

was validated by the data from a teacher’s account: Kalau menurut pengamatan saya selama ini yang disukai siswa itu juga merupakan tes tertulis. Kalau tes performance speaking, mereka itu biasanya nervous atau mungkin tidak tidak percaya diri untuk me mengikuti tes tersebut, jadi mereka lebih senang kalau tesnya tes tertulis. Kadang-kadang ya itu protesnya itu kalau saya memberi pertanyaan tambahan yang menurut mereka dianggap mereka terlalu susah karena mereka tidak siap sebelumnya dengan tes speaking, jadi kadang-kadang saya mesti kasih pertanyaan tambahan sebelum saya kasih comment. Saya kasih pertanyaan tambahan anak ini betul-betul tahu atau cuma hanya sekedar apalan gitu lho, nanti akan kelihatan. Kalau sing kalau yang di rumah cuma cuma betul-betul menghafal saja ya tanpa mengetahui konten dari suatu modul mungkin jadi kalau dikasih pertanyaan tambahan mereka sering kaget, na itu kadang-kadang protesnya di situ. Kok jadi melenceng, katanya mereka. Pada hal itu kan cuma untuk me mencari tahu anak ini siap atau tidak, tahu betul atau tidak. (Teacher Iin) [As I observe, what students enjoy is the written test. They are usually confused and may be not confident to take the speaking performance test. In short, they prefer the written tests. Well, sometimes they protest if I ask additional questions which they consider too difficult as they are not ready yet before the speaking test. I feel I have to ask additional questions before commenting on their performance. It is to ensure whether the student can really speak or just murmur without a good understanding. It will show that those who have practised memorizing without understanding the content of their module will feel shocked when I give additional questions. And they stage a protest. They say, ”Why is the question out of track?” I argue everything I do is to find out this student is ready or not, really know or not]

together with my observation A guidance and counseling teacher contacted me and required information about the implementation of sub-competency English test for the thirdgraders. She exposed the problem that there were students who would soon try to get a university’s admission. One of the requirements was to submit their schoolreport’s copy. Since they had some blanks or no marks in a number of competencies, she told me that one of them protested. In my opinion, they were afraid their application would be rejected. (my diary)

and by the student’s copy of school report (as shown by Appendix 9.5).

105


Stage 6. Producing an account by drawing diagrams, tabulating tables and writing texts and incorporating them into a coherent whole This activity marked the end of data processing. It resulted in: 1. The portrait of individual student participant in English testing and learning consisting of emotional experience and learning experience 2. Themes in English testing, which in addition to the prefigured themes test backwash, qualities, test content, learning, autonomy, includes status of test, fairness, time allocation, achievement, competitiveness, and preparedness 3. My interpretation of what testing means to the students as well as teachers of the vocational school which at a certain point clearly shows a point of clash.

E. DISCUSSION: DESCRIPTION AND INTERPRETATION The discussion includes (1) the description of student participants’ experience in English testing and learning, which not only shows relationships among participants’ narratives, but also other support data,

(2) thematic

description and illustrative quotations, which present relationship among participants’ narratives and behaviors based on prefigured and emergent themes, and (3) my interpretation about the participant’s meaning of testing. 1. Student participant portraits in English testing and learning a. Student Atik Student Atik (SA) was born to a family working as elementary school teachers. She was categorized as a student with achievements. (See a copy of her school report at Appendix 9.5). She always scored high in any high-stakes tests although she had a difficulty in oral communication (speaking).


When interviewed, SA was in the third grade. Realizing that learning should bring enjoyment, she did not allocate a particular time in her daily life. In her view, if we were compelled to do it, in turn we would hate it. She preferred studying on her own with the reason that having a companion meant having a chat and made her forget to study. She knew that in learning English, she could do it through TV. She pointed to the fact that the national TV, TVRI offered a lot of English educational program. In learning reading, she consulted her dictionary to find the meaning of the words she encountered. To sharpen her listening skill, she watched foreign films at high volume. She wanted to have a listening skill which would make her understand what was being spoken when watching TV programs. She learned it from TV. She realized that she could not speak well yet. For her, to master English needed good theoretical foundation, knowledge of grammar, vocabulary and listening. In her view, speaking came later when the listening skill was acquired. She believed in speaking we had to think of the grammar, instead of merely producing words. In learning English she expected to have a speaking skill with grammatically correct sentences, instead of a merely understandable talk. To achieve the goal, she said she was only learning the theory then. She had not learned to practice yet. She had a belief that tests brought influence to the way one learned and motivated to study. Her style in learning English was also applicable for competency test (speaking test). To prepare for the test, what she did was reading the module she owned. She told that in the competency test, she had


to compose her own answers to do the task. She read the sample dialogs available in the module. In addition, she asked her friends who had already taken the tests. In relation with the competency test, she shared her experience. She said she wondered with her score. She felt surprised to get a good mark although the assessor had criticized a lot. She also wondered that she found an unsatisfying mark although the assessor gave no comment on her performance. Realizing her potential and weaknesses, she was not interested in taking her classroom speaking tests which at that time were not compulsory for the students in the higher levels. She said she preferred TOEIC test and the like, which tested listening, grammar, reading. She knew speaking was also important, but it was not interesting for her to prepare for the dialogs. The reason was that there was no obligation to take such a test. As informed earlier, there was a possibility to graduate without taking all the competency tests. The other reason was that there was no extrinsic motivation. She pointed out that she was so relaxed and not motivated to take the test as a result of her friends’ influence. She was not challenged as no friend made higher achievements and she has no one to ask. Another important reason was that she felt traumatic in taking the test. She did not think that the teacher’s question was relevant, which made her confused to answer. Further she explained, in the test with the topic of “describing people”, she was asked a question what she thought of her school principal and was asked if she dared propose something to him, which she


thought had nothing to do with the test topic. This experience of having too many questions to answer, which she thought were not relevant, made her scared to take the similar test. She experienced a difficulty in proposing for university admission designated for students with achievement as she had blank spaces in her schoolreport. As a consequence she was required to take the equivalent competency test. With this experience,

she showed objection with the

competency test implementation. In her opinion, since the beginning, such kind of a test should be made obligatory with the tight scheduling. She knew that speaking should be trained, and the test should be scheduled. To prepare for TOEIC test, as she knew what it would test, she learned materials which she believed to encounter. According to her, such a test was important since it allowed us to know our level and the certificate would be useful for career development. Before taking a test, there was some anxiety in her, but she had only a low degree of anxiety as she believed the materials would not be difficult and they would be within students’ reach. Nevertheless, as compared to what she had experienced in her previous school, SMP, the degree of anxiety was higher, since the materials had been simpler. Before taking the test, she increased her learning time, but she did not do it all night long. She expected to have a calm feeling and atmosphere during the test implementation In the last try-out test, she felt disturbed by those who had left before time was called. She thought they should have stayed in class to double-check the work. She did not think that the supervisor disturbed her. In


the try-out test, she felt relaxed as she thought she could do the test although she felt confused by the tryout test, which presented too many sentences in the item. She thought that TOEIC test was much simpler. Nevertheless, in the TOEIC test, she felt sad, dizzy and having a high burden as she had a lot of difficulty and could not finish it so that she was afraid to get a low mark. Nevertheless, she thought TOEIC was more impressive than the tryout test. In the last examination, she felt nervous before doing the test. She felt nervous as she thought she would have difficult questions. But as she read the items and found them easier than those from the previous examination, she felt relieved. She also found the listening materials very clear although she failed to understand one. b.

Student Siti Student Siti learned English in order to understand English well and

to get a job. She thought speaking, listening, reading was important. She usually studied on her own. She did not do it with friends with the belief that studying would be harder with a friend around since she would not be able to focus on her concentration. She was afraid that studying with a friend would lead to a chat instead. When first interviewed, Student Siti was in her second grade. As a vocational school student, her experience in English testing included competency tests at the end of studying with the modules and summative test with written texts. She said she hardly learned at home and learned only accidentally. She played more. When taking a (speaking) competency test, she looked nervous


because she was not prepared. Nevertheless, she did not feel the same when taking the written test. She said it gave opportunity to read, understand and did correction. This was different from what she experienced in the speaking competency test, in which she was thinking about how to pronounce the words and what if she was making a mistake. Nevertheless, she confessed that she felt content when she finished the test. In her opinion, the competency test was good to motivate students since she felt challenged to compete with friends. In addition, the material was also relevant with daily life and not difficult. She held a belief that English testing could increase students’ knowledge. She also said that the written test was also acceptable as the material had been learned and students did not have any difficulty in doing it. In her view, TOEIC test was good since it could measure one’s ability in listening and understanding what the speakers said and it had a reading section which measured how we found an answer from texts and paragraphs. In the speaking competency test, she would prefer a quiet situation. She would feel disturbed by others’ conversation when she was speaking. She would be attracted to hear what others were saying. In the quiet situation, she would be able to think clearly: she could think when it was noisy. Nevertheless, she once experienced anxiety when taking a test in a quiet situation. This was due to her worries about failure. Before the competency test, she tried to study on her own, practiced pronunciation and asked friends for checking grammar and pronunciation and did her correction. She consulted a dictionary when she found


difficulty in understanding the meaning. In learning English, she tried to read the module before the test, and the night before the lesson. She felt sorry and sad to learn she failed in a test although she had tried to do her best. In her reflection, this failure was due to lack of studying. She knew she should have studied at least one hour a day. In a high-stakes test, she would say nothing when her friends were making a noise. She would prefer the supervisor to remind the noisy students to keep quiet and do their own. She hoped no friends to disturb her. She confessed some friends asked her to inform the answers during a test, and only answered when she knew the answer. In her last test (TOEIC test), she found the test difficult although at home, she had learned to do practice test for hours the night before. According to her, the time allocation to do each listening test item was so limited that she almost had no sufficient time to mark her answer. It really made her dizzy and forget the previous question in order to hear the next number. She said her nervousness in taking the test was due to her feeling of uncertainty about: success or failure. She said she had learned the test materials. As the test ended, she thought she had not done the test well and wondered if she could pass. She was also eager to know is she was successful or not. c.

Student Endang Student Endang’s goal in learning English was to be able to

practice, speak in English and compose English words correctly. She was


aware, however, she did not study English intensively enough. She had other lessons to learn, but she still put English on top priority. She learned English once in a while and learned it little by little. When learning, what was important to her was that she could achieve a high degree of understanding. When first interviewed, Student Endang was in the first grade. She said she enjoyed her high-stakes test and other educational tests. In her view, the high-stakes test enabled students to learn independently and trained our hearing to be sensitive to catch spoken English words and learn English quickly. In her expectation, she wanted the test to train her to use English fluently and enable her to use everything well. She had also undergone four classroom speaking tests, which she also liked. Prior to a speaking competency test, she prepared with a text and learned it by heart (memorized it), although she knew there were many erroneous uses of tenses. In studying, she always read any handout she got and when she found a word of which meaning she would like to know, she consulted a dictionary and sometimes asked friends. But overall, she learned on her own at home. She also needed her friends’ assistance to do corrections. She studied less than 1.5 hour every day. What she usually did was reading the modules. She described her speaking competency test implementation as noisy: friends were learning to prepare for the test and propose for a test to their teacher one by one. In such a situation, she did not feel so confident with her skill, being afraid of making mistakes. Nevertheless she felt all


right and undisturbed. She felt tensed because she had to compete with her friends to get her turn. Those who could work faster would go to the teacher first. As the test ended, she felt dissatisfied with her score. She understood she could not speak as well as her friends. Even so, she found such a method acceptable. In her high-stakes test, she admitted that she made speculation in marking the answers. (See a copy of her work in Appendix 9.2). The reason was that she did not master English well, especially the vocabulary. The words were too difficult for her to understand. During the test, she got tensed and shocked as she was not accustomed to listening. Such indicates she tried to do her best to answer the questions. She did not have a particular target: for her, what was more important was she became more familiar with English through the test. There was a possibility she would do speculation again next time, depending on the situation. In her opinion, all kinds of tests she had taken were good, but vocabulary became her main problem, especially in the last promotion test. She said her score was not satisfying. As she confessed, this was due to the low intensity of her learning. In the summative assessment, she had to take a makeup test. She told she had found gerund difficult but said the test material was all right. When the result of test was announced, she was first shocked but later she realized she had not done the test well. She did not expect to have a makeup test, but she knew she had to do it. She then prepared for the makeup test by trying to recall where she went wrong and finding the


meaning of difficult words. She did not think of the score she would get from the makeup exam. With her achievement, she realized she needed to improve her learning. So far she only borrowed her friends’ notes: she did not try to borrow learning materials from school which were available in the laboratory with the reason that she had no companions to do it. In the provincial tryout test, Student Endang scored

lower

than the national passing grade. For this reason, the school provided her with an extra English class for special treatment. Nevertheless, she did not take the opportunity. After the national exam, she reported she scored a 7, which was above the national passing grade. For her absence during the previous extra class, she argued it was because she could not stay at school all day long, telling that she was afraid there would no passenger service in the afternoon. 2. Thematic description and illustrative quotations a. General description of emergent categories All the student participants in this project were exposed to English testing in their learning environment. Students took classroom tests (See Appendix 9.6) and high stakes tests in the form of discrete point tests (See Appendix 9.7). The classroom speaking competency test items had been released before the student performance was assessed, thus they allowed the students to prepare to compose their answers. The high stakes test items had always been kept confidential before being presented to students. Both types, as this research indicates, brought with them their physical as well as


non-physical attributes which were practically influential to the lives of the research participants as they became the major testing environment. Referring to the observation blueprint, the field notes show the physical as well as non-physical components around the time of the classroom and high-stakes test are not identical (See Table 4.13). Table 4.13 Components, categories and indicators constituting environment around English testing implementation No. 1.

Test Type Classroom test

Component Physical

Non-physical

Category class population room size light facilities test material external concerns atmosphere

Indicator densed spacious abundant test evaluation sheet speaking and writing nature of measurement noisy less formal, relaxed

feedback

score teacher’s comments

2.

High-stakes test

Physical

class population room size light facilities

sparse spacious abundant test booklet tape-recorder

Non-physical

test material

listening and reading

concerns

graduation rate achievement competitiveness

atmosphere

quiet formal serious

emotion

terrified worried grief sympathy

strategy feedback

clarification on the rule of the game score rank

116


The participants’ narratives, behavior and documents - in addition to the prefigured themes test backwash, qualities, test content, learning and autonomy - produced categories as shown in Table 4.14. Table 4.14 Emergent categories from primary and secondary data from primary data sources learning motivation learning process learning feedback learning enforcement learningachievement learning autonomy status of test fairness time allocation achievement competitiveness preparedness level of difficulty test content reliability validity test backwash guidance certificate

117

from secondary data sources status of test attitude concern about quality improvement passing grade reliability validity fairness honesty self evaluation subsidy concern about graduation rate technical matters less concern with capability manipulation cheating leakage speculation demonstration clarification on the rule of the game situation replica test practice availability appropriate strategies in doing test concern about achievement policy competitiveness preparedness assembly special management team spirit to solve the problem seriousness grief sharing test content level of difficulty time allocation high expectation evaluation speculation the cause of test result assistance assurance autonomy


I would like to closely depict a high-stakes test since this type of test was richer in data which might affect students’ lives. The themes available here were generated from secondary sources. The four major themes with categories and indicators around were concerns about quality improvement, graduation rate, achievement and test status. The notion of reliability, validity, fairness, honesty, passing grade, subsidy and self evaluation showed concerns about quality improvement. Technical matters, test practice availability, leakage, speculation, special managerial team marked the existence of concerns about graduation rate. The existence of competitiveness, preparedness and policy indicated concerns about achievement. Test status brought with it attitudinal indicators such as grief, anxiety, seriousness, disappointment and sympathy. I illustrate this with Figure 4.3.

quality improvement

test status

Highstakes test

graduation rate

achievement

Figure 4.3 High-stakes test and thematic description, gleaned from secondary data sources b. Students’ emotional experience


1) Anxiety The most common emotional experience among the participants was anxiety. Anxiety in testing resulted from various factors, ranging from unpreparedness, making mistakes, competitiveness, confusion and noisy situation. This happened when they were doing both types of test, their classroom test as well as high-stakes test. Mungkin karena belum siap, belum terlalu siap, gitu. […] kalau yang reading itu kan bisa dibaca lalu dipahami, kayak gitu. Jadi kita itu nggak terlalu nervous. Kalau speaking itu kan nanti bagaimana pengucapannya kalau salah terus berfikir gitu lho, Bu. Kalau yang reading kan nanti kalau tulisannya kan bisa diperbaiki lagi, gitu kalau reading. [Maybe it is because I was not ready, not well-prepared... In the reading section, we have enough time to read and understand, something like that. So we do not feel so nervous. In the speaking test section, we wonder how to pronounce the words, keep on thinking what if we go wrong, like that Miss. In the reading test, say we are miswrite the words, we have time to correct them.]

(Student Siti, after taking a speaking test) Ya kurang PD gitu, kan masih takut kalau punya kita itu masih ada yang salah gitu. Takut kalau disalahkan teman-teman gitu. Ya, tapi masih biasa kok, belum terganggu. Deg-degan tegang. Ya soalnya majunya itu harus dulu-duluan gitu lho, Bu. Tidak, tidak apa urut gitu tidak. Siapa yang cepat dia itu langsung maju, yang bisa langsung njawab, gitu. [Well, not so confident as still afraid of making mistakes. Feel afraid of being blamed by friends. But it is OK, not disturbing. Tensed... because we must compete with each other to propose for the test. We are not called one by one according to the student numbers. Instead, those who master the lesson earlier can propose for the test.]

(Student Endang, describing her speaking test) Dan saya juga pernah agak trauma. Karena pertanyaan menyimpang jauh dan membuat saya bingung menjawabnya. Tapi setelah itu saya jadi takut, Bu. Terlalu banyak pertanyaan itu saya takut. [And I have a traumatic experience. It is because the questions were not relevant and made me confused to answer.]

(Student Atik, describing her speaking test experience)

119 Kalau suasananya tenang, e suasana perasaan cukup tenang itu sudah baik. Tapi yang saya tidak suka kalau ada misalnya pas penjajakan waktunya sudah selesai, tapi teman-teman sudah keluar mengganggu


konsentrasi. [If the situation is calm, so is my mood, that’s good. But what I hate is like this, in the previous tryout test, those who finished the test earlier went out of the class and destroyed my concentration.]

(Student Atik, talking about her tryout test) Awalnya nervousnya itu takut karena tidak bisa mengerjakan karena membayangkan soalnya akan sulit. [Nervousness starts as a result of being afraid of failure to do the test and thinking that we will find it difficult]

(Student Atik) “Perasaan saya tadi waktu mengerjakan tes agak nervous ya, karena menurut saya ya gimana ya, soalnya cukup sulit walaupun sudah belajar semalaman tap sepertinya masih agak sulit, Bu.” [I felt a bit nervous as it seemed to me the test items were so difficult although I had studied all night long, Ma’am.]

(Student Siti, after taking TOEIC test) 2) Tensity Tensity is due to competitiveness and new experience. Student Endang told how she felt during her classroom test Deg-degan, tegang, gitu. Ya soalnya majunya itu harus dulu-duluan, gitu lho, Bu. Tidak, tidak apa urut gitu tidak. Siapa yang cepat dia itu langsung maju, yang bisa langsung njawab, gitu. [Feeling my heart beat, tensed. It’s because we compete with each other to finish the test, instead of doing it like in a queue. Whoever can learn fast, they can propose for the test and answer the questions.]

and during her high-stakes test, “Ya, tegang, kaget, gitu, lah, Bu. Untuk mendengar gitu, belum biasa.” [Yeah, tensed, shocked, something like that, Ma’am. Not accustomed to hear that way...] 3) Dissatisfaction Dissatisfaction resulted from low achievement (low score). Ya masih kurang puaslah wong bijinya apa nilainya itu belum bagus, belum semaksimal, belum seperti teman-teman yang lancar berbahasa Inggris. [Not satisfied yet as my score is not good enough, not maximal, not as good as what my classmates who speak English fluently can get.]

(Student Endang)


Setelah penjajakan ya mungkin perasaan saya agak tenang, karena saya merasa agak bisa, tapi kalau seperti tadi. Ada rasa sedih, ada rasa di sini pusing, di sini rasanya berat sekali, karena tidak bisa. Karena belum selesai mengerjakan, takut nilainya jelek. [I felt a little relieved after taking the tryout test. It is because I felt I could do it. But what I experienced just now, there was sadness, a headache, and a big burden here, because I could not do it well. As I did not finish it, I am scared if I get a bad score.]

(Student Atik) Kecewa karena prediksi saya hanya satu yang salah. Dan prediksi saya kan hanya satu. Yang listening itu saya jelas-jelas saya lepas, gitu, sama sekali kok nggak dengar kata-kata itu. Yang reading itu kan itu saya yang ragu-ragu itu satu. Tapi sudah saya tanyakan sama guru Bahasa Inggris dan e beliau, guru Bahasa Inggris itu juga membetulkan jawaban saya gitu. Tetapi ternyata salah dua. [...] [Disappointed because I predicted I had only one mistake. I am sure one answer in the listening section is wrong as I had never caught the words. In the reading section, I had a doubt in answering one question. But I asked it to an English teacher and the teacher confirmed that my answer was correct. But I found then that I had two mistakes.]

(Student Atik) 4) Reluctance Reluctance to take a specific test might result from the test content and extrinsic motivation. Kalau saya pribadi lebih menyukai soal-soal seperti TOEIC. Jadi listening, dan apa namanya, grammar, reading dan seperti itu. Ya, speaking itu juga penting, ‘tapi e kan untuk membuat dialognya itu agak malas, ‘gitu. Dan itu ‘kan seperti dari awal itu seperti tidak wajib. Jadi, katanya, walaupun tidak selesai tetap lulus, gitu Bu. […] Kita masih santai: teman-teman juga masih A5. Dan itu juga pengaruh teman. Kalau temannya nggak tinggi ya ‘nggak. [ I personally prefer TOEIC-like test. It tests listening, and what is it... grammar, reading and the like. Well, speaking test is also important, but I feel reluctant to prepare the dialog. And it seems that it is not compulsory to take. I heard that although we cannot finish them, there is an opportunity to pass, Ma’am.... We relax, the others are still studying A5. And it’s also friends’ influence. As the others are not a higher level, I don’t (take it)]

(Student Atik)

121 Student Atik preferred another type of test which provided the materials she were studying. This was due to her past experience and


in line with her own concept of learning. In addition, the existence of awareness of her own capability was contributive to this. E, misalnya kemampuan Bahasa Inggris itu ya harus didasari dengan teori yang bagus, harus dengan pengetahuan tentang grammar, tentang vocab, kemudian dengan listening. Setelah listening baru speaking. Dan untuk saat ini saya untuk speaking belum cukup mampu. […] [Er...say, English communication ability must be laid on a strong theoretical foundation, must go with a knowledge of grammar, vocabulary, and listening. After listening is mastered, there comes speaking. And for this time, I still cannot master speaking]

(Student Atik) Selain itu speaking tetapi tidak hanya speaking seperti penjual yang melayani turis. Itu kan speakingnya waton dimengerti ‘gitu lho Bu. Saya ingin speaking yang grammarnya juga betul, gitu. [Moreover, the kind of speaking mastery is not that of a seller who is serving a tourist. That kind is merely understandable, Ma’am. I want a capability of speaking with correct grammar.]

(Student Atik) 5) Confusion Confusion was due to test validity and reliability. The student and teacher surprisingly did not share the similar perception of test validity and reliability. While the teacher was trying to increase the level of the test validity and reliability, the student thought that the test items were not relevant to the ability the test intended to measure. Student Atik said, “Karena pertanyaan menyimpang jauh dan membuat saya bingung menjawabnya. […]”[” ... as the questions were much irrelevant and made me confused to answer.”

122 6) Excitement This resulted from a sense of accomplishment and


achievement. Having an opportunity

to take a high-stakes test

brought a joy to Student Siti. It was considered a reward to the success of her learning achievement. She described her excitement, “[...] waktu mengetahui saya ikut tes ini itu sebenarnya ya senang ya, Bu, ya. Karena tidak semuanya bisa ikut.” [”... as I learned I was invited to take this test, I felt excited, Ma’am. You know not everybody could join.] She

did not even care about

what score she would get from the test. She confirmed, Saya ya lebih senang kalau yang seperti ini karena kita itu bisa sedikit bangga, wah saya ternyata bisa ikut tes yang nggak semuanya bisa ikut [laughing] Walaupun nanti tidak tahu hasilnya seperti apa. [ I prefer a test like this as we can feel proud, ”Wow I had an apportunity to take a test while not everybody could join[laughing]...” although I did not know how high the score would be...

7) Doubt It was due to the test materials and content. Nevertheless, doubt led to curiosity. Again, Student Atik said, Maksudnya itu masih bingung saya. 24 jam buka terus atau dalam 24 jam jadi, gitu lho. Saya bingung. Kalau yang reading itu saya hanya ragu-ragu satu itu, tapi entah pada akhirn ya itu yang salah atau bukan, saya tidak tahu. [I was not sure what it means.... open 24 hours or finished within 24 hours. I was confused. In the reading section, there was only one item in which I was in doubt ; but I don’t know it was the one where I was wrong or not.]

She added, “Tapi sudah saya tanyakan sama guru bahasa Inggris

dan e... dan beliau guru Bahasa Inggris itu juga

membetulkan jawaban saya itu.” [“But as I asked an English teacher, he/she said my choice was correct.”] 8) Relief


Worries and anxiety were reduced by the lower level of difficulty and achievement, thus leaving a relief. Student Atik told her experience in a high-stakes test, “Ya perasaan nervous itu sedikit terkurangi karena saya melihat bahwa soal ujian nasional itu malah lebih mungkin

mudah dari pada soal-soal ujian nasional ya

tahun-tahun lalu.”[Well, I felt relieved as I found the

national test items easier than those from the previous year.]. Similarly, Student Siti described her experience in her classroom test,”Ya ada, tapi kalau sudah biasanya kalau sudah selesai tesnya itu baru lega gitu lho Bu, rasanya […]” [ ... well there was anxiety, but when it was done, I felt relieved.”]. Observation data of events and behavior from test location as Appendix 4.1 shows confirm what Siti expressed. 9) Disappointment Disappointment was mostly caused by lower achievement, as Student Atik clarified, “Kecewa, karena prediksi saya hanya satu yang salah [...] Tetapi ternyata salah dua...” [Disappointed... I predicted I had two erroneous answers but I find I had two...”] 10) Inconfidence Inconfidence occured in relation to potential failure. Student Endang said, “Ya kurang PD, gitu, kan masih takut kalau punya kita itu masih

ada yang salah, gitu” . [“Inconfident, worried, in

case I will make a mistake”] . 11) Sadness


It was due to

failure to finish the tasks and

lower

achievement. Student Siti revealed, “Ya sedih aja kok nggak lulus padahal sudah berusaha.” [“So sad I didn’t make it although I had tried. Another student said, Setelah penjajakan ya mungkin perasaan saya agak tenang, karena saya merasa agak bisa, tapi kalau seperti tadi, ada rasa sedih ada rasa di sini pusing, di sini rasanya berat sekali. Karena tidak bisa. Karena belum selesai mengerjakan, takut nilainya jelek. [After the tryout test, I felt all right as I could do it, but what I just experienced, there was sadness, dizziness, and a heavy burden. It was due to my unability. As I failed to finish it, I was afraid I would get a bad score]

(Student Atik, after a high-stakes test) c.

Students’ learning experience 1)

Learning motivation Student Atik’s story revealed that learning motivation was

triggered by the familiarity of test format and content, competitiveness and test result. She was familiar with what her classroom tests and high-stakes tests measured and she was aware each offered a different degree of competitiveness and reliability on the way the work was scored. As she knows the test format and materials, she knew what to do and was motivated to learn. Curiosity to the test problem she experienced during the test also arouse her learning motivation by discussing it with her friends and teacher. Student Siti’s story tells that learning motivation was triggered by the competitiveness the test brought to the class. The result of the test which showed her level of competency motivated her to learn more. She did not want to be considered as underachiever


as compared to her friends. Student Endang was motivated to learn by her awareness of the goal of studying English and her achievement in the test. When her achievement was lower, she made an evaluation. She recalled what she was unable to do in the test and tried to get a feedback from her study materials. 2) Learning process Student Atik’s story shows that her learning process was influenced by mood and learning style. She could only learn when there was a will to study and this was done with her preference in the way she studied prior to the test. The content and level of difficulty she encountered during the test brought her to a moody situation which consequently either facilitated or slowed down the process of learning. Student Siti’s learning was chance and

plenty time to

facilitated when there was a

do revision.

As compared to

competency classroom test and the listening section stakes test;

the reading section

the

of the high-

in the high-stakes test was more

contributive to her learning process. Situation

around

the test

implementation was also said helped her to think clearly. For Student Endang, each occasion in English testing was English learning. Student Endang’s learning process was facilitated by a considerable amount of digestible test material, therefore excessive incompatible test content had little meaning in her learning process.


3) Learning feedback Student Atik got her learning feedback from test content, level of difficulty, test score and teacher’s comments. She told that the learning feedback from the teacher related to her performance int the test was oftentimes surprising . Student Siti and Atik shared a similar way

in getting

feedback. They preferred to study on her own. Siti told that studying with friends would be less successful. Nevertheless, they obtained their learning feedback from discussion with friends prior to, and after the implementation of the classroom as well as highstakes tests. 4) Learning enforcement Student Atik ‘s got her learning enforcement from her score in the high-stakes tests. She believed the high-stakes test score which was based on objective judgement was much more reliable and therefore represented her achievement. As a result, she felt secure with it and the test was influential in her lives since it offered stronger enforcement to her learning. Student Siti’s learning was enforced by test materials with a considerable level of difficulty, relevant with the lessons she had learned and daily activities. Student Endang learned English words by consulting

her

dictionary. She was shocked by many unfimiliar words in her first high-stakes test. Yet, this experience was valuable since it became


her learning enforcement. 5) Learning achievement Student Atik’s learning achievement was much supported by the intensity in studying, and influenced by the clarity of the rules of the game in the test. She believed in a high-stakes test with objective scoring to measure her learning achievement. A feeling of relief and joy marked Student Siti’s learning achievement. It unnecessarily resulted from a good score. A sense of accomplishment for finishing the test, excelling as a learner from her group were her learning achievement. Student Endang did not worry about her score although she knew it was below her friends. She emphazised on her increased understanding in English as an indicator of her learning achievement. So to her, every little new thing in English testing would be contributive to her learning achievement. 6) Learning autonomy Student Atik has a high sense of learning autonomy. She has critical reflection on what learning situation should be. In her view, learning English should be enjoyable. There should be enjoyment in learning English. She has independent action to fulfill her needs, to make her own decision on what and how she should learn. She knew that testing was contributive to her learning. Student Siti was aware of her goals in learning English

and

how she fulfilled her needs. She knew in order to concentrate her


thought she needed calmity. She knew quite well that English test could satisfy the needs and improve learning. Student Endang’s learning autonomy was characterized by her awareness of the way her learning proceeded and her freedom and capacity to control

her learning. She was also aware of her

goals in learning English, yet she did not worry about scores and target. She saw that every English testing occasion was English learning. 3. Participant’s meaning of testing Questioned about what a test actually is and for what purpose a test serves, the two teachers responded that it is a means of measurement. Teacher Nanik explained, “Tujuannya antara lain mengetahui, mengukur, seberapa jauh kemampuan siswa dalam menguasai materi yang sudah disampaikan atau sudah diterangkan”which means that the purpose of a test is to find out, measure how deep the students master the lesson delivered and explained. Teacher Iin said, Tujuan tes itu kan untuk mengevaluasi siswa, untuk menilai siswa sejauh mana materi yang diajarkan itu ditangkap oleh siswa dan bisa kalau bisa diaplikasikan dalam kehidupan sehari-hari. [The purpose of a test is to evaluate students, to assess how wide the delivered lesson can be understood and applied in daily activities]

In addition to a means of measurement, it is necessary to note that Teacher Nanik also perceived a classroom speaking test as a means to train students in speaking. Teacher Nanik told: Kalau speaking itu sebetulnya anak-anak... kalau pas dialog mungkin dia ya senang, disamping itu juga melatih kemampuan berbicara juga kadang apa namanya rileks. Kalau pas yang tertulis ya writing yang masuk tenses-tenses dia nggak bisa rileks harus terfokus soal-soal. Kalau speaking kan kita kadangkadang bisa tertawa, bisa apa pun, misalnya tangannya dan lain sebagainya.


[Actually, in the speaking test, having a dialog is enjoyable to students... Besides training them in speaking, it enables them to do the task relaxedly. When they are working on a written test and dealing with tenses, it is impossible for them to relax; instead they must focus on the test items. In the speaking test, they can laugh and do anything for example with their hands...]

Thus to her, as we observe, a test also means learning facilitation. Student Atik agreed that it is a means of measurement “[…]mengetahui kemampuan seseorang, mengetahui prestasi seseorang sejauh mana memahami materi yang diajarkan” which means ”to find out one’s ability, their achievement, to what extent they understand the teaching materials already delivered”. Similarly, Student Siti told that a test measures her ability. Talking about a high-stakes test, she said: Menurut saya bagus karena bisa mengukur kemampuan kita dalam berbahasa Inggris. Karena di dalam tes itu ada listeningnya, berarti bisa mengukur bagaimana pendengaran kita dan memahami apa yang dikatakan oleh speaker, misalnya lalu ada tes reading itu bagaimana kita menemukan suatu jawaban di dalam teks atau e alenia-alenia atau paragraph yang ada di situ, gitu Bu. [In my opinion, it is good as it measures our mastery in using English. As the test includes listening material, it can measure how well our hearing can catch what the speaker is saying. There is a reading test where we should find an answer in the text or paragraph.]

In addition to a means of measurement, Student Atik and the other two students, surprisingly, also considered testing as something else, i.e. learning. Student Atik said that it is a means to plan one’s study: Untuk e … merencanakan mungkin cara belajar. Mungkin kalau nilainya terlalu rendah, jadi dia kan harus belajarnya semakin giat, harus sangat ditingkatkan. [It may be to plan the way we study. Say, if their score is too low, they must study harder and improve it.]

and highlighted the function a test serves: [...]kalau tidak ada tes seperti itu semangat belajarnya kurang, karena tidak ada apa ya e hasil yang konkrit, jadi tidak diketahui saya itu misalnya kalau tes terus nilainya sembilan, Kan oh berarti saya cukup bisa, kalau nilai enam berarti saya nggak bisa. Kan saya berarti saya kurang dong belajarnya, berarti saya harus belajar lagi. Tapi kalau tidak ada tes seperti itu kan kita nggak tahu. [...if there were not such a test, we would not be motivated in studying, as there would be any real result. We wouldn’t know anything like when I get a 9


indicating I master the lesson or when I get a 6 indicating I don’t, meaning I should study harder. But if there weren’t such a test, we wouldn’t know it.]

From this we see that to Student Atik a test is her learning motivation as well as her learning feedback. Similarly, Student Siti responded, Ya menurut saya kalau uji kompetensi yang modul-modul itu bagus, bu karena bisa memotivasi siswa gitu, bu. Wah kalau teman saya sudah berapa gitu levelnya njuk saya kok masih di sini, jadi memotivasi gitu lho bu, agar belajarnya lebih ditambahi lagi, gitu. [Well, in my opinion, the competency tests related to the modules are good, Ma’am, as they motivate us to study. If I see my friend is on a higher level and I stay on the same lower level, the tests motivate me to study more.]

She added that an English test is to broaden the students’ knowledge, ”Menambah pengetahuan siswa”. It indicates that the meaning of a test to Student Siti is triggering her learning motivation and facilitating her learning process. To Student Endang, a test meant a learning opportunity, or in a wider sense, it was accomodating and facilitating her learning process. Here is what she expressed, Misalnya tes TOEIC atau tes apa itu pendidikan yang lainnya itu. Tes TOEIC itu bisa untuk kita supaya berlatih mandiri untuk belajar Bahasa Inggris supaya tidak tergantung kepada teman dan melatih pendengaran kita supaya kita itu peka terhadap perkataan dengan menggunakan kata-kata Inggris yang tidak secara tulisan, secara lisan. [For example TOEIC test or the other educational tests. TOEIC test encourages us to study English independently and not to rely on friends, trains our listening ability so that we become sensitive in catching what others say orally, something spoken, instead of written]”

Viewing a test as a measurement, Teacher Iin and Nanik were deeply concerned with the test variables reliability and validity. In a particular classroom speaking test, Teacher Iin tried hard to modify the available questions and tasks. She spoke her mind: [...] Jadi kesannya siswa yang pinter sama yang tidak itu dalam speaking test


kelihatannya sama. Karena apa? Karena mereka cenderung menghafal, mencatat di rumah ya. Terus kalau tidak dikasih pertanyaan tambahan jadi tidak jelas perbedaannya antara siswa [...]. Kalau yang speakingnya bagus ya itu bisa menjawab panjanglah, istilahnya apa pun yang ditanyakan gurunya bisa respon di dalam maksudnya kasih respon yang betul. Kalau yang tidak bisa speaking ya biasanya kalau dikasih pertanyaan kaget, terus misalnya pertanyaan A, jawabannya Z, biasanya seperti jauh jauh sekali dari dari apa yang jawaban apa diharapkan oleh pertanyaan tersebut. [It seems to me that students who have the real ability and those who have not are alike. Why so? Because those who have not the real ability tend to do memorization upon their note before they come to the speaking test. So if I didn’t ask additional questions, it would not clear to see the difference. Those who has a good speaking ability, they can provide a long description; they can respond to whatever questions asked by the teacher. Those who has not, they usually get surprised and shocked: for example, a question about A is given an answer about Z. It has nothing to do with the question...]

She was aware that the way she was conducting the classroom speaking test caused anxiety among students and therefore she received complaints from students: Ya anak-anak kadang-kadang kan aduh bu kadang-kadang juga keluar, Bu kok tanyanya jadi itu. Kadang-kadang kan terus Bu, kok mbok yang kayak ... kadangkadang malah seperti mengajari, bu mbok pertanyaannya sama dengan yang itu. Kadang-kadang kan mungkin mungkin siswa yang maju itu mungkin bercerita sama temannya, tadi saya dikasih pertanyaan bebas tentang ini, jadi mungkin mereka juga sudah siap. Mungkin kalau ditanya ini jawabannya tentang ini, tapi karena ganti terus jadi kadang-kadang kalau yang kaget ya spontan kaget, ndak bisa njawab. [Well, they complained and expressed their feelings, why the questions had gone that way. Then, as if they were teaching me, they told the questions should be the same as the previous. May be those who had taken the test before told their friends the extra questions. But as the questions were changing, they could not answer.]

In addition, in another classroom (written) test,

she always

controlled the situation and the behavior of her students with strict rules, as she said, Saya sangat strict untuk hal-hal seperti itu, misalnya untuk e masalah sepele saja, misalnya anak harus atau kedua tangan anak harus berada di atas meja. Ya. Tidak boleh toleh kanan, tidak boleh toleh kiri. Ya, semua hal, misalnya mmm tidak boleh itu tadi pinjam meminjam alat ya tidak boleh. Ibaratnya mungkin ada anak yang duduk di dekat tembok, bersandar di tembok itu juga tidak boleh. Jadi anak harus posisi tubuh mereka pun harus harus harus saya tekankan biar mereka karena kadang-kadang kan anak memasukkan tangan ke laci, dengan tujuan mencari contekan. Na, itu saya tekankan bahwa kalau kalian tes, semua tangan ada di atas meja. [I am very strict in such trivial matters, such as asking students to put their


hands on the table. Well. They are not allowed to turn their heads around, to lend and borrow things. Those who are sitting next to the wall are not allowed to lean on the wall. They should sit upright. I emphasize these as sometimes I see some students put their hands into the drawer to do cheating. I say, if you are at a test, please put all your hands on the table.]

Teacher Nanik, holding the similar belief that the reliability and validity should be preserved, told: Karena kalau ulangan itu baik reading itu saya tidak pernah memperbolehkan membawa kamus atau melihat kamus. Jadi kan sudah dipersiapkan dulu. Besuk ulangan anak-anak harus siap tidak boleh membuka kamus. [When I give, say, a reading test, I never allow students to refer to or have a dictionary with them. So they must be prepared. I say that tomorrow they will have a test and should be prepared since they will not be allowed to refer to a dictionary. ]

To those who failed in the classroom test, she gave a different assignment as the make-up exam to the test, arguing: Kalau diberikan tugas ya... rata-rata anak seperti itu, terus kompromi dengan yang lainnya... apa.. makanya kadang-kadang saya berikan tugas yang berbeda satu anak dengan yang lain, tapi tingkat kesulitannya sama, tapi bentuk soal, artinya berbeda.. [When given an assignment, the same task,...most my students compromised to make the similar work...so sometimes I give different tasks with different formats and meaning but the same level of difficulty]

Such things however, were responded by the students negatively. Student Atik, for example, said: Dan saya juga pernah agak trauma. Karena pertanyaan menyimpang jauh dan membuat saya bingung menjawabnya. Tapi setelah itu saya jadi takut, Bu. Terlalu banyak pertanyaan itu saya takut. [And I felt traumatic. The questions were not relevant at all and made me confused to answer. Since tnen I have become scared. Too many questions make me afraid.]

Teacher Iin noticed the existence of student anxiety or nervousness during her classroom speaking test, stating: Kalau tes performance speaking, mereka itu biasanya nervous atau mungkin tidak tidak percaya diri untuk me mengikuti tes tersebut, jadi mereka lebih senang kalau tesnya tes tertulis. [In the speaking performance test, they usually look nervous and inconfident. So they like the written tests better.]


Although two students and teachers shared a mutual understanding that a test is a means of measurement, there were different concerns among them. Student Siti, for example, did not seem to care about the nature of measurement. Instead, she indicated that the nature of learning and achievement takes precedence over the nature of measurement. Questioned what if her friends around her were talking, discussing something during a test, she answered, ”Gimana ya? Ya kalau bisa ya walaupun tanya ya jangan sampai mengganggu temannya yang lain ’gitu, Bu.”. [”What should I say? Well, asking a question is all right, as long as they do not disturb the others...”]. Further, prior to a high-stakes test which was financially subsidized by the government (See Appendix 9.1 and 9.2) she expressed her feeling: [...] sebenarnya waktu mengetahui saya ikut tes ini itu sebenarnya ya senang ya, Bu, ya. Karena tidak semuanya bisa ikut. Lalu saya terus ya saya berfikir wa bagaimana caranya saya harus belajar atau apa gitu. [...actually when I was informed I was allowed to take the test, I felt overjoyed, Ma’am, since not everybody could join. Then I thought of something, somehow I should study...]

Here it shows that having an opportunity to take a high-stakes test is seen as a rewarding experience which in turn brings learning enforcement and motivation. Her happiness was due to competitiveness. Student Endang confirmed her view about English testing as her learning experience. In a high stakes test, which she did most items speculatively (see Appendix 9.3), she made a justification: ”Saya tidak mengejar target. Yang penting saya itu bisa tahu tentang Inggris, ’gitu lho.” [”I do not set any target. What is important is I eventually have a knowledge of English”]. This clearly means she does not care about


measurement and even achievement in terms of score. Further, she explained,”Saya itu kalau belajar sedikit demi sedikit asal dhong, ’gitu lho.” [“I always learn little by little with good understanding”]. Student Endang was aware of her constraints and problems when doing the tests. She did experience anxiety, yet she did not find her experience as traumatic. She was even determined to use the same speculation strategy to do tasks the other time when she had difficulty. She did not show any regret at all for doing that. She was neither sorry for nor ashamed of the way she took. In response to a question related to what she had done if she might use the same strategy when facing the same difficulty she innocently said, ”Ya mungkin. Melihat keadaan.” [Well, may be, depending on the situation.]. For her, every opportunity in an English test was seen as a learning experience and felt all right with it. Questioned what benefit she could obtain from her classroom tests, she said, ”Ya, bisa berlatih untuk lancar menggunakan bahasa Inggris, bisa semuanya bisa menggunakan apa itu dengan baik bahasa Inggris” [Well, it enables me to practice speaking, using English in a good way]. Questioned how

she enjoyed the high-stakes test, she confirmed what she had said, “Ya, saya suka. Untuk melatih pendengaran kita supaya kita itu cepat mudah mempelajari Inggris.” [Yes, I like it. It is to train our hearing so that we can study English more easily].What emerges here, again, is her concern about learning. All these therefore indicate the existence of strong self-awareness in her. The kind of interest as what Student Endang possessed, however, received an unpredictable response. Teacher Iin, when questioned what a


test serves to students with low achievement, commented: Mungkin hanya sekedar katakanlah rutinitas atau mungkin malah juga formalitas, yang penting mereka itu kadang-kadang yang penting rapornya ada isinya. Kadang-kadang kalau anak sudah begitu kan memang sudah backgroundnya juga sudah… jadi di dikasih apapun anak anaknya memang sudah susah gitu lho. Jadi kadang-kadang mereka menghadapi tes ya asal… asal mengerjakan tes, tidak ada beban e..nilainya jelek. Jadi kadang-kadang asal dilalui mereka sudah sudah aman, katakanlah. [Maybe a test is, say, a routinity or formality for them. What is important for them is that their schoolreport is completely filled out. When one behaves that way, it tells their personal background... it will be difficult for them to accept whatever given to them. So when they are facing the test, they just do it, having no burden that the score will be low. In short, as they are finished with it, they feel, say, secure...]

The different interest among teachers and students as such potentially creates a point of clash when they are engaged in the same particular testing. This research shows the clash happened in classroom tests as Student Atik experienced. When it does happen, then, it potentially hurts student’s learning, except when there is a sense of autonomy in them. Nevertheless, in a different test situation, especially in the context of high-stakes implementation, students and teachers shared similar concerns. While students’ concern in learning and achievement remained consistent, the teacher’s concern was surprisingly shifting from the nature of measurement to sense of achievement and graduation. More surprisingly, the shift of concerns was mainly attributed to the status of the test. A high-stakes test was perceived as having a higher value and level status, since it was about a competition not only among students, but also among teachers, institutions, officials and regions. Teacher Iin put it this way, Kalau saya pribadi unas itu tetap perlu diadakan karena e yang namanya unas itu kan standar ya untuk membuat standar standar dalam satu wilayah mungkin untuk memberi standar pada siswa mana yang layak lolos sama ndak, jadi tidak ada unas juga pendidikan malah akan kacau karena standarnya apa yang dipakai yang nggak ada itu yang susah.


[Personally, I will say that we need the national examination as it is the evaluation standard in a region to decide which students are eligible for graduation and which students are not. There will be a disorder in education if there is no standard test. It will be difficult. In my opinion we must go on with the national exam, and more specifically I will say it is a good idea to go back to the written test with the orientation to the national examination, because so far a kind of English test which is widely approved is the written test. So, although the students have a good speaking ability, but if they are poor in writing, everyone will see that their scores in the national exam are poor.]

The perception that the high-stakes tests were considered as having a higher value and status level were confirmed by the teachers’ state of mind. Questioned how she felt prior to, during and after a high-stakes test, Teacher Iin answered, Lebih ...lebih... derajat ketakutannya lebih tinggi sekarang... Perasaan kalau ulangan harian kan ibaratnya kita kan cuma tolok ukur tingkat sekolah, tingkat kelaslah katakanlah. Tapi kalau kalau unas ini kan ini kita kan langsung dinilai dinilai paling ‘ndak kan tingkat kabupaten. Jadi kita ada perbandingan antara o, sekolah ini bagaimana, sekolah itu bagaimana kan. Jadi kita tolok ukurnya sudah lebih tinggilah katakanlah seperti itu... [The level of anxiety is getting higher now ...I feel the classroom test is only a school-level measurement, but in the national exam, we are directly assessed and monitored at least at the regency level. There will be a comparation among schools. Let’s say the level is higher...]

In relation to this, Teacher Nanik responded, Ya, harap-harap cemas karena dalam kesehariannya artinya dalam memberikan les itu kan kemampuan siswa itu seberapa, gitu. Anak ini kira-kira bisa mengerjakan Unas atau tidak? [Well...expecting the best with anxiety, as from my observation during the extra lesson, I know their low capability. Can they do the national exam well or the reverse?]

Furthermore, Teacher Iin and Nanik explained: Saya sering merasa merasa berdosalah kalau ada anak yang sampai nggak lulus walaupun itu mungkin siswanya memang betul-betul low, tapi kan kayaknya kok saya kok nggak bisa sih membuat anak ini jadi lulus. Mungkin ada PR kayaknya masih menjadi semacam PR besar buat saya untuk lebih meningkatkan kemampuan (Teacher Iin) [I often feel guilty when I see one does not pass, although it is due to the low level of the student’s ability. But it seems to me that I fail to make her pass. There is something like big homework for me to improve my ability ] [...] cuma merasa kasihan kok kenapa dia tidak lulus.Untuk tahun yang kemarin itu anak tersebut ikut apa termasuk ikut lomba debat itu, tapi dia


bahasa Inggrisnya tidak lulus, jadi kan kami bertanya-tanya kenapa kok sampai dia tidak lulus. Kesalahannya di mana. Akan tetapi kalau memang biasanya anak kan ada yang tidak lulus, tapi dia memang kurang to kemampuannya jelas kurang itu ya. Perasaan ya cuma kasihan saja, tapi nggak begitu terus gimana.Ya memang karena kemampuan dia sejauh itu. [...]Yang penting saya lihat e apa yang kita persiapan kita yang kita leskan dengan soal yang itu tidak meleset jauh, artinya kita sesuai dengan kisi-kisi yang di..Kalau anak ya memang dia kok kenapa dia seperti itu barangkali ya dia salah le mengarsir atau mungkin dia pas lagi kondisinya ndak baik, nggak fit, kan kita tidak tahu. Jadi bukan karena situasi atau fasilitas yang kurang baik itu ya. Jadi mungkin kesalahannya semacam kesalahan teknis siswa di dalam mengerjakan.” (Teacher Nanik) [...only feel sorry why they could not pass. Last year, a student who was active in the debate contest failed. We surely wondered why and what was wrong. Normally those who failed were less capable. I just felt sorry, nothing else. That is the limit of their ability. [...] What is important for me is what we prepared in the class was not much different from the blueprint. But she experienced that was possibly because the way she marked the answer was erroneous, or because of her physical health. So it was a kind of technical problem in doing it.]

Now we see that in the context of a high-stakes test, students and teachers not only had similar concerns, but also shared similar state of mind. As the research shows such a shift was originally triggered from the external situation. Appendix 3 Observation data of events from official meetings and environs shows the heavy concern on graduation rate, which brought a significant influence to the teacher’s interest. Seriousness, grief, anxiety, spirit to solve the problems, strategies, clarity of rules were all brought to the test site. Instead of a point of clash, it is a gentleman’s agreement which emerged, resulting in a reduced level of concern in the validity and reliability of the test. Questioned what they did before the implementation of a high-stakes test, Teacher Nanik said, “Pas tiga hari di sekolah itu biasanya materi-materi Unas semua...” [“Three days in a week the materials I presented were related to the national exam”]. Teacher Iin told that she shared special strategies or tricks with the students when working on the national examination:


Preparation ya mungkin semacam semacam itulah, pokoknya segala jenis soal yang berhubungan dengan persiapan unas, model-model how to solve problem in unas sama mungkin trik-trik, trik-trik dalam mengantisipasi soal grammar itu juga perlu mungkin dikasih trik khususnya kalau ini kata ini ketemu ini, kata ini ketemu ini. Itu biasanya perlu juga. [Preparation...well...something like that. In short, any kind of practice test models related to the national examination preparation, how to solve the problems together with some tricks, strategies to anticipate grammar problems such as when they meet a certain kind of word, they should match it with this.]

The secondary data from my diary (Appendix 6 Researcher’s diary) and observation confirm the existence of the concern reduction. I heard Teacher Iin report with a pride that what she had presented in the test preparation course appeared in the last regional test, informing that there were a big numbers of the test items are similar as what they had discussed in the class. Teacher Yayuk, my colleague, told me that she had met our fellow teacher (Teacher Lucy) the day before. I heard that Teacher Lucy had kept on asking for the test booklet and cassette of the previous second-opportunity national exams. That was the third time for her to beg us to show what it looked like. “What’s up?”, I questioned myself. Since December 2007, there have been a number of demands for the same test material. There has been a rumor that the coming national exam will use either similar or exactly the same material.

From this description we see that teachers and students had different meanings and concerns in the context of classroom testing. In the context of high-stakes testing, however, they had a similar concern and state of mind. Both brought their own consequences. As the illustration of this, Figure 4.2 makes clear:

139 teacher’s meaning of testing

concern about about the nature of measurement

concerns concerns about achievement and graduation


POINT OF CLASH

concerns oncerns about learning and achievement

GENTLEMAN’S GENTLEMAN’S AGREEMENT

concerns concerns about learning and achievement

student’s meaning of testing Figure 4.4 Different interests within testing The clash, as this study indicates, took different forms such as avoidance, defense, and ignorance. As observed, Student Atik was not interested in and missed speaking competency tests. In other words, she tended to avoid them. This was due to her view in learning English, meaning of testing, critical reflection, different concerns, and test content. Student Atik shared the similar perception with the teachers about what a test is, i.e. as a measurement. She agreed a test aims to assess one’s mastery in understanding the lesson. She was already aware of her low capability in speaking; consequently, she did not think it was necessary to measure that particular skill. She clearly argued, E mungkin yang terakhir itu memang memang speaking itu penting, tapi untuk sampai tingkatan tertentu. Misalnya untuk tingkatan seperti saya itu sangat kesulitan. Katanya orang kalau belajar itu mendengar dulu, baru menulis, baru speaking, baru ngomong. Jadi harus yang mendengar sama yang menulis ini harus kuat dulu.


[Er.. maybe speaking is also important eventually when we come to a certain stage. For anyone on my level, it is very difficult. People say, we learn by listening first, and then writing, and speaking. So listening and writing ability should be strengthened first.] E, misalnya kemampuan Bahasa Inggris itu ya harus didasari dengan teori yang bagus, harus dengan pengetahuan tentang grammar, tentang vocab, kemudian dengan listening. Setelah listening baru speaking. Dan untuk saat ini saya untuk speaking belum cukup mampu [...]. [Er.., say English ability should be based on the theory mastery of grammar, vocabulary, and listening. After listening is mastered, comes speaking. And for the time being, I still cannot speak...]

She had a critical reflection on what learning situation should be. In her view, learning English should be enjoyable and bring enjoyment, ”Karena yang saya tahu, apabila belajar itu dipaksakan, justru kita akan menjadi benci dengan belajar...[Because as far as I know, if we are compelled to study, we will hate studying]” Student Atik’s experience in the classroom tests was unpleasant one. There was a conflict of interests, a point of clash between her concern and the teacher’s, more specifically between her view and the teacher’s in the nature of measurement. As she realized it she preferred to miss her oppotunities to take the tests to avoid the conflict. To her, they did not bring any enjoyment and therefore it hurt her learning. Nevertheless, it is her autonomy which helped save herself from destruction. She had a high independency in learning and capacity to make her own decisions: 141 Tapi e kalau tiba-tiba di awal kita speaking, tanpa kita mengetahui dulu seperti apa to ilmunya seperti kita belajar menulis, kan belajar membuat kalimat itu kan lebih lebih mudah dengan menulis dulu, ditulis duluan kan? [But if at the very beginning we are expected to speak without a knowledge of how to say it…. Isn’t easier for us to learn to make a sentence by writing it first?]


Selain itu speaking tetapi tidak hanya speaking seperti penjual yang melayani turis. Itu kan speakingnya waton dimengerti ’gitu lho Bu. Saya ingin speaking yang grammarnya juga betul, gitu. [In addition, speaking is not merely like a seller serving a tourist. That kind of speaking is merely understandable. I want to have a speaking ability with grammatically correct sentences] Sebenarnya bisa kita langsung ngomong, tapi nanti kualitasnya beda. Mungkin hanya seperti katakanlah seperti tukang-tukang becak di Malioboro itu bisa ngomong. Tapi ya ngomong yang bahasa Inggris yang seperti apa dulu. Mereka kan ya hanya yang penting orang itu mengerti artinya, gitu. Bayi yang baru lahir juga bisa ngomong tapi mungkin kan tata bahasanya juga anu salah, gitu. [Actually, we can master speaking in a short time, but with a different quality of speaking Maybe it is only like, say, the speaking ability of pedicab drivers in Malioboro. So what kind of English speaking we mean. They are satisfied as long as the person they are talking to understand. Newly-born babies can speak too, but their grammar is wrong.]

Student Siti had an opinion that the classroom tests were good to motivate students to learn, since they provided a challenge to compete with her friends. Ya menurut saya kalau uji kompetensi yang modul-modul itu bagus, Bu, karena bisa memotivasi siswa gitu, Bu. Wah kalau teman saya sudah berapa gitu levelnya njuk saya kok masih di sini, jadi memotivasi gitu lho Bu agar belajarnya lebih ditambahi lagi, gitu. [Well, I think the (classroom) competency tests presented in the modules are good, Ma’am; since they motivate students. I feel challenged to study a lot more as I watch my friends are at higher level, asking myself: why am I still here?]

She admitted that during the classroom test she experienced anxiety which in her view resulted from unpreparedness, and worries about failure. ”Mungkin karena belum siap, belum terlalu siap, gitu.” [”Well, I think it is because I was not ready, well-prepared...”] 142 Responding to the question what if she were asked to share her answers during the test, she explicitly stated,”Kalau... ya tergantung, Bu. Kalau saya bisa kasih tahu...” ”Ya kalau bisa jangan sampai


ketahuan pengawasnya, Bu ”[Well, it depends on the situation. If I know the answer, I will let them know...” ” trying not to be noticed by the test supervisor, Ma’am”]. This narrative indicates that the nature of measurement did not really matter to her. As potentially arising

we

here took the form of defense to

see, the clash protect

her

concerns in learning and achievement from an external ’intruder’. Another form as the indicator of clash is ignorance. Student Endang, speaking of the existing tests, found everything in the test acceptable, although she had hard time during her classroom as well as high-stakes tests and her scores were lower than the others in her class. She stated, ”Sepertinya sudah cocok semua, kok Bu. Bisa mengikuti kegiatan itu”. ”Ya bisa berlatih untuk lancar menggunakan bahasa Inggris”. [”I feel everything all right, Ma’am. I can accept that kind of activity. Well, it enables me to practice and use English in a good way.”] Student Endang, considering every moment of English testing was her opportunity to learn and practice English, ignored whatever external interests about English testing were. She did not care about what others would do on her with the test and whether she would score low.

143 CHAPTER V

CONCLUSIONS, IMPLICATIONS, AND RECOMMENDATIONS


This concluding chapter mainly presents the summary of the analysis results to the research problems. Nevertheless, with the spirit of openness in doing this research, I will begin this chapter with a short report of my reflection on what I had planned and how I actualized the plan. Related issues considered significant, although not part of the research problems, and recommendations for further research will follow. This research underwent the process of changes, modification, addition, reduction, shifts, and enrichment throughout the course of the study. This is in line with the progressive nature of this study. To ensure openness, I would like to present five major points I made related with the plan and implementation. First is the development and refinement of research questions. Research questions were continually revised, updated in response to what happened in the setting. FHI (2008:3) acknowledges such flexibility in qualitative study, “[Qualitative] study design is iterative, that is, data collection and research questions are adjusted according to what is learned”. Second is the abortion of a specific method, grounded theory. Since not possible methods were later applicable, I was determined to abort the grounded theory from being used in this study. The idea to adopt grounded theory was that in the beginning, I thought this study had an opportunity to produce a theory. Methods were proved to be emergent in my study. Third is the reduction of number of participants. I felt comfortable working with six people. Nevertheless, one participant (teacher participant) moved to another school, therefore I had less access to her. I decided to reduce the


number. Fourth is the extension of enquiry periods from 3 months to 11 months, divided in two periods. This prolonged engagement even ensures validity of this research. This happened owing to several factors such as the complexity of data, my own capacity in doing this project.

A. CONCLUSIONS English testing is a living phenomenon in English education which is problematic in both theory and practice. The most significant problem in the body of knowledge is the prominence and domination of the literal meaning of English testing resulting in the ignorance of its contextual meaning. As a result, learners and learning factors receive less attention than they should, which clearly indicates sort of suppression and oppression upon the primary stakes holder of education. There should be an agenda for reform to change their lives by including their participation in the reform. As well, there is an urgent need to promote learner’s learning autonomy in the recent context of education. To obtain deep understanding of how student participants experience English testing and learning, this research has tried to answer carefully formulated research questions “What is the lived experience of vocational school students in English testing and learning?” theoretically as well as experientially. 144 Relevant materials on previous research and recent issues have been thoroughly traced, studied, reviewed and acknowledged to obtain the theoretical answers, revealing a gap or disconformities among theories which potentially


marginalize the primary stakesholders of education, students. Qualitative progressive research with advocacy/participatory approach and phenomenological study, and narrative design as strategies of inquiry has also been employed to carefully explore the research question and answer the problems empirically. To ensure the trustworthiness of this study, data as well as method triangulation and thick description were employed. I utilized data not only from the student participants’ narratives

as the primary sources

but also from

secondary data sources such as from the teacher participants. I utilized not only interview

as the main instrument to collect the data but also employed

observation and document review. The research eventually reveals that despite the similar concerns among the student participants in testing, learning and achievement, there were unique emotional experience and learning experience; therefore I produce the individual student portraits comprising their emotional experience and learning experience prior to, during and after English testing implementation. Next, this study also reveals emergent themes around the time of English testing which, in addition to the prefigured themes test backwash, qualities, test content, learning and autonomy, includes status of test, fairness, time allocation, achievement, competitiveness, preparedness, which

are gleaned

from primary as well as

secondary data. The general description of prefigured and emergent themes in English testing were

then

produced showing concerns about learning,

measurement and achievement, students’ emotional experience such as anxiety, tensity, dissatisfaction, reluctance, confusion and excitement

and students’


learning experience comprising learning motivation, learning process, learning feed back, learning inforcement, learning achievement and learning autonomy. In addition, my interpretation of what testing meant to the students as well as teachers shows that there was a shift of concerns among teacher participants in different types within different contexts of testing. In the context of classroom testing, the teacher participants had different concerns from the student participants, validity and reliability, while the students were concerned about learning and achievement. In the context of high-stakes testing, however, teachers and students shared similar concerns, learning and achievement. That classroom testing potentially created a point of clash and hurt student learning as its implication was due to the different concerns in testing between the teachers and students. That high-stakes testing potentially created a gentleman’s agreement was due to the similar concerns. The descriptive as well as interpretative texts, incorporating discussion about emotions, eventually provide a holistic picture of the participants’ experiences, which represent the ultimate goal of qualitative research. The results of this study are not to be generalized in that the results are valid only within the unique context and boundary of the study. Trochim (2006:1) affirms,

147 The detail in most qualitative research is both a blessing and a curse. On the positive side, it enables you to describe the phenomena of interest in great detail, in the original language of the research participants [...]. On the negative side, when you have that kind of detail, it's hard to determine what the generalizable themes may be. In fact, many qualitative researchers don't even care about generalizing- they're content to generate


rich descriptions of their phenomena.

B. IMPLICATIONS This research should bring certain implications in both theory and practice. The analysis results from this study provide a strong argument to promote students’ learning needs in the current context of emancipation and liberation. The student participants of this study have indicated their concerns about their learning and achievement around the time of their English testing experience. The results of this particular study therefore suggest an agenda for a change by presenting a fair justification for learning to find a place in any stages within English education in the vocational school or elsewhere. Bachman (1995:297) indicating that for both theory and practice, the challenge is to develop tests that reflect current views of language and language use says, As language testing researchers, the challenge facing us is to utilize insights from linguistics, language learning and language teaching to develop tests as instrument of research that can lead to a better understanding of the factors that affect performance on language tests. Further he sets up a conceptual foundation for answering practical questions regarding the development and use of language test, which include three broad areas, (1) the context that determines the uses of language tests (2) the nature of the language abilities we want to measure (3) the nature of measurement. 148 In an educational context such as vocational school, such foundation needs highlighting. Educators may, thus, have different concerns. Therefore, in an educational context, I wish it read:


As language educators, the challenge facing us is to utilize insights from linguistics, language learning and language teaching to develop tests as educative instruments of learner’s assessment that can lead to a better understanding of the factors potentially arising that affect performance on learning….. The idea that characteristics of a good test, take for instance, are on its validity, reliability and practicability (Brown, 1994:253-254) is too simplistic to be placed in an educational context. It must go far beyond that ‘literal meaning of testing’. In other words of saying, it is the ‘contextual meaning’ which educators have to highlight. For me as a researcher, the research results have proved to develop my understanding of the student participants’ lived experience in English testing interacted with their learning. In turn, this understanding enables me to emancipate in building the students’ learning autonomy by setting forth the following recommendations.

C. RECOMMENDATIONS For education policy makers, it is recommended to control the use of English test at the vocational school, by assessing students’ learning needs to promote their autonomy. For students and teachers, it should enable to develop awareness of themselves and their learning environment and encourage them to play their social role at work and in the community (cf. Delors et al., 1998). For future researchers, it will be good to do similar research with participants of different sex, of different study program. It will also be beneficial to conduct a naturalistic study with a different method to advance the existing testing theory.


For test designers and specialists, the outcome of this study should bring new horizons, enlightenment and even a challenge to prepare a progressive testing applicable not only for students of the vocational schools but also those of all kinds and levels of school.

150

BIBLIOGRAPHY Adi Utarini, 2000. Metode penelitian kualitatif. Yogyakarta: Magister Kesehatan Ibu dan Anak, UGM. Ahmad Fauzan. 2009. Book Review: Qualitative data analysis Ian http://www.geocities/zulkardi/fauzan.html [Retrieved September 12, 2009]

Dey.


Aprendizagem, Avaliacaoda. 2005. Principles and indicators for student assessment Systems. www.serprofessoruniversitario.pro.br/ler.php?modulo=4&texto=1472 [Retrieved August 26, 2009] Ary, Donald. Jacobs, Lucy Cheser and Razavieh, Asghar. 2002. Introduction to research in education. Belmonth: Wadsworth/Thomson Learning. Bachman, Lyle F. 1995. Fundamental considerations in language testing. Oxford: Oxford University Press. Barkhuizen, Gary P, 1998. Discovering learners’ perceptions of ESL classroom teaching/learning activities in a South African context. TESOL quarterly Vol. 32. No. 1. Spring 1998 Benson, Phil. 2001. What is autonomy?. http://ec.hku.hk/autonomy/what.html [Retrieved June 5, 2007] Biagi, Shirley, 1992. Media/Impact: An introduction to mass media. Belmonth, California: Wadsworth Inc. Bond, Linda Ann. 1995. Critical Issue: Rethinking assessment and its role in supporting educational reform. North Central Regional Educational Laboratory. http://www.ncrel.org/sdrs/areas/issues/methods/assment/as700.htm[Retrieved May 12, 2007] Borgatti, Steve. Example of field and code notes. http://www.analytictech.com/mb870/handouts/exfieldnote.htm [Retrieved May 12, 2007] Borgatti, Steve. Introduction to grounded theory. http://www.analytictech.com/mb870/introtoGT.htm [Retrieved May 12, 2007] Brown, H Douglas. 1994. Principles of language learning and teaching. New Jersey: Prentice Hall Regentls. Prentice Hall, Inc. Brown, James Dean. 2005. Testing in language programs. A comprehensive guide to English language assessment. New York: McGraw-Hill ESL/ELT. Brumfit, Christopher. 2001. Individual freedom in language teaching: helping learners to develop a dialect of their own. Oxford: Oxford University Press. Caine, Nicholas A. 2005. EFL examination washback in Japan: Investigating the effects of oral assessment on teaching and learning. The University of Manchester: Language and literacy studies in education. www.asian-efljournal.com/thesis_washba [Retrieved January 24, 2007] CARLA. …… Theory of language assessment. The Regents of the University Minnesota www.carla.umn.edu/assessment/VAC/research/theory.html [Retrieved June 5, 2007]


Conttia, Lai Man Wai. 2007, The influence of learner motivation on developing autonomous learning in an English-for-special-purpose course. Hong Kong: University of Hong Kong Chalhoub-Deville, Micheline, 2003. Second language interaction: current perspectives and future trends. Language Testing. Iowa: Arnold Chenail, Ronald J. 1995. Presenting qualitative data. The Qualitative Report, Volume 2. Number 3, December 1995. http://www.nova.edu/ssss/QR/QR2-3/presenting.html [Retrieved January 17, 2008] Chenail, Ronald J. 1995. Writing up qualitative research. http://blog.lib.umn.edu/swiss/archive/Chenail.pdf [Retrieved April 22, 2008] Coombe and Hubley, 2008. The cornerstones of testing. http://taesig.8m.com/createii.html [Retrieved February 18, 2008] Coughlin, Mark.2006. Creating a quality language test. Tokyo. http://www.usingenglish.com/teachers/articles/creating-quality-language-test.html [Retrieved September 12, 2009] Creswell, John W. 2003. Research design. Qualitative, quantitative, and mixed methods approaches. California: SAGE Publications, Inc. Delors, Jacques. 1998. Education: the necessary utopia in Learning: the treasure within: report to UNESCO of the international commission on education for twenty first century. UNESCO Publishing. Departemen Pendidikan Nasional, 2003. Pelayanan kurikulum 2004. Penilaian kelas. Jakarta: Pusat Kurikulum Depdiknas. Departemen Pendidikan Nasional, 2005. Penilaian pembelajaran Bahasa Inggris. Jakarta: Ditjen Pendidikan Dasar dan Menengah. Departemen Pendidikan Nasional. 2006. Bahan bimbingan teknis penyusunan KTSP dan silabus Sekolah Menengah Kejuruan. Jakarta: Departemen Pendidikan Nasional. Dey, Ian. 1993. Qualitative data analysis. London: Routledge. Direktorat Pembinaan Sekolah Menengah Kejuruan. 2008. Sekolah Menengah Kejuruan. Direktorat Jendral Manajemen Pendidikan Dasar dan Menengah Jakarta: Departemen Pendidikan Nasional. Douglas, Dan. 2000. Assessing language for specific purposes. Cambridge: The Press Syndicate of the University of Cambridge. Doyle, Paul Grahame. Goh, Christine C.M. and Zhang, Lawrence Jun. 2004. Language


education concepts. A guide for English language teachers. Singapore: Prentice Hall Pearson Education South Asia. Dubin, Fraida and Olshtain, Elite. 1994. Course design. Developing programs and materials for language learning. Cambridge: Cambridge University Press. Egbert, Joy. 1999. Classroom practice: Practical assessment in CALL classroom. CALL environments research, practice, and critical issues. Alexandria: TESOL, Inc. Ellis, Rod. 2003. The study of second language acquisition. Oxford: Oxford University Press. Feez, Susan and Joyce, Helen. 2002. Text-based syllabus design. Sidney: AMES, National Centre for English Language Teaching and Research. FHI, …. . Qualitative research methods: a data collector’s field guide. Module 1 . Qualitative research method overview. http://www.fhi.org/NR/rdonlyrest/etl17vogszehu5s4stpzb3tyqlpp7rojv4waq37elpb yei3 [Retrieved April 22, 2008] Finney, Denise. 2002. The ELT curriculum: A flexible model for a changing world in Richards, Jack C and Renandya, Willy A. Methodology in language teaching, An anthology of current practice. Cambridge: Cambridge University Press Grix, Jonathan. 2004. The foundations of research. New York: Palgrave Macmillan. Hall, Peter Clark. 2000. Learner-centredness in the disciplines: Implications for teaching and curriculum development. University of Guelph. http://www.uoguelph.ca/tss/tli/tli00/program.pdf [Retrieved September 15, 2009] Hancock, Beverly. 2002. Trent focus for research and development in primary health care: an introduction to qualitative research. Trent Focus. Harmer, Jeremy. 2007. The practice of language teaching. Harlow: Pearson Education Limited. Hastings, Wendy. 2004. Emotions and the practicum: the cooperating teachers’ perspective. Teacher and Teaching: theory and practice. Vol. 10. No. 2 Carfax Publishing. 153 Healey, Deborah. Theory and research: Autonomy in language learning. http://www.public.iastate.edu/~bdaisy/510/510_healey_answers.htm [Retrieved June 5, 2007] Holliday, Adrian. 2002. Doing and writing qualitative research. London: SAGE Publications Ltd. Horby, A.S. 2000. Oxford advanced learner’s dictionary of current English. Oxford:


Oxford University Press. Hughes, Arthur and Ur, Penny. 2003. Testing for language teacher. Cambridge: The Press Syndicate of the University of Cambridge. www.amazon.com. [Retrieved June 7, 2009] Hutchinson, Tom and Waters, Alan. 1994. English for specific purpose. Cambridge: Cambridge University Press. Johnson, Burke ....... Chapter 17 Qualitative data analysis. www.southalabama.edu/coe/bset/johnson/dr_johnson/lectures/lec17.pdf [Retrieved April 2, 2008] Knight, Brian. Types of testing session. www.philseflsupport.com/testingtypesession.htm [Retrieved January 17, 2008] Little, D. 1991. Learner Autonomy 1: Definition, issues and problems. Dublin:Authentik Little, David and Dam, Leni. 1998. Learner Autonomy: What and Why. JALT98. Johns, Ann M and Price-Machado, Donna. 2001. ESP: Tailoring courses to student needs and to the outside world. Boston: Heinle & Heinle Thomson Learning. Maughan, Sarah. 2009. Closing the gap between assessment and learning. Cambridge: University of Cambridge International Examination. http://www.spbea.org.fj/aceab/maughan [Retrieved September 12, 2009] McLoghlin, David. 2003. The use of small-scale research studies as a means of investigating language learners’ causal attributions for failure. Research in ELT. Thonburi: the School of Liberal Arts and the Continuing Education Centre. King Moonghut’s University of Technology. McNamara, Tim. 1996. Measuring second language performance. Harlow: Pearson Education Ltd. McNamara, Tim. 2003. Looking back, looking forward: rethinking Bachman. Book review. Language Testing. McNamara, Tim. 2004. Language testing. Oxford: Oxford University Press. 154 Mitchell, Rosamond and Myles, Florence. 2004. Second language learning theories. London: Arnold Moleong, Lexy J. 1990. Metodologi penelitian kualitatif. Bandung: PT Remaja Rosdakarya. Morrissey, G. and Higgs, J. 2006. Phenomenological research and adolescent female sexuality: Discoveries and applications. The Qualitative Report, 11(1), 161-181.


http://www.nova.edu/ssss/QR/QR11-1/morrissey.pdf [Retrieved September 12, 2009] Murtagh, Patricia. 2003. Analysing qualitative data. www.shef.ac.uk/~mcn/359/lectures/359_lect11_analysing_qualit_data.pdf May 12, 2008]

[Retrieved

Mynard, Jo. …. Investigating learner autonomy in a virtual EFL classroom: a grounded theory approach. Zayed University, United Arab Emirates Nunan, David. 1994. Research methods in language learning. Cambridge: Cambridge University Press Nunan, David. 2000. Autonomy in language learning. Cartagena: ASOCOPI Onwuegbuzie, Anthony J, and Daniel, Larry G. 2003. Typology of analytical and interpretational errors in quantitative and qualitative educational research. Current Issues in Education Volume 6, 2003. College of Education. Arizona State University. http://cie.asu.edu/volume6/number2/ [Retrieved April 2, 2008] Presiden Republik Indonesia. 2005. PP No. 19 Tahun 2005. Standar Nasional Pendidikan. Jakarta. Paul, David. 2003. Teaching English to children in Asia. Hong Kong: Longman Asia ELT. Pulist, S.K. 2009. Learner-centredness: an issue of institutional policy in the context of distance education. Indira Gandhi National Open University. http://www.tojde.anadolu.edu.tr/tojde4/pulisttxt.html [Retrieved September 15, 2009] Punch, Keith F. 2001. Developing effective research proposals. London: SAGE Publications Ratcliff, Don. 2006. Qualitative research. http://don-ratcliffs.net/qual [Retrieved May 12, 2007] Rayment, Tabatha. 2006. 101 Essential lists on assessment. London: Continuum. 155 Riddell, David. 2003. Teach yourself. Teaching English as a foreign/second language. London: Hodder Education Senior, Rose M. 2006. The experience of language teaching. Cambridge: Cambridge University Press. Shrader, Stephen R. 2003. Learner empowerment – A Perspective. The internet TESL Journal Vol. IX No. 11. November 2003. http://iteslj.org/articles/shrader-


empowerment.html [Retrieved January 17, 2008] Smith, M.K. 2005. The encyclopedia of informal education. www.infed.org/biblio/learning-behaviourist.htm; -cognitive.htm; -humanistic.htm; -social.htm [Retrieved May 9, 2007] Tepsuriwong S and Srisunakrua T. 2009. Degrees of learner-centredness in Thai tertiary English courses. http://www.wll.nie.edu.sg/esea2008/sessions/TEPSURIWONG_S_and_SRISUNA KRUA_t.pdf [Retrieved August 30, 2009] Trochim, William M.K. 2006. Research methods knowledge base. www.socialresearchmethods.net/kb/contents.php [Retrieved January 17, 2008] Thanasoulas, Dimitrios. 2000. What is learner autonomy and how can it be fostered? The Internet TESL Journal. http//iteslj.org/Articles/Thanasoulas-Autonomy.html Tudor, Ian. 1997. Learner-centredness as language education. Cambridge: Cambridge University Press. http://www.gettextbooks.com/isbn_0521480973.html [Retrieved September 18, 2009] University of the West England, 2006. Data Analysis. http://hsc.uwe.ac.uk/index.asp [Retrieved April 30, 2008] Woods, Peter. 2006. Qualitative research. www.edu.plymouth.ac.uk/resined/qualitativemethods/qualrshm.htm [Retrieved January 17, 2008] Wardhaugh, Ronald. 1992. Introduction to sociolinguistics. Oxford: Blackwell. Wharton, Sue. 2007. Characterising your language test. Language testing unit 1. http://www.philseflsupport.com/testing1.htm [Retrieved May 22, 2007] Wiggins, Grant. 1998. Educative assessment. Designing assessment to inform and improve student performance. San Francisco: Jossey-Bass Publisher. www.josseybass.com Wilson, Reid. 2000. A summary of Stephen Krashen’s “Principles and Practice in Second Language Acquisition”. Language Impact. www.languageimpact.com/articles/rw/krashenbk.htm [Retrieved 1 January 2008] An introduction to qualitative research. www.faculty.uccb.ns.ca/.../MBAGO3_files/IntroQualitativeResearch.pdf [Retrieved 1 January 2008] Qualitative research method, qualitative analysis, qualitative data. http://www.resolutions.co.nz/analysing-qualitative-research.htm.[Retrieved May 9, 2007]


-

2000. The American heritage dictionary of the English language. http://education.yahoo.com/reference/dictionary/entry/vocational%20school http://education.yahoo.com/encyclopedia/entry/entry/cvocatled

http://www.kompas.com/kompas-cetak/o704/28/utama/3489354.htm [Retrieved May 9, 2007] http://suaramerdeka.com/cybernews/harian/0610/17/nas2.htm [Retrieved May 9, 2007] www.thejakartapost.com [Retrieved May 9, 2007]

157

APPENDICES Appendix 1. Interview probes 1.1 Questions to students 1. 2. 3.

Menurut pendapatmu, tes ulangan harian yang kamu tempuh itu bagaimana? Menurut pendapatmu, tes unas/penjajakan/TOEIC/regional itu bagaimana? Menurut pendapatmu, tes yang ideal itu seperti apa?


4. 5. 6. 7. 8.

Sebelum, saat dan sesudah mengikuti ulangan harian, bagaimana perasaanmu? Sebelum, saat dan sesudah menempuh ulangan harian, apa yang ada di benakmu? Sebelum, saat dan sesudah mengikuti ulangan harian, apa saja yang kamu lakukan? Sebelum, saat dan sesudah mengikuti ulangan harian, apa yang kamu alami? Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/regional, bagaimana perasaanmu? 9. Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/regional, apa yang ada di benakmu? 10. Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/regional, apa saja yang kamu lakukan? 11. Sebelum, saat dan sesudah mengikuti tes unas/penjajakan/TOEIC/regional, apa yang kamu alami?

1.2 Questions to teachers 1. 2. 3. 4.

Why and for what reasons do you give tests? What are the purposes of your tests? What do you think the test your students like best? What makes an ideal test in terms of goal, process and material? Do you think tests bring impacts on your teaching and student’s learning? What impacts does each test bring? Which tests bring positive impacts on your teaching and student’s learning? 5. How would you describe your classroom test situation? 6. How would you describe high-stakes test situation? 7. How would you respond to your student’s work? 8. How would you respond to your student’s behavior during a test? 9. How would you react to each test 10. Describe how you feel and what you do when conducting and supervising tests? 11. How have you experienced your classroom test and high-stakes test?

158 Appendix 2. Coding Blue Print 2.1. Participant: Student Functions (I) Participant

Classroom Test (A)

Highstakes Test (B)

Measurement Qualities ( II ) Classroom Test (A)

Highstakes Test (B)

Content and implementation ( III ) Classroom Test (A)

Highstakes Test (B)


Atik (Sat) Siti (Sit) Endang (Sen) SGroup (Sgr)

Think

Sat-I-A-?

Sa-t-I-B-?

Sat-II-A-?

Sat-II-B-?

Sat-III-A-?

Sat-III-B-?

Think

Sit-I-A-?

Sit-I-B-?

Sit-II-A-?

Sit-II-B-?

Sit-III-A-?

Sit-III-B-?

Think

Sen-I-A-?

Sen-I-B-?

Sen-II-A-?

Sen-II-B-?

Sen-III-A-?

Sen-III-B-?

Think

Sgr-I-A-?

Sgr-I-B-?

Sgr-II-A-?

Sgr-II-B-?

Sgr-III-A-?

Sgr-III-B-?

Before (1)

Participant Atik (Sat)

Siti (Sit)

Endang (Sen)

SGroup (Sgr)

Feel (1) Think (2) Do (3) Exp (4) Feel (1) Think (2) Do (3) Exp (4) Feel (1) Think (2) Do (3) Exp (4) Feel (1) Think (2) Do (3) Exp (4)

During (2)

After (3)

Classroom Test (A) Sat-1-1-A-?

Highstakes Test (B) Sat-1-1-B-?





Sat-2-1-A-?

Sat-2-1-B-?

Sat-2-2-A-?

Sat-2-2-B-?

Sat-2-3-A-?

Sat-2-3-B-?

Sat-3-1-A-?

Sat-3-1-B?

Sat-3-2-A-?

Sat-3-2B?

Sat-3-3-A-?

Sat-3-3-B?

Sat -4-1-A-?

Sat -4-1-B-?

Sat -4-2-A-?

Sat -4-2-B-?

Sat -4-3-A-?

Sat -4-3-B-?

Sit-1-1-A-?

Sit-1-1-B-?

Sit-1-2-A-?

Sit-1-2-B-?

Sit-1-3-A-?

Sit-1-3-B-?

Sit-2-1-A-?

Sit-2-1-B-?

Sit-2-2-A-?

Sit-2-2-B-?

Sit-2-3-A-?

Sit-2-3-B-?

Sit-3-1-A-?

Sit-3-1-B?

Sit-3-2-A-?

Sit-3-2B?

Sit-3-3-A-?

Sit-3-3-B?

Sit -4-1-A-?

Sit -4-1-B-?

Sit -4-2-A-?

Sit -4-2-B-?

Sit -4-3-A-?

Sit -4-3-B-?

Sen-1-1-A-?

Sen-1-1-B-?

Sen-1-2-A-?

Sen-1-2-B-?

Sen-1-3-A-?

Sen-1-3-B-?

Sen-2-1-A-?

Sen-2-1-B-?

Sen-2-2-A-?

Sen-2-2-B-?

Sen-2-3-A-?

Sen-2-3-B-?

Sen-3-1-A-?

Sen-3-1-B?

Sen-3-2-A-?

Sen-3-2B?

Sen-3-3-A-?

Sen-3-3-B?

Sen -4-1-A-?

Sen-4-1-B-?

Sen -4-2-A-?

Sen-4-2-B-?

Sen -4-3-A-?

Sen-4-3-B-?

Sgr-1-1-A-?

Sgr-1-1-B-?

Sgr-1-2-A-?

Sgr-1-2-B-?

Sgr-1-3-A-?

Sgr-1-3-B-?

Sgr-2-1-A-?

Sgr-2-1-B-?

Sgr-2-2-A-?

Sgr-2-2-B-?

Sgr-2-3-A-?

Sgr-2-3-B-?

Sgr-3-1-A-?

Sgr-3-1-B?

Sgr-3-2-A-?

Sgr-3-2B-?

Sgr-3-3-A-?

Sgr-3-3-B?

Sgr -4-1-A-?

Sgr -4-1-B-?

Sgr -4-2-A-?

Sgr -4-2-B-?

Sgr -4-3-A-?

Sgr -4-3-B-?

2.2. Participant: Teacher Functions (I) Participant Iin (Tin) Nanik

Measurement Qualities ( II )

Content and implementation ( III )

Think

Classroom Test (A) Tin-I-A-?

Highstakes Test (B) Tin-I-B-?

Classroom Test (A) Tin-II-A-?

Highstakes Test (B) Tin-II-B-?

Classroom Test (A) Tin-III-A-?

Highstakes Test (B) Tin-III-B-?

Think

Tik-I-A-?

Tik-I-B-?

Tik-II-A-?

Tik-II-B-?

Tik-III-A-?

Tik-III-B-?


(Tik) TGroup (Tgr)

Think

Nanik (Tik)

TGroup (Tgr)

Tgr-I-B-?

Tgr-II-A-?

Before (1)

Participant Iin (Tin)

Tgr-I-A-?

Feel (1) Think (2) Do (3) Exp (4) Feel (1) Think (2) Do (3) Exp (4) Feel (1) Think (2) Do (3) Exp (4)

Tgr-II-B-?

Tgr-III-A-?

During (2)

tgr-III-B-?

After (3)

Classroom Test (A) Tin-1-1-A-?

Highstakes Test (B) Tin-1-1-B-?





Tin-2-1-A-?

Tin-2-1-B-?

Tin-2-2-A-?

Tin-2-2-B-?

Tin-2-3-A-?

Tin-2-3-B-?

Tin-3-1-A-?

Tin-3-1-B?

Tin-3-2-A-?

Tin-3-2-B?

Tin-3-3-A-?

Tin-3-3-B?

Tin -4-1-A-?

Tin -4-1-B-?

Tin -4-2-A-?

Tin -4-2-B-?

Tin -4-3-A-?

Tin -4-3-B-?

Tik-1-1-A-?

Tik-1-1-B-?

Tik -1-2-A-?

Tik -1-2-B-?

Tik -1-3-A-?

Tik -1-3-B-?

Tik-2-1-A-?

Tik -2-1-B-?

Tik -2-2-A-?

Tik -2-2-B-?

Tik -2-3-A-?

Tik -2-3-B-?

Tik-3-1-A-?

Tik -3-1-B?

Tik -3-2-A-?

Tik -3-2B?

Tik -3-3-A-?

Tik -3-3-B?

Tik-4-1-A-?

Tik -4-1-B-?

Tik -4-2-A-?

Tik -4-2-B-?

Tik -4-3-A-?

Tik -4-3-B-?

Tgr-1-1-A-?

Tgr -1-1-B-?

Tgr -1-2-A-?

Tgr -1-2-B-?

Tgr -1-3-A-?

Tgr -1-3-B-?

Tgr-2-1-A-?

Tgr -2-1-B-?

Tgr -2-2-A-?

Tgr -2-2-B-?

Tgr -2-3-A-?

Tgr -2-3-B-?

Tgr-3-1-A-?

Tgr -3-1-B?

Tgr -3-2-A-?

Tgr -3-2-B-?

Tgr -3-3-A-?

Tgr -3-3-B?

Tgr -4-1-A-?

Tgr -4-1-B-?

Tgr -4-2-A-?

Tgr -4-2-B-?

Tgr -4-3-A-?

Tgr -4-3-B-?

159 2.3 Observation on test implementation Test Type

Classroom Test (C)

Before During After (Be) (Du) (Af) Physical Non Physical Non Physical Non (P) Physical (P) Physical (P) Physical (NP) (NP) (NP) C/Be/P/? C/Be/NP/? C/Du/P/? C/Du/NP/? C/Af/P/? C/Af/NP/?


High stakes Test (H)

H/Be/P/? H/Be/NP/? H/Du/P/? H/Du/NP/? H/Af/P/? H/Af/NP/?

2.4 Inductive codes BB

= Beneficial backwash

HB

= Harmful backwash

LMOT

= Learning motivation

LPRO

= Learning process

LFEE

= Learning feedback

LENF

= Learning enforcement

LACH

= Learning achievement

LAUT

= Learning autonomy

160 Appendix 3. Observation data of events from official meetings and environs 3.1 Official meeting 1 (external) Observation Data and Comment Thursday, 25th January 2007. Dinas Pendidikan Provinsi DIY’s Auditorium. 08.30 – 11.30 Approximately 110 EFL teachers from the province’s vocational schools had an assembly with local authorities as a part of preparation for the national exam, Unas. Among those who were present were the deputy head of vocational school principal organization, K3SK, the head of vocational school EFL teacher association MGMP, chief of high education, Kabid Dikmen. They were seated with the following configuration

Code H/Be/NP/assembly


A

B

C

D

E

F

Audience

The head of MGMP invited the audience to form what he called “Tim Tangguh” or “Tim Sukses” for the coming national examination and to listen to the key note speaker.

H/Be/NP/to form special management team

The Kabid Dikmen added by reminding and emphasizing that the team’s service period was limited from the day of the meeting through the day before the national examination, 25th January 2007 to 16th April 2007.

H/Be/NP/fairness assurance

Meanwhile, the deputy head of K3SK informed anything about Provincial Unas try-out scheduled to take place on March 6,7, and 8. It would use a replicate of the real national examination. There would be two test-packages used, A and B. Student test takers with odd registration numbers would do different test package from those with even numbers. The student answer sheets would be sent to the local office, which would then forward this to the provincial office. He advised all facilities such as tape player and speakers be prepared and checked. It was expected that with the participation of all EFL teachers and Dikmen, the number of successful students would increase.

H/Be/P/test replica H/Be/NP/situation replica H/Be/P H/Be/NP/high expectation H/Be/NP/concern with graduation rate

1) Unas, UN, UAN (Ujian Nasional = National Examination). This examination is for all third-year students of the Public as well as Vocational Schools where English is one of the subjects tested and which that year took place on March 6,7, and 8. 2) K3SK ( Kelompok Kerja Kepala Sekolah Kejuruan = Vocational School Principal Workshop) is an organization of vocational school principals which aims to discuss all the matters they have in common. 3) MGMP ( Musyawarah Guru Mata Pelajaran = Specialist Teacher Assembly ) is an association of teachers who teach a specified subject, which is in this case English. 4) Kabid Dikmen ( Kepala Bidang Pendidikan Menengah = Secondary Education Chief ) is one of the Provincial Education Office chiefs who is concerned with all the educational matters in the secondary education.

161 3.2

Official meeting 2 (external) Comment

Friday, 30 March 2007. 09.00 – 14.00. SMK 4 Yogyakarta A meeting attended by approximately 150 English teachers of the province’s vocational school is held to welcome the coming Unas test. Seated with the following configuration, an

Narrative ( Translated ) Official: “It really makes us sad. Only 32% pass. [ The tryout test ] brings such a result. Let’s analyze, evaluate why so. Are the test items too difficult? Is it us who is not ready? Is it you who is not ready? Or the students? We feel terrified in case it brings bad

Code H/AfBe/NP/assembly H/Af/NP/grief sharing H/Af/NP/evaluation H/Be/NP/seriousness H/Af/NP/grief H/Af/NP/small number scored below passing grade in the tryout


official of the provincial education office informs that only 32% of the vocational school students pass in the previous tryout test. He expresses his grief and invites the audience to make an evaluation.

impacts to the national examination.

There some non technical problem. Students who could have done the test well face with a problem of blackening the circles in the answer sheet.

The configuration shows how the high stakes-test has been taken seriously. The officials’ big concern with graduation rate would potentially bring consequences such as psycho-effects to teachers and students within their testing and learning situation. I would consider this powerful thing as non-physical testing and learning environment

To what extent is the students’ mastery against Direktorat /Puspendik’s competency standards? Can Bu Yy raise from 32% to 80% or 90%? We still have 2 weeks. Who is wrong? Who is to blame? What’s the solution so that on the D day it will be good?

H/Af/NP/spirit to solve the problem and do best

H/Af/NP/self evaluation

All of you (English teachers) have 90% contribution in raising the percentage. The school which is in rank 2, has failure rate 60%! When you get back to school, report to your principal whether you are satisfied [with all of these]. At schools, there are small Tim Sukses, which should end on 16 April. Which number is most students fail to do? Mrs Yy, if there is any test practice from Puspendik, have it photocopied and distributed to schools. We invite you here to give you facilitation. You can do your Friday prayer here. The main agenda for today is essential material and it will be conducted by Mrs Yy.

Comment

H/Af/NP/analyzing, evaluating, test result, program evaluation H/Af/NP/terrified, worried H/Af/NP/it may bring bad impact H/Be/NP/speculation on the cause of the test result: students may not be skillful in technical matters

Narrative ( Translated )

H/Be/NP/concern on test practice availability H/Be/NP/assistance assurance

(cont’d) Code


Since 1984, Ebtanas has become yearly routine. Every year, there is always something wrong. Last year, we had 51 cases. Only 3 schools complained.”. [He then explains anything about how to write the test participant’s identity and number. He informs there will be two test packages and student with odd number will take a different test package from those with the even number. He reminds that students should not miss a single test item although when they make a speculation, they may find it wrong. Why so, he explains that when the one test item can only be done by less than 5% of the total participants, the test item will be annulated, which means every student who has tried to do the item, no matter the answer is correct or incorrect, they will get a bonus score.]

H/Be/NP/concern with technical matters

“Those who leave it blank do not get the bonus. Please inform your students and ask them not to leave it blank.”

H/Be/NP/concern with appropriate strategies in doing the test

H/Be/NP/clarification on rule of the game H/Be/NP/speculation welcome H/Be/NP/ less concern with capability

[He reminds that all of us are parts of a system and those who are around are only implementer. He continues, “If the minister says A, we are to implement it.”] “This year, unas is complicated. The guide is complicated, the policy is complicated. Your task is to save and raise the graduation rate: how to raise from 32% to 90 or 95%. Please do whatever you will do with this. Don’t touch the policy. Check whether the test level of difficulty is too high or too low. The competency standard is high or low. We will facilitate you till afternoon. There might be something to clarify.”

164 3.3 Official meeting (external)

H/Be/NP/concern with graduation rate


Comments

Description of events and behavior

Thursday, 19 April 2007. 08.00 – 10.00. Local Education Office Hall.

This is the day when English national examination for students of public high and vocational schools is conducted. This day, the Secondary School English teachers are gathered in the local education office hall. The officials reportedly make a policy to ‘sterilize’ the test sites from subject-matter teachers. The meeting is one of the ways to save the Unas test implementation. It is hold to quarantine the teachers from the test sites, for experience shows that so far there has been some indication of unfair implementation of the previous national examination which involves English teachers. The situation is merry and cheerful.. The meeting starts with a prayer. The English teacher superintendent hosts the meeting. In his speech he presents all the recent problems arising in the national examination as well as in the daily teaching and learning program. He says that from TV he learns that there has been some negative

This activity is to make sure and guarantee that students can do the test on their own. It implies that the teachers are seen as having potential threats to reliability. It may also mean that teachers have a big concern in the students’ graduation, since this will be their special achievement in teaching.

Code

H/Du/NP/reliability

3.4 Official meeting 4 (internal)

Comment March 15, 2007. 10.00 – 11.00. Visitor Room of SMK N 1 Pengasih The spirit of “success” in Unas echoed from the school management, which in turn might influence students’ behavior


Code

School management and teachers of English, Indonesian, Mathematics and Economics had a serious meeting to discuss a plan to put third graders in quarantine. The situation is serious but friendly. The schedule is set.

H/Be/NP/quarantine program idea H/Be/NP/seriosity

165 Appendix 4. Observation data of events and behavior from test sites


4.1. Classroom test 1 Comments


Thursday, 1st February 2007. 10.15. SMK 1 Pengasih’s language lab Student’s feeling has no relationship with the test environment. This is due to psychological component: worrying about failure. Feeling of success makes testing as a positive, enjoyable experience

In a speaking competency test, Siti had to answer the assessor’s questions on describing her favorite teacher, school principal and favorite artist. The situation was fine and quiet. In the room there were only three people: Siti, the student test-taker, the assessor and a friend of Siti’s, who was sitting behind her. During the first minutes of the test, she looked nervous and trembling, and the assessor had to signal to convince her that she need not feel that way. To the assessor’s surprise, she spoke smoothly. She did make minor grammatical mistakes but her language was understandable and acceptable. The assessor decided that she deserved an A. Noticing her success, she screamed to joy, and with a pride, she told her friend her attainment. She left the room saying a big thank you to the assessor.

Code

C/Du/NP/fine,quiet C/Du/P/there are three people Sit-1-2-A-nervous Sit-3-2-A-trembling Sit-1-3-A-relieved, overjoyed

Sit-3-3-A-sharing her test experience with a friend with pride

4.2. Classroom test 2 Comments Friday, 2nd February 2007. 09.00. Room 19. SMK 1 Pengasih


Code

Situation prior to a speaking competency test : A class of 19 students majoring in multimedia with novice level of English competence were preparing for a speaking test assessing their communication skill in expressing their feeling. Students were reading, looking up their dictionaries and having a dialog, memorizing and practicing speaking. They had pair dialogs, group discussions of three or four. Some were laughing while practicing, some were thinking deeply. One student, who was sitting alone, was supporting her jaw with her left hand, then changed her gestures by lying her head on the table. When the teacher got closer asking whether she was ready for the test, she gave no response but then continued to read and write. Next, the first pair was approaching to propose for the test. Before the test was conducted, the teacher assessor asked one of them, by practicing to the student’s learning agenda, why she was late to take the test. [In her learning agenda, she planned to take the test on 19th December 2006]. She told she had a new brother. After assessing the first pair of students, the teacher saw another student put her head on the table. She approached her and asked, “Are you sleeping?” She answered no. She told she was practicing by memorizing what she had to say.

4.3. Classroom test 3

Sgr-3-1-Areading, using a dictionary, having a dialog, discussing, memorizing proposing for a test Sgr-1-1-Aserious


Comments


Code

Saturday, 3rd February 2007. 13.30 – 14.05. Room 19. Test-takers standing in a queue for a speaking test although school ended means the test supported learning. They may have felt tired, but their physical state was less intense than the emotional condition. It indicates high degree of autonomy: they know they want to fulfil their needs. There is a kind of competition which they may think can lead to their achievement and sense of success.

A class of 36 students majoring in office administration. When speaking competency test was in progress, the bell rang, indicating school was over. The teacher assessor soon ended the lesson right away, but gave a sign that anyone wishing to have their competency tested could propose for a test as soon as the meeting was formally closed. No less than 10 were standing in line. The teacher asked them to submit their test form and sit down. She called one after another interviewing the students and asking their personal background.

Sgr-4-1-Amotivation

4.4. Classroom test 4 10 March 2007. 07.52. SMK 1 Pengasih Observation Data Comment

Speaking competency test 1

Code

This is a classroom where speaking competency test (classroom test) is hold. Both student test taker and teacher are seated on a wooded chair in a very spacious, well ventilated room. The teacher asks questions and the student is to answer. The teacher has an old folio book and a piece of paperwhere she records the student’s score. There is a rectangular wooded table separating students and teacher who are communicating face to face. The score or result of the test, with teacher’s commentaries and suggestions, is given to the student test takers right away

C/Du/P/fine

4.5 Classroom test 5 10 March 2007. 07.58- 08.02. SMK 1 Pengasih Observation Data Comment

Code C/Du/NP/moody

The students are conducting a conversation in front of the class. The teacher is watching their performance.

Speaking competency test 2

4.6. High-stakes test 1 Comment

Observation Data

Code


Thursday, February 8, 2007. SMK N 1 Pengasih. Local Try-out of National Examination

Observation was done in and from Room 3. School has a two-storied building and this room is on the first floor. There were 40 wooden chairs. It was ….X…. meters wide which accommodated 40 students and a teacher. Students were seated in such a way that they faced the same direction. The room is well-ventilated with a number of glass windows. It has two doors: one at the back and another on the students’ right. A rectangular board was located in front of the class. A grey loud-speaker was on the wall above the board. The floor was composed of grey square tiles with 20X20 cm dimension Bell rang at 7.25. 18 students majoring in Accounting were ready to the test. Morning prayer was conducted.

Technical breakdown makes testing as a negative experience

At 7.27 a late student came in and made an excuse. Over the speaker, the committee informed anything about the time allocation, number of test items, and options. He asked testees to report if they found their test booklet incomplete. As the test booklet had been distributed, one asked if they were to write in ink or pencil, and if they were to put a cross on or blacken the letters to mark their answers. As this matter was forwarded to the committee, the committee replied that they were to put a cross on the letter in the circle, for the circles were too big. At 7.35. The committee started to play the cassette to present the listening section of the test. After the first few minutes, sound of dissatisfaction was heard from another room (Room 2) for not being able to hear the voice clearly. Teacher supervisors of other rooms looked nervous and anxious and then approached the room where the committee played the cassette asking to make it better. The cassette was then rewinded after presenting 4 number of test items. The effort was successful, for the sound quality increased significantly. Everything run well.

At 08.00, the listening section ended. One student exchanged the test booklet for the pages were incomplete. At 08.40, a great number of students were seen out of the test room, and discussed the answers of the test.

168 4.7 High-stakes test 2

Sgr-3-2-B-asking for a clarification Sgr-3-2-B-getting feedback

Sgr-1-2-B-dissatisfied Sgr-3-2-B-complaining about test content clarity Tgr-1-2-B-nervous, anxious Tgr-3-2-B-approaching operator, demanding for betterment Tgr-3-2-B-getting feedback Sgr-3-2-B-getting feedback Sgr-3-2-B-requesting a new test booklet Sgr-3-3-B-discussing what they have done and experienced


Comment


Thursday, March 8, 2007. SMK N 1 Pengasih. Provincial Try-out of National Examination

It is a bright sunny day. 281 students are ready to take the second opportunity to take the English national examination try-out in 15 test rooms. The test had been scheduled to take place from 08.00 through 10.00 a.m. 07.30 the bell indicating the testees and test supervisors to enter the test rooms rang. I, as a researcher got permission from the committee head to do recording. 07.40 praying was conducted. 08.00 bell to mark the starting time to do the test rang. The committee head played the cassette presenting the listening section of the test. The situation is conducive in spite of the noises from the building project. Technical problem arouse. The sound quality was not maximal. The committee head rewinded and replayed the cassette. The solution to the problem was found.

Making a communication during a test is widely known as ‘cheating’, but here I would say it is a “defense” indicating that learning actually takes place within testing Students’ , including Atik’s concentration in doing the test must dispersed Students had finished the test before time was up. Atik must have been disturbed.

08.30. I observed Room 9 and took a picture. The testees majored in Office Administration. Two supervisors (nonEnglish teachers were standing at the door. As the listening section ended, a small number of testees tried to make a communication with some others. 08.40. Over the loudspeaker, an excuse for making interruption was heard. It presented school information which had nothing to do with the examination.

09.10. A small number of testees were seen out of the room. Their faces showed no sign of anxiety. Around room 8 and 9, they were talking in small groups. 09.20. Observation was done to Room 3. Here the students majoring in Accounting were doing their test. 3 students had already left the test room. I approached the three students and asked, “Bagaimana soalnya tadi?” One of them answered, “Ya, lumayan sukar. Terutama pada error recognition.” 09.25. Test supervisors of Room 10 had left the room and were seen walking to the committee room to submit the students’ work. 09.40. Observation was done in the committee room. The supervisors of Room 3 and 5 entered the room and submitted the work. 09.30. As I checked Room 11, 12, 13, the places were found empty. The rooms were where the marketing department students had done the tests. 09.45. The supervisors of Room 14 submitted the testees’ work in the committee room.

4.8 High-stakes test 3

Code

Sgr-4-2-Btechnical problem Sgr-3-3-Bmaking a communic ation with others Sgr-4-2-Bdistracted by announce ment Sgr-1-3-Bshowing no anxiety Sgr-3-3-Btalking in small groups Sgr-2-3-Berror recognition is most difficult


Comment Setting: SMK N 1 Pengasih. 22 March 2007. TOEIC test

Observation Data 1.

The test implementation was so formal and serious. There were many rituals 2.

I believe that students found the test challenging. They had potentials to do the test well. The level of difficulty was acceptable. They felt sorry that time allocation had prevented them to finish the task. The learning process was interrupted.

3. 4. 5. 6. 7.

Test Committee Room, 07.25. 13 proctors, 1 test supervisor and the coordinator are present. It is raining outside. Proctors are required to sit in an order. 07.30 supervisor opens the test packages. Progress report is signed. 07.40. Supervisor requests the presence lists be filled out clearly during the test and students be reminded to silence the communication devices. Only pencils and erasers are allowed to be used. Students are also not allowed to choose their own seat. 265 = country code. 328 = language code. 09.30 the bell to instruct students to enter the room rings. In Room 1, students begin to complete the form with their own personal data. 09.45 test starts. 09.50 sound of car is heard. 10.40. It is cloudy, silent with bird noise. 11.00. Power cut. Dark. Voice of disappointment is heard. 11.45. Test time is over. Students should submit their work. As they get out of the room, they murmur and pour out their feeling in a loud voice and noises expressing their regrets for not being able to finish all the test items. In response to my question, they say “waktunya kurang” [“short of time”]

170 Appendix 5. Observation data of physical and non-physical environs

Code

H/Du/P/Power cut

Sgr-3-3-B-pouring out regrets Sgr-1-3-B-sorry, disappointed Sgr-3-3-B-saying time allocation is inadequate


5.1 Situation prior to classroom test 1 Comment Saturday, 9 February 2008. 10.05.

In such condition with stuffy, dark, dusty room and noisy situation, students seem absorbed studying. They do not look disturbed. They know what to do and know how they should learn. The poor quality of physical environment do not mean a thing. The class atmosphere is good. Everybody in the group is the actor in the classroom test, and as a group everybody is another motivator.


Code

I had a schedule to teach A II 1. I entered a classroom located one block away from the teacher’s room. It was a room with brown concrete wall furnished with 40 seats, 22 tables, a blackboard, a bucket, plastic flowers, posters, and slogans. There was no table cloth. I went into a class whom I did not think it was responsive. In the previous two meetings, I felt so depressed to learn that they didn’t even respond to my questions and comments. As I entered my class, I found students of A1 were studying. A student soon cleaned the blackboard. When I asked, “What are you doing?” One answered, “I’m fine”. I repeated, “what are you doing?”. There was a silence. “Are you preparing for the test?” They answered, “Yes.” Soon I took a seat. In a minute, it got noisy. Students were conversing, having discussions, writing. Sometimes, bursts of laughter were heard. Everybody was serious. No one was sleepy. There was no anxiety. They seemed to relax and undisturbed. 10.15 a student stood up and passed a piece of paper to her friend behind her. As she got back, I asked: Are you ready for the test? She smiled and shook her head, saying “Not yet”. 10.15 a student made an excuse for coming late as she had to meet another teacher. It is near the road where we can clearly hear the moving trucks and other vehicles. It is noisy too outside. The noises came from the neighbor class and building work. Suddenly the wind blew and slammed the door. Although well ventilated, it is a bit dark and dusty. It was cloudy out. The room is located in the first floor, next to the stairs and parking area. An accounting teacher came and called one student. One minute later, she is back with a note. As she took a seat, she looked back and talked to other students to inform something. Not long, she resumed work, holding a pen and memorizing. The bell rang. The first period passed by. No one approached me to propose for a test. They kept practicing, speaking in English for the competency test which asked them to tell and ask their daily activities. I kept on waiting. The bell marking time was up, but they kept practicing

5.2 Situation prior to classroom test 2

Sgr-3-1-Aholding a pen Sgr-3-1-Ahaving a discussion Sgr-3-1-Awriting Sgr-3-1-Alaughing Sgr-3-1-Amemorizing

C/Be/P/dusty C/Be/P/dark C/Be/P/noisy C/Be/NP/no anxiety C/Be/NP/seriou s C/Be/NP/undist urbed C/Be/NP/relax C/Be/P/strategic C/Be/P/well furnished C/Be/P/well ventilated


Comment Monday, 28 May 2007. 12.45. Room 9. SMK N 1 Pengasih Students are motivated to learn materials they will face in the test

Observation Data

Code

It is 12.45 after the second break’s time. In one month’s time, it will come to the end of the semester. The second graders who major in marketing (belong to P II 1) look very busy practicing their English. There is no sign that they have just enjoyed their break time. There are 16 students there. Some are studying by themselves and some others are practicing with each other in small groups. Before long, there comes a student proposing for a competency test.

Sgr-3-1-Apractising speaking in English in their break time LMOT LAUT

5.3 Situation prior to high-stakes test 1 8 February 2007 08.37. SMK 1 Pengasih Observation Data

Comment

Code

These students get

H/Be/NP/preparedness

together at tables reading and discussing learning materials. They look serious. Test time is still one and half hour away. It shows high independency on learning and tells that a test motivates learning. Such a group activity may also motivate an individual to do the

Situation 1 (prior to regional test)

5.4 Situation prior to high-stakes test 2

same

LAUT


Comment

Observation Data

Code

Anxiety about failure as well as concern about graduation rate among teachers and school official potentially leads to nonsensible instructional programs like this.

This is the second day of Unas when students take Mathematics test at 08.00 – 10.00. As scheduled, upon the completion of the test, those who scored low in English in the last try-out are required to get additional practice to prepare for the coming English test. 21 are expected to attend the class.

Sgr-3-1-A-low level of interest in attending extra lesson

I felt disappointed with this situation. But I guessed those who did not come may study on their own and therefore have a high learning autonomy

10.15. The laboratory is closed. None is seen around. 10.25. Only 4 students appear. One follows. The teachers ask, “Where are the others?”. An answer is heard, “Downstairs.”.

This student is not confident to take the coming exam, but she has a strong will to learn.

One student asks if she can join as she actually does not belong to the class. The teacher gives no clear reply. The teacher says she is afraid that the student may disturb the others. The student remains in the class.They wait for the others. 10.35. One student comes. Then, comes another. A few minutes later, a call over the loudspeaker is heard, inviting the rest of students to attend the class.

Wednesday, 18 April 2007. 10.15 – 13.30 SMK N 1 Pengasih’s language lab

. Less students come to the test preparation class indicate that learning autonomy is already there, and the test preparation class is nonsense for them. In fact it indicates teachers and officials’ big concerns about graduation

The class is at last attended by only 10 students. This is the transcript of the first part of the teaching learning process: Baik. Marilah kita berdoa dulu untuk memulai aktivitas kita hari ini. Berdoa dipersilahkan. Selesai. Baik. Selamat pagi semua. It’s good to see you again today. I hope all of you are very well and ready for the test tomorrow. All right. First of all, I would like to check your presence: Ana Uswatun, Mujiati,Tri Yuniati absent! Ratri Apriyana, Sulismiyatun, Novia Erliani, Aprilia Puji Rahayu, absent! Arin Yuliani, Maryanti, absent! Tinuk Farianti, Nurhidayah, Siti Muniroh, Suprihatiningsih, Titin Septina, Nurrohmah, Rohmatik e Rohmiatik? All right. So, only how many students here? One, two, three, four, five, six, seven, eight, nine, ten. Ten students!

173 5.5 Situation after high stakes-test 1 Comment

Observation Data

Code


Thursday, 19 April 2007. SMK N 1 Pengasih’s Students are seen talking in groups after the national examination. They may be sharing their experiences and expectation in doing the test. It means learning process continues out of the testing room.

BB LAUT LFEE LPRO

5.6 Situation after high-stakes test 2 Comment Thursday, 19 April 2007. 11.00. SMK N 1 Pengasih The test score means students’ learning feedback and enforcement

Observation Data

Code

When I am working at a table in the teacher’s room, two students of the Accounting class approach me. The students look curious to know the result of the test they have just done and ask a question. I feel sorry for I do not know the whole test material yet.

Sgr-3-3-B-checking whether their work is fine Sgr-1-3-B-curious to know the correct answers

5.7 Situation after high-stakes test 3 Comment

Observation Data

22 March 2007.

Code -

This activity shows concern about confidentiality which is expected to increase the level of validity and reliability of the test

USED TEST BOOKLETS DEMOLISHED

174 Appendix 6. Researcher’s Diary: Date

Description

Interpretation


Someday in February 2007

10/2/2007

22/3/2007

18/4/2007

12/4/2007

13/6/2007

16/6/2007 20/6/2007

16/6/2007 16/.../2007 21/7/2007 27/6/2007 21/7/2007 21/6/2007 30/6/2007 9/2/2007

A guidance and counselling teacher contacted me and required information about the implementation of sub-competency English test for the thirdgraders. She exposed the problem that there were students who would soon try to get a university’s admission. One of the requirements was to submit their schoolreport’s copy. Since they had some blanks or no marks in a number of competencies, she told me that one of them protested. In my opinion, they were afraid their application would be rejected. Second graders contact their teacher to see the result of the test asking whether they pass or not. Some showed happiness. Here, I thought it is clear that testing is in the context of communication and learning. A few moments after the TOEIC test was over, I interviewed student A. I chose her my research participant with the reason that I saw an interesting phenomenon in her. I know she is smart, but I wonder why she doesn’t seem interested in taking the classroom tests. When I was on the way to school, I met my student who had finished her English exam. I asked how she found the test. She answered it was OK. Arriving at school, when I was sitting at my table, two students approached me and with a curiosity asked how their test work was. I was a bit annoyed and said sorry that I was working and had not finished looking to the test materials. I attended the teacher meeting discussing the result of the national exam. The school principal informed that no students failed. He told he had got a number of SMS messages from students who asked the result of the exam. It confirmed that testing occurs in the communication and learning situation Students reportedly received an envelope announcing their graduation at school. School’s previous plan actually is to inform students the graduation by sending the letter to their home. But as no one failed, the plan was changed. Some celebrated the graduation by having bakso. No scratches on the shirts. Confirm Student A’s first interview When I was walking downstairs out of the computer room, a group of my students approached and followed me asking about the make-up test. I then collected them in Room 10 and asked about their feeling. I interviewed Teacher Iin I interviewed Teacher Nanik I interviewed Teacher Nanik I interviewed Teacher Rima I interviewed Student Atik I interviewed Student Siti I interviewed Student Endang I observed my class (A II 1) before the classroom test was conducted

Concern about achievement

The curiosity leads to learning feedback demand -

Students need a learning feedback

Communication act in testing LACH

Communication act in testing Real-life experience

Learning feedback

-

(cont’d) 175 (cont’d) Date 9/2/2007

Description I observed my class (A II 1) before the classroom test was

Interpretation -


11/2/2007

12/2/2008 13/2/2008

13/2/2008 13/2/2008

Date

conducted I observed Student Endang in the classroom test. Her hands are playing with her head coverings I interviewed Teacher Iin I observed Teacher Nanik’s class When I came to the lab, Teacher Nanik was walking to and fro supervising. The students were writing and holding a piece of paper. I was curious. I thought ”There must be a test.”. Little small talks were heard. A few minutes later, while trying to reach the glass cupboard where teaching materials were kept, I asked one of the students who sat at the back. ”Ulangan ya mbak?”. [Is it a test?]She answered ”Ya”. [Yes]. ”Do it by yourself!” I heard Teacher Nanik say. Before, I heard the teacher and students interacting to make a clarification of something. I was sitting in the small room adjacent to the lab. Through the glass window I could see her standing in front of the class holding a paper. The bell rang. Teacher Nanik was leaning on a table in the first row. The class remained quiet, but I saw some students turn their heads to the left and to the right. Laughters and complaints filled the air. Teacher Nanik moved to the center. Outside, some other students were making noises but it did not last long. Teacher Nanik kept standing and reading a piece of paper. At 09.25 I got out of the room. When I returned at 09.40, the class was noisy. Some students were standing and walking. ”Is there any difficult thing?” she asked. ”On part 2 you make question with do/does + verb 1.” she explained. ”Any question about simple present to talk about daily activities?” Next she moved to the new lesson, telephoning. Laughters were heard. She explained the lesson in English and Indonesian. The bell rang. Some students stood up. Teacher Nanik did not end the lesson right away. She kept explaining. One moment later at last she ended her lesson by saying ”See you next time”. Students answered, ”See you.” When she sat down. I approached her and asked,”Ulangan, Bu?”. ”Tidak. Cuma latihan.” [“No. It’s just a practice”]Then she said, ”Khawatir materi tidak selesai, selak PSG” I observed Teacher Iin’s class I interviewed Teacher Iin I heard Teacher Iin report with a pride that what she had presented in the test preparation course appeared in the last regional test, informing that there were a big numbers of the test items are similar as what they had discussed in the class.

Description

Endang’s learning style in a classroom test -

Concern about validity and reliability in a classroom test

defence LPRO LACH

Concern about her teaching achievement in a high stakes test Concern about reliability was reduced in a high stakes test

(cont’d) Interpretation


16/2/2008

I observed three participants before, during and after the local tryout exam was conducted. They were Teacher Nanik, Teacher Iin and Student Siti. I arrived at school early, at 6.50. Test was scheduled to take place at 07.30 to 9.30. At the gate, I met somebody from another vocational school hold a pile of test materials. I entered the teachers’ room and met no one. Next, I entered test supervisor’s room next door. I was not on duty then, but I had planned to do observation on my research participants. I met some colleagues there and extended greetings. I was there to find out in which rooms my participants would be. I tried to look up in the schedule put on the board to see their names written there, but what I coud read were only numbers. A bit disappointed, I returned to the teachers’ room to decipher the code. I looked up a list informing teachers’ names and their codes. At last I found that Teacher Iin would be in Room 10, Teacher Nanik would be in Room 8 and Student Siti would be in Room 9. I felt lucky, since the locations of the rooms would enable me to do observation easily. It is 07.00. It was still quiet. Only few students could be seen. I went upstairs to get a cassette and headed to Room 9 to see if Student Siti has already been there. I saw somebody reading a notice on the door. When I asked, ”Ini Ruang 9 ya?”, which got a reply ”Ya”, I soon found that it was Siti herself. ”Sudah belajar?” I asked. ”Sudah. Doakan ya, Bu.” I left Siti and returned to my seat in the teachers’ room. At 07.10, Teacher Nanik came. Smile was on her face. She approached me to shake hands. She moved and sat in front of the announcement board and talked about the weather with a colleague. Then she went to the supervisor’s room. Teacher Nanik sat by her colleague in the centre row. She was having a pencil case with her. Then she came closer to the announcement board. Meanwhile, other teachers came one by one and shook hands. Soon the room got alive. At least 32 teachers were expected to supervise the English tryout examination. Most of them were non-English teachers. Only 5 of 7 English teachers were on duty that day. They did not wear uniforms for it was Saturday. At 7.15 an announcement was heard over the loudspeaker. It informed the test rooms each group os students should use. Teacher Iin came. I met her at the door of the teachers’ room. She was heading for the supervisors’ room. Smile was on her face. As she had the test materials, she walked slowly alone to the test room. 07.22. I went to Room 9. No supervisor was seen there. The class was a bit noisy. I tried to find out where Student Siti sat. She was in the first row opposite the teacher’s desk. One minute later, one supervisor came. I noticed Siti was calm. She talked to a friend behind her. The bell rang. The supervisor distributed identity cards. Siti was holding her card. Her arms were not on the table. She was playing with a pencil. Then she was moving her body, standing and pushing her chair to the left and exchanged it with another. (cont’d)


Morning prayers would be done through the loudspeaker by school official. Outside a group of students were seen coming late. In the prayers, they begged to achieve ”predikat kelulusan yang memuaskan” [satisfying grade]. As the prayer ended, another supervisor arrived. She then helped distribute the test answer sheet right away. Siti started writing her identity calmly. Two students rushed to the room. There were only 17 students from 20 expected to come. The bell rang. Latecomers were walking hastely to find where she had to go. In Room 8, Teacher Nanik was distributing test booklets one by one. She was working with her partner. As she finished it, she approached me at the door, expressing a wish with a little disappointment, ”Ndak ada sisa soal! Mau ngrungokke, buat kunci sekalian.” [ “No spare copy. I wish I could listen too and prepare the answer key] I went back to Room 9. The supervisor was presenting the guide and rules for the test-takers. Each point was responded with a laughter. I tried to find a free seat and sat down. One student asked to the class, ”Saiki Bahasa Inggris atau Bahasa Indonesia, ya?” [Are we doing an English or Indonesian test?] Some laughed and an answer replying to the question was heard, ”Inggris...” [“English...”] I noticed the student didn’t take the right seat. I asked her the reason. She answered giving a sign that indicated the table was almost broken. In front of the class was a pile of students’ bags. Since the latecomers had a difficulty in finding their rooms, the supervisor suggested, ”Besuk jangan sampai.”[Don’t be late again next time] 07.40. A voice coming from the committee was heard over the loud speaker, informing that the committee was trying to do a checking on the technical equipment, by informing that international TOEIC test would be conducted on February 24, 2008. The class was a bit noisy. Small talks were heard. Outside, the construction workers make noises. 07.42 The last student arrived at the class. Another announcement was heard. It informed the code number for each subject tested in the national exam. 20 for English, 10 for Indonesian and 33 for Mathematics. First part of the test, was about to begin. The committee was ready to play the CD. Everybody kept silent, Everyone tried to listen. When listening test was in progress, the class was disturbed by the arrival of another student, while there was no more space available for any other test-taker. She had to go to find out where to go.

Date

Description

Concern about graduation Concern about achievement in a high stakes test

She might be interested in the test materials and want to assess the validity A high stakes test gives an opportunity to validate as what I believe teacher’s achievement It seems a silly question, but in fact I will say that the student may consider the test as a learning opportunity, therefore it is unnessary to make a special preparation

Preparedness, concern about graduation and achievem ent May be she went to school to learn, not to be measured.

Interpretation


In Room 10, Teacher Iin was supervising the class. She sat at a table. Her co-supervisor was in front of her. She looked serious. She was holding a pen and supporting her jaw and forehead. Then she was holding a test booklet. A bang was heard from outside. The class kept doing their work. 08.00 Listening section ended. Another bang was heard from outside. There were noises coming from the building project. Teacher Iin then stood up and went to another corner approaching a student to hand something for administration. A minute later, she was putting her hands on her waists for a while and then rested her hand on the student’s table. She was walking around to do checking. Two lights on the ceilings were on. The windows were open. I moved back to Room 8. Teacher Nanik sat at a table while her co-supervisor was standing. In Room 9, Student Siti kept doing her work seriously. A friend on her right made a sign to borrow something from her and she had to serve her. She remained silent. A few minutes later, there seemed to be a problem with the test. Talks discussing something to clarify were heard. 08.19 Teacher Iin sat at a table looking down in silence, then she had a small talk with her partner. She continued to write and complete the room administrator. She was holding a piece of administration paper. She put her head on her arm for a while. She rose to her feet and walked to the centre to talk to her friend. 08.30 Room 9.Student Siti was still doing the test seriously. Meanwhile in Room 8, Teacher Nanik still sat at the table. I approached her. She told she had just got the extra copy she had wanted. I went out and sat on a chair available near the door.The noises and laughters coming from the construction work were still heard. Meanwhile, the sun started to shine. The breeze blew. 08.40 My collegue Mr X who handled the TOEIC matters passed by and put an announcement on the board located next to the student’s mini shop. To my surprise, the student who was the last to come to the class had finished the test. One by one followed her. I soon went back to Room 9, but Siti seemed absorbed and undisturbed. She was holding the test paper in upright position and then lowering it. Next she was putting her left hand under her left forehead. 08.46. Five more students got their bags and went out one by one. Siti relaxed and remained serious. Twelve students had left. The supervisor started to collect students’ work from the tables. Another student went out of the room. At 08.50 Siti finished the work and left the room. She went with her bag hanging on the shoulder to the announcement board to check whether her name was printed out among those who would be sent to take the international TOEIC test. As I was walking to follow her, Mr. X stopped me and discussed anything for the coming international test. As I got closer to Student Siti, I asked her, ”Bagaimana? Ada ndak?” Siti answered ”Ada” ”Kok bisa? Berdasarkan apa to?” she asked me. ”Berdasarkan ranking tes regional” I answered. Then she exclaimed, ”My God!!!”. Date

Description

LPRO They were absorbed in studying. They had intensive learning

If this had been a classroom test and a teacher like Iin and Nanik had seen it, Siti and friends may have given a warning or punishment.

LPRO

Learning achievement in a high stakes test It confirms that a high stakes test is considered as a test with a higher status and prestige which enable to provide a sense of accomplishment. Interpretation


08.55. Teacher Nanik was busy collecting the work. There was almost no one in the room. Announcement was heard again, informing that students had to return the test booklet and other information. 08.56. I returned to Room 10. All students were still there. One of the supervisors were talking to her guests at the door. Students had some little talks but I think it was still under control. Teacher Iin still sat at the table, writing and reading the test booklet. 08.58. One student got out, followed by other students. Teacher Iin stood up folding her arms and soon collecting the work. I left the room at 09.00. I returned to the supervisor room. I found Teacher Nanik having a talk with two friends. I was curious. Then I realized they were discussing the test content with two English teachers. 09.05 Teacher Iin entered the room and handed the work to the committee. She helped herself with a glass of tea available on the front row and took a seat not far from me. She read the test booklet again and asked me the meaning of a word found in the text, but she could guess what it meant. She then said something about whether she got a task to do correcting students’ work. A non English teacher asked me about who was speaking in the test material.

24 February 2008

Test was scheduled 8.00-10.00. It was a high-stakes test with 92 student test takers coming from three vocational schools. 07.30. When I arrived to do my duty in the event of a highstakes test, the program was about to begin. Piles of test booklets were on the tables in the committee room. A testagent from a reputable company started with his introduction, explaining his status as a supervisor and some terms used. He explained the formalities to be carried out during the test implementation, guidelines and the rules of the game. A number of teachers who were to act as a room administrators were listening attentively. Some rules said only student test takers are allowed to break the seal of the test booklet, it is prohibited to do a test on behalf of somebody else and anyone breaking the rule should be sent out of the class and if this happened, test-score would not be provided. 08.10. The bell rang. Each room administrator went to the respective room. They were required to present the rules available in the guide to the test takers. It was a bright day. Bird voices filled the air. There were 20 test takers in the lab. They started it with a prayer. 08.15. All the room administrators were expected to return to the committee room. Some were seen standing, discussing with the supervisor (test agent) around the table. Some were holding materials, expressing complaints and need for clarification. Some of them had reportedly distributed the answer sheet while they were not expected to do so. Realizing her mistake, Teacher Nanik expressed her feeling of guilty and confusion, “Gemrobyos aku!” [ Sweaty am I!]

Concern about confidentiality, practicality, validity and reliability

Concern about learn ing feedback, enforcement

This employed a lot of rituals. The formalities this testing event brought to the site might be seen as increasing the status level.

A trivial matter could become serious in a highstakes test Cont’d


Date

26/2/2008

13/3

15/3/2008

Description As the test was over , all the room administrators returned to the committee room. There were piles of used test booklet. I was curious to see the materials and soon I learned something went wrong . Ooops! I felt my blood rushing to my head. I hold my head when the agent pointed to my “fault” of opening the test booklet. With great suspicion, he even grabbed the small tape player I put on the table. Teacher Yayuk, my colleague, told me that she had met our fellow teacher (Teacher Lucy) the day before. I heard that Teacher Lucy had kept on asking for the test booklet and cassette of the previous second-opportunity national exams. That was the third time for her to beg us to show what it looked like. ”What’s up”, I questioned myself. Since December 2007, the have been a number of demands for the same test material. There has been a rumor that the coming national exam will use either similar or exactly the same materials. My colleague expressing words of anger reported the implementation of the regional test which did not run well. After tryout implementation, the neighbor school hold the test. Disrespect, students leaving the testroom before the time ended, asking ”what are all of these for?” A number of copies were missing. When I came to the room, Student Endang was serious. She was sitting next to the wall. On her table was her black bag. There were 18 students there. The room looked dark. Electric fan was turning around. The supervisor, sitting next to the door was sleepy. One of the students put her head on the table for a long time. Endang was reading the test paper. Her mouth was murmuring, reading something in the test paper.

Interpretation Concern about confidentiality in a high stakes test

Concern about graduation rate Concern about validity and reliability reduced

This is a situation when students do not really need a type of high stakes test

Being serious in a test activity means being serious in studying. Finding some difficulties means finding some things new

Student Endang’s table

A few minutes later... Sometimes she put her hand on her head. When somebody was passing by in front of the door, the supervisor stopped and made a conversation with the supervisor. Endang kept serious. She was not influenced or disturbed. She kept working. A few minutes before time was up, the supervisor said ”Boleh keluar” [“You can go out”], but Endang even still looked more serious.

Endang must have learned a lot of new things from the test She might have wanted to learn English more. Learning it through English testing was challenging.

181 Date

Description

(cont’d) Intepretation


One by one student went out, the supervisor collected the work from the students’ tables. When the supervisor asked for her work, she soon went out. As she got to the door, I stopped her and asked,”Bagaimana Endang?” [“How did you find the test?”] She smiled and said ”Ya lumayan susah....” [“Well, a bit difficult”]. I asked for repetition, ”Apa?” [What?] ”Lumayan susah....” [”A bit difficult”].

182 Appendix 7 Interview data 7.1 Student Atik 1

As Endang was concerned about learning, ” a bit difficult” meant the test brought new lessons to her.


Comment

Narrative

Interviewer: Researcher Interviewee: Student A Setting: SMK N 1 Pengasih. 22 March 2007. 13.50 – 14.25

[...] E, di rumah biasanya cara belajar Mbak Atik itu bagaimana?

SA has a high sense of learning autonomy. She has critical reflection on what learning situation should be. In her view, learning English should be enjoyable. There should be enjoyment in learning English. She has independent action to fulfill her needs, to make her own decision on what and how she should learn. She knows that testing is contributive to her learning. As she knows the test format and materials, she knows what to do

Ya, jadi bagaimana belajar itu kan tidak sebenarnya dalam Bahasa Inggris khususnya, sambil nonton TV pun bisa, karena sekarang di TVRI itu ada program pendidikan dan bahasa Inggrisnya banyak.

Kalau mengenai waktu, saya tidak kontinyu. Jadi tidak harus setiap hari misalnya belajar dua jam, tidak, tapi tergantung mood karena yang saya tahu, apabila belajar itu dipaksakan, justru kita akan menjadi benci dengan belajar.

Code

LAUT

Jadi disesuaikan dengan mood? LAUT

Banyak bahasa Inggrisnya, ya.Belajarnya biasanya sendiri atau ada diskusi dengan teman atau bimbingan orang tua. Saya lebih senang belajar sendiri karena untuk belajar dengan teman nanti justru akan e mengobrol. Dan untuk orang tua, karena orang tua saya hanya guru SD, jadi tidak mampu lagi membimbing saya yang sudah SMK.

LAUT

Kalau belajar Bahasa Inggris di rumah, biasanya misalnya ya reading. Itu terus membaca keras atau cuma dalam hati gitu? Yang dipelajari apa? Kalau reading, saya membaca dalam hati dan kemudian kalau ada kata-kata yang tidak saya mengerti, saya cari di kamus. Dan untuk listening, saya sering nonton film luar negeri dengan volume suara yang keras.

LAUT

Kalau mendengarkan radio siaran luar negeri? Oh, di rumah saya tidak ada radio. Oh, adanya televisi. Suka browsing internet tidak? Suka, tapi tidak untuk pelajaran. Oh, ya sebetulnya di situ banyak materi-materi Bahasa Inggris dan bisa apa saja. All right. [...]He’e ya. Baik. Sekarang mulai masuk ke masalah tes, ya. Dari sejak SMP sampai sekarang kan banyak ragam tes, ya. Kira-kira kalau kamu tahu akan dites Bahasa Inggris itu ada pengaruhnya tidak di dalam cara belajar. Wah ini, saya mau tes, terus usaha mati-matian atau belajar yang belajar tidak tergantung dan lain sebagainya. Sedikit banyak tes itu mempengaruhi cara belajar. Bagaimana pun kita akan berusaha belajar lebih intensif.

BB LMOT

Jadi kalau misalnya yang diteskan seperti tes TOEIC tadi, kirakira persiapannya apa? (Cont’d) (cont’d) Belajar mengenai soal-soal tes TOEIC. Kan e beberapa kali saya pernah menjalani tes TOEIC, jadi sudah tahu apa kita-kira soalnya seperti apa, misalnya soal ya grammar, reading. Jadi

Sat-3-1-B-doing test practice


belajar mengenai yang kira-kira keluar. Oh ya. Baik. Itu dilakukan sendiri ya tanpa diskusi dengan teman, atau gimana ya?

Sat-2-1-B-she knows what the test is like and what will be tested

Sendiri

Sat-3-1-B-studying alone

Sendiri, ya. Menurutmu alasan mengikuti tes itu apa ya? Seberapa penting ikut dalam tes Bahasa Inggris itu? Menurut saya cukup penting karena untuk mengetahui kemampuan kita dalam Bahasa Inggris. Kita sudah sampai sudah level apa? Selain itu untuk tes TOEIC menurut sepengetahuan saya juga sertifikatnya berguna untuk mencari pekerjaan.

Sat-I-B-assess one’s ability Sat-I-B-know level Sat-I-B-career development

Oh, ya. Ha, sekarang yang bukan tes TOEIC ya seperti misalnya competency test, speaking test. Menurutmu, bagaimana itu kalau menempuh tes seperti itu? Ada kaitannya dengan belajar tidak, ya? Ada pengaruhnya untuk belajar tidak itu? Ada, sebetulnya ada. Kan competency test, walaupun itu speaking, tapi kan dalam speaking kita juga harus memperhatikan grammarnya, tidak asal ngomong, harus memperhatikan tatabahasanya. It means that grammar and speaking are interrelated

Tapi kalau menjelang tes kompetensi itu apa yang kamu lakukan, biasanya? Biasanya, e, membaca modul. Itu kan tes competency test kan biasanya ber.. berupa pertanyaan yang jawabnya seperti mengarang. Walaupun dialog, tapi dialognya harus mengarang. Dan di dalam modul itu kan banyak contoh-contoh dialog. Jadi kita belajar sebagai contoh kita membuat dialog. Selain itu kita juga tanya-tanya sama teman-teman yang sudah tes duluan.

Sat-3-1-A-reading modules, learning from model dialogs Sat-3-1-A-asking friends who has taken the test

He’em, ’gitu ya. Baik ya. Menurut pendapatmu, mana yang lebih menarik? Dengan speaking test seperti itu atau soal-soal seperti TOEIC, atau soal-soal seperti Unas, atau mungkin model-model tes yang lain? Kalau saya pribadi lebih menyukai soal-soal seperti TOEIC. Jadi listening, dan apa namanya, grammar, reading dan seperti itu. Ya, speaking itu juga penting, ‘tapi e kan untuk membuat dialognya itu agak malas, ‘gitu.

Sat-1-1-A-reluctant to prepare for speaking test

Oh, agak malas, ya. Dan itu ‘kan seperti dari awal itu seperti tidak wajib. Jadi, katanya, walaupun tidak selesai tetap lulus, gitu Bu. Dan temanteman juga masih di A5. Kita masih santai: teman-teman juga masih A5. Dan itu juga pengaruh teman. Kalau temannya nggak tinggi ya ’nggak.

Sat-2-1-A-not necessary to take speaking test Sat-4-1-A-not motivated

Artinya tidak termotivasi, wah saya ingin cepat-cepat selesai. Masih banya kompetensinya yang ingin saya raih. ’Nggak ada pikiran seperti itu? (cont’d) ’Nggak. Ya karena kan belum ada teman yang di atasnya jadi belum ada yang ditanyai, gitu lho Bu. Biasanya tanya hasil competency test. Terus tanya, kemarin ditanya apa sama Bu

Sat-1-3-Atraumatic


Guru? Terus gimana gimana. Dan saya juga pernah agak trauma. Trauma? Karena pertanyaan menyimpang jauh dan membuat saya bingung menjawabnya.

Sat-1-2-A-confused to answer

Kok menyimpang bagaimana? Pertanyaannya terlalu jauh. Misalnya tentang ... sebentar ... A5 itu apa Bu?

Sat-2-2-A-questions are not relevant

A5 Describing people The excessive number of questions may lead to harmful backwash

Kalau ndak salah itu malah ditanya – saya dulu pengurus OSIS – sebagai pengurus OSIS gimana menurut kamu kepala sekolah kita? Terus gimana, berani ndak kalu mengajukan usul kepada kepala sekolah, yang tidak ada hubungannya dengan describing people. Tapi kan itu hanya pertanyaan tambahan kan. Jadi yang describing peoplenya mungkin sudah selesai, terus, bapak Ibu guru atau temanmu? Bapak Ibu guru yang penguji ingin menggali lebih banyak ya kemampuan Bahasa Inggrismu. Tapi setelah itu saya jadi takut, Bu.

Sat-1-3-A-afraid

Kok takut? Terlalu banyak pertanyaan itu saya takut.

Sat-1-3-A-afraid of excessive questions

Ha, terus menurut pendapatmu, kemampuan bahasa Inggris yang ingin kamu raih itu yang seperti apa? Terus nanti bayangan untuk masa depanmu itu bagaimana? Her concept of learning English, her concern about learning achievement, her view about reability, interacted with her experience leads to autonomy

E, misalnya kemampuan Bahasa Inggris itu ya harus didasari dengan teori yang bagus, harus dengan pengetahuan tentang grammar, tentang vocab, kemudian dengan listening. Setelah listening baru speaking. Dan untuk saat ini saya untuk speaking belum cukup mampu untuk...

LAUT

Makanya itu saya ingin tahu kemampuan Bahasa Inggris apa yang inginkamu raih sebetulnya? Ya, untuk ke depannya.. untuk kedepannya saya juga ingin listening yang benar-benar bisa listening yang dengar bisa paham misalnya nonton TV.

LAUT

Jadi berarti. . . In her view, speaking correctly can only be mastered only if she masters its theory (grammar and vocabulary)

Selain itu speaking tetapi tidak hanya speaking seperti penjual yang melayani turis. Itu kan speakingnya waton dimengerti ’gitu lho Bu. Saya ingin speaking yang grammarnya juga betul, gitu.

LAUT

Oh, ya. Terus usahamu kalau menempuh cita-citamu itu apa?

(cont’d) Sejauh ini kalau soal listening itu dengan tadi dengan nonton. Terus dengan speaking, baru mempelajari teorinya. Untuk prakteknya belum

LAUT


Jadi tidak pernah berlatih dengan temannya? Tidak pernah reading aloud, artinya membaca dengan nyaring, ya, untuk melatih pronunciation. Tidak pernah dilakukan? Belum Oh, ya. Baiklah ya. Menurutmu teman-temanmu itu cara belajarnya kira-kira sama dengan belajarmun atau tidak? Reading is the main activity

Saya kira teman-teman saya belajar dari membaca Baik, sekarang saya ingin tanya, perasaanmu, ya, suasana hatimu menjelang, pada saat dan setelah tes berakhir.Ya kan sudah menempuh berbagai macam mulai dari Unas, ya itu perasaanmu bagaimana? Apa takut, atau biasa-biasa saja atau suatu hari yang dinanti-nanti, ya. Mohon diberi penjelasan, gitu ya, suasana hatimu menjelang, pada saat dan setelah tes, ya. E, kalau menjelang tes, perasaan takut itu pasti, mesti, ada, tapi kalau saya tidak terlalu takut karena ya apa pun itu yang diteskan kan sudah disesuaikan dengan kemampuan anak SMK. Jadi saya yakin tidak akan terlalu sulit bagi anak bagi anak SMK.

Sat-1-1-B-not too afraid Sat-2-1-B-materials within reach Sat1-1-B-sure test not be difficult

Baik. Kalau waktu di SMP dulu, jenis-jenis tesnya itu bagaimana ya, barangkali lebih detail lagi mengenai tes waktu SMP. Ulangan, ya, ulangan, kalu ibu guru memberi ulangan dengan kertas dibagikan atau bagaimana? Coba diingat-ingat. Atik had been engaged in a paper-and-pencil English test. She might have felt secure with it.

E, ada soal itu yang dengan kertas dibagikan itu. Terus soalnya berbentuk beberapa berbentuk multiple choice, melengkapi kalimat dan ada essaynya sedikit dan essaynya itu simpel.

Perasaanmu menghadapi tes seperti itu dulu merasa takut tidak berhasil atau punya gambaran yang seperti apa. Coba diterangkan. Atau biasa saja. Ya ada rasa takut, tapi lebih takut yang sekarang yang di SMK daripada dulu. Oh, gitu.

Simplicity in the test might reduce anxiety thus bring comfort

Karena dulu itu lebih ya simpel, gitu, belum ada grammar, jadi masih simpel sekali. Jadi menjelang ujian perasaan cuma sedikit takutnya kan tidak begitu parah jadi belajarnya juga terus tidak didobeli dari biasanya. Cuma biasa-biasa saja? Ya ditambah sedikit. Tidak sampai wayangan begitu lho, tidak. Tidak sampai wayangan ya. Baik kalau begitu belajarnya memang tiap hari kontinu beljar, menjelang ujian ditambah itu. Jadi perasaat takut atau minder tidak pernah ada, ya. Ada, sedikit. (cont’d) Ada sedikit oh sedikit. Baik. Pada saat tes, menurut pendapatmu situasi Kadang-kadang apa merasa jengkel dengan sesuatu tau apa. Boleh pada saat uji kompetensi, boleh pada saat tes TOEIC seperti itu boleh pada saat penjajakan dan lain sebagainya.


Kalau suasananya tenang, e suasana perasaan cukup tenang itu sudah baik. Tapi yang saya tidak suka kalau ada misalnya pas penjajakan waktunya sudah selesai tapi teman-teman sudah keluar mengganggu konsentrasi. Pada hal kita sebaiknya menghabiskan waktu tes itu. Walaupun kita sudah selesai. Kita menghabiskan waktu itu untuk mengulang terus mengulang. Terganggu adanya siswa yang keluar lebih dahulu.

Sat-1-2-B-disturbed by other’s leaving LPRO

Kalau Bapak/ibu guru pengawas? Barangkali ada perilaku yang bisa mengganggu selama ini? Kalau tidak ada ya katakan tidak ada, tapi kalau ada tolong gambarkan. Sejauh ini perilaku Bapak ibu guru yang mengawasi tidak ada yang mengganggu, namun perasaan saya sendiri, misalnya kemarin pas pada penjajakan Bahasa Indonesia. Yang mengawasi kan Bu Yuli, guru. Bagaimanapun ada sedikit nervous, gitu. Jadi termasuk juga pengawas itu juga menentukan suasana pada saat. .. Saya kan kemarin takut kalau nilai Bahasa Indonesia saya jelek. Yang ngawasi pas waktu itu pas Bu Yuli. Kalau misalnya mengerjakan tes itu diawasi oleh pengawas yang bagaimana? Pengawas yang bukan guru mata pelajarannya dan justru tidak mengajar, jadi kurang mengenal saya, begitu Bu. Ya artinya waktu di kelas itu Bapak /Ibu pengawas itu memberikan perhatian yang lebih artinya melarang apa pun yang agak apa di luar. Wah bagaimana yang cara Bapak/Ibu guru memberi memberi perhatian yang lebih banyak kepada para peserta itu atau yang agak kendor pengawasannya dan lain. Kalau lebih suka yang agak kendor pengawasannya karena. Sebenarnya misalnya saya nggak nurun nak kerjasama tapi kalau dilihati bapak/Ibu guru bagaimana ya nervous. Ada lagi pengalaman yang lain mengenai tes Bahasa Inggris. Perasaan senang, sedih, gembira, terserah atau jengkel mengenai pemberian nilai dari Bapak Ibu guru. Kalau jengkel sih enggak, Bu tapi ada yang di luar dugaan gitu Bu. Misalnya ada komentarnya banyak terlalu banyak mengritik, tapi kok nilainya bagus. Ada yang diam sepertinya saya tidak melakukan banyak kesalahan, tapi ternyata nilainya jelek. Seberapa penting sih nilaimu itu terhadap pengaruh belajarmu? Misalnya tiba-tiba saya kasih mbak nilai 6 atau 5. Terus perasaanmu itu ingin memacu belajar lagi atau jengkel kepada yang memberi nilai? Atau bagaimana perasaanmu. Kalau masalah nilai yang kamu alami sampai sekarang. Kalau tidak ya. Kalau saya lebih ke meningkatkan nilai lebih dari 6.

Sat-1-3-Asurprised to see the score Sat-4-2-A-being criticized Sat-4-2-A-getting no comment on her performance

BB

(cont’d) Kalau sudah baik? Kalau misalnya 9, 10? Mempertahankan. Saya ingin mempertahankan. mempertahankan lebih sulit daripada meningkatkan.

Justru

BB


Jadi tidak berhenti belajar. Jadi belajarnya terus. All right, Sekarang mengenai penyampaian materi Bapak/Ibu guru di kelas, ya. Apakah ada sesuatu yang berkesan dan lain sebagainya atau cara mengajar yang kurang pas dengan minat mbak .Atau wah bagus sekali ya, disajikan ini, disajikan itu. Ada yang bagus waktu belajar listening. Setelah misalnya mendengarkan 10 soal listening. Setelah itu pembahasan dan waktu pembahasan saya lebih senang kaset itu diputar lagi. Jadi misalnya nomor satu, ibu guru memberi jawaban A tapi setelah itu diulangi lagi mendengarkan apakah betul A itu. Kan ada guru yang memberikan hanya kuncinya saja. Nomer 1 A, 2 B.

LENF

Terus mengenai perasaaan kalau sudah selesai mengerjakan tes? Tergantung pada tesnya. Haa ya. Yang sudah kamu alami itu bagaimana? Kalau tesnya susah. Haiya. Lebih spesifik, misalnya tadi atau waktu penjajakan dulu. Perasaanmu bagaimana? Coba diingat-ingat. Setelah habis mengerjakan Setelah penjajakan ya mungkin perasaan saya agak tenang, karena saya merasa agak bisa, tapi kalau seperti tadi, ada rasa sedih ada rasa di sini pusing, di sini rasanya berat sekali. Karena tidak bisa. Karena belum selesai mengerjakan, takut nilainya jelek Belum selesai, berarti waktunya kurang. 2 jam 200 soal, ya. Memang kalau penjajakan kan 2 jam berapa soal?

Sat-1-3-B-relax Sat-1-2-B-can do the test Sat-1-3-B-sad,dizzy Sat-1-2-B-cannot do the test Sat-4-2-B-fail to finish the test Sat-1-3-B-afraid to get low marks

Ya. Kalau tes itu sebetulnya e ada tidak ya di dalam itu pikiran bahwa kamu sendang melakukan komunikasi dengan pembuat tes? Sejauh ini saya belum berfikiran seperti itu. Oh, belum ya. Menurut pendapatmu tes yang mengesankan itu yang seperti apa? Atau sama saja yang satu dengan yang lain? Menurut saya kalau dibandingkan antara TOEIC dan tes penjajakan menurut saya lebih mengesankan TOEIC. Kamu mengerti gunanya TOEIC? Kok kamu harus mengikuti tes TOEIC tahu tidak manfaatnya untuk apa. Kalau yang saya tahu ya untuk mengetahui sejauh mana kemampuan kita dalam Kan TOEIC ada readingnya kemampuan kita dalam reading. Vocabnya juga dan grammar. Cuma yang belum ada dalam TOEIC kan speaking.Maksud saya orang ramai-ramai, termasuk mbak mengikuti tes TOEIC e faedahnya apa di luar untuk mengetahui. Faedahnya apa mengikuti tes seperti itu? (cont’d) Apa ya? Belum mengerti? Oh ya. Mengenai sikapmu seperti competency test ingin mengetahui dan meningkatkan antara tujuan seperti itu


dengan pelaksanaan dengan materi tesmu. Menurut pendapatmu bagaimana sikapmu. Kalau misalnya ya sekolah ini ya. Sekolah ingin siswanya pinter ngomong, pinter. Lalu diberikan tes speaking competency tes. Menurut pendapatmu sikapmu terhadap tujuan dan pelaksanaan tes itu bagaimana? Sikap bagaimana? Maksudnya apakah kamu termasuk yang mendukung wah ganti saja modelnya Saya agak keberatan dengan metode seperti itu. Saya lebih suka kalau ya mungkin ada Yang waktunya dibatasi saja, menurut saya mulai dari kelas satu bulan I semua harus lulus A1, bulan II. Ya memang yang ideal Tapi pertanyaan sebetulnya sekolah punya cita-cita.Lalu diadakan tes kompetensi itu. Pertanyaan saya sikapmu terhadap antara tujuan dengan yang di...Bukan masalah idealnya seperti tapi kenyataan selama di sekolah tidak bisa berjalan siswanya kan sikapnya lain-lain. Sekolah ingin memberi kemampuan speaking lalu dites. Ya menurut saya bagus. SMK kan kebanyakan langsung kerja, jadi penting apalagi menghadapi globalisasi katanya banyak orang luar yang masuk sini. Dunia kerja pekerja dan untuk meraih tujuan sekolah mengadakan competency test memang seperti itu. Speaking harus dilatih. Itu tadi waktunya diharuskan.

Sat-I-A-rules of the game

Ditarget siswanya berarti. Jadi itu menyangkut proses. Kamu mendukung kalu prosesnya kurang berjalan selama ini. Menurut saya, speakingnya tidak hanya di tes, tapi dipelajari sebaiknya ada pelajaran. Apa nggak ada. Masa. Please practise. Membacakan pernah Membuat dialog. Ya, pernah, pernah Bu. Tapi itu. .. Terus yang berikutnya materinya. Materi di masing-masing tes materi di apa di kompetensi tes, materi di penjajakan, materi di TOEIC, mungkin materi Unas. Kira-kira menarik tidak isinya? Atau kalau Materinya kamu lihat materi Umpamanya TOEIC tes. Sedikit banyak tentu masih bisa mengingat. Kalau yang saya inginkan memang sebenarnya. Concern about linguistic knowledge and achievement

Misalnya error recognition itu misal sebenarnya yang diarah oleh soal itu misalnya mengenai gerund, tapi kalau di TOEIC itu sepertinya itu tidak terlalu banyak kalimatnya itu lho, Bu. Jadi kalau gerund langsung bisa ditangkap kalau itu yang. Kalau di penjajakan error recognition panjang-panjang. Itu membingungkan. Tapi kan kalau listening diulang-ulang kali. Menurut pendapatmu sebetulnya?

More concerned about achievement

Menurut pendapat saya lebih baik yang satu kali. Secara fisik penampilan buku soalnya, tentang yang TOEIC maupun huruf-hurufnya, kualitas cetakannya dan lain sebagainya. (cont’d) Mengenai [tes] yang regional sama penjajakan itu kan hampir sama mengenai gambar. Materi [tes] gambarnya dibuat yang seperti TOEIC internasional. Terus materi listeningnya. Materi itu juga sama. Paling jelas dari penjajakan yang TOEIC


international. Kalau yang penjajakan? Paling tidak jelas. Ah masa? Tidak jelas? Dibandingkan dengan TOEIC international. Tapi kan masih ada keuntungan itu 2 kali. Baiklah untuk hari ini sekian dulu ya. Kalau Setelah Terima kasih atas. Lain waktu kita ngomong-ngomong lagi ya.

190 7.2 Student Atik 2 Interview Transcript


Comment Interviewer: Researcher Interviewee: Student Atik Setting: SMK N 1 Pengasih. 21 July 2007. 10.00 – 11.00 Lower level of difficulty reduced anxiety

Narrative

Code

Mbak Atik, coba berikan gambaran seperti apa perasaanmu waktu mengikuti unas yang terakhir, terutama pada saat emm pelaksanaan ujian Bahasa Inggris berlangsung. E sebelumnya sebelum ujian itu memang saya ya agak nervous, nervous juga ada tapi ketika saya sudah baca soalnya ya perasaan nervous itu sedikit terkurangi karena saya melihat bahwa soal ujian nasional itu malah lebih mudah dari pada soalsoal ujian nasional ya mungkin tahun-tahun lalu. Jadi lebih sederhana.

Sat-1-1-B-a little nervous Sat-1-2-B-relieved Sat-4-2-B-getting easier questions

Ya nervousnya itu karena apa kira-kira? Awalnya nervousnya itu takut karena tidak bisa mengerjakan karena membayangkan soalnya akan sulit.

Sat-2-1-B-test will be difficult

Tapi pada saat melak.. melaksanakan tes itu sendiri perasaanmu? Sedikit hilang karena ya soalnya tadi sepertinya cukup sederhana. Dan untuk listeningnya juga sangat jelas walaupun ya saya ada yang lepas sedikit, tapi sangat jelas

Sat-4-2-B-clarity of listening material

Materi apa kira-kira di dalam unas itu yang membuat kamu merasa, oh ini mudah, oh ini sukar. Bagaimana? Yang saya anggap mudah itu bagian grammar, tapi yang susah itu yang ada teksnya di atasnya itu ada teks mungkin iklan iklan atau apa terus ada pertanyaannya di bawahnya.

Sat-4-2-B-gram easy Sat-4-2-B-reading difficult Sat-4-2-B-difficulty

Reading itu ya? Bagian reading? Ya, reading. Kira-kira yang menyebabkan kesulitan apa? She might have got materials not taught in the class.

Pemahaman. Pemahaman ya? Kosa katanya barangkali?

Doubt is due to the test material.

Ya, soalnya apa yang dimaksud dengan ini, itu. Kemarin yang soal kemarin yang saya susah memahami itu ada e 24 jam service itu. Maksudnya itu masih bingung saya. 24 jam buka terus atau dalam 24 jam jadi, gitu lho. Saya bingung.

Sat-1-2-B-doubtful

Oh. Tapi kamu bisa menyelesaikan semua soal dengan baik, itu?

Ya e saya rasa ya. Kalau yang reading itu saya hanya ragu-ragu satu itu, tapi entah pada akhirnya itu yang salah atau bukan, saya juga tidak tahu. Tapi yang listening saya juga lepas satu. OK. E dalam waktu 2 jam itu waktunya bisa apa namanya

(cont’d) Sat-1-3-B-doubtful where went wrong Sat-1-3-B-sure where made mistake


digunakan sebaik-baiknya atau masih kurang? Kalau nggak salah, terakhir itu sangat waktunya sangat cukup. Dari waktu masih setengah jam kalau nggak salah, teman-teman sudah keluar semua. O, jadi waktu sangat sisa.

Sat-4-2-B-abundant time

Jadi mbak[ ] juga termasuk yang keluar dulu. Ya, karena teman-teman semuanya sudah keluar. Karena menurut wawancara saya yang terdahulu kan sebaiknya mengerjakan sampai waktunya habis gitu ya? Ya Tapi waktu waktunya dihabiskan sampai Tidak itu karena memang saya memang mengusahakan sampai waktu habis tapi teman-teman keluar dan tinggal dua orang. Jadi tidak mungkin saya menghabiskan waktu sendirian.

Sat-3-2-B-trying to use up allocated time Sat-2-2-B-impossible work alone in class

Oh ya, jadi kira-kira lebih cepat berapa menit bisa menyelesaikan. Keluar dari ruangan itu kira-kira waktu yang tersisa masih berapa? Sekitar setengah jam.

Sat-4-2-B-having spare time

Cukup lama kalau gitu ya? Itu pun sudah sudah saya ulang beberapa kali, gitu.

Sat-3-2-B-doublechecking the work

Jadi selesai mengerjakan terus dicek lagi. Kerjanya gitu di selama. Baik. Tiga puluh menit itu sudah terpaksa keluar karena mengikuti teman-teman yang semua sudah keluar.

Sat-1-2-B-had to go out before time was called.

Kalau kamu lihat di lingkungan sekitarnya juga banyak yang sudah keluar atau hanya khusus kelasmu sendiri, kelas AI1. Karena mungkin canggih-canggih terus.. Saya rasa anu malah A1 itu A1 itu justru yang akhir. Kelas lain sudah keluar sejak tadi. Waktu 2 jam itu ternyata banyak tersisa ya, Jadi sangat cukup istilahmu ya. Concerned achievement

about Mungkin kalau dikerjakan itu dan tidak perlu mengulang berkali-kali satu jam saja cukup. Oh, mungkin sudah terbiasa dengan soal TOEIC test yang 200 jumlahnya. Kan sama 2 jam tapi 200 soal. Itu berapacuma 50. Nah, pertanyaan saya, setelah keluar itu apa yang kamu lakukan?

E, berdiskusi dengan teman-teman, mencocokkan jawaban. Misanya nomor sekian jawabannya apa. Jawabanku apa gitu.

(cont’d) Sat-3-3-B-discussing answers

Yang didiskusikan itu soal yang bagaimana kira-kira? Sat-3-3-B-discussing


Yang susah, yang ragu-ragu.

difficult questions

test

O, terus setelah mendapat jawaban dari temanmu Tentu saja yang mungkin jawabannya beda dan ketahuan kalau saya yang salah, terus saja kecewa. Oh ya. Jadi mungkin kalau ada temannya o sama Oh ya jadi merasa lega gitu.

Sat-1-3-B-hate find mistakes Sat-1-3-B-relieved finding her answer similar to friends’

Ya. Oh. Jadi setelah selesai mengerjakan unas, di luar sama temannya berdiskusi mengenai jawaban-jawabannya.Kalau mengenai instruksi tesnya sendiri? Pengalaman di situ bisa diceritakan tidak pada teman-teman. Misanya selama di kelas tadi ada trouble atau apa. Tidak ya. Kalau keluar biasanya memang hanya mendiskusikan jawaban soal.

Sat-3-3-B-discussing the answers

Baik. Na setelah ini agak meloncat sedikit ya. Setelah mengetahui hasil pengumuman ya kan nilainya kan dicantumka ya setelah lama terus dicantumkan pengumuman. Kira-kira reaksi dirimu bagaimana? Kecewa.

Sat-1-3-Bdisappointed

Kecewa? Kecewa karena prediksi saya hanya satu yang salah. Dan prediksi saya kan hanya satu. Yang listening itu saya jelas-selas saya lepas gitu, sama sekali kok nggak dengar kata-kata itu. Yang reading itu kan itu saya yang ragu-ragu itu satu. Tapi sudah saya tanyakan sama guru bahasa Inggris dan e dan beliau guru Bahasa Inggris itu juga membetulkan jawaban saya gitu. Tetapi ternyata salah dua. Reading itu saya yang ragu-ragu itu satu.

Sat-3-3-Bcontact English teacher to confirm

Kamu berpendapat itu kamu salah dua. Yakin? Kalau nilai yang kamu lihat itu berarti kamu menyimpulkan oh salah saya sekian itu atau punya persepsi yang berbeda? Dengan nilai 9.6 itu kan ya saya salah dua.

Sat-3-3-B-make a conclusion, relating the score with her mistakes

Gitu? Ya. Apakah tidak begitu? Ya, ya mungkin.Ya karena di dalam unas itu ya ini sedikit berikan pengertian ya e kriteria mengenai penilaiannya kan memang tidak diinformasikan ya. Bisa jadi bisa saja itu memang kamu salah satu, tapi yang jelas yang bisa saya tangkap kamu merasa kecewa dengan nilanya kok begitu.Ya. Baik. Terus menurutmu kalau begitu perlu tidak diberikan informasi kriteria untuk penilaiannya.

(cont’d) She is concerned about her learning achievement, therefore she needs a

Perlu. Perlu, sangat perlu. Jadi ya yang saya tahu nilainya itu sama. Jadi kalau salah satu ya dikurangi 0.2. Kalau salah 2 ya dikurangi 0,4. Tapi kalau seperti yang ibu katakan tadi. Bisa saya salah satu, ya menurut saya ya perlu diinformasikan bagaimana


clarity of the rules of the game

sebenarnya penilaiannya itu. Apakah bobot, misalnya yang depan itu nilainya kecil-kecil lalu semakin belakang semakin besar. Itu mempengaruhi strategi pengerjaan. Saya pernah mengerjakan mengerjakan satu soal tapi bukan Bahasa Inggris itu dengan bobot. Jadi di depan sedikit, di belakang besar nilainya. Saya mengerjakan dari belakang dulu, soalnya pada waktu awal kan konsentrasinya masih bagus. Saya mengerjakan soal yang bernilai besar dulu. Jadi menurutmu, kriteria penilaian itu perlu diinformasikan kepada siswa?

It implies that a test is considered as something to conquer, not merely a measurement of English competency

Perlu. Kalau memang itu tadi bobot bisa mempengaruhi trik mengerjakan. Baik. Yang kedua sekali lagi apa yang kamu lakukan menjelang unas itu? Sebelum mengikuti tes, baik jauh hari malam menjelanya pagi harinya. Coba diingat-ingat. Tolong diceritakan. Diingatingat. Jauh-jauh hari ya belajar seperti biasa, ya mendekati tes itu lebih meningkat belajarnya. Belajarnya dari materi yang diberikan Bapak/Ibu guru atau berusaha mencari di luar yang disajikan Bapak/Ibu guru. Saya e mencari tambahan di luar. Jadi tidak tidak hanya yang disampaikan Bapak/Ibu guru.

Sat-3-1-B-study extensively LAUT

Dari mana itu yang kalau dari luar? Dari.. Saya kan mengikuti bimbingan belajar. Jadi bimbingan belajar itu saya cari buku-buku misalnya buku tentang grammar.

Sat-3-1-B-reading books on grammar LAUT

Banyak ya, yang fokusnya ke grammar? To her, grammar could satisfy her learning needs

Ya, saya yang lebih saya fokuskan ke grammar. O. Baik, terus apa lagi? Kalau malam menjelang tes ya saya memang belajar intensif, tapi tidak sampai terlalu malam, nanti kalau terlalu malam justru merugikan paginya. Terus kalau paginya saya usahakan sampai di sekolah ya buka-buka sedikit, tapi jangan sampai terlalu serius. Mungkin itu malah membuat kacau pikiran. Jadi sedapat mungkin seperempat jam sebelum tes dimulai itu saya malah berhenti belajar. Jadi menenangkan diri.

Sat-3-1-B-study intensively BB Sat-3-1-B-stop studying fifteen minutes before test to relax

Jadi artinya kamu dating di tempat tes tidak terlambat ya masih ada masih menyisakan waktu untuk membuka sedikit. Tensity related preparedness

to Usahakan datang seawal mungkin karena kalau sudah waktunya mepet kan juga membuat perasaan kan jadi tegang, gitu.

Sat-3-1-B-come to school early allowing to relax

Selama mempersiapkan pada saat hari tesnya itu tetep sendiri sendiri atau berbagi dengan teman, diskusi dengan teman. Diingatingat yang kamu lakukan saja. Self-study satisfied her learning needs

Saya sendiri, karena memang saya lebih senang belajar sendiri.

Sat-3-1-B-study her own

Lebih dominan, ya. Kalau teman-teman ada yang tanya pada mbak Atik. Ada sedikit.

Sat-4-3-B-being asked

on


Ada?E kesulitannya tadi ya kesulitan waktu mengerjakan soal. Cuma pada reading dan satu listening tadi. Nah kembali ke pertanyaan setelah tes selesai, kamu kan mendiskusikan jawaban dengan temanmu tadi ya. Ada juga membuka catatan barangkali di catatan ada?

Ya, membuka catatan. catatannya, saya buka.

Kalau memang saya ingat ada

Sat-3-3-B-open and read notes LFEE

Apa yang berkesan dari unas tadi, baik pelaksanaannya atau instruksi soalnya atau apa peranannya. Atau ada yang menjengkelkan, misalnya. Jadi apa yang berkesan dari ... Clarity in test materials facilitated her learning needs.

Yang saya ingat, listeningnya lebih jelas, dan mm soalnya itu lumayan tidak membingungkan seperti mungkin soal-soal ujian yang pernah saya baca sebelumnya. Jadi cukup jelas.

Sat-4-2-B-clearer materials

Pelaksanaannya lancar? Lancar, ya. Artinya tidak misalnya kamu salah mene apa menorehkan jawaban harusnya di nomor satu tapi masuk ke nomor dua dan seterusnya. Jadi apa harus menghapus. Teknik pelaksanaan saya kira lancar-lancar saja.

Sat-4-2-B-run smoothly

Lancar?Ya, tidak ada apa-apa ya? Terus materi tes yang berkesan? Yang apa?[long pause] Yang paling berkesan materinya?[long pause] She was consistent in her focus in learning English

Mungkin tentang grammar, Bu.

Sat-III-B-grammar

Tetap anu pada grammarnya, ya? Terus mengenai instruksinya?[long pause]Instruksi itu misalnya disuruh apa mencari jawaban yang salah atau yang benar dan lain sebagainya. Instruksi tadi.[long pause] Ada yang berkesan di situ?[long pause] Biasa saja, Bu. Oh, biasa. Tidak ada sesuatu yang menjengkelkan, mungkin pengawasan di ruangan?Tidak?Bisa digambarkan yang dilakukan oleh pengawas di ruang tes dulu? E ya pengawas itu duduk di depan. Jumlahnya berapa? Dua. Dua?

Dan tidak terlalu ketat. Tidak anu pengawasannya juga tidak cuek dengan peserta, tapi juga tidak terlalu terus mengawasi, terus memandang daalam kan juga ada pengawas yang memandang terus, tatapan yang curiga. Ada juga kan Bu? Yang saya rasakan kemarin biasa saja, Bu.

(cont’d) Sat-4-2-B-acceptable supervision manner


Ada juga yang minta ijin keluar dari ruangan? The testing situation was conducive

Tidak. Semuanya ada di kelas ya selama pelaksanaan itu. Ya. Baik. Sekarang yang keempat, harapan apa kepada unas dengan tujuan dan fungsinya, materi atau pelaksanaannya.Harapan!

She enjoys challenges

Harapan saya walaupun kemarin saya masih ada yang kesulitan tapi menurut saya e soal unasnya itu dibikin lebih kompleks sedikit. Saya rasa lebih baik. Malah lebih kompleks, ya? Iya. Terus apa namanya format unasnya yang seperti itu dengan memberikan satu pilihan jawaban yang betul? Sudah kamu rasa bagaimana? [long pause] Atau mungkin perlu diubah sehingga menjadi semacam tes esai atau apa ABCD? Kalau satu pilihan yang betul itu saya rasa kalau untuk kemudahan memang mudah yang seperti itu. Mmm. Multiple choice gitu kan Bu. Tapi mungkin untuk memberikan sedikit tantangan mungkin bisa dirubah. Ada beberapa soal yang seperti tipe SPMB itu lho Bu. Bagaimana SPMB? Itu misalnya itu kan jawabannya sampai D, kalau TOEIC itu D atau E, bu? Sampai D. Kalau yang di soal SPMB jawabannya sampai E. Mmm. Jadi kalau nanti dari pilihan ABCDE itu yang betul ada dua jawaban betul. Jadi A dan B betul, ngisinya di jawaban A, terus nanti yang betul A dan C, B. Atau A dan D C, gitu bu.

Sat-IV-B-improving rule of the game

Seperti itu? Ya itu seperti ya lebih menantang gitu bu.

Oh, lebih menantang. Challenges in tests might encourage learning

Dan mungkin bisa juga dipakai itu lho Bu. Jadi misalnya betul nilai 4 kalau salah kurangi satu Dengan pengurangan nilai ya, kalau salah?. Kalau pilihannya

Sat-III-B-less challenging (cont’d)


tetap ABCD seperti itu apa tidak cenderung apa si pekerja soal it terus ngawur kalau tidak tahu, berspekulasi. She had an opinion that multiple-choice tests were more reliable in scoring

Ya ada. Kemungkinan gambling itu ada tapi kan e kalau ABCD itu lebih jawaban pasti, kalau soal esai itu memang mengoreksinya kan jadi subjektif, kan tergantung yang ngoreksi, bu.

Sat-II-B-objective test is more reliable

Jadi tetap memilih pada tes yang objektif, yang milih ABCD itu. Baik. Tapi dibuat seperti soal SPMB. Saya ingin tanya, umpamanya tes Bahasa Inggris seperti itu tidak diunaskan. Apa kira-kira e semangat belajarmu akan berkurang. Umpamanya ya ini hanya if, mengandaikan. Kamu giat belajar itu hanya karena itu diujikan secara nasional atau memang kamu ingin punya target yang lain? Untuk dirimu sendiri, bukan untuk orang yang lain. Maksudnya tetep ada tes, tapi bukan nasional? Ya. She is not only concerned about the test status. It clarifies and confirms that she is eager to make a learning achievement

Kalau saya masih masih soalnya kan saya tentu saja ingin memperoleh nilai yang nilai yang lebih baik dari pada temanteman, tidak hanya di nilai unas. Jadi walaupun itu sifatnya tidak nasional, tapi tetap ya tetap ada pengaruhnya dengan belajar. Soalnya TOEIC itu kan juga tidak nasional to, Bu? Internasional. Yang regional, gitu lho. Ya. Apa lagi ya? Sekarang yang umum ya. Sebetulnya tes yang ideal itu yang seperti apa sih? Tes Bahasa Inggris yang ideal? Materinya itu yang sesuai dengan yang diajarkan, namun juga tidak terlalu sederhana. Jadi ada pengembangannya agar siswa itu lebih kreatif.

Sat-III-A/B-relevant, less simple materials Sat-III-relevant (previously taught but not too simple)

Ya. Terus apa lagi? Disesuaikan dengan waktu. Jadi artinya kalau waktunya dua jam, soalnya itu tidak terlalu banyak, juga tidak terlalu sedikit. Mungkin yang kemarin waktunya kan sangat sisa, itu mungkin waktunya yang terlalu panjang atau soalnya yang terlalu sedikit.

Sat-IV-reasonable time allocation

Terus, tes itu sebaiknya untuk apa?[ long pause ]. Fungsi tes itu sebaiknya untuk apa?

(cont’d) Mengetahui kemampuan seseorang, mengetahui prestasi seseorang sejauh mana memahami materi yang diajarkan. Terus manfaatnya sendiri, manfaat bagi si pengerja soal, yang mengambil tes itu, apa sih?

Sat-I-know one’s achievement, level of understanding


BB

Untuk e merencanakan mungkin cara belajar. Mungkin kalau nilainya terlalu rendah, jadi dia kan harus belajarnya semakin giat, harus sangat ditingkatkan.

Sat-I-plan how study Sat-I-B-LFEE Sat-I-B-LMOT

to

Selama ini pernah mendapat nilai rendah? Pernah. Pernah? Terus apa yang kamu lakukan di situ kalau nilainya rendah? Beneficial Backwash Autonomy

Saya cari-cari materi Bahasa Inggris gitu lho kok yang saya ndak bisa itu apa. Kalau kelas satu dulu tentang vocab, tentang grammar gitu saya senang Saya nggak tahu apa lagi yang TOEIC yang pertama itu belum ada pelajaran Bahasa Inggris yang di kelas. Dari ke TOEIC itu kan sering sekali

Sat-3-3-B-studying the similar materials which she thinks difficult LAUT

Jadi ada ya e feedbacknya? Jadi dari hasil tes itu terus ada dorongan untuk belajar. Umpamanya kalau umpamanya kalau sekarang ini umpamanya lagi diberi pelajaran Bahasa Inggris tapi tidak pernah dites. Misalnya Bapak/Ibu guru masuk di kelas, terus memberikan pelajaran Bahasa Inggris terus. Tapi Bapak/Ibu guru memberi nilai dari pengamatan penampilan sehari-hari. Saya tidak setuju. Gimana? Tidak setuju seperti itu. Tidak setuju? Beneficial Backwash

Tidak. Karena, yang pertama, kalau tidak ada tes seperti itu semangat belajarnya kurang karena tidak ada apa ya e hasil yang konkrit, jadi tidak bisa diketahui saya itu misalnya kalau tes terus nilainya sembilan. Kan oh berarti saya cukup bisa, kalau nilai enam berarti saya nggak bisa. Kan saya berarti saya kurang dong belajarnya, berarti saya harus belajar lagi. Tapi kalau tidak ada tes seperti itu kan kita nggak tahu.

Sat-III-highly important, reliable

Ya, tapi kan Bapak/Ibu guru nanti memberikan tes, ya jadi bukan tes, tapi nilainya. Jadi misalnya semester ini belajar Bahasa Inggris terus Bapak/Ibu guru kan hanya mengamati, oh penampilan si A ini aktif di kelas, terus diam-diam dicatat. Yang ini kalau pelajaran cuma diam saja, ini nilainya sekian saja. Kan penilaian seperti itu biasanya kan subjektif dan yang ketiga, siswa yang aktif itu belum tentu pandai. Mengawali teman-teman di sekolah di kelas itu ada yang pelajaran aktif, tapi nanti saat tes, ternyata nilainya jeblok. Dan ada mungkin yang dia aktif, berani bicara, tapi yang dia katakan itu salah. Jadi tetap tes itu ada manfaatnya? Ya. Baik. Tests were highly important in her studying. She did not want her learning concern was hurt by irrational assessment

Mungkin kalau ada penilaian Bapak/Ibu guru itu sifatnya hanya nilai tambah. Misalnya anak itu aktif, tapi nilai Bahasa Inggris tesnya kan jelek. Jadi nilai keaktifan itu bisa untuk menambah nilai yang jelek.

Sat-II-B-reliable


Jadi digabung? Digabung, tapi e misalnya ada dua nilai, yang satu nilai tes, yang satu nilai penampilan, apa? Nilai penampilan sehari-hari di kelas. She believes that a test is reliable to measure her ability and accomodates her learning

Ya, mungkin nilai tes yang tadi dikali tiga, terus tambah nilai penampilan, tapi ini cuma kali satu, terus dibagi empat. Jadi porsinya tetap besar yang tes. O, mm. OK. Baik. Ya, terus materi apa saja yang harus diujikan, kira-kira dalam tes itu, khusus untuk pelajaran Bahasa Inggris. Reading, baca-baca dan terus vocab, grammar.

Sat-III-A-reading, vocab, grammar

Vocab, grammar. Kalau error recognition itu masuk grammar nggak ya kan beda. Kalau grammar ada yang … yang grammar yang ngisi titik-titik itu, terus vocab yang ngisi titik-titik terus yang baca naskah, terus yang error recognition itu. Itu kan cuma formatnya saja, tapi materinya? Kan bisa reading itu bisa disajikan dengan pilihan ABCD, mungkin bisa juga dengan meringkas, meringkas bacaan secara esai. Jadi materinya yang saya tanya. Materi dari Bahasa Inggris itu kan misalnya-OKvocab, bisa apa namanya speaking, bisa reading, ada listening, ada writing. Di antara seperti itu kira-kira materi apa saja yang perlu diujikan dan mana juga yang tidak. Mungkin bisa menyebutkan dari skala prioritasnya, yang harus diujikan. Misalnya reading yang diunaskan itu, terus grammar, terus vocab, terus yang listening itu, yang dibacakan naskah itu, terus yang listening yang percakapan, terus reading reading yang percakapan misalnya kalau ditanya “Who are you?” Jawabannya apa titik-titik itu. Terus mungkin bisa ditambah apa writing.

Sat-III-A-reading, grammar, vocab, listening, writing

Speakingnya yang terakhir? Speaking tests came last.

Iya. (with a slow, low tone).

Sat-III-A-speaking less motivating

Iya? Nanti kalau kan sebetulnya belajar Bahasa Inggris itu untuk komunikasi. Nanti kalau cuma belajarnya grammar saja atau tapi waktu terjun di dunia kerja, di kehidupan sehari-hari ditanya terus tidak bisa merespon apa termasuk e kategori siswa yang berhasil dalam belajar? Atau atau kamu berpendapat speaking kan bisa kalau mempelajari yang lain otomatis bisa ngomong.

(cont’d) She knows speaking ability is important, but it is not necessary to measure it before the mastery of other skills

E mungkin yang terakhir itu memang memang speaking itu penting, tapi untuk sampai tingkatan tertentu. Misalnya untuk tingkatan seperti saya itu sangat kesulitan. Katanya orang kalau belajar itu mendengar dulu, baru menulis, baru speaking, baru ngomong. Jadi harus yang mendengar sama yang menulis ini harus kuat dulu.


Tapi kalau kita perhatikan misalnya dari lahir bayi itu pertama mendengar terus latihan ngomong, kan belum membaca dulu, belum menulis dulu. Gimana? Tapi e kalau tiba-tiba di awal kita speaking.tanpa kita mengetahui dulu seperti apa to ilmunya seperti kita belajar menulis, kan belajar membuat kalimat itu kan lebih lebih mudah dengan menulis dulu, ditulis duluan kan? Ah buktinya bayi yang baru lahir itu belum menulis dulu sudah bisa ngomong. Itu kan beda. Beda? She is concerned about the quality of her speaking,

Sebenarnya bisa kita langsung ngomong, tapi nanti kualitasnya beda. Mungkin hanya seperti katakanlah seperti tukang-tukang becak di Malioboro itu bisa ngomong. Tapi ya ngomong yang bahasa Inggris yang seperti apa dulu. Mereka kan ya hanya yang penting orang itu mengerti artinya, gitu. Bayi yang baru lahir juga bisa ngomong tapi mungkin kan tata bahasanya juga anu salah, gitu. Gitu ya. Juga mungkin nanti juga akan saya tanya lebih detail lagi ya mengenai yang itu. Tapi sekarang saya tanya lagi yang lain. Soal dan formatnya sekarang seperti apa? Tadi materinya sudah ya, sekarang format. Bentuknya? Bentuknya, ya bisa mulai dari tadi ya seperti tes TOEIC bisa objektif ABCD dan lain sebagainya. Kalau e tadi kamu juga mengatakan ada tes writing. Sebaiknya seperti apa yang writing? Formatnya! Saya rasa kalau itu kan sebagai hanya inovasi tambahan. Itu yang sederhana saja, yang mungkin menerangkan gambar. Jadi ada narasi saja. Jadi untuk gambar 1 orang ngapain, 2 ngapa, tiga melakukan apa itu, disambung. Membuat kalimat? Ya, membuat cerita. Jadi tidak terlalu susah. Kenapa kok milih yang tidak terlalu susah?

She is concerned about learning achievement

Karena e kan selama ini selama ini belum ada kan yang tes mungkin ujian nasional yang ditambah writing. Jadi itu inovasi baru. Jadi jangan sampai inovasi baru itu malah menjatuhkan nilai. Kalau terlalu susah kan malah menjatuhkan.

Sat-III-B-less complete Sat-III-A-not cause to get lower marks

200 7.3. Student Siti 1 Comment Interviewer: Researcher Interviewee: Student Siti Setting: SMK N 1 Pengasih. 21 June 2007. 10.15 – 10.25

Interview Transcript Narrative

Mbak Siti, kamu di sekolah mengikuti pelajaran Bahasa

Code


Inggris biasanya tesnya apa saja, tesnya berupa apa? She informed that she had competency tests for each module and summative tests. She mentioned the test material, “reading” In response to the question whether she studied regularly or accidentally, she honestly answered “accidentally” The literal meaning of ”nggak pernah” is ”never” but I would prefer interpreting this as ”rarely”

Tesnya biasanya itu yang kompetensi modul-modul itu lalu juga kalau ada ulangan umum reading juga ada. Jadi tes speaking dan tes reading.Baik. Selama ini setiap hari belajar bahasa Inggris secara rutin atau dadakan kalau cuma kalau mau ada ujian atau kalau mau maju ujian. Dadakan, bu. Kok dadakan? Kenapa?

Karena nggak pernah belajar, bu di rumah, Bu. Ya, mengapa?Di rumah kok tidak pernah belajar? Sibuk main, Bu. Kan bisa ya, main sambil belajar.E, saya pernah melihat kok kelihatannya mbak siti takut waktu maju ujian. Alasannya kenapa?

She explains that she felt nervous in taking the test owing to unreadiness

Karena nervous, Bu.

Sit-1-2-A-nervous

Ya nervous itu kenapa? Mungkin karena belum siap, belum terlalu siap, gitu.

Sit-2-A-nervous, (cont’d) not ready for the test

Itu tadi. Cuma main. Iya kan? Ha itu. Iya. Belum siap. Apakah perasaan nervous atau gelisah itu juga ada waktu ujian yang non speaking? Artinya seperti ulangan umum seperti kemarin kan tes tertulis, ya to? Ya. Selama itu ada perasaan gelisah tidak waktu mengerjakan tes seperti ini? Agaknya nggak bu karena kalau yang reading itu kan bisa dibaca lalu dipahami, kayak gitu. Jadi kita itu nggak terlalu nervous kalau speaking itu kan nanti bagaimana pengucapannya kalau salah terus berfikir gitu lho, bu. Kalau yang reading kan nanti kalau tulisannya kan bisa diperbaiki lagi gitu kalau reading.

Sit-1-2-B-not nervous ;gave an opportunity to understand. Sit-1-2-A-nervous haunted by making mistakes LPRO (cont’d)

She admits that there is always a kind of nervousness when taking a speaking test

(cont’d) Oh ya. Jadi speaking testnya kan berulangkali ya. Na apakah di setiap maju speaking test mesti ada perasaan nervous itu?. Ya ada, bu.

Sit-1-2-A-nervous

Juga ada rasa senang atau.. . . Sit-1-3-A-feeling


Maybe it’s due accomplishment

to

Ya ada tapi kalau sudah biasanya kalau sudah selesai tesnya itu baru lega gitu lho Bu rasanya.

content LACH

Ya menurut pendapatmu bagaimana sih sebaiknya tes Bahasa Inggris itu? Ya menurut saya kalau uji kompetensi yang modul-modul itu bagus, bu karena bisa memotivasi siswa gitu, bu. Wah kalau teman saya sudah berapa gitu levelnya njuk saya kok masih di sini, jadi memotivasi gitu lho Bu, agar belajarnya lebih ditambahi lagi, gitu. Terus mengenai apa namanya materi tes yang ada kira-kira dirasa cocok atau dirasa bagaimana oleh mbak Siti? Ya, saya rasa bagus juga, bu. Kan berkaitan dengan kehidupan sehari-hari kita, jadi nggak sulit-sulit, gitu, Bu. Kamu mengerti apa tujuannya tes Bahasa Inggris itu sebetulnya? Menambah pengetahuan siswa.

Sit-IV-A-good to motivate st Sit-I-A- competing Sit-1-1-Achallenged by others’ achiev. LMOT Sit-III-Aexisting test good Sit-III-A- relevant with daily life, not difficult LENF Sit-I-A/Bincreasing knowledge

Menambah pengetahuan siswa. Baik. Kalau sekarang yang tes tulis seperti ini, menurut pengamatanmu materinya ini sudah cocok atau belum atau merasa bagaimana kalau mengerjakan tesnya. Saya rasa juga cocok karena sudah pernah dipelajari, gitu, bu. Jadi siswanya nggak kesulitan dalam mengerjakannya. Sekarang saya mau tanya mengenai suasana lingkungan pada saat tes berlangsung. Menurut pendapatmu sebaiknya seperti apa ya tes suasana tes itu?Untuk speaking maupun untuk yang tes reading. Kalau untuk speaking itu lebih baik suasananya lingkungan itu tenang, gitu, bu jadi berfikirnya itu jernih gitu bu. Kalau berisik kan aduh gimana gitu Bu.

Sit-III-B-acceptable material Sit-III-B-has been learned Sit-III-B-user friendly LENF Sit-IV-A-should be done in quiet situation LPRO

Gimana bagaimana? Artinya merasa terganggu gitu? Terganggu gitu, bu. Kalau berbicara terus ada orang-orang berbicara kan nanti malah ah mau dengerin dulu, gitu.

Sit-1-2-A-disturbed by conversation LPRO

Kalau waktu mengerjakan tes ulangan umum ini suasana tesnya bagaimana? Suasananya kalau bisa ya tenang, jadi berfikirnya nggak ruwet. Kalau berisik kan bagaimana bisa berfikir, bu kalau mau berfikir tapi suasananya itu berisik gitu rasanya mengganggu sekali.

Sit-1-2-B-calmed by quiet situation LPRO

(cont’d) Kalau yang berisik teman-temanmu sendiri bagaimana, sibuk tanya kiri kanan? Diamkan saja.

Sit-1-2-B-reluctant to control others’ behavior

Diamkan saja tapi merasa terganggu tidak seperti.. .. Ya sebenarnya merasa terganggu tapi gimana ya? Sama

Sit-3-2-B-nothing


teman itu?

to disturb friends

Ya kalau pengawasnya harapannya kepada peserta yang gaduh seperti itu bagaimana? Di dalam ruang. Ya lebih baik pengawas itu menegur siswanya itu lho supaya tidak berisik lagi kalau bisa ya mengerjakan sendiri.

Sit-2-2-Bsupervisor to keep situation calm

Tapi kalau soalnya terlalu sulit terus teman-temannya kanan kiri bertanya dan lain sebagainya berisik? She is consistent with her concern in learning and achievement.

Gimana ya? Ya kalau bisa ya walaupun tanya ya jangan sampai mengganggu temannya yang lain gitu, bu.

Sit-2-2-B-asking all right as long as not disturbing. LPRO

Sering tidak waktu mbak Siti kesulitan di kelas itu terus kanan kiri tanya atau mungkin ditanya oleh temanmu. Ya ada. Ada beberapa. Terus kamu kasih tahu? Kalau ya tergantung, Bu. Kalau saya bisa kasih tahu.

Sit-4-2-B-difficulty Sit-4-2-B-asked by friends LFEE Sit-3-2-B-sharing ans.

Meskipun ada pengawasnya di situ? Nekat saja kalau misalnya ini lho jawabannya. Ya kalau bisa jangan sampai ketahuan pengawasnya, Bu.

Sit-3-2-B-hiding from supervision in sharing answers LFEE

Saya tanya yang terakhir ini terjadi ya. Kira-kira pada waktu tes ini tentu kamu masih ingat karena belum lama. Ada tidak kanan kirimu yang bertanya? Ada teman-teman tapi tidak bertanya pada saya. Tapi kegiatan tanya jawab. Ada sama yang lainnya, Bu.

Sit-4-2-B-seeing friends sharing answers

Terus kembali ke speaking test tadi. Di dalam speaking tes itu kan ada kecenderungan karena soalnya kan sudah dipaparkan di situ. Ada kecenderungan teman-temanmu atau kamu sendiri yang tanya bagaimana tadi pelaksanaannya terus saya minta contoh sini dialogmu bagaimana? Seperti itu? Ada, Bu. Ada? Atau kamu sendiri? Saya nggak. Oh tidak.

Saya kalau bisa ya buat sendiri, Bu. Nanti cuma tanya temannya itu bagaimana bentuk-bentuknya itu sudah benar atau belum, gitu Bu. Nanti kalau belum ya dikoreksi lagi, gitu.

Itu tulisannya ya barangkali?. Tapi kalau mungkin pengucapannya?

(cont’d) Sit-3-1-Adiscussing with friends, asking for correction LFEE


Kalau pengucapannya saya minta teman saya itu mendengarkan, nanti bagaimana yang salah mananya. Kamu minta umpan balik dengan temanmu? Atau temanmu yang bertanya pronunciation Ya saya bertanya sama teman saya ini yang kurang tadi yang mananya, yang salah itu yang bagaimananya.

Sit-3-1-A-practicing pronouncing words Sit-3-1-A-friends listen, check LFEE

Sit-3-1-A-asking friends for feedback LFEE

Oh, pengucapannya yang salah mana, gitu? Diskusi dengan teman? Iya. Pernah tanya sama bapak ibu guru mengenai kesulitankesulitan yang dialami di situ? Misalnya tidak tahu kata terus tanya, menghubungi Bu Tri, Bu Lis? Tidak. Kalau saya cari di kamus, Bu.

Sit-3-1-Aconsulting dictionary LFEE

Oh, cari di kamus. Kalau di rumah ada pernah membaca dengan nyaring atau tidak, misalnya di dalam modul kan ada contoh-contoh dialog. Terus mencoba mengucapkan itu atau cukup membaca dalam hati? Kalau saya kalau pas itu harus diucapkan ya sebisa mungkin saya mengucapkannya sambil latihan gitu bu melatih bagaimana mengucapkannya.

Sit-3-1-A-practicing pronunciation LMOT

Secara rutin itu? Oh, tidak. (laughing) Kamu bilang belajarnya sedikit. Sedikit, seberapa ya? Kalau mau ulangan saja? Ya saya mau tes tapi kalau paginya itu ada pelajarannya sebisa mungkin saya baca dulu.

Sit-3-1-A-reading LMOT

Terus pernah kamu merasa gagal tidak berhasil di dalam mengikuti tes? Pernah.

Sit-4-3-B-failing in the test

Waktu kapan itu? Waktu tes TOEIC itu. Saya nggak lulus.

Sit-4-3-B-did pass LFEE

not

(cont’d) Oh, ya. Terus perasaanmu bagaimana? Ya menyesal, Bu. Menyesal? Ya sedih aja kok nggak lulus padahal sudah berusaha tapi karena kesalahan saya sendiri. ( laughing )

Sit-1-3-B-sorry,sad, know why she fails

Sit-1-3-B-sad


Kesalahannya apa itu kira-kira? Test functions as a learning feedback

Kurang belajar.

Sit-2-3-B-having learned less LFEE

Idealnya kalau belajar itu seberapa banyak sih? Kok kurang belajar itu? It means that she learn less than one hour everyday.

Ya paling nggak satu jam sehari satu jam itu kalau belajar. Terus mengenai pengawasan di ruang kelas ya kalau tes yang ulangan umum kemarin itu pengawasnya kira-kira selalu mengawasi dengan ketat atau kadang-kadang ditinggal keluar? Itu masing-masing guru Bu, ada yang selalu mengawasi dengan ketat itu. Jadi siswanya tidak ada yang bisa mencontek gitu, tapi ada juga pengawas yang keluar kelas, malah yang pengawasnya kan nggak cuma satu, bu, malah ditinggal ngobrol sendiri. Jadi malah mengganggu konsentrasi muridnya, gitu.

LPRO

Oh, ya. Ngobrol sendiri ya. Waktu tes bahasa Inggris saja ya, bukan mata pelajaran yang lain. Pada waktu yang terakhir itu kamu merasa terganggu oleh suasana seperti itu atau tidak? Tidak, pas waktu Bahasa Inggris tenang.

Sit-4-2-B-silent atmos, LPRO

Tenang? Oh, ya. Makanya bisa berhasil dengan baik, ya. Baik. Di kelas kamu kan termasuk yang hasilnya bagus, ya. Pernah juga teman-temanmu bertanya padamu? Bertanya? Kamu kok bisa, seolah-olah curiga atau bagaimana. Ya, cuma bertanya kok bisa nilainya bagus? Ya saya jawabnya ya cuma latihan aja gitu, lalu dipahami secara benar.

Sit-3-1-A-practice to understand LENF

Kalau di rumah kurang belajar sering tidak kamu mengunjungi sudut yang ada di situ dan melihat membacabaca? Tidak pernah (laughing) Tidak pernah? Karena kalau pas istirahat itu saya gunakan buat shalat, Bu. (cont’d) Oh, ya. Tidak pernah mengunjungi terus pinjam materi untuk di bawa pulang? Belum pernah. Kamu itu sebetulnya tujuan akhir belajar bahasa Inggrismu itu apa sih?


Ya bisa mengerti bahasa Inggris.

LAUT

Ya artinya kira-kira mau digunakan untuk apa kalau sudah bisa bahasa Inggris? Sebagai bekal kalau nanti misalnya cari kerja atau di luar sudah lulus dari sekolah biasanya kan cari kerja. Semoga saja bisa berguna. Bisa membayangkan materi apa yang kira-kira bermanfaat untuk di dunia kerja nanti. Materi bahasa Inggris. Kira-kira ketrampilan apa yang akan digunakan? Saya rasa yang speaking. Speaking? Listeningnya juga. Listening!Kalau readingnya? Readingnya ya saya rasa juga berguna nanti kalau tahu pengucapannya tapi nggak tahu bagaimana cara menuliskan juga susah to bu? Baik untuk hari ini cukup sekian dulu ya, mungkin masih kurang saya cari yang belum sempat saya dapatkan dari informasi tadi. Terima kasih. [ ]

206 7.4 Student Siti 2 Comment

Narrative […] tolong digambarkan perasaanmu tadi waktu mengikuti tes yang terakhir, ya. […] Dijawab yang sejelas-jelasnya, perasaan saat mengerjakan tes bagaimana. Perasaan saya tadi waktu mengerjakan tes agak nervous ya karena menurut saya ya gimana ya soalnya cukup sulit

Code

Sit-1-2-B-a little nervous Sit-III-B-difficult


walaupun sudah belajar semalaman tapi sepertinya masih agak sulit, Bu [laughing]

Sit-3-1-B-study for hours LFEE

Di bagian mana yang sulit? Bagian listeningnya karena waktunya itu terlalu cepat itu lho, Bu. Jadi baru menjawab di lembar jawab itu sudah masuk ke soal berikutnya. Itu yang membuat saya jadi pusing [laughing]

Sit-III-B-shortage of time Sit-4-2-B-disturbed by another test item Sit-1-2-B-dizzy LPRO

Pusing? Ya. Masih memikirkan jawaban yang tadi terus pada saat datang soal berikutnya sudah tidak konsentrasi lagi? Tidak itu tidak mengganggu kalau yang jawaban yang tadi ya sudah dilupakan saja, seperti itu.

Sit-3-2-B-clear up the previous problem

Persiapanmu apa kok bilang melakukan persiapan? Saya mengerjakan soal-soal yang ada di rumah saya yang saya pinjam dari perpustakaan sekolah, Bu.

Sit-3-1-B-doing practice test LMOT

a

Terus mmm belajarnya sendiri atau dengan teman? Sendiri. Kalau dengan teman itu saya malah jadi agak sulite Bu, nanti temannya kalau malah mengajak ngobrol atau apa gitu yang lainnya jadi nggak bisa konsentrasi malahan.

Sit-3-1-B- studying alone LAUT

Jadi sebelumnya tetap belajar ya? Ya. E, beda tidak kalau saat mengerjakan tes seperti itu dengan tes ulangan harian seperti speaking test itu? Kalau… Perasaannya? Perasaannya ya sepertinya sama saja sama-sama nervous.

Sit-1-2-B-nervous

Kok bisa ya kok bisa nervous, alasannya kenapa kok bisa nervous? Anxiety is due to uncertainty to get through the passing grade

Ya kenapa ya…? [laughing]. Mungkin karena nggak yakin kalau bisa lulus atau tidak, gitu. (cont’d) Jadi ada semacam beban itu ya terhadap hasilnya. OK. Na, e menurut pendapatmu tes seperti itu bagaimana? Menurut saya bagus karena bisa mengukur kemampuan kita dalam berbahasa Inggris. Bagus, ya? Mmmm terus menurut pendapatmu tes yang ideal itu yang seperti apa sih?

Sit-1-B-measure one’s English skill LENF


Ya, yang … Kok itu kamu katakan bagus. Alasan bagusnya kenapa? Concern about content and coverage

Karena di dalam tes itu ada listeningnya, berarti bisa mengukur bagaimana pendengaran kita dan memahami apa yang dikatakan oleh speaker, misalnya lalu ada tes reading itu bagaimana kita menemukan suatu jawaban di dalam teks atau e alenia-alenia atau paragraph yang ada di situ, gitu Bu.

Sit-IV-B-good, acceptable LENF

Mmm ya. Terus selama pelaksanaan tes tadi kamu menemui apa mengalami peristiwa apa? Selain hanya sekedar mengenai perasaan, apa ada materi baru atau apa yang kamu jumpai di dalam tes tadi? Sepertinya materinya itu e pernah dipelajari.

Sit-III-B-relevant materials

O. E sekarang perasaanmu masih tetap nervous? Setelah selesai seperti ini? Ya saya kalau sekarang itu malah berfikir wah bagaimana ya tadi kayaknya saya nggak bisa mengerjakan, besuk bisa lulus atau tidak, seperti itu Bu, sekarang perasaannya [laughing]

Sit-1-3-B-doubtful LFEE

Tidak merasa gembira atau menunggu-nunggu kepingin tahu hasilnya atau gimana? Ya kalau ingin tahu hasilnya sudah pasti tentu, ingin tahu bagaimana sebenarnya lulus atau tidak tadi itu.

Sit-1-3-B-curious

Ya. E sebelum tes ini tadi, perasaanmu bagaimana apa ya tetap sebelumnya juga nervous, pada saat mengerjakan nervous, sekarang juga nervous. Apa sama itu ya sebelum sebelum mungkin tadi malam atau waktu seminggu yang lalu mengetahui namamu tercantum disana. Itu apakah juga nervous tetap mengganggu? Happiness is due to competitiveness Taking a high-stakes test is seen as a rewarding experience which in turn brings learning enforcement

Tidak sebenarnya waktu mengetahui saya ikut tes ini itu sebenarnya ya senang ya, Bu, ya. Karena tidak semuanya bisa ikut. Lalu saya terus ya saya berfikir wa bagaimana caranya saya harus belajar atau apa gitu.

Sit-1-1-B-happy LENF

Jadi tidak sama ya sebelumnya dan saat dan setelahnya. Jadi perasaan nervousnya hanya pada saat mengerjakan dan setelahnya. Ya.

(cont’d) Tadi selain hanya sekedar tingkat kesulitannya yang e tinggi atau mungkin yang lain yang kira-kira di dalam secara teknis mengerjakannya kira-kira bagaimana? Ada hambatannya apa tidak? Sepertinya tidak. Tadi kan belum mulai ya dari waktu yang ditentukan sempat molor. Perasaannya bagaimana?


Ya itu agak gimana ya? Deg-degan aja kok lama sekali itu kenapa? Apakah ada sesuatu atau memang waktunya diundur atau apa itu menunggunya jadinya lama.

Sit-4-2-B-heart beating

Menurut pendapatmu seperti itu bagaimana? Kalau ujian seperti itu? Menurut saya … Pelaksanaannnya, pelaksanaannya… Menurut saya ya kurang bagus karena tidak mengikuti apa ya jadwal yang sudah ditentukan.

Sit-III-B-less good as it did not run as scheduled

Tidak kamu apa kesempatan untuk belajar lebih banyak waktunya agak mundur. Kan kitanya sudah disuruh masuk di dalam ruang kelas, jadi ya sudah tidak bisa buat belajar lagi.

LPRO

O, gitu. E, waktu mengerjakan tadi tidak ada hambatan di dalam memilih jawaban? Ya, sepertinya ya tidak Bu ya, hambatan ya mungkin karena jawabannya itu hampir-hampir sama gitu. Jadinya ya itu hambatannya [laughing]

Sit-4-2-B-similar options LPRO

E, bisa kamu bedakan ya apa yang kamu lakukan antara mengikuti tes e seperti itu tadi dengan tes yang speaking test itu. Kegiatan, persiapannya atau apa yang membuat beda? Kalau persiapan ya sepertinya ya sama ya belajar gitu, lalu apa ya memahami setiap pelajaran yang pernah diberikan kepada saya.

Sit-3-1-B-studying, understanding every lesson LMOT

Tidak ada perbedaan ya tetep ada apa belajarnya disana ya. Menurut pendapatmu bagaimana antara dua tes tadi, yang satu hanya ulangan harian, yang satunya kan banyak diikuti siswa, bahkan dari sekolah luar juga. Ya bagus, Bu. Kamu merasa nyaman atau yang mana, suka yang mana? Having an opportunity to take a high-stakes test makes her proud, reducing concern about test result

Saya ya lebih senang kalau yang seperti ini karena kita itu bisa sedikit bangga, wah saya ternyata bisa ikut tes yang nggak semuanya bisa ikut [laughing] Walaupun nanti tidak tahu hasilnya seperti apa [laughing]

Sit-1-1-B-happy LACH

E Ada rasa optimis melihat skornya nanti?

She showed happiness although she did expect much.

Sedikit [laughing]

(cont’d) Sit-1-3-B-a little optimistic

Kira-kira targetnya berapa skornya? Berapa ya? Dua ratus lima puluh mungkin. [laughing] Berapa? Coba diulangi lagi.

Sit-3-3-B-setting a low score target


She did scores

not care about

Mungkin dua ratus lima puluh sudah Alhamdulillah… [laughing] OK. Mungkin ada yang perlu disampaikan sehubungan dengan tes yang seperti itu tadi?

It indicates curiosity preparedness .

and

Ya pengetahuan, e, pemberitahuannya itu yang lebih awal lagi gitu.

LPRO

Maksudnya? Ya pemberitahuan kalau ada tes seperti itu jadwalnya atau apa itu lebih awal lagi, misalnya kalau yang kemarin satu minggu, misalnya satu bulan sebelumnya sudah dikasih tahu, gitu.

Sit-4-2-B-calm situation

Saat di ruangan, pengawasannya bagaimana menurut pendapatmu? Pengawasannya bagus ya karena di ruangan saya itu tertib, tidak ada yang ramai atau pun contek-contekan itu tidak ada.

LPRO

Fasilitasnya? Fasilitasnya juga bagus. OK. Saat sudah ya,. Na, sebelum mengikuti tadi apa yang tergambar dalam pikiranmu?Mungkin kalau perasaanmu saya tadi sudah tanya ya? Sebelum, sebelum mengikuti tadi mungkin tadi malam, mungkin beberapa hari yang lalu ya apa yang kamu pikirkan tentang tes yang akan kamu ikuti tadi? Saya berfikir itu apa itu peraturannya itu sangat ketat, ternyata benar. Itu sangat ketat [laughing]. Lalu ya suasana yang tenang dan sepi, itu ternyata ya memang seperti itu.

Sit-2-1-B-strict rules, conducted in a quiet situation LPRO

Oh, ya. Mengenai materinya? She had a positive learning feedback and enforcement from the test materials

Kalau materinya ya sama kok Bu, yang seperti yang diajarkan, ada materi-materinya. Kemudian, na setelah kamu mengikuti tes tadi apa yang kamu lakukan setelah dari ruangan?

Being in a test meant working hard although it was enjoyable

Menghirup udara [laughing]

Sit-3-3-B- removing her burden by having a chat

Yang lain, selain menghirup udara?

(cont’d) Ya ngobrol-ngobrol sama teman bagaimana tadi bisa nggak atau [laughing] … Apa yang didiskusikan pada saat ngomong-ngomong dengan temanmu? Tadi, diingat-ingat. Gimana soalnya, wah soalnya ternyata susah sekali [laughing] Lalu ya, terus bisa nggak tadi ngerjainnya. Wah, ya ya biasa-biasa aja gitu.

LFEE


Yang tanya kamu atau temanmu? Saya yang nanya.

Sit-3-3-B-sharing her problem

O. Ada temanmu yang tanya pada kamu? Ya sebaliknya gitu lho, Bu. Lha kamu sendiri gimana, gitu. Terus ya, saya sama saja. [laughing]

LFEE

Ngomongnya sama siapa saja tadi? Tadi saya sama DJ, dengan UL. Oh. Jadi mungkin membahas tentang soalnya. Yang dibahas itu biasanya tes apa? Ya, keduanya, yang listeningnya juga. Soalnya terus soal apa ya yang listeningnya sama readingnya juga. Kalau listeningnya itu ya dibahas masalah waktunya gitu, lho Bu. Wah terlalu cepat ini.

LPRO

Terlalu cepat waktunya? Ya. Oh. Kalau yang reading yang dibahas apa? Lengthy test materials could be tiring

Paragrafnya itu terlalu panjang, gitu.

LPRO

Sukanya yang pendek-pendek kalau gitu? Terus setelah itu apa lagi? Setelah itu saya itu hanya nunggu teman itu di depan. Tidak ada kegiatan apa saja ya? E, misalnya mendengar ndak keluhan dari temanmu di itu. Atau keluhan, atau rasa gembira untuk mengikuti tes seperti tadi? Keluhan ada keluhan tadi teman saya itu Wah saya tidak yakin bisa lulus, seperti itu.

LACH

Tentunya tidak ada istilah lulus dan tidak lulus ya itu cuma ngerti levelnya saja. OK. Selama e mengerjakan tes, yang kamu lakukan apa tadi? Saya cuma memahami soal-soalnya itu.

Sit-3-2-Bunderstanding test items

Memahami soal. Yang kamu lakukan? Tidak ada. (cont’d) Tidak ada? Dua jam tidak melakukan apa-apa? Apa ya? [laughing] Ya, cuma itu mengerjakan soal-soalnya itu tadi. Kalau sebelum tes seperti tadi pengalaman apa yang kamu jumpai? Misalnya mengerjakan .. Apa dikunjungi temanmu untuk latihan bersama atau apa


yang ada kaitannya dengan kegiatan dengan tes tadi? Tidak, tidak ada teman yang berkunjung. Saya ya cuma belajar sendiri saja di rumah.

Sit-3-2-B-studying alone

Saat e mengikuti tes juga tidak mengalami peristiwa yang istimewa? Tidak. Setelahnya juga tidak ada peristiwa yang istimewa? Ya, istimewanya ya senang aja sudah keluar dari ruangan itu [laughing].

Sit-1-3-B-relieved LACH

Waktu mengerjakan itu kamu bisa berhasil mengerjakan dua ratus soal? Tidak. Tidak sampai dua ratus. Ada yang berapa nomor ya tadi saya cuma blok saja karena waktunya sudah habis.

Sit-3-2-B-use speculation strategy LPRO

Sudah habis? Ya. Maksudnya diblok saja itu gimana? [laughing] She had no more time to show her competency.

Ya, apa ya milih terserahlah apa A, B, C, D itu tidak tinggal pilih salah satunya [laughing] Tidak melihat soalnya, maksudmu? O, itu di bagian reading atau listening yang banyak ketinggalan? Yang reading.

LPRO

Reading. Kalau yang listening selesai ya? Ya, selesai.

LPRO

Cuma reading di bagian yang mana? She was short of time and had no opportunity to do the last part.

Bagian yang akhir-akhir itu, nomor-nomor akhir. Mengerjakannya urut ya, tidak pilih yang mudah dahulu, atau apa.

She was concerned in learning, not in achievement in terms of score

Tidak, Bu, tidak kepikiran [laughing].

Oh. Banyak tadi yang sempat ndak kejawab?

Sekitar sepuluh nomor. Oh. Nomor kamu isi apa? Speculation strategy to fight for colonizing time allocation

Blok saja. A, B, C, D A, B, C, D gitu. [laughing] Tidak A, A, A, A semua itu, tapi A, B, C, D A, B, C, D? Ya.

(cont’d)


Itu main spekulasi ya kalau gitu [laughing] OK, ya. Kalau dibandingkan dengan saat tes ulangan harian, gimana ya selain hanya perasaanmu. Menurut pendapatmu bagaimana kedua tes tadi? Menurut saya ya semuanya bagus kok Bu. Kalau ulangan harian itu kan bisa mengetahui keg apa kemajuan kita dari semester satu ke semester dua misalnya. Kalau yang tes seperti ini kan mengetahui keseluruhan kemampuannya kita.

Sit-I-A-monitor her progress from semester to semester Sit-I-B-know the accumulation of skill

OK, baik. Saya kira itu dulu ya. Nanti saya minta informasi lagi kalau ini yang saya cari masih kurang. Ya. Terima kasih.

213 7.5 Student Endang Comment

Narrative

Interviewer: Researcher Interviewee: Student Endang Setting: SMK N 1 Pengasih. 30-6- 2007. 9.40 – 10.00

Mbak Endang, saya ingin bertanya mengenai pengalamanmu menempuh tes bahasa Inggris, ya. Kan ada bermacam-macam tes. Nah. Menurut pendapatmu mana yang paling disukai di antara berbagai jenis tes-tes tersebut.

Code


She enjoys high-stakes tests as well as classroom tests

Misalnya tes TOEIC atau tes apa itu pendidikan yang lainnya itu. Tes TOEIC itu bisa untuk kita supaya berlatih mandiri untuk belajar Bahasa Inggris supaya tidak tergantung kepada teman dan melatih pendengaran kita supaya kita itu peka terhadap perkataan dengan menggunakan kata-kata Inggris yang tidak secara tulisan, secara lisan

Sen-I-A/B-learning independently Sen-I-B-training listening skill

Jadi kamu juga menyukai tes yang berupa wawancara atau tanya jawab/dialog itu, tes lisan? She answers she likes speaking tests

Ya, saya suka. Selama ini sudah menyelesaikan berapa tes speaking atau tes berbicara itu? Sudah sekitar empat kali. Oh, empat kali? Belum sampai 10 kali? Sudah sampai A6 kalau nggak A7. Belum sampai A10 itu? Belum? Belum Baik. Sekarang saya ingin bertanya tentang tes semacam tes TOEIC yang jumlahnya 200 itu ya. Kamu menyukai tes seperti itu? Ya, saya suka. Untuk melatih pendengaran kita supaya kita itu cepat mudah mempelajari Inggris. Tapi itu kan cuma a,b,c,d. Apakah selama ini kamu serius memilih jawaban ini, oh, ini a, ini b, ini c, ini d, atau hanya spekulasi saja? Spekulasi. Soalnya saya itu belajar bahasa Inggris itu belum terlalu bisa, gitu lho Bu.

Sen-I-B-training to listen, and learn English faster Sen-3-2-B-doing test speculatively because she realized she still had difficulties in learning English.

Belum terlalu bisa, ya. E, soalnya saya pernah ya, maaf, menemui pekerjaanmu, kok pilihannya hanya aaa, bbb, itu. Alasannya kenapa kok bisa seperti itu? Belum, ya saya itu kosakata-kosakata Inggris itu belum terlalu tahu, gitu lho. Terus saya itu menjawabnya sebisanya.

Sen-3-2-B-do best she can

the

Apakah kamu juga karena takut ya, tidak bisa mengerjakan tes? Selama mengerjakan tes itu perasaaannya santai atau merasa waduh ini saya harus bisa nilai baik? Saya tidak mengejar target. Yang penting saya itu bisa tahu tentang Inggris gitu lho.

(cont’d) Sen-3-2-B-has no target Sen-3-2-B-learning to know English

Selama ini pelaksanaan di sekolah menurut pendapatmu bagaimana, baik yang mulai dari yang duaratus soal itu, mungkin juga tes yang e apa wawancara atau berbicara, juga ada tes kenaikan kelas yang terakhir. Ya, itu semuanya itu sudah baik, tapi yang yang terakhir itu masih kurang saya pahami, gitu lho Bu. Kosakata-kosakata itu masih banyak yang sama, masih banyak yang kurang dipahami. Itu masih banyak yang menyalahkan gitu lho. Ya, a,

Sen-IV-A/B-good Sen-IV-B-difficult vocab, distracting options


b, seperti bahasa Indonesia itu soalnya mengecoh gitu. Selama ini nilai-nilai yang kamu dapatkan di tes tersebut baik?

Belum. Belum belum terlalu bagus, gitu lho Bu. Belum memuaskan.

Sen-4-3-B-not satisfying score

Kenapa kok belum memuaskan? Soalnya saya itu belajar bahasa Inggris itu belum maksimal, gitu. Masih kurang.

Sen-3-1-B-study little

Belajarnya tiap apa sih? She found many unfamiliar in the test and consulted her dictionary

Ya, setiap ada soal lembaran itu dibaca, kadang-kadang mempelajari kamus.

Sen-3-1-B-reading test handout, learning from a dictionary

Itu belajarnya secara rutin ataukah hanya menjelang saat tes? Sekarang kadang-kadang, bu. Maksud kadang-kadang? Ya, tidak terlalu rutin, gitu, bu. Biasanya belajar yang lain gitu. Tapi Inggris masih yang dipentingkan.

Sen-3-1-A/Bstudying in a very limited time

Maksudnya? Artinya tiap hari atau hanya menjelang pada saat ulangan umum, terus sekarang ngebut belajar? She is concerned about learning.

Tidak. Saya itu kalau belajar sedikit demi sedikit asal dong gitu lho.

Sen-3-1-A/Bstudying little by little with a good comprehension

Oh, sedikit-sedikit. Penyampaian ibu guru ya di kelas juga sudah cukup baik pada bahsa Inggrisnya? Sudah. Setiap mau ujian, ya, tadi kamu mengatakan mau ujian apa mau tes speaking itu apa persiapannya yang dilakukan? Ya latihan membuat teks, menghafalkan gitu, tapi masih banyak kosa kata yang itu belum tenses benar gitu, Bu, masih kurang.

Sen-3-1-A-writing text, memorizing

Ya, memang harus begitu kalau dalam belajar seperti itu, ya, kalau sudah bias tentu tidak belajar. Biasanya belajar sendiri atau bersama dengan teman kalau mau ujian tes berbicara itu? Ya kadang-kadang kalau ada kosakata yang belum tahu, tanya teman, tapi apa selebihnya masih belajar sendiri. Banyak belajar sendiri? Kalau belajar sendiri terus mau ujian berbicara apa cuma ngomong sendiri, menghafal, mengingatingat apa yang harus diucapkan?

Ya, kalau di rumah itu membuat sendiri, tapi kalau sudah di sekolah minta bantuan teman untuk memperbaiki.

Kamu belajar bahasa Inggris itu rata-rata sehari berapa jam?

(cont’d) Sen-3-1-A-asking friends, autonomous learning Sen-3-1-A-doing preparation on her own Sen-3-1-A-asking friends to correct her work


Satu jam sampai satu setengah, kadang-kadang. Kadangkadang nggak nyampai, bu. Tapi sedikit-sedikit, gitu, ya. Selama ini kamu dengan temantemanmu itu sama kecepatannya atau kamu jauh tertinggal atau lebih baik dari teman-teman? Sama Sama? Jadi teman-temanmu juga sudah selesai 7 kompetensi maju modulnya? Belum sampai mencari lokasi dan lain sebagainya itu? Kalau di rumah, modul tersebut diapakan? Dibaca-baca, kadang-kadang ya dipahamilah Bu, isinya yang itu.

Sen-3-1-A-read module

Sebetulnya kamu belajar bahasa Inggris itu kemampuan apa yang ingin kamu raih? She could identify her learning goals.

Ya bisa berlatih, bisa berbicara dengan bahasa Inggris, bisa menyusun kata-kata Inggris itu dengan benar, gitu lho. Baik. Sekarang mengenai pelaksanaan tes berbicara itu di dalam kelas. Suasana waktu kamu mengikuti tes tersebut bisa kamu gambarkan seperti apa? Ya majunya itu sendiri-sendiri, tidak sama teman gitu ya ramailah, Bu. Ada teman-teman yang sedang belajar untuk maju, gitu. Merasa terganggu tidak dengan suasana seperti itu atau biasa saja? Ya kurang PD gitu, kan masih takut kalau punya kita itu masih ada yang salah gitu. Takut kalau disalahkan teman-teman gitu. Ya, tapi masih biasa kok, belum terganggu.

Sen-4-2-A-noisy situation, friends are studying, preparing for the test Sen-1-2-A-not confident with her skill Sen-1-2-Aundisturbed in noisy situation

Tidak terganggu situasi seperti itu. Jadi meskipun pada saat maju ujian bersama teman atau bersama ibu guru, temantemannya ramai berlatih, kamu tidak merasa terganggu. Pada saat kamu maju tes itu perasaan apa yang kamu rasakan? Bisa digambarkan? Deg-degan, tegang, gitu.

Sen-1-2-A-tensed (cont’d)

Selalu? Setiap maju mesti deg-degan gitu? She felt tensed because she should be competing with friends

Ya soalnya majunya itu harus dulu-duluan gitu lho, Bu. Tidak, tidak apa urut gitu tidak. Siapa yang cepat dia itu langsung maju, yang bisa langsung njawab, gitu. Dulu-duluan? Tidak dipanggili oleh ibu guru misalnya, [...], maju tes. Tidak Jadi kamu maju sendiri? Ada teman-temanmu yang merasa tidak cocok dengan cara seperti itu, atau …?


Sepertinya sudah cocok semua, kok Bu. Bisa mengikuti kegiatan itu.

Sen-III-A-find acceptable

test

Jadi artinya apa tes itu memberi manfaat bagimu? Kemudian harapanmu apa yang akan ingin kamu dapatkan dari tes tersebut? Her interest was in learning

Ya, bisa berlatih untuk lancar menggunakan bahasa Inggris, bisa semuanya bisa menggunakan apa itu dengan baik bahasa Inggris.

I pointed to a self-access corner where a glass cupboard with learning materials is provided for students

Jadi selain modul di situ kan ada materi-materi yang bisa kamu lakukan. Kamu pernah mengunjungi di situ dan pinjam materi?

Sen-I-A-practice use English well

to

Belum pernah membawa pulang, tapi pernah mendengarkan. Oh, pernah mendengarkan. Sekarang kalau kamu tadi mengatakan bahwa pada saat maju itu deg-degan ya. Setelah Bapak atau Ibu guru memberi nilai kepadamu, selesai, perasaanmu bagaimana? Ya masih kurang puaslah wong bijinya apa nilainya itu belum bagus, belum semaksimal, belum seperti teman-teman yang lancar berbahasa Inggris.

Sen-1-3-Adissatisfied with the score

Belum bagus? Ya masih ingin mendapat nilai yang lebih bagus lagi Tapi kan setiap maju lulus? Atau pernah setiap maju terus ada tidak lulus? Tidak. Lulus

Sen-4-3-A-pass each test

Sebelum kamu mengerti levelmu bahasa Inggris sekarang? Belum Dulu waktu masuk kan diberi tes placement itu ya, kan? Ya, pokoknya saya menemukan mbak DW ya. Saya tunjukkan, bisa?Ya sebetulnya skornya baik, ya di elementary ya. Ini kan seperti ini. Ini kok seperti ini. Ini alasannya kenapa kok seperti ini? Soalnya itu nganu e bu, untuk memahami itu susah.

Sen-1-2-B-hard understand

to

(cont’d) Oh, karena susah, pokoknya asal isi saja, tidak ingin dibiarkan kosong. Saya pikir ini kan oh, anak ini terlambat datang apa bagaimana ya, kok seperti ini? Tidak Tidak. Memang anu ya tapi nilainya bagus kok lho ya. Jadi ini karena ini kan sudah tiga ratus, dua ratus. Kan jadi tertarik seperti ini, yak an. Jadi tidak karena terlambat ini ya? Jadi kamu ingat-ingat lagi waktu mengerjakan tes ini perasaanmu bagaimana seperti itu? Mengerjakan kan kamu bilang tidak bisa dan lain sebagainya. Bisa digambarkan perasaan saat itu, saat mengerjakan ini? Sen-1-2-B-tensed


Ya, tegang, kaget, gitu, lah, Bu untuk mendengar gitu. Belum biasa.

because listening is a new experience

Belum biasa. Waktu di SMP belum. Belum pernah ada ya seperti itu? Jadi kan hanya untuk mengukur seberapa sih kemampuan. Baik. Jadi waktu mengerjakan ini memang karena anu ya anunya belum begitu jelas ya. Semuanya juga baru. Kalau umpamanya sekarang kamu maju, maju tes seperti ini lagi, apakah kamu juga akan melakukan hal yang sama dengan ini? Ya mungkin. Melihat keadaan. O. Tapi tidak ada permasalahan karena mungkin sound systemnya kurang bagus, atau pelaksanaan di situ situasinya ramai seperti ini? Tenang ya waktu itu? Terus yang berikutnya mengenai tes kenaikan kelas kemarin ulangan umum. Bagaimana, kamu bisa mengerjakan dengan baik? Belum, masih kurang, Bu. Nilainya kemarin aja mengulang itu.

Sen-4-3-B-getting low marks

Mengulang yang mana? Mengulang ulangan her lagi.

Sen-3-3-B-doing make-up test

Waktu bersama bu Suwartiningsih itu? Kan termasuk … Berapa nilainya? Berapa ya? Lima koma berapa itu. Terus waktu diumumkan her lagi itu, perasaan bagaimana? Ya kaget itu. Tetapi saya ya nggak begitu kaget, soalnya mengerjakan bahasa Inggris itu belum terlalu bisa itu.

Sen-2-2-B-she cannot do the test well

Terus tindak lanjutnya waktu diberitahu. Oh ini DW harus her harus mengulang, bagaimana? Ya … Tindak lanjutnya? Ya mengikuti teman-teman, gitu. Tapi sebenarnya tidak ingin wong ya kalau sudah terjadi ya gimana lagi gitu.

Sen-3-3-B-doing same with friends Sen-1-3-B-obliged to do something not expected to do (cont’d) (cont’d)

Ya, tidak ingin. Tapi terus tindak lanjutnya apa? cuma diam saja atau terus diskusi sama teman, terus apa. Coba diingat-ingat. Setelah ya latihan mengingat-ingat soal yang ndak bisa itu. Dicari di catatan gitu. Oh Cari kosakata yang belum tahu. Terus kira-kira sekarang kamu bisa memprediksi wah kalau sudah her ini nilainya berapa atau skornya berapa. Belum

Sen-3-3-B-trying to recall where she went wrong, finding the meaning of difficult words


Tapi tidak ada perasaan takut? Biasa saja? Jadi ternyata kamu her lagi ya. Di materi apa yang kamu pandang dari tes tersebut? Gerund, gitu. Susah gitu. Cocok nggak materi itu untuk e keinginanmu untuk bisa bahasa Inggris? Pelajaran itu. Cocok? Kamu rasa waa saya mau mempelajari materi yang lain, kok dikasih ini. Sudah. Sudah. Sudah cocok.

Sen-III-A-all right

Sudah cocok seperti itu? Cuma belajarnya mungkin perlu ditambah. She wanted more learning materials

Ya, perlu tambah materi yang misalnya bagus ke masalah materi itu lho ke permasalahan itu difokuskan lagi. Supaya lebih dong. Di kelas kamu tanya sama ibu guru tidak, kalau misalnya dia memberi nilai yang kamu rasa kok tidak cocok? Tidak Jadi apa yang didapat dari bu guru terus langsung diterima? Tidak tanya itu apa sebabnya, bu? Atau ibu guru yang sudah menjelaskan. Misalnya waktu kamu maju speaking. Sudah. Sudah dikasih tahu. Memang saya itu bahasa Inggris memang. Tapi tindak lanjutnya apa kalau seperti itu? Tidak ada. Tidak ada? Tidak pinjam materi di tadi di lab itu kan banyak materi. Oh saya mau pinjam buku ini, saya mau pinjam kaset ini. Belum. Paling pinjam catatan teman yang belum dipunyai. Pinjam punya teman gitu aja. Belum mencari di lab itu belum mencari.

Sen-3-1-A/Bborrowing from friends

notes

Kok belum? Alasannya tidak punya waktu atau kenapa? Atau kurang tertarik?

(cont’d) Kalau tidak ada temannya itu saya itu malese, bu mau kemanamana ya. Oh. Sekarang mengenai yang akan datang ya. Tadi kan mungkin waktu akan lulus di SMK kan ada tes unas ya. Menurut pendapatmu bagaimana tes hasil tes unas menentukan kelulusan itu. A high-stakes test was considered as contributing her future life.

Sudah bagus. Soalnya itu kan untuk mencapai ke masa depan kita gimana cara yang baik gimana nilai yang bagus itu kan memang pokok itu. Sudah ada persiapan dari sekarang, mulai misal melihat soal tes unas itu seperti apa atau belum? Belum.


Belum pernah? Belum punya gambaran, kalau gitu. Baik untuk hari ini saya rasa cukup sekian ya dulu. Mungkin kalau ada materi susulan. Nanti saya ..

220 7.6 Teacher Iin 1 Comment Interviewer: Researcher Interviewee: Teacher Iin Setting: SMK N 1 Pengasih. 16 June 2007. 9.40 – 10.00

Narrative

Code

Selamat siang, bu Iin. Saya mau tanya tentang seluk beluk tes Bahasa Inggris dan penerapannya di sekolah. Anda tahu apa yang merupakan tujuan dari tes Bahasa Inggris itu? Tujuan tes itu kan untuk mengevaluasi siswa, untuk menilai siswa sejauh mana materi yang diajarkan itu ditangkap oleh siswa dan bisa kalau bisa diaplikasikan dalam kehidupan sehari-hari.

Tin-I-A/Bevaluation/assessment, measurement of achievement

Ya terima kasih. Kalau misalnya siswa nilainya kurang, terus itu artinya bagaimana Kalau nilainya kurang ya berarti siswa tersebut belum ya belum

Tin-I-A/B-program


bisa menangkap secara maksimal tentang materi yang diajarkan oleh guru. Berarti di sini mungkin e perlu pemantauan atau mungkin apa review. Istilah review apakah itu atau mungkin dalam gurunya menyampaikan materi itu belum maksimal atau mungkin belum bisa diterima siswa. Atau mungkin juga dari faktor siswanya itu sendiri. Atau memang siswa itu yang termasuk dalam kondisi lambat atau memang apa mungkin background siswanya itu sudah berbeda istilahnya SDMnya itu berbeda. Mungkin kendalanya di situ.

evaluation

Perasaan ibu bagaimana kalau ada siswa yang tidak bisa mengerjakan, ya kesulitan. ( laughing). Perasaan saya pribadi? Ya. Ya itu mungkin tergantung prosentasi siswanya berapa. Berapa misalnya yang ndak bisa itu seluruh kelas. Mungkin kesalahan pada gurunya Ya pertama ya mungkin yang dicari permasalahannya dulu. Prosentasenya banyak. Oh, ini berarti kesalahan pada gurunya, berarti yang harus review itu gurunya, yang harus menindaklanjuti ke dalam gurunya sendiri. Tapi kalau mungkin mungkin juga sedikit prosentasenya itu juga mungkin ada senengnya. Ada senengnya? Karena yang berhasil lebih banyak berarti. Gembiranya dilihat dari prosentase sedikit. Oh, sedikit berarti ini memang anaknya yang mungkin SDMnya kurang ditindaklanjuti sendiri.

Tin-1-3-A/B-happy with the class’ good achievement

Kalau melihat situasi pada saat ujiannya atau ulangannya bagaimana itu? Yang kira-kira cocok untuk pelaksanaan tes.Maksudnya untuk ya ya lingkungan apa kapan di . . . Kalau saya cenderung kalau writing tes cocoknya setelah dua materilah dikasih tes writing. Terus untuk speaking itu mungkin bisa per unit, per modul, gampangannya per modul bisa dikasih tapi untuk writing mungkin bisa gabungan dari beberapa modul.

Tin-III-A-at the end of each lesson unit (speaking) Tin-III-B-two units or more

Mengenai lingkungannya lingkungan mengikuti tes selama ini bagaimana? Lingkungan sekolah atau lingkungan kelas? Ya bisa lingkungan sekolah, bisa lingkungan kelas. Kira-kira suasana yang cocok untuk mengadakan tes itu yang seperti apa? (cont’d) Untuk tes writing mungkin jelas ketenangan. Untuk pelaksanaan tes. Tapi untuk tes speaking itu mungkin biarlah siswanya yang tidak menghadapi tes ya tidak kondisi pas gilirannya barangkali bisa berlatih jadi sedikit lebih crowded tapi nggak masalah asal itu tidak mengganggu yang dites itu tidak berpengaruh kalau saya.

Tin-III-B-calm situation-writing Tin-III-A-noisy is OK for speaking

Oh, tetap jalan meskipun suasananya riuh. Tetap jalan, ya. Kalau tes speaking kan memang dituntut seperti itu. Selama ini ada siswa yang protes-protes atau tidak? Tidak.

Tin-4-2-A receives no protest for noisy situation


Sekarang masih mengenai pelaksanaan tes di kelas atau di sekolah ibu, ya. Kira-kira pelaksanaan selama ini bagaimana e cukup berjalan baik atau ada problema apa selama melakukan di sekolah. This teacher is deeply concerned with the validity and reliability. Her statement also implies that classroom test brings motivation to practice and to learn

Mengenai itu sudah bagus, cuma permasalahannya kadangkadang kalau saya pribadi permasalahan terletak pada speaking test seperti yang saya utarakan tadi. Kalau mereka diberikan suatu topik itu sepertinya kita itu cuma mendengarkan rekaman dari siswa. Jadi kadang-kadang kalau gurunya tidak memberikan pertanyaan yang bisa menggali kemampuan siswa itu gurunya tidak bisa dibedakan antara siswa pinter dalam writing sama yang tidak bisa dalam writing. Jadi kesannya siswa yang pinter sama yang tidak itu dalam speaking test kelihatannya sama karena apa? Karena mereka cenderung menghafal, mencatat di rumah, ya. Terus kalau tidak dikasih pertanyaan tambahan jadi tidak jelas perbedaannya antara siswa.

Tin-II-A-not much reliable Tin-3-2-A-giving more questions Tin-3-2-A-modifying questions Tin-II-A-test is not reliable Tin-II-A-not valid

Kalau ada pertanyaan tambahan e itu dari guru itu bebas tanya apa saja atau terfokus pada yang diujikan tadi. Ya. Biasanya pertanyaan mengarah ke hal-hal yang diujikan. Oh, jadi misalnya anak ujian A5 misalnya terus pertanyaannya itu sampai yang lain bagaimana menurut pendapatmu ya what do you think of our headmaster sampai di situ atau yang... Biasanya karena itu tes describing people ya itu kadang-kadang kita juga e coba kamu deskripsikan pak ini kita kasih yang guru lain misalnya, jadi kan mungkin kan siswa juga mengetahui guruguru di sekolah. Jadi kan kadang-kadang ketahuan ini siswa itu tahu betul atau cuma berbekal hafalan. Biasanya kan langsung ketahuan anak yang bisa ya bisa, yang bermodal cuma hafalan ya ketahuan, nggak bisa njawab. Selama ini ibu sudah mengamati ya reaksi anak-anak waktu ditanya pertanyaan tambahan itu bagaimana? Kalau yang speakingnya bagus ya itu bisa menjawab panjanglah, istilahnya apa pun yang ditanyakan gurunya bisa respon di dalam maksudnya kasih respon yang betul kalau yang tidak bisa speaking ya biasanya kalau dikasih pertanyaan kaget, terus misalnya pertanyaannya A, jawabannya Z, biasanya seperti jauhjauh sekali dari dari apa yang jawaban apa diharapkan oleh pertanyaan tersebut.

Tin-4-2-A-find modified questions brings students fall into two categories Tin-4-3-A-those who can really speak can do the modified test well

(cont’d) Ya, waktu dia kaget itu ekspresinya bagaimana?Siswa-siswa? Ya anak-anak kadang-kadang kan aduh bu kadang-kadang juga keluar, Bu kok tanyanya jadi itu. Kadang-kadang kan terus Bu, kok mbok yang kayak ... kadang-kadang malah seperti mengajari, bu mbok pertanyaannya sama dengan yang itu. Kadang-kadang kan mungkin mungkin siswa yang maju itu mungkin bercerita sama temannya, tadi saya dikasih pertanyaan bebas tentang ini, jadi mungkin mereka juga sudah siap. Mungkin kalau ditanya ini jawabannya tentang ini, tapi karena ganti terus jadi kadangkadang kalau yang kaget ya spontan kaget, ndak bisa njawab. Jadi e itu mengenai speaking test ya? Ya.

Tin-4-3-A-receiving students’ complaints for modifying test questions HB


Mengenai writing test ada siswa yang komplain atau bagaimana? Paling juga komplainnya tentang ini, bu soalnya kok sulit sekali, misalnya kalau siswanya bu ini soalnya kok sulit sekali jadi banyak yang nggak bisa. Tapi mungkin juga karena kalau misalnya ini kan cuma kamu nggak teliti saja jadi kamu kurang cermat dalam membaca mungkin juga terlalu istilahnya membacanya cuma apa berulangkali membaca tetapi tidak tidak mengacu ke pertanyaan jadi malah kadang-kadang juga beda atau mungkin juga kadang-kadang dibaca sekilas jadi tidak fokus dengan pertanyaan.

Tin-4-3-B-students’ complaints for difficult test Tin-3-3-B-comforting, speculating what has happened to students

Ibu yakin kalau yang membuat sukar itu materi tes itu sendiri? Apakah mungkin juga karena instruksinya tes yang kurang jelas? Bisa juga mungkin instruksinya. Jadi sebetulnya instruksinya tidak dimengerti oleh siswa atau mungkin dalam instruksinya itu bisa punya arti ganda, mungkin. Mungkin bisa seperti itu juga. Biasanya apa ya kalau yang punya mungkin dalam writing ya mungkin dalam esai model kayak gitu biasanya yang kadangkadang anak punya pe penangkapan yang berbeda-beda mungkin. Ya, terus kalau ibu yang bikin soal itu terus kira-kira terus melihat dari o instruksi tesnya ini yang memang kurang jelas atau bagaimana atau terus ibu tindaklanjutnya terus nggenahke ke siswa? Oh, yang dimaksud di sini ini lho, bukan begini. Itu biasanya kalau writing test itu kan kalau ketahuan kan kalau sudah koreksi. Kok anak-anak ini semua, berarti mungkin perintahnya yang tidak jelas, jadi ya mungkin dikembalikan ke siswa saja maaf kemarin mungkin perintahnya yang tidak jelas jadi mungkin ada penangkapan yang berbeda dari kalian. Ya mungkin kalau saya solusinya ya dikasih poin masing-masing saja, ndak apa-apa. Terus saya tanya ibu memberikan speaking test, writing test itu sebetulnya mengharapkan apa sih sama siswa-siswa ibu? Kok memberikan writing test? Ya tujuannya itu mau apa ya selain cuma sekedar mengetahui seberapa jauh siswa dapat belajar bahasa Inggris saya kira motifnya itu apa? Kalau saya ya motifnya motifnya ya itu tadi untuk mendapatkan feedbacklah feedback ke kita maupun ke siswa atau juga mungkin ya mengetahui itu tadi sejauh mana materi yang kita berikan dapat diterima anak dan sejauh mana anak itu bisa menangkap hal-hal yang diberikan oleh guru.

Tin-I-A/B-program evaluation

(cont’d) Ya. Tapi pertanyaan saya kok mengapa tes tersebut harus diberikan selain sekedar mencari e seberapa jauh siswa dapat belajar. Manfaatnya apa?Manfaat dari tes? Iya dari... Kalau saya ya ... Dari speaking test maupun writing test, manfaatnya apa?

Manfaatnya ya kalau saya ya cuma cenderung itu saja untuk memberi membuat apa persiapan mungkin dalam proses pengajaran selanjutnya mungkin memberikan materi yang lebih tinggi mungkin? Mm. Baik. Selain writing test dan speaking test yang ibu lakukan di kelas kan ada juga test-test lain ya seperti unas misalnya, atau

Tin-I-A/B-program evaluation


penjajakan unas. Na, menurut persepsi ibu bagaimana itu tes-tes yang seperti itu ya? Kira-kira baik untuk siswa atau atau bagaimana tentang e unas? Pendapat saya tentang unas? Heem. Kalau saya pribadi unas itu tetap perlu diadakan karena e yang namanya unas itu kan standar ya untuk membuat standar standar dalam satu wilayah mungkin untuk memberi standar pada siswa mana yang layak lolos sama ndak, jadi tidak ada unas juga pendidikan malah akan kacau karena standarnya apa yang dipakai yang nggak ada itu yang susah. Pendapat saya ya unas itu perlu terus terutama juga kembalikan ke tes-tes writing itu tadi aplikasi dari tes writing itu tadi yang mengarah ke unas karena selama ini kan bahasa Inggris tesnya yang diakui kan tertulis. Jadi kalau ibaratnya anak-anak speakingnya bagus, tapi writingnya jelek nanti kan ketahuannya kan di unas nilainya jelek.

Tin-I-Badministration Tin-I-B-the absence of implementation means disorder in the world of education

Tin-I-B-legal, prestigious

Menurut ibu, tes-tes seperti itu layak dipercaya itu ke validity validitasnya? Layak dipercaya.

Tin-II-B-reliable

Kalau umpamanya ada siswa yang tidak lulus ya perasaan ibu bagaimana? Kan ada yang mengeluh wah tiga tahu belajar Cuma ditentukan oleh waktu seratus dua puluh menit, seperti itu. Perasaan kalau ada siswa ibu yang tidak lulus bagaimana? Ya sedih jelas sedih, terus e kalau pendapat yang mengatakan sekolah tiga tahun ditentukan oleh dua jam itu juga salah karena yang namanya pendidikan itu tetap perlu ada standar tertentu standar kelulusan terus jadi memang apa-apa itu harus ada standarnya kalau nggak ada nanti bagaimana kita nanti menilai, bagaimana kita mau mencari hasil, istilahnya mengelompokkan siswa yang kalau misalnya kalau cuma kenaikan kelas. Itu mungkin anak mana yang layak naik kelas, mana yang tidak, mana yang lulus, layak lulus, mana yang tidak, itu memang tetap perlu.

Tin-1-3-B-sad when students fail

E berarti menjelang tes itu ada persiapan-persiapan tertentu untuk agar siswa ibu bisa lulus ya. Tadi kan mera ibu mengatakan bahwa sedih kalau ada siswanya yang tidak lulus. Tentu sebelum peristiwa itu terjadi mungkin ada langkah-langkah antisipasi. Apa saja yang dilakukan menjelang tes seperti unas itu? (cont’d) Ya kalau saya sebagai guru itu mungkin mempelajari dari modelmodel soal ebtanas terdahulu, kira-kira apa yang cenderung sering ke luar, mungkin dari grammarnya apa yang sering keluar, terus mungkin juga speaking dialog-dialog yang mungkin respon dan mungkin dalam simpati responnya apa, agreement-agreement itu responnya apa, terus juga dari itu kita bisa membuat apa latihan, mengarahkan ke siswa mungkin. Oh dari tahun ini modelnya cenderung ke sini, tahun ini cenderung ke sini, jadi kita me mengakumulasikan semula dari ... yang muncul dari ebtanas tiap tahun jadi kita trainingkan ke siswa, jadi model-model soal sebagai bekal di ebtanas mereka mungkin sudah familiar.

Tin-2-1-B-studying model test

Semacam drill soal? Ya. Drill soal.

Tin-2-1-B-giving


training, test practice drill Drill soal itu berupa apa saja selain itu tadi grammar yang sering keluar, model-model dialog, pelaksanaan di kelas itu implikasinya bagaimana? Ya cenderung mempelajari soal-soal yang terdahulu, ditambah mungkin dari buku-buku. Kan sekarang banyak sekali buku siap unas buku apa mungkin semacam apa apa istilahnya kalau siap unas itu mungkin? Untuk pembelajaran semester itu apa namanya? Jadi lali. Buku ...

Tin-3-1-B-learning previous test materials, reading test preparation materials

Preparation? Highly concerned about graduation and achievement

Preparation ya mungkin semacam semacam itulah, pokoknya segala jenis soal yang berhubungan dengan persiapan unas, model-model how to solve problem in unas sama mungkin triktrik, trik-trik dalam mengantisipasi soal grammar itu juga perlu mungkin dikasih trik khususnya kalau ini kata ini ketemu ini, kata ini ketemu ini. Itu biasanya perlu juga. Mulai sejak kapan tes e persiapan seperti itu dilakukan/ Apakah sejak dari kelas satu, kelas dua, kelas tiga, atau kapan? Kalau saya cenderung mulai masuk kelas 3. Ya jadi diarahkan ke materi unas.

HB/BB

225 7.7 Teacher Iin 2 Comment Interviewer: Researcher Interviewee: Teacher Iin Setting: SMK N 1Pengasih.

Narrative

Code

Ibu, sekarang sebentar lagi kan mengikuti tes penjajakan tingkat kabupaten, ya. Perasaan ibu bagaimana saat ini dalam hubungannya dengan e menyongsong tes penjajakan tadi? Yang pertama adalah … ada dua hal ya… Dua-duanya ada optimis dalam kaitannya karena kemarin dalam KBM saya sudah me memberikan beberapa pelatihan yaitu dengan mengerjakan tugastugas mengerjakan soal-soal Unas terdahulu atau mungkin soalsoal penjajakan dahulu yang model-model soalnya mengarah ke materi Unas. Tapi e kita juga pesimis juga karena apakah e sejauh ini pelajaran yang kita berikan atau soal-soal latihan yang kita berikan itu sudah sudah ibaratnya sudah membekali siswa untuk menghadapi penjajakan tersebut. Ya, jadi ada dua yang penting

Tin-1-1-Boptimistic Tin-3-1-B-test practice Tin-1-1-Bpesimistic


optimis bagaimana kok pesimis sama optimis. Optimisnya saya kemarin sudah memberikan bekal, pesimisnya takutnya apakah bekal kemarin sudah di sudah bisa diaplikasikan siswa apakah mungkin sudah me sudah cukupkah, bekal yang saya berikan kemarin untuk bekal siswa untuk menghadapi unas. Kadangkadang juga masih tanda tanya besar karena kemampuan anak kan beda-beda. Ada yang cepet ada yang lama menelaah soal jadinya dua hal itu perasaan saya Mmm. Bedanya kalau menjelang mengikuti ulangan harian? Concern about test status

Beda. Perasaan kalau ulangan harian kan ibaratnya kita kan cuma tolok ukur tingkat sekolah, tingkat kelaslah katakanlah. Tapi kalau kalau unas ini kan ini kita kan langsung dinilai dinilai paling ‘ndak kan tingkat kabupaten. Jadi kita ada perbandingan antara o, sekolah ini bagaimana, sekolah itu bagaimana kan. Jadi kita tolok ukurnya sudah lebih tinggilah katakanlah seperti itu. Jadi e ada semacam beban yang berbeda antara… Ya,ya. Lebih lebih derajat ketakutannya lebih tinggi sekarang. (tertawa)

Tin-1-1-B-scared

O.

Worry about failure

Takutnya siswa nggak mampu gitu. Ha kalau yang ulangan harian? Kalau apa ndak ada perasaan takut siswanya tidak mampu? Ya takut juga, tapi kan nanti kalau ulangan harian kan nanti bisa ada ibaratnya apa ada remidi atau apa. Kan ada ada penanganan tersendiri setelah, setelah apa, setelah ulangan harian. Jadi kelasnya maksudnya tingkatnya juga lebih lebih lebih mudah bagi kita untuk memberikan treatment bagi siswa. Tapi kalau sudah unas kan sudah nanti kita harus menelaah model soalnya bagaimana soal apa yang siswa itu ndak bisa misalnya kalau grammar grammar apa yang masih sulit, model dialog apa yang masih anak-anak masih bingung menjawabnya. Itu kan kita harus lebih-lebih sulit lagi tidak hanya satu ibaratnya tidak hanya per unit tapi kayaknya sudah all sudah semua. Pelajaran yang diberikan mulai dari kelas satu sampai kelas tiga.

Tin-3-3-A-remidial

Tin-III-A-easier follow-up Tin-III-B-wider coverage

(cont’d) Kalau yang penjajakan itu apa tidak ada kesempatan untk remidi juga? Saya kira ndak ada. Cuma nanti mungkin itu hanya dijadikan tolok ukur bagi gurunya kalau hasilnya masih jelek. O mungkin anak-anak saya belum mampu, bekal yang saya berikan masih kurang berarti sebagai guru harus meningkatkan apa to yang yang dari segi apa to yang anak-anak banyak nggak bisanya.Jadi mungkin lebih lebih memberikan pelatihan yang e yang semua hampir semua siswa salah menjawabnya. Jadi cenderung ke situ me memberikan bekalnya diarahkan ke semua apa hal-hal yang kebanyakan siswa ndak mampu.

Tin-3-3-Bevaluation, training

more

Baik. E nanti setelah penjajakan berakhir rencana ibu bagaimana? Yang akan dilakukan? Ya masih masih seperti kemarin memberikan yang pertama

Tin-3-3-B-material


melihat itu tadi melihat ketidakmampuan anak paling banyak dalam hal apa. Jadi kita ulang katakanlah kita review ke yang situ. Kita kasih latihan lagi untuk apa pengayaan lagi kepada siswa tentang hal-hal yang anak mungkin e paling sulit dikerjakan oleh siswa. Terus lagi mungkin juga akan memberikan lagisoal-soal Unas yang terdahulu lagi. Dengan mungkin mungkin mungkin listeningnya anak banyak yang salah pada hal apa terus readingnya misalnya kalau reading pada hal apa. Jadi kita kita combine apakah listeningnya apakah readingnya.

review, enrichment

Mm. Na kalau maksud saya supaya siswa-siswa itu tahu tentang tes sendiri itu bagaimana? Maksudnya? Kan itu rencananya setelah selesai langsung memberikan materi yang siswa tidak bisa. Nanti siswanya bengong. Bagaimana kok dikasih ini. Dulu ini kok dikasihkan lagi. Ya mungkin kita kasihkan dengan model soal yang berbeda. Jadi mungkin dengan dengan teori yang sama tapi mungkin yang namanya soal kan beda-beda. Jadi mungkin kita mungkin diambilkan dari sumber lain. Mungkin yang agak-agak berbeda bobotnya.

Tin-3-3-B-giving test practice with different sources and format

Baik kalau yang itu tadi ya mengenai rencana KBMnya. Kalau untuk tes itu sendiri terhadap yang dilakukan yang ada hubungannya dengan tes itu sendiri. Yang mau dilakukan apa? Tindak lanjutnya hasil tes. Sekarang maksud saya yang ada hubungan langsung dengan tes itu sendiri. Setelah nanti hari Sabtu selesai penjajakan terus apa terus langkah-langkah tindakan apa yang mau dilakukan? Terhadap tes? Siswa kan mengerjakan. Terus setelah itu apa? Tindakan konkrit yang mau dijalankan, bukan hubungan dengan KBM . Berarti kita apa anak itu mengerjakan kita harus yang namanya anak mengerjakan kita nggak bisa tahu pada saat anak mengerjakan. Kita kan tahunya setelah ada hasil dari siswa to? Ya yang dilakukan terhadap pekerjaan siswa itu apa?

Menganalisa tadi. Kira-kira kesalahan terbanyak pada hal apa. Jadi kan kita harus tahu bentuk soalnya, kita analisa ini model soal ini, soal ini misalnya tentang conditionalnya berapa soal, tentang speaking berapa soal. Kita analisa terus kita kita cari tahu tadi berapa atau soal apa yang mungkin kebanyakan siswa itu salah menjawabnya. Saya cenderung ke situ. Kita lihat kita lihat bentuk soalnya seperti apa. Kita-kita apa kita lihat aplikasi ke materi pelajarannya yang paling banyak sulitnya itu yang yang mana. Kita lihat satu per satu. Mungkin prosentase prosentase siswa menjawab benar berapa, prosentase siswa menjawab salah berapa. Jadi untuk untuk feedback kita mana to materi yang anakanak mesti salah menjawabnya. Maksud saya terhadap pekerjaan tes itu yang dilakukan ibu itu apa? Tidak ada hubungan dengan mengajar atau menyajikan materi. Biasanya apa?Dengan lembar kerja siswa? Ya.Apa yang dilakukan?

(cont’d) Tin-3-3-B-error analysis


Ya, paling kita kan mengoreksi. Koreksi. Tapi kan yang koreksi kan sudah ada tersendiri to. Kalau kalau apa penjajakan kan sudah ada apa, mungkin kita cuma cuma bisa lihat kalau di sekolah ada mungkin ada lembar jawaban yang khusus untuk sekolah. Kan ada lembar jawaban yang mungkin dikirim ke Kabupaten atau ke mana. Itu mungkin kita lihat yang lembar jawaban yang sekolah. Yang lulus berapa yang ndak berapa. Bisa diulangi lagi tadi? Jadi apa yang ibu lakukan setelah selesai pelaksanaan tes itu? Peristiwa apa yang terjadi? Sehubungan dengan hasil kerja siswa tadi? Mengoreksi. Setelah mengoreksi? Melihat prosentase siswa yang bisa lolos nggak, walaupun mungkin dengan sistem penyekoran kita kan mungkin beda dengan sistem yang dilakukan panitia tapi yang jelas secara secara gampangnya dulu kira-kira berapa siswa yang sudah masuk passing grade, siapa yang belum

Tin-3-3-B-checking the number of students who get through the passing grade

Setelah itu kalau sudah tahu hasilnya? Terus kegiatan apa yang dilakukan? Ya, mengoreksi, tadi kan sudah mengoreksi, terus menganalisa, terus ini itu. Setelah itu. Setelah dianalisa, diprosentase siswanya sekian-sekian. Mengelompokkan siswanya kira-kira yang nggak lulus dikelompokkan sendiri, yang lulus dikelompokkan sendiri, mungkin kita lihat dikasih treatment yang berbeda, mungkin?

Tin-3-3-Bcategorizing, putting students in groups Tin-3-3-B-giving different treatment

Yang lebih riil lagi apa kira-kira setelah itu? Mengoreksi, ya kayaknya cuma itu yang namanya guru kan sudah ada pekerjaan dikoreksi, menilai.

(cont’d) Sekali lagi, pertanyaan saya, prosedurnya itu dijelaskan secara runtut. Setelah pelaksanaan tes ya hari Sabtu, terus Bu Iin menerima pekerjaan ataau pekerjaannya disampaikan ke orang lain. Tahaptahapnya apa? These formalities might increase the level of test status.

Mungkin mengoreksi, terus menilai, setelah menilai mungkin di e diberikan ke panitia mungkin panitia yang di sekolah mungkin. Apakah hasilnya bagus atau nggak itu kan harus kita laporkan ke panitia. Jadi mungkin dari panitia mungkin ada semacam treatment apakah mungkin ada hal-hal yang harus saya lakukan terhadap siswa-siswa. Mungkin dilaporkan ke sekolah dulu kirakira hasil siswanya bagaimana. Terus bagaimana siswa akan tahu e bahwa prestasinya bagus dan jelek? Kalau ibu tidak melakukan apa-apa? Jadi kan mengelompokkan. Tiba-tiba kan siswanya tiba-tiba dikelompokkan, kenapa?


Ya sebelum kita kelompokkan mengumumkan hasil dulu ke siswa. Mengumumkannya di mana? Kalau saya lebih senang di kelas. Di kelas kita panggil, misalnya A nilainya sekian, B nilainya sekian, C nilainya sekian, dari mungkin dari sistem penyekoran yang kita punya. Gambarannya seperti itu. Terus, kalau ini kembali ke masa yang lalu ya? Kalau selama tahun yang lalu ya, sehabis penjajakan tes lalu mengumumkan seperti itu reaksi siswa bagaimana? They were happy to get feedback which might lead to learning motivation

Biasanya kalau yang selama ini terjadi kalau misalnya ada siswa A nilainya tinggi, sorak-sorak. Nilainya jelek juga sorak-sorak [laughing] Siswa jelek juga sorak-sorak? Ya, mereka sorak-sorak. Itu sorak-soraknya “Huuu”… Gimana gitu popkoknya responnya kayaknya itu. Sama. Ya sorak-sorak semua. Mungkin, mungkin siswa yang bersangkutan mungkin sedih ya jelas mungkin nilainya jelek. Tapi kan itu kan namanya biasa. Terus yang bagus, “ Huaaa..” misalnya seperti itu. Nah kalau sekarang pada saat ibu mengawasi tes penjajakan atau tes TOEIC atau apa pun. Tapi bukan tes ulangan umum harian ya selama di kelas itu kira-kira yang dirasakan apa? Kita lihat yang terakhir ibu amati dilakukan di kelas itu merasa bagaimana atau apa yang dipikirkan? Merasakan apa dulu? Kalau saya kalau pas ngawasi itu apa pun pelajarannya mungkin saya lihat reaksi siswanya dulu. Kalau siswanya banyak mengeluh, o berarti soalnya sulit. Kalau siswanya istilahnya gimana ya tidak bersikap manislah katakanlah dalam tes itu mungkin ehm soalnya terlalu tinggi atau terlalu susah atau malah kalau heboh sekali mungkin terlalu mudah atau bagaimana. Jadi mungkin ya cuma mengamati saja. Kira-kira siswanya tidak apa mungkin dalam hal mengerjakan tesnya itu tidak atau mengganggu itu ya mungkin kita kasih peringatan atau bagaimana. Perasaannya ya biasa-biasa saja.

Kalau ada siswa yang tadi ibu mengatakan mengganggu?

Tin-3-2-B-see students’ reaction

Tin-2-2-B-level of difficulty Tin-1-2-B-all right

(cont’d) (cont’d)

Concern orderliness

about

Ya kita kasih peringatan misalnya mungkin katakanlah mereka nggak tanya tapi mungkin pinjam apa peralatan temannya. Itu kan kalau terlalu apa tingkah lakunya terlalu misalnya sampai jalanjalan itu kan tidak dibolehkan.

Tin-3-2-B-warning

Ada yang sampai jalan-jalan? Ada kalau saya, pinjam tipek ya jalan betul. [Laughing] atau mungkin..

Tin-4-2-B-see students do other activities

Perasaan ibu kalau seperti itu? Tin-1-2-B-annoyed Concern about orderliness to make class situation calm so that the result of the test is good

Jengkel, anak-anak kok seperti itu, ndak bisa diatur gitu ya. Misalnya. Kalau pinjem, pinjemnya malah saya keras, Kalian tidak boleh pinjam meminjam alat. Biar anaknya diam.


Ada pernah mengalami kejadian yang apa semacam kejadian yang luar biasa pada saat pelaksanaan tes? Teacher tries to control the situation

Ya, paling pernah menemui anak ngepek, anak nyontek. Kalau ada anak nyontek, mungkin ketahuan, kertasnya saya minta. Mungkin saya sertakan malah saya jadikan satu dengan lembar jawaban siswa tersebut. Ya, kalau masih bisa diperingatkan ditarik kertasnya, tapi dengan catatan khusus. Dengan catatan khusus bahwa anak nomor sekian, bernama sekian nyontek. Tapi biar guru yang bersangkutan yang e memberi tindak lanjut kepada siswa tersebut. Tapi yang jelas di dalam ruangan diperingatkan. Mm.

Tin-4-2-B-see students cheating Tin-3-2-B- warning irregularities

Untuk sementara saya ingin pindah ke yang ulangan harian. Apa yang ibu lakukan sebelum ulangan harian? Mengadakan les untuk persiapan penjajakan besuk hari Sabtu.

Tin-1-1-B- giving extra lesson

Itu siswanya sendiri yang minta? Ya.

Tin-4-1-Bdemanded by students to give the extra lesson

Oh.Berapa jenis tes yang ibu berikan kepada siswa? Jelaskan apa saja. Teacher employs two kinds of classroom test which measure reading and speaking skills

Pada dasarnya ada dua jenis tes yang saya berikan kepada siswa. Yang pertama yaitu tes e writing atau readinglah gampangannya tes reading, sama tes speaking. Ya. Yang mmm yang bisa diambil setiap akhir akhir modul atau akhir akhir pelajaran. Misalnya untuk tes reading itu bisa mencakup grammarnya, apa vocabularynya. Tes dua jenis tes yang pertama reading, yang kedua tes speaking ya. Tes speaking saya berikan untuk setiap akhir sama untuk reading. Untuk tes speaking diberikan pada tiap akhir modul tapi nanti ada dua jenis yang berbeda yang digali dari masingmasing tes itu. Kalau tes speakingnya biasanya pada performance siswa, misalnya ekspresi. Jadi kita harus tahu ekspresi apa yang ada di modul A misalnya. Jadi anak-anak ya harus tahu accuracy, apa how to respondnya a bagaimana juga harus tahu, terus yang kedua tes reading. Reading biasanya menya mencakup ini vocabulary, grammarnya, tata bahasanya dan juga mungkin mm reading itu sendiri. Bacaan-bacaan yang mungkin mengacu ke topic. Kayaknya cuma itu. Terus kalau listening itu biasanya cuma saya kasih, tidak ada nilai tersendiri. Kalau saya cuma mungkin saya tambahkan sebagai pelatihan saja pada unit unit tertentu. (cont’d) Alasannya apa ibu memberikan tes-tes itu? Apa tujuan tess-tes tersebut? Alasannya untuk saya kira kalau alasan sama tujuan sama, yaitu untuk mengetahui e sejauh mana mm apa penangkapan siswa terhadap materi yang kita berikan. Sejauh mana penguasaanpenguasaan mereka terhadap unit-unit atau modul-modul yang kita berikan.

Tin-I-A/B-measure the mastery of the given lesson

Menurut ibu, tes yang baik, yang ideal itu yang seperti apa? She seems dissatisfied with the implementation of the speaking test and considers that the oral test is inadequate to

Itu menurut saya ya yang mencakup semua aspek. Yang mencakup semua aspek baik itu mm reading, ya baik itu juga listening, tetapi kalau di model-model tes secara tertulis itu bagusnya antara speaking sama readingnya juga termaktub jadi satu. Misalnya ada reading, misalnya penguasan penguasaan itu tadi kosa kata sama grammarnya. Terus pada speaking, bisa bisa speaking tapi bisa

Tin-IV-A/B-testing all aspects


measure the student’s speaking ability. She thinks it will be better to include a written test to validate the oral test.

selain performance, penampilan siswa, bisa juga dimaksudkan dalam bentuk dialog. Jadi anak tahu responnya bagaimana itu tes speaking itu dalam bentuk tertulis. Saya kira itu lebih bagus jadi antara kombinasi antara yang reading sama speaking tapi dalam bentuk tertulis.

Menurut ibu, tes yang disukai siswa itu tes yang seperti apa? Kalau menurut pengamatan saya selama ini yang disukai siswa itu juga merupakan tes tertulis. Kalau tes performance speaking, mereka itu biasanya nervous atau mungkin tidak tidak percaya diri untuk me mengikuti tes tersebut, jadi mereka lebih senang kalau tesnya tes tertulis.

Tin-III-Bpreferred by students Tin-4-2-A-nervous, inconfident

Tapi dari yang saya dapat dari ibu dari wawancara terakhir ibu menjelang pelaksanaan tes itu juga ada perasaan takut. Ya.

Tin-1-1-B-afraid, worried

Alasannya kenapa kok itu takut, padahal anak-anak kan lebih suka mengerjakan tes yang seperti itu. Worry about failure

Takutnya nggak bisa mengerjakan dengan baik. [laughing]. Takutnya ya itu tadi apakah anak bisa menangkap selama ini dengan training-training yang kita berikan itu apakah bisa diaplikasikan ke anak ke soal-soal yang diberikan. Cuma ya nervous saja, tapi kadang-kadang juga yakinlah kalau anak-anak bisa mengerjakan soal yang kita yang diberikan e berdasarkan apa pelatihan-pelatihan yang kita berikan terdahulu.

Tin-1-1-B-worried Tin-1-1-B-sure

Jadi ibu tetap berpendapat bahwa tes yang tertulis seperti tes penjajakan, unas itu masih cukup penting walaupun itu bisa menimbulkan ketakutan? Ya. Penting.

Tin-I/II/III-Bimportant

Seberapa pentingnya itu? SANGAT penting. [laughing]. Sangat penting. Terus pertanyaan yang lain. Ibu kadang-kadang takut kalau siswanya itu anu jelek hasilnya, pada hal di lain ketika saya Tanya waktu di dalam tes, kemudian ada anak ramai dan lain sebagainya, ibu merasa terganggu dan siswanya tidak boleh pinjam barang ya dan lain sebagainya. Lha nanti kalau itu ada dampak dampak ke hasil tesnya bagaimana? Kok kelihatannya menurut saya kok agak kontradiktori ya, yang satu pihak ingin nilainya bagus, di lain pihak di dalam ruang tes cuma pinjam apa juga ndak boleh? Concern about reliability as well as achievement Teacher tries to manage the situation to increase the reliability of the test result

Ya maksudnya begini untuk anak itu, e preparationnya itu kalau bisa ya sebelum. Jadi kalau bisa ya anak itu all All semua itu me mempunyai barang-barang yang harus, barang-barang wajib yang harus dibawa pada tes itu harus punya. Jadi maksud saya begini. Kalau mereka sudah punya sendiri, itu kan juga akan konsentrasi mereka akan tertuju hanya pada soal tes itu. Jadi tidak terpengaruh pada hal-hal lain yang bisa menimbulkan misalnya. Ya bisa menimbulkan mungkin ada teman lain yang terganggu dengan polah tingkah seseorang. Ya jadi menurut saya ya hasil mungkin itu perasaan bisa bisa mempengaruhi hasil tes. Nanti akan saya tanya lagi mungkin mengenai hal itu ya. Sekarang saya Tanya yang lain lagi. Tolong digambarkan suasana tes harian yang ibu berikan pada saat ini.

Tin-I/II/III-B-very important

(cont’d)


Teacher tried to manage the situation

Kalau saya memberikan ulangan harian itu saya ya seperti biasa. Siswa kita berikan tes itu suasana tenaaang. Terus yang, terus anak juga tidak membuat ulah yng mungkin. Karena kalau saya sebelum tes biasanya memberikan batasan-batasan apa yang bisa dilakukan siswa sama apa yang tidak bisa dilakukan siswa. Jadi saya selalu berharap mmm atau mmm bersikap keras untuk halhal yang sudah saya tekankan bahwa ini tidak boleh dilakukan oleh siswa. Jadi saya selalu menitik beratkan di situ sehingga situasinya jadi tenang. Misalnya apa tidak boleh apa kalau di dalam ulangan harian yang ibu berikan. Kalau saya …

Tin-3-1-Ainforming rules of the game

Yang tidak boleh misalnya? Concern about the nature of measurement

Kalau saya ada batasannya itu tadi. Saya sangat strict untuk halhal seperti itu, misalnya untuk e masalah sepele saja, misalnya anak harus atau kedua tangan anak harus berada di atas meja. Ya. Tidak boleh toleh kanan, tidak boleh toleh kiri. Ya, semua hal, misalnya mmm tidak boleh itu tadi pinjam meminjam alat ya tidak boleh. Ibaratnya mungkin ada anak yang duduk di dekat tembok, bersandar di tembok itu juga tidak boleh. Jadi anak harus posisi tubuh mereka pun harus harus harus saya tekankan biar mereka karena kadang-kadang kan anak memasukkan tangan ke laci, dengan tujuan mencari contekan. Na, itu saya tekankan bahwa kalau kalian tes, semua tangan ada di atas meja.

BB HB

Itu pada saat tes, istilahe ‘njenengan tes yang tertulis? Tertulis. Ya. Ya.

BB

Kalau yang harian, yang untuk speaking test? Kalau yang speaking itu mm saya bebaskan untuk anak-anak yang belum mendapat giliran maju itu untuk berlatih sebelumnya ya berlatih dulu atau mungkin kalau mereka mungkin sudah hapal di rumah mungkin sudah hapal dialognya, mungkin ditekankan lagi mungkin untuk intonasinya, atau rising fallingnya itu harus harus berlatih lagi biar nanti kalau ke tempat sudah sudah sudah siap, betul-betul siap, tidak hanya cuma tampil saja, tapi tampilnya maksimal untuk tes speaking. Jadi untuk gampangannya, untuk yang belum maju sudah berlatih dulu. Jadi ada perbedaan antara situasi ruang ya daripada saat ulangan harian dengan apa tes tertulis?

(cont’d) Ada, ada. Kalau speaking, kelasnya agak-agak crowded ya tapi kalau tertulis biasanya tenang, sangat sangat tenang. Jadi waktu ulangan harian itu ibu tidak marah ya misalnya kalau ada e siswa yang tadi ya misalnya tidak menaruh tangannya di sini. OK. Yang berikutnya bagaimana ibu merespon pekerjaan siswa? Ya entah itu yang tes tertulis atau pun tes yang speaking itu? Maksudnya merespon setelah tes berakhir? Ya bagaimana ibu mereaksi dari hasil pekerjaan siswa, kan pada saat menguji kan ibu melihat ini itu. Respon ibu bagaimana kalau yang tes ulangan harian speaking tes?

She hold an

E mungkin saya begini. E untuk pelaksanaan tes speaking ya biasanya kan anak-anak sudah membuat drafnya. Ya, buat drafnya di rumah. Terus mereka menghafalkan draf tersebut untuk


opinion that speaking comes after writing.

ditampilkan di depan kelas atau di depan saya atau di depan teman-temannya mungkin. Jadi e saya minta draft yang mereka buat ya sama ini mereka harus maju, jadi kadang-kadang saya lihat cocok ndak dengan draft yang mereka buat, terus lagi juga e kalau mereka mungkin ada ekspresi tertentu yang salah, saya lihat. Juga di draftnya juga salahnya di mana. Jadi nanti setelah setelah selesai speaking itu nanti mesti saya kasih comment. Jadi siswa ada feedback ya siswa memperoleh… Ya. Kamu salahnya di sini, terus ekspresinya kalau misalnya ekspresi ini responnya harus begini. Itu saya kasih feedback untuk setiap tes speaking.

LFEE

Terus tanggapan siswa waktu ibu beri masukan atau komentar bagaimana? Speaking was perceived as the application of linguistic knowledge

Ya mereka senang kan, jadi jadi tahu ya misalnya o, ternyata selama ini apa mungkin antara mungkin mereka sudah tahu aturan ya, kadang-kadang sudah tahu aturannya begini, tetapi mungkin secara teoritik. Secara praktek mereka masih belum bisa mengaplikasikan dalam bentuk tulisan atau misalnya dalam bentuk speaking. Ya masih cuma baru sekadar sekedar tahu, tapi aplikasinya belum,. Misalnya anak-anak di situ.

HB

Tidak pernah ibu menerima protes ya atau misalnya reaksi siswa kok terus negative, marah-marah atau … Concern about reliability

Kadang-kadang ya itu protesnya itu kalau saya memberi pertanyaan tambahan yang menurut mereka dianggap mereka terlalu susah karena mereka tidak siap sebelumnya dengan tes speaking, jadi kadang-kadang saya mesti kasih pertanyaan tambahan sebelum saya kasih comment. Saya kasih pertanyaan tambahan anak ini betul-betul tahu atau cuma hanya sekedar apalan gitu lho, nanti akan kelihatan. Kalau sing kalau yang di rumah cuma cuma betul-betul menghafal saja ya tanpa mengetahui konten dari suatu modul mungkin jadi kalau dikasih pertanyaan tambahan mereka sering kaget, na itu kadang-kadang protesnya di situ. Kok jadi melenceng, katanya mereka. Pada hal itu kan cuma untuk me mencari tahu anak ini siap atau tidak, tahu betul atau tidak. Selain sekedar tahu atau tidak, apa tidak ada tujuan yang lain dari pelaksanaan tes itu? Tes speaking? Baik dua-duanya. (cont’d) Dua-duanya. Kan selama ini saya tanya tes itu untuk memberikan supaya mengerti level siswa, supaya mengerti dia bisa atau tidak. Pertanyaan saya, selain sekedar mengetahui bisa atau tidak, ada tidak misalnya untuk e tujuan-tujuan yang lain dari tes. Ya mungkin, tujuannya mungkin … Atau fungsi, manfaat dari hasil tes itu? Mungkin dari hasil tes tes itu kita bisa me ya jadi tahu, katakanlah o ini ternyata kalau di Bahasa Inggris bagus, ya mungkin ada anak-anak tertentu yang mempunyai bakat atau kecakapan dalam Bahasa Inggris. Ya nanti mungkin setelah itu bisa kita arahkan mungkin kalau di sekolah ada penunjukan untuk mewakili lomba PKS atau apa nanti bisa kita jadikan sebagai apa sebagai bekal, o

Tin-I-B-program evaluation


anak ini bagus o anak itu tidak bagus itu kan kita jadi jadi tahulah kemampuan masing-masih anak, ya mungkin layak untuk diikutkan lomba misalnya debat atau apa. Jadi mungkin itu tujuannya. Pada saat melaksanakan atau mengawasi tes, apa yang anda rasakan? Ya itu tadi perasaannya mesti ke takut tadi. Takut takut e takut kalau anak misalnya pas itu pelajaran kita pelajaran yang kita ampu jadi kan kadang-kadang takut lihat-lihat anak tak bisa mengerjakan. Perasaan saya sepanjang selama ini cuma itu saja.

Tin-1-2-B-worried about failure

Tidak ada perasaan lain? Tidak ada. Cuma takut? Takut (laughing). Jadi merasa takut terhadap hasil siswa, tapi di lain pihak juga merasakan bahwa tes itu sangat penting? Sangat penting… (laughing) Sulit jadi guru… Setelah pelaksanaan tes ya baik itu yang ulangan harian, maupun yang ulangan kenaikan kelas atau juga unas, apa perasaan ibu? Apakah juga sama dengan yang dirasakan pada saat menjelang dan saat melaksanakan tes? Pada saat menjelang pelaksanaan tes? Perasaan saya? Setelah selesai memberikan tes baik itu ulangan harian atau unas atau ulangan tertulis lainnya. Setelah tahu hasil tes itu atau mungkin sebelum dikoreksi? Ya pokoknya semuanya SETELAH mungkin dari detik terakhir bel selesai?

A high-stakes test was considered as highly reliable not only to assess students but also teachers. It offered competitiveness

Ya yang jelas perasaannya lega, lega, tesnya sudah berakhir. Terus mungkin juga tapi kelegaannya masih diliputi tanda tanya besar, ya tanda tanya besar ( laughing ) tentang ada siswa berhasil atau tidk itu masih masih merupakan e kalau saya kok merasa ini seperti beban. Bebannya bukan seperti takutnya ya itu tadi ya Cuma sebatas ah anak saya bisa nggak. Tapi saya tetap menganggap tes itu penting, tes tertulis pejnting, tapi ya itu untuk acuannya ke diri saya sendiri, untuk ke koreksi saya sendiri. Ah saya itu sudah mampu belum untuk me... me... misalnya mengampu pelajaran ini. Untuk misalnya kelas 3 itu, memang saya layak nggak dikasihkan kelas 3. Anak-anak mampu nggak. Anakanak kalau saya ajar hasilnya bagaimana itu kan mungkin nanti saya tetap, tetep membandingkan mungkin dengan dengan klasklas lain. Ya itu mungkin untuk keke arah diri saya sendiri. Mungkin ada koreksi, mungkin saya kurang begini, kurang begini. Ke diri saya sendiri malahan.

Tin-1-3-B-relieved

Jadi kalau ada siswa yang nilainya tidak lulus it terus e ibu merasa bahwa itu kesalahan ibu? A high-stakes test was considered as assessing teaching

Bisa juga seperti itu. Saya sering merasa merasa berdosalah kalau ada anak yang sampai nggak lulus walaupun itu mungkin siswanya memang betul-betul low, tapi kan kayaknya kok saya kok

Tin-1-3-B-guilty if failure


competency

nggak bisa sih membuat anak ini jadi lulus. Mungkin ada PR kayaknya masih menjadi semacam PR besar buat saya untuk lebih mengingkatkan kemampuan. Sebetulnya saya tanya lagi, tes yang ideal itu yang seperti apa sih?( long pause ).Ibu tadi sudah menjawab tentang tes ideal yaitu misalnya ada reading dan lain sebagainya, materinya seperti itu. Selain yang sekedar itu?Mungkin dilihat yang seperti apa? Jadi apakah tidak ada keinginan misalnya waduh, siswa saya karena banyak yang tidak lulus, terus soalnya tidak usah yang sukar-sukar atau yang seperti apa? Dengan level tingkat. Kalau saya tes ideal itu yang mempunyai yang mempunyai tolok ukur yang jelas, gitu misalnya.Kalau memang siswanya itu rendah, itu berarti memang kita harus yang meningkatkan itu gurunya, kalau saya. Tetep, tetep gurunya yang harus harus ber berusaha memperbaiki karena yang namanya standar kan namanya sudah standar tes itu biasanya sudah-sudah average lah sudah… maksudnya tidak terlalu tinggi, tidak terlalu rendah, tapi mencakup semua jenis siswa. Ya jadi tes yang bagus ya yang itu. Kalau sampai ada…

Tin-III-B-moderate level of difficulty

Tidak mesti soalnya yang yang terlalu gampang itu tes yang bagus Malah itu meragukan kalau tes terlalu mudah itu. Meskipun akibatnya siswa ibu lulus semua? Tidak. Tidak semua soal mudah itu bagus. (laughing). Jadi meskipun ibu katakan takut, tapi juga ada senangnya mestinya, ya to? Ya. Dari pembicaraan itu… Jadi meskipun itu menggelisahkan, atau menakutkan, takut tidak lulus ternyata juga kalau diminta untuk soalnya gampang juga tidak setuju kelihatannya. OK pertanyaan berikutnya, apakah saja yang biasa ibu jumpai, yang paling berkesan kalau melaksanakan tes, baik yang ulangan harian maupun yang e tes tadi, Unas? Waktu melaksanakan dan setelah? Ya, saat pelaksanaan tes itu ada yang menonjol apa, menemui apa, gitu? Ya mungkin perlu dipikir sebentar. Saya matikan dulu. (cont’d)

She expected her students to do their best in a high-stakes test. She was sorry to see them do the test speculatively

Paling dalam pelaksanaan tes itu saya sering menjumpai hal-hal yang kadang-kadang mungkin berkesan, mungkin tidak berkesan, misalnya yang selalu menjadi perhatian saya itu adalah kalau menjumpai siswa yang mungkin ya waktu belum habis tapi sudah keluar, ya. Atau mungkin katakanlah mungkin anak itu tidak.. mungkin anak itu terlalu pandai, mungkin soal dianggapnya terlalu mudah, atau mungkin malah mungkin juga terlalu sulit. Jadi anak tersebut tidak mau berfikir, jadi asal asal menjawab itu. Kadang-kadang kalau anaknya keluar itu saya saya kalau pelajaran yang saya ampu, lho ya sehingga saya jadi tahu betul salahnya paling ndak anak itu ngawur atau memang betul. Kadang-kadang saya lihat soalnya sama saya lihat jawabannya ya itu kira-kira banyak betulnya atau banyak salahnya. Kalau banyak salahnya berarti anak itu memang sekedar ngawur ya itu. Terus lagi mungkin ada anak yang ini mungkin dari dari apa siswa yang pandai ya mungkin di kelas itu terkenal selama ini mungkin banyak nilai yang bagus-bagus ternyata pada tes itu mungkin


malah belakangan. Na itu mungkin malah mungkin menurut pendapat saya mungkin anak itu malah memanfaatkan waktu sebaik-baiknya. Jadi memaksimalkan waktu yang diberikan untuk me mengerjakan soal tersebut. Jadi malah anak yang yang pandai di kelas itu biasanya malah lebih lebih hati-hati di dalam mengerjakan soal dalam melaksanakan tes, dari pada anak yang kadang-kadang memang yang levelnya rendah itu. Jadi cuma itu mungkin yang menjadi jadi perhatian saya. Mungkin yang keluar terlalu cepat atau mungkin malah yang kayaknya anak pandai tetapi malah tidak-selesai-selesai, itu katakanlah mungkin karena mungkin dia mengecek ulang jawaban setelah ditulis, atau apa. Selain dari siswa itu sendiri, mungkin ibu bisa menyebutkan lingkungan atau apa yang lainnya? Atau tes materi tesnya itu sendiri ya? Ya mungkin lembar jawab atau yang lain? Yang menonjol, yang perlu ibu utarakan? She was also sorry to see her students disturbed by inconvenience

Saya kira mungkin kalau lingkungan itu sudah terkondisi ya di mana kalau sekolah itu ada tes mesti sudah dikondisikan lingkungan setenang mungkin setenang mungkin tidak ada gangguan suara. Mungkin… dari lingkungan saya kira tidak ada masalah kecuali hal-hal yang khusus suara… Misalnyya suara mobil lewat atau apa itu kan tidak bisa di tidak bisa di apa tidak bisa dicegah, misalnya. Terus lagi mungkin pada waktu mungkin tes listening itu tiba-tiba ada tapenya suaranya nggak jelas, atau bagaimana itu kadang-kadang anak-anak pasti tambah comment sehingga semakin tambah gaduh, gitu lho misalnya suara nggak jelas. Mungkin anak-anak mengeluh sambil … ibaratnya mengeluh tapi tidak … mengeluh cuma dari sikap ya tidak tidak dari suara mungkin dari sikap yang nggak nggak jelas. Jadi mungkin itu saja perhatian. Mmmm menurut ibu tes itu membawa dampak yang bagaimana terhadap belajar siswa? Ada pengaruhnya tidak? Itu penting? Ada pengaruhnya karena… Kalau gini lho, bagi siswa yang tidak mengerjakan tes, apa manfaat dari tes itu? Mungkin bagi yang pandai itu jelas ya nilainya bagus. Sekarang pertanyaan saya untuk siswa yang katakanlah kurang pandai manfaat tes itu sendiri apa? Mungkin hanya sekedar katakanlah rutinitas atau mungkin malah juga formalitas, yang penting mereka itu kadang-kadang yang penting rapornya ada isinya. Kadang-kadang kalau anak sudah begitu kan memang sudah backgroundnya juga sudah… jadi di dikasih apapun anak anaknya memang sudah susah gitu lho. Jadi kadang-kadang mereka menghadapi tes ya asal… asal mengerjakan tes, tidak ada beban e..nilainya jelek. Jadi kadangkadang asal dilalui mereka sudah sudah aman, katakanlah.

(cont’d)

Kalau siswanya merasa ada beban kalau nilainya jelek, waduh merasa bisa tapi nilainya jelek. Kira-kira manfaatnya apa? Error in technical matters might suggest that the score was not reliable. It might cause harmful backwash as well as beneficial backwash

Ya mungkin manfaatnya me.. memberi warning kepada siswa.. ya memberiperingatan kepada siswa ketoke bisa tapi ternya nggak, merasa bisa tapi nggak… Itu mungkin e mungkin kalau pakai logika mungkin me..me… memberi jawaban melingkari jawabannya mungkin kurang kurang tebal atau bagaimana… Atau mungkin kadang-kadang mereka tidak teliti.. Tidak telitinya mungkin ada soal yang sulit dilewati,… terus soal yang mudah, salah nomor. Itu kan kadang-kadang mereka juga harus tahu kesalahan mereka. Jadi nggak teliti dalam ini… me… mentransfer jawaban mereka ke lembar jawaban.

LFEE


Baik untuk saat ini sementara itu dulu ya. Terimakasih.

237 7.8 Teacher Nanik 1 Interview Transcript Comment

Narrative

Interviewer: Researcher Interviewee: Teacher Nanik Setting: SMK N 1 Pengasih. 16 2007. 9.40 – 10.00

Bu Nanik, Bu Nanik sebagai guru bahasa Inggris tentunya sudah sangat sering melakukan kegiatan tes bahasa Inggris, entah itu di ruang kelas atau juga dalam ruang lingkup sekolah. Saya ingin tahu apa sih sebetulnya tujuan tes-tes tersebut. Tujuannya antara lain mengetahui, mengukur seberapa jauh kemampuan siswa dalam menguasai materi yang sudah disampaikan atau sudah diterangkan. Baiklah. E ie untuk itu apa yang dilakukan ibu di dalam penyampaian pelajarannya.

Code

Tik-I-Ameasure mastery of materials


Her teaching materials covered reading, speaking, writing, grammar

Oh, dalam penyampaian pelajaran itu kita memberikan materi terlebih dahulu. Misalnya materi reading. Anak saya berikan satu passage kemudian mencari kata-kata sulit atau e difficult vocab kemudian setelah itu understanding passage. Kemudian setelah siswa paham sudah tahu maksud isi bacaan tersebut kemudian menceritakan kembali retell the passage setelah menceritakan kembali, siswa sudah bisa, kemudian kita berikan beberapa pertanyaan berdasarkan passage tersebut. Kemudian e dalam materi speaking biasanya e saya berikan dialog pendek e short conversation, setelah itu kami menerangkan apa maksud-maksud dari pada setiap tanya jawab atau short conversation tersebut. Ini menunjukkkan apa misalnya e misalnya pesan hotel misalnya itu bagaimana kita mengekspresikan atau bagaimana kita mengungkapkan apabila kita akan memesan sebuah kamar hotel. Kemudian dalam writing, writing itu biasanya saya masuk ke bentuk sentence pattern atau tenses. Dari reading, kemudian speaking, writing. Kemudian writing masuk ke sentence pattern. Bentuk-bentuk kalimat yang digunakan dalam bacaan ataupun dalam conversation atau dialog itu dalam bentuk apa misalnya bentuk present tense atau past tense atau bentuk future dan sebagainya.

Tik-3-1-Apresenting relevant materials

Jadi e berapa macam tes yang diberikan ibu kepada siswa. Jadi kan tadi sudah mengajarkan reading, mengajarkan speaking, writing ya. Terus implikasi di dalam tesnya bagaimana? Di dalam. Setelah penyampaian materi tadi kami dipilah-pilah sendiri-sendiri. Kalau pas bahasan reading, reading selesai berarti nanti kita melakukan tes reading. Setelah itu kita selesai writing kita mengadakan tes writing. Setelah itu kita menyelesaikan speaking, kita mengadakan tes speaking. Kalau yang biasa sering itu nek speaking itu kan langsung conversation, sedang kalau yang writing writing itu biasanya materi satu materi tertentu dalam bentuk misalnya saya memberikan sentence pattern dalam bentuk present tense, maka nanti kami mengaplikasikan pada misal making daily activity in the form of present tense kemudian anak-anak membuat daily activity kegiatan sehari-hari mereka dari pagi sampai malam. Terus bentuk soalnya seperti apa itu? Bentuk soalnya biasanya saya misalnya memberikan make daily acitivity your daily activity itu misalnya, kalau yang apa writing. Kalau reading, reading saya berikan satu passage tadi kemudian saya berikan pertanyaan berdasarkan passage tersebut. Kemudian kalau speaking nanti conversation langsung praktek langsung saya berikan situasinya misalnya tentang apa conversation in the market atau story book. Atau yang lain-lainnya. Kemudian anak berbicara langsung dengan pasangan masing-masing. (cont’d) Terus sebelum melakukan tes itu siswa diberitahu tidak atau siswa inisiatif maju sendiri atau usul atau bagaimana? She expected her students to learn more before the test

Biasanya diberitahu, bu, karena kemampuan dari siswa-siswa kami yang terbatas sehingga kalau misalnya besuk pagi mau diadakan tes jadi materi itu sebelumnya diberikan terus dari awal mulai reading, reading selesai selesai satu topik bahasan terus kemudian besuk kita ulangan reading, readingnya sudah anu terus kembali lagi untuk pertemuan berikutnya writing, untuk pertemuan berikutnya speaking. Diberitahu misalnya oh tanggal sekian, jam sekian kita akan tes Bahasa Inggris. Ya, ya. Atau dadakan misalnya siswa tiba-tiba kaget wah sekarang ulangan atau bagaimana

Tik-3-1-Ainforming the plan to test


She refered to the situation to make a decision. She was concerned about achievement.

Ya kadang saya dadak, kadang-kadang juga saya beritahu tergantung nanti situasinya. Satu kelas dengan kelas lainnya itu berbeda-beda apa kemampuannya. Kalo kalau misalnya kelas akuntansi kalau di sekolah saya saya dadak biasanya. Kemudian yang saya beritahu itu kelas AP. Nek kelas AP selalu saya beritahu karena jelas kalau tidak diberitahu anaknya itu tidak siap. Jadi kita selalu mengulang, mengulang, mengulang terus, hasilnya tidak akan memuaskan. Reaksinya bagaimana kalau siswa diberitahu akan tes hari ini? Ya akhirnya kalau anak sudah diberitahu otomatis anak sudah siap dan dia ya berarti setiap begitu masuk kita mau memberikan tes dia sudah tidak lagi ramai, tidak ribut wa nggak diberitahu dan sebagainya. Nanti kalau sudah diberitahu situasinya kondusif. Begitu masuk, please prepare a paper and then please do this exercise atau do this test dan sebagainya. Menurut pengamatan ibu kira-kira tes yang apa yang kira-kira disukai anakanak mungkin malah sebaliknya. Wah kok tesnya seperti ini itu?

Concerned about test reliablity

Kalau yang disukai itu biasanya ya tes tertulis itu tadi yang writing yang mengerjakan apa yang tertulis misalnya apa misalnya passage seperti daily activity membuat daily activities itu tadi itu. Anak-anak malah lebih enjoy seperti itu tadi daripada tes kalau reading, pusing. Nih kata-kata sulitnya banyak yang tidak tahu. Karena kalau ulangan itu baik reading itu saya tidak pernah memperbolehkan membawa kamus atau melihat kamus. Jadi kan sudah dipersiapkan dulu. Besuk ulangan anak-anak harus siap tidak boleh membuka kamus. Jadi tidak pernah open book gitu ya tesnya jadi misalnya. Tidak.Saya berikan satu passage yang baru belum pernah saya berikan waktu KBM. Kemudian anak-anak mengerjakan sambil mencari itunya kemudian apa menjawab pertanyaannya. Kalau misalnya meretell itu ya retellnya itu ya with your own sentence, tidak boleh sama persis tapi dengan kata-katamu sendiri tapi maksudnya sama.Kalau speaking itu sebetulnya anak-anak. Kalau pas dialog mungkin dia ya senang, disamping itu juga melatih kemampuan berbicara juga kadang apa namanya rileks. Kalau pas yang tertulis ya writing yang masuk tenses-tenses dia nggak bisa rileks harus terfokus soal-soal yang. Kalau speaking kan kita kadang-kadang bisa tertawa, bisa apa pun, misalnya tangannya dan lain sebagainya. Kalau ibu menilai soal writing, pekerjaan siswa itu kriterianya apa disitu?

Setting a competency standard by applying scores

Kriterianya? Kalau tenses ya biasanya kalau dia itu betul artinya kalimatnya betul, kemudian kalimatnya juga sempurna, sesuai dengan tenses dan sesuai dengan pertanyaan yang ada, saya memberikan satu satu soal skornya dua. Kalau yang speaking ya nanti kita ya lihatnya bagaimana conversationnya, tensesnya benar atau tidak, juga fluently, kelancaran terus apa speakingnya keras atau tidak dari situ untuk tambahan speakingnya. Kalau speakingnya keras dia salah tapi speakingnya bagus kemudian ada lagi dia tensesnya bagus tapi speakingnya lambat, lirih itu tetap saya memberi nilainya ya kurang. Saya masing ingin tanya mengenai apa namanya writing tadi kan fokusnya pada tensesnya. Sekarang mengenai banyak sedikitnya kata yang ditulis siswa ada pengaruhnya ndak dengan nilainya? Banyak sedikitnya? Ada siswa yang tensesnya bagus tapi kemampuannya cuma bisa menulis tiga kalimat dengan yang lain yang satunya bisa 20 kalimat tapi tensesnya ya banyak salahnya.

Tik-2-2-Astudents are ready for the test Tik-2-2-A- get no complaints Tik-2-2-Aconducive situation Tik-3-2-A/Bforbidding students to use a dictionary

Tik-II-A- val. reliability Tik-3-2-A-prevent leakage Tik-2-2-Astudents happy, relaxed Tik-I-A-train to speak Tik-III-A-bring laughter, enabling to use gestures Tik-III-Bdoesn’t allow to relax (cont’d)


That she focuses on the accuracy potentially cause harmful backwash effects on learning.

Ya kalau saya artinya sedikit tensesnya bagus, dan itu ada korelasinya artinya perhubungan kalimat yang satu dengan kalimat lainnya, artinya tidak acak biasanya da misalnya bangun jam berapa, terus habis itu apa, itu kan ada urut-urutannya. Tapi kalau anak itu ber urut-urutannya tidak pas tetap kurang. Yang jelas sedikit akan tetapi tensesnya bagus lebih dihargai. Lebihlebih nilainya mendapat nilai yang lebih daripada banyak, nggladrah akan tetapi tensesnya ibaratnya kalau anak-anak saya itu hanya nyusun kata. Katakata dijentrek-jentrek tapi dia tidak punya arti, misalnya gini, orang Indonesia: ”Saya pergi ke pasar”. Anak-anak biasanya I am go to market, misalnya begitu. Ketoke luwes, tapi kan salah to? Anak-anak biasanya seperti itu. Kata-kata dijejer-jejer ibaratnya sudah jadi kalimat tapi kan lebih bagus yang kalau tensesnya benar, sedikit tapi benar, daripada nggladrah. Sekarang tensesnya benar tapi tidak menjawab soal misalnya tell about your daily activities, tapi ternyata di dalam dia menjawab malah menceritakan kegiatan sehari-hari orangtuanya. Jadi tensesnya bagus, bahasanya bagus, tapi tidak nyambung soalnya. Terus sikap ibu bagaimana seperti itu? Ya harus apa memberi memahamkan anaknya supaya tahu ini yang dimaksud itu bukan menceritakan setelah sudah dikoreksi. Setelah dikoreksi artinya kalau sudah dikumpulkan kemudian satu per satu kita berikan tak beritahu kesalahannya. Kamu kesalahannya di sini. Sedangkan kamu kesalahannya ini harusnya menceritakan activity kamu sendiri kenapa ini menceritakan daily activity orang lain. Kemudian kalau ini kamu salahnya kamu membuat kalimat tidak berdasarkan tenses tidak menggunakan struktur kalimat yang benar. Itu Kembali ke yang tadi pertanyaan saya, kalau misalnya siswanya itu tidak menjawab soalnya ya, tapi bahasanya bagus ibu kan memanggil terus menunjukkan terus pertanyaan saya nilai yang diberikan kepada siswa tersebut bagaimana? Nilainya ya misalnya saya satu kalimat dua tapi karena itu tensesnya benar. Saya kasih nilai ya setengah. Artinya siswa tersebut masuk kategori lulus atau tidak lulus? Tidak. Terus setelah tidak lulus, apa yang ibu lakukan?

Kami melakukan perbaikan lagi.

Oh, perbaikan.

Perbaikan lagi. Melakukan perbaikan bagi anak-anak yang belum sampai nilai minimal kita kan Nek yang lain-lainnya artinya misalnya dalam satu kelas yang sudah mendapat nilai bagus yang satunya sambil menunggu yang perbaikan tadi. Kalau sehari-hari ibu sering memberikan tes Atau hanya terbatas yang

Kalau yang abcd akan tetapi kalau masuk ke sumatif tes saya menggunakan abcd. Misalnya error recognition. Itu satu kalimat tidak saya pilih a,b,c, d. Ada yang salah dalam satu kalimat

(cont’d) Tik-3-3-A-give remidial


Selain tes yang sehari-hari tes Dan juga ujian nasional. Kan di dalam tes-tes seperti itu ada materi listening. Pertanyaan saya di dalam KBM itu siswa mendapat kesempatan belajar itu tidak.

Dalam satu bulanlah paling tidak cuman tiga kali. Dan formatnya paling ini melihat gambar. Kemudian yang berikutnya itu biasanya melengkapi dengan men. Satu bulan Kalau memberikan tes, ibu instruksinya menggunakan bahasa Indonesia atau menggunakan bahasa Inggris? Dalam perintahnya itu. Bahasa Inggris. Dengan menggunakan bahasa Inggris baik itu tes sehari-hari misalnya tes formatif setelah selesai satu pokok topik dan apa melakukan tes formatif itu selalu menggunakan bahasa Inggris, misalnya ya apa complete the following sentence itu or...

Tik-3-2-A-use English

Terus siswa di dalam memahami instruksi itu sering mendapat kesulitan atau tidak? Misalnya di dalam kelas itu ini maksudnya bagaimana, seperti itu. Belum ada sampai sekarang tidak ada yang merasakan kesulitan karena perintah-perintah yang saya berikan itu sangat simpel. Dan ada tidak siswa yang kurang memahami instruksinya itu jadi mengerjakannya jadi salah seperti yang saya utarakan tadi misalnya instruksinya tell your daily activities, tapi yang diceritakan malah daily activities orang lain. Selama ini

Tik-4-2-A-no student experiencing difficulty in understanding instructions Tik-III-A-use simple instructions

Selama ini belum ada. Baik sekarang saya mau tanya mengenai apa namanya pelaksanaan tes tersebut di sekolah? Berapa kali ibu memberikan tes writing , tes speaking dan juga tes sumatif itu dalam satu semester.

Kalau dalam satu semester itu kalau pas tidak PSG itu 4 kali, kalau pas ada kegiatan PSG itu cuma dua kali. Yang apa itu? (cont’d) Ya semuanya.

Writing tesnya 4 kali, speakingnya test 4 kali. Terus.Jadi kalau yang PSG 2 kali.Terus setelah memberikan tes itu siswa mendapat kesempatan untuk melihat kembali jawaban dan juga skornya atau juga cuma skornya saja? Setelah tes itu apa kertas yang dikumpulkan tadi, lembaran lembar jawab yang dikumpulkan itu selalu saya berikan ke anak dengan dicantumkan sekalian nilainya. Jadi anak itu bisa mengetahui oh, salah saya di sini makanya saya mendapat nilai sekian.

Itu semuanya baik-baik? Ya, sebagian kecil yang baik. Sebagian besar ya mesti empat, lima, paling baik ya delapan, tujuh, enam itu rata-rata itu misalnya dalam 1 kelas kan tiga puluh enam orang. Dari tiga puluh enam orang itu yang biasa mendapat nilai enam sampai ya enam ke atas itu biasanya dua puluh, kemudian sisanya itu empat ke bawah.

Tik-3-3-A-give feedback Tik-3-3-Areturn students’ work Tik-3-3-A-give feedback


Reaksinya bagaimana siswa yang nilainya kurang?

Concern about reliability

Anak yang kalau sana ya biasa, bu, kalau swasta itu ya santai saja itu tidak terus punya wah bagaimana saya bisa nilainya bagus seperti yang lainnya, tapi ya biasa saja nanti kalau misalnya sudah perbaikan beberapa kali misalnya dua kali perbaikan saya memberikan maksimal dua kali perbaikan kok belum bagus ya belum dapat biasanya kan diberikan tugas. Kalau diberikan tugas ya rata-rata anak seperti itu terus kompromi dengan yang lainnya apa. Makanya kadang-kadang saya berikan tugas yang berbeda satu anak dengan yang lain, tapi tingkat kesulitannya sama tapi bentuk soalnya artinya, berbeda.

Tik-3-3-A-give make-up exam and assignment Tik-3-3-A-give different tasks to students Tik-3-3-A-give different materials of the same level of difficulty

Perasaan ibu sendiri bagaimana kalau melihat siswa-siswanya apa mendapat nilai yang katakanlah kurang dari yang ibu maksud. Perasaan ya seorang guru itu ya mengharapkan siswanya itu semua mendapat nilai yang baik akan tetapi kan sadar ya kemampuan anak ya berbeda-beda, tingkat apa namanya daya tangkap juga berbeda, maka ya kadang-kadang ya pasrah saja dengan catatan kita selalu selalu berusaha bagaimana caranya untuk supaya anak tersebut tidak begitu ketinggalan dengan yang lain supaya barangkali ya dikumpulkan jadi satu kemudian kita apa kembali menerangkan lagi apa apa topiknya itu pelajarannya itu kita kumpulkan kembali sendiri, berapa anak, misalnya 10 anak, terus ditanya satu per satu e yang belum jelas mana yang belum. Kira-kira yang belum faham mana kemudian silakan tanyakan masing-masing. Kita ulangi materi kembali. Kemudian kita berikan lagi perbaikan. Hasilnya bagaimana kalau memang hasilnya tidak ada peningkatan ya memang mungkin kemampuan anak seperti itu kemudian kita membantu supaya memenuhi target nilai yang diharapkan.

Tik-3-3-A-do the best so that none is left behind. Tik-3-3-Aquestioning students about their problems Tik-3-3-A-do material review Tik-3-3-A-help to achieve the minimum score target Tik-1-3-Atolerant

Sekarang tes yang saya maksud tadi tes Unas. Tentunya ibu pernah mengalami katakanlah kegagalan siswa atau kesuksesan siswa di dalam menempuh unas itu terutama unas bahasa Inggris. Saya ingin diceritakan pengalaman-pengalaman ibu baik pelaksanaan maupun antisipasi atau kegiatan sebelum mengikuti tes tersebut.

(cont’d) Untuk kegiatan sebelum atau antisipasi unas itu di sekolah kami sejak begitu kenaikan kelas sampai kelas tiga awal tahun seperti ini, itu sudah diberikan les-les. Anak-anak itu diberikan tambahan pelajaran yang satu jam minimal itu dan dilaksanakan setelah KBM kita tambahkan satu jam misalnya hari di tiap hari Senin setelah KBM berakhir jam setengah dua kemudian satu jam ke depan itu untuk menambah ada tambahan pelajaran Bahasa Inggris pada anak-anak. Kemudian setelah setelah semester berikutnya yang menjelang menjelang Unas itu kita efektifkan. Kita selalu biasanya ada penelitian di luar itu tiga hari penelitian di luar untuk membuat tes akhir itu apa namanya proposal itu?

Tik-3-1-A-give extra lesson for thirdgraders since the beginning.

Tugas akhir? Tugas akhir, misalnya tiga hari ke luar, tiga hari pelajaran di sekolah. Pas tiga hari di sekolah itu biasanya materi-materi Unas semua apabila sudah ujian produktif juga. Misalnya Senin, Selasa,Rabu seperti kemarin. Senin itu materinya cuma Bahasa Indonesia, bahasa Inggris, Matematika. Selasa juga. Cuma ganti-ganti, artinya tiga hari itu untuk tiga unas tersebut dan Bahasa Inggris itu biasanya paling minta jam yang banyak misalnya kan matematika itu satu minggunya empat jam atau lima jam. Bahasa Inggris itu mesti

Tik-3-1-A-ask for the longest teaching hour


melebihi enam jam atau tujuh jam. Terus itu tadi pertanyaan saya mengenai keberhasilan atau kegagalan siswa dalam Unas Bahasa Inggris. Untuk keberhasilan Perasaan ibu bagaimana? Ya, harap-harap cemas karena dalam kesehariannya artinya dalam memberikan les itu kan tahu kemampuan siswa itu seberapa gitu. Anak ini kira-kira bisa mengerjakan Unas atau tidak? Kadang-kadang untuk menghadapi itu kita menunggu hasil itu juga harap-harap cemas, artinya semoga anak-anak juga diberikan kemudahan, diberikan kesehatan dan

Tik-1-3-Bworried, expecting best

the

Ibu berdoa terus? Hanya berdoa, ya artinya berdoa supaya anak-anak itu bisa lulus dengan baik.

Tik-3-3-B-pray

Terus selama pelaksanaan tesnya? Pelaksanaan tes? Maksudnya? Ya sesuatu yang ada di situ itu cukup kondusif atau tidak dengan apa namanya dengan tuntutan tes itu dilaksanakan? Ya fasilitasnya, ya tempat duduk siswanya bagaimana. Untuk tahun ini ya kelihatannya apa fasilitas sudah sudah sudah lumayan meningkat dan anak-anaknya pun sudah alhamdulillah meningkat dari tahun-tahun kemarin. Untuk tahun ini fasilitas apa yang listeningnya juga sudah paralel, kemudian apa namanya anak-anaknya juga kelihatannnya cukup siap gitu menghadapi unas tersebut tidak terus grogi nanti saya bisa mengerjakan dan tidak tapi saya sendiri kurang tahu kalau lihat sikap-sikap anak itu kayaknya biasa. Beda dengan kalau misalnya pelajaran yang lain kalau waktu kemarin saya di sekolah itu Bahasa Inggris kayaknya anakanaknya ya biasa-biasa saja nggak seperti kalau pas matematika. Wah sudah bingung apa-apa ada yang banyak ketinggalan anak-anak itu. Pelaksanaannya nggak begitu.Jadi pelaksananan dan suasana tes tersebut sudah cukup baik.

(cont’d) Cukup baik. Terus mengenai mengenai tadi pertanyaan saya mengenai kesuksesan dan keberhasilan. Pada saat hasil unas diumumkan dan ternyata ada siswa yang tidak lulus apakah ada perasaan ibu merasa takut atau gelisah atau bagaimana? Ya, pada saat itu ya perasaan takut itu tidak cuma kadang-kadang cuma merasa kasihan kok kenapa dia tidak lulus.Untuk tahun yang kemarin itu anak tersebut ikut apa termasuk ikut lomba debat itu, tapi dia bahasa Inggrisnya tidak lulus, jadi kan kami bertanya-tanya kenapa kok sampai dia tidak lulus. Kesalahannya di mana. Akan tetapi kalau memang biasanya anak kan ada yang tidak lulus, tapi dia memang kurang to kemampuannya jelas kurang itu ya. Perasaan ya cuma kasihan saja, tapi nggak begitu terus gimana.Ya memang karena kemampuan dia sejauh itu.

Tik-1-3-B-a pity for failure Tik-4-3-Bfailure of student debater Tik-4-3-Bwonder Tik-2-3-Bfailing students usually has a low ability

Tapi menurut ibu materi tes itu sendiri valid atau tidak? Nek menurut saya ya. Ada siswa yang aktif di debat tapi kok tidak berhasil di unas. Menurut saya ya saya kira yo sudah anu to sepantasnya sudah valid yang menilai karena ya saya sendiri pengalamannya kurang begitu banyak.

Tik-II-B-valid


Yang penting saya lihat e apa yang kita persiapan kita yang kita leskan dengan soal yang itu tidak meleset jauh, artinya kita sesuai dengan kisi-kisi yang di..Kalau anak ya memang dia kok kenapa dia seperti itu barangkali ya dia salah le mengarsir atau mungkin dia pas lagi kondisinya ndak baik, nggak fit, kan kita tidak tahu. Jadi bukan karena situasi atau fasilitas yang kurang baik itu ya. Jadi mungkin kesalahannya semacam kesalahan teknis siswa di dalam mengerjakan.

Tik-2-3-B-the failure may be due to technical problem and physical health.

Baiklah.

244 7.9. Teacher Nanik 2 Comment Interviewer: Researcher Interviewee: Teacher Nanik Setting: SMK N 1 Pengasih. 21 June 2007. 10.15 – 10.25 This teacher wants her students to have a feeling of success in doing it

Narrative

Code

Excuse, me. What do you think, what makes an ideal test? What are the criteria of a good English test? I think we can say the ideal test it means that er ideal test can measure can measure how far the students can master the material of the topic. And then criteria good English test, first I think it is simple. Second it means simple the students can the students are available to do the test well and then second, the test based on the material which is taught by the teacher. Do you think that your English test is already simple and then the students can also do the the test? Yes. I think so. I mean the simple test here that the test

Tik-III-A/B-should be simple:students can do it Tik-III-A/B-materials which have been taught


is easy to understand by students such as on vocab or maybe on the form of the sentence, so the students can do the test well. Is that all? Pardon the last question? My question is Is your own tes simple enough for your students to understand? Yes. I think if we give a test as simple as possible and it means that a simple as possible here simple it means the students is easy to understanding the vocab such as the vocab not e the vocabulary always used in everyday not difficult vocabulary. I think the students can do the test well. Yes, but my question is do you think that the test you have given to your students is simple or is it too difficult for them. Yes? Yes. Good, thank you. Now. So because students have different ability, do you make a different test for them or do you make the same question or the same test to them. The first test all of students the same. I give the same test but after that if maybe the result result test is bad, I give the different test. Oh, I see. For different program I mean for different department, say for example accounting students will receive different test from administration. Say for example like that.

(cont’d) Yes. I think on the different class such as accounting the students is clever than the other class so the accounting I give the test more difficult than the other class but may be on administration or any other we give e different test than maybe it is easier easier than the accounting class.

Tik-3-1-A/B-adjust the material with level of competency

Ok, thank you very much. What do you usually do in the testroom when you give a test to your students, what do you usually do, what are you doing while the students do the test? I usually wait and watch out the students do the test in the class until the students finish do the test.

Tik-3-2-A/B-waiting and watching

What are you doing if you see your students are talking to each other while doing a written test? E, I will call them not to discuss with the other or maybe if there is a student cheat cheating I think I will call them not to cheat. Why because e this time is a test. So I mean the test to measure how far you can master

Tik-3-2-A/B-asking students not to do cheating. Tik-I-A/B-measure


the material well.

students’ mastery

Did you happen to experience this? Er yes I did. It was happen that the time is test, so they discuss the test with the other than I call them to stop. I never to stop to do the test that I ever do that if after I call them then they stop not cheat or not discuss with the other then they continue the test. OK, thank you. Do you notice, I mean did you notice what is his or her reaction when you ask him or her to stop cheating. Were they afraid or shy or just relax? Yes I think that he is afraid he or she is afraid and also shy with the other students. Sometimes there is a student that relax. I think it is a usual event etcetera.

Tik-4-2-A/B-students feel afraid, shy, relax

Have you ever found that the testroom is quite noisy ya because your students felt any problem or difficulty in doing the test. Just remember try to remember I think I haven’t…On my class I think that after I explain about the instruction of the test, I think the student understand what must they do and then they can keep silent on the class. When you are in the classroom, I mean in the testroom, do you always walk around the class or just sit at your table or stand what may be you can explain. Yes. Sometimes walk around the class and then sometimes I only sit on the chair but I think it is not it is impossible if the teacher wait the test and a apa walk around the class always walk around the class because it can disturb the students’ concentration.

Tik-3-2-A/B-walk around the class for some time Tik-III-B-need condusive situation

(cont’d) OK. Did you ever find this experience? You gave a wrong score to your students? Say for example, your students actually got a seven but in the school report you gave only six or seven and so on? I give a wrong mark? Yes Then on report give mark 6 or 7? No, I mean did you give a wrong mark to your students. Have you ever found this? I think never. Oh, never. Never give a wrong mark to the students such as that on on report and on test it is different or no I never give a wrong mark to student.


OK, thank you. Now let’s talk about unas. What do you think of unas especially English unas, English test in unas or the national examination. OK. I think unas is important and valid to the students, but sometimes it makes the students afraid if they are not pass or may be they do not do the test. Why is it? Because English is a foreign language. Maybe they find out a difficult vocabulary and they cannot understand the meaning of the words, so it makes they have difficulty, so I think many students is doubt with their capability although he or she had studied for three year or maybe more than three year because I think English e had been studied on SMP or then SMK or SMU so I think not only three years they study English but more than more than three years been studied English. I think he feel e of he get he get a difficult if difficult I think he feel he feel he still he still feel difficulty about English.

Tik-II-B-valid Tik-I-B-important Tik-I-B-makes students afraid of failure

Is that all? OK, thank you. Now, my question is like this. You said that you never go around the class when watching your students doing the test. You never walk around the class all the time because you are afraid that it will disturb your students will disturb your students’ concentration. OK. What do you think the things make disturb then in in the unas test. Maybe my question is like this. E, you don’t want to disturb your students’ concentration. Why don’t want to disturb their concentration when doing the test? She wanted to establish good situation in test room

E, I think maybe if walk around the class the students will shy if we look their answer sheet, maybe their answer is wrong or any other and maybe it makes noise by because the teacher walking around the class or may be it can… that maybe the teacher will give the answer to the students or any other.

(cont’d) Now, my question is like this, actually. You don’t want to disturb your student’s concentration by walking around the class all the time. Why do you not want to disturb the student’s concentration? Concerned about achievement

Oh, oh, ya. I don’t disturb the student’s concentration because I think that all of my students can do the test. When the result of the test is good, may be they have good mark to the test. So, what happen if the student cannot concentrate on the test? What do you think?

She wanted the score really represented students’ ability

I think they will get bad mark or maybe they will o maybe they the time of the test is over but they don’t finish do their test. Thank you. Can you describe activities which may disturb your student’s concentration when doing the test any other than walking around the class.May be you can mention? Yes, for example if there is a student are some questions to the teacher about the answer of the test, or if there is


a student are the answer to the other students or maybe if there is someone knock the door, or may be wants to meet me or meet the students.

248 Appendix 8 News stories and information 8.1 Radio news Comment 17 April 2007 RRI Pro 3 07.00 In respond to the reported leakage among students, the official countered the news, instead of doing doublechecking and making clarification. This can be meant to make assurance on the test quality.

Data Pukul 7 WIB. Radio Republik Indonesia dengan warta berita dibacakan Eko Wahyuwanto. Sari berita: Warga […] Pelaksanaan ujian nasional tingkat SLTA yang berlangsung mulai pagi ini di wilayah Surakarta sempat diwarnai isu beredarnya info pesan singkat tentang bocoran kunci jawaban. Hal tersebut dilaporkan Wiwid Widaningsih. Info melalui pesan singkat tentang kunci jawaban soal ujian nasional yang telah beredar masyarakat seharusnya tidak perlu ditanggapi peserta orang tua murid karena informasi tersebut menyesatkan. Kepada wartawan Kepala Drs Ansori SH M.Pd mengatakan masalah yang mungkin timbul dalam penyelenggaran UN termasuk kebocoran materi ujian dan contekan jawaban melalui alat komunikasi modern telah diantisipasi penyelenggara. Dalam satu ruangan ujian yang digunakan 20 peserta memakai sistem nomor ganjil dan genap yang berbeda urutan soal sehingga tidak mungkin saling mencontek. Siswa juga tidak diperbolehkan membawa telepon genggam di dalam kelas, sementara pengawasan dilakukan 2 orang untuk masing-masing kelas menggunakan sistem silang murni dan tim pemantau independent dari perguruan tinggi juga dilibatkan 1 orang untuk satu sekolah. Selain itu menurut Ansori pihaknya juga meminta pengawas sekolah untuk mengedepankan kejujuran sehingga tidak melakukan pengatrolan kelulusan secara sistematis. “ Saya bahwa pendidikan harus menjadi pelopor karena dengan kejujuran kualitas pendidikan diharapkan”

Code


Soal ujian nasional telah tiba di kota Surakarta hari Minggu sore lalu dari propinsi jawa Tengah dengan dikawal petugas kepolisian. Selanjutnya didistribusikan ke tiga sub rayon baik SMA maupun SMK dijaga petugas khusus dan aparat polisi. Dan pagi menjelang ujian baru diambil oleh sekolah masing-masing. Di kota Surakarta ujian nasional diikuti 41 SMA dan 7 MA dengan 7212 siswa serta 42 SMK serta 6775 siswa. Masih soal ujian nasional. Meskipun bebas dari kemungkinan terjadinya kebocoran soal, pelaksanaan ujian nasional di Kabupaten natuna Provinsi Kepulauan Riau diperkirakan hanya mampu menghasilkan 55% tingkat kelulusan. Reporter Ali Arsyadi dari Ranai menyampaikan laporannya. Seluruh pelajar SMA dan sederajat di wilayah kabupaten Natuna mulai hari ini dipastikan mengikuti ujian nasional UN. Tidak seperti rumor yang berkembang di berbagai daerah, pelaksanaan ujian nasional di kabupaten yang berbasis kepulauan dinilai akan berjalan lancar tanpa kemungkinan terjadinya bocoran soal. Ketua Forum Komunikasi Guru Natuna Zaenal Abidin S.Ag kepada RRI menyatakan. Dari pengamatan FKGN pelaksanaan ujian nasional yang dimulai serempak hari ini secara nasional tersebut berjalan normal. “Saya mengamati selama ini belum ada indikasi ada kebocoran soal, belum ada indikasi ada rekayasa. Dengan telah dibentuknya tim pemantau independen ujian nasional menurut Zainal Syah kelulusan pelaksanaan ujian tahun ini murni dari hasil belajar siswa dengan tingkat kelulusan sekitar 85% lebih tinggi dari prediksi semula yang hanya 55%. “Artinya ada kesempatan ujian nasional yang dinyatakan tidak lulus Ketua FKGN Zainal Abidin berharap dengan pelaksanaan ujian nasional yang berlangsung objektif para peserta ujian nasional justru mampu menunjukkan hasil prestasi belajar yang terbaik dan dinyatakan lulus sesuai standar kelulusan 4.5.

249 8.2 Television news 8.2.1 Television news 1 Comment Thursday, 19 April 2007. 16.00.Lintas Peristiwa, TPI. 83 Vocational school students in Padang boycott National Exam It indicates that external factor brings certain effect on English testing implementation, and in turn brings negative effects on their own learning and may influence another’s learning

Data

Code

Ujian Akhir Nasional di Padang menolak mengikuti ujian nasional hari ke tiga. Para siswa memboikot ujian nasional karena menduga terjadi kebocoran soal ujian. 83 siswa SMK Dhuafa yang mengikuti ujian nasional di SMK 2 Padang memboikot pelaksanaan ujian hari ke tiga. Para siswa menuduh terjadi kebocoran soal ujian. Menurut mereka peserta ujian nasional dari SMK 5 menerima jawaban soal. Namun Kepala Sekolah SMK 5 Yefrison membantah adanya kebocoran soal. Menurut Yefrison yang membuat kegaduhan saat ujian justru guru dan siswa SMK Dhuafa. Dugaan kebocoran soal ujian nasional dan kecurangan di SMK 5 Padang kini dalam penyelidikan polisi bersama Dinas Pendidikan Provinsi Sumatra Barat. Demikian Lintas Peristiwa. Laporan selengkapnya mengenai ujian nasional akan mengisi Lintas 5 satu jam dari sekarang. Terima kasih atas perhatian anda. Sampai jumpa.

Sgr-3-2-Brefusing to do the test Sgr-2-1-B-the test implementation is not reliable Sgr-3-2-Baccusing for leakage

Sgr-4-2-B-chaos Sgr-III-Breliability assurance


5.2.2 Television news 2 Comment Thursday, 19 April 2007. 17.00.Lintas Lima, TPI. Anxiety about low graduation rate may cause this to happen. Validity and reliability come next.

Data

Code

Hari terakhir ujian nasional tingkat SMA, pelanggaran terjadi di Kupang NTT. Sejumlah siswa SMK Negeri 3 mengaku mendapat jawaban soal ujian dari guru Bahasa Inggris. Tapi tim pengawas independen membantah pelanggaran tersebut. Hari terakhir pelaksanaan ujian nasional ujian nasiona di Kupang diwarnai pelanggaran. Sejumlah siswa SMKN 3 mengakui mengisi jawaban sesuai arahan dua guru Bahasa Inggris. Menurut siswa, guru tersebut lebih dulu mengerjakan soal Bahasa Inggris, kemudian jawabannya dibagikan kepada para siswa. “Siapa yang kasih tahu?” “Ada pengawas 2 orang.” “Pengawas siapa?” “Entahlah ada satu dua rambut pendek pakai baju kuning.” “Kasih tahu kenapa?” “Tidak tahu” “Nomer 1 sampai 50 dikasih tahu jawabannya semua.” “Soal apa itu?” “Bahasa Inggris” “Bahasa Inggris. Semua?” “Ya.” “Dan yang paling menjengkelkan itu ketika orang itu kasih tahu semua jawabannya yaitu betul” Namun, tim pengawas independent Manopo membantah pengawas memberi kunci jawaban. “ dulu itu semua.” “Tidak ada kebocoran” “Tidak ada kebocoran.” “Menurut pengakuan siswa bahwa mereka diberitahu oleh guru pengawas untuk mata pelajaran tertentu ada 50 nomor, Pak Yang dibagi?” “Betul. Nanti ada orang khusus ya yang akan memonitor. Tetapi kalau ada pengakuan nanti itu” Usai ujian, para siswa dikumpulkan di lapangan. Mereka dilarang memberikan keterangan kepada wartawan. Kontributor TPI Kupang NTT melaporkan.

Sgr-3-3-Badmitting cheating Sgr-3-2-Breceiving answerkeys from teachers Sgr-3-3-Badmitting to have got teacher’s directions in doing the test Sgr-3-3-B telling that teachers did the test and distributed the answers to testtakers Sgr-4-3-Binstructed to attend assembly Sgr-4-3-B-not allowed to tell anything to reporters

250 8.2.3 Television news 3 Comment Thursday, 19 April 2007. 17.00. Lintas Lima, TPI. They did not need the test since they believed the test meant nothing in terms of competitiveness among the group.

Data Sementara di Padang, siswa SMK Dhuafa memboikot ujian nasional karena menduga terjadi kebocoran soal. Menteri Pendidikan Nasional menegaskan, kebocoran soal akan dipidanakan. Hanya ada 2 pengawas dalam ruang ujian mata pelajaran Bahasa Inggris di SMK 5 Padang. Sejak kemarin 83 siswa SMK Dhuafa yang semestinya mengikuti Ujian Nasional di sini memboikot ujian nasional sebagai protes dugaan kebocoran soal di kalangan siswa SMK 5. “Saya tidak pernah merasa rugi. Karena apa yang kami lakukan saat ini saya rasa ini benar. Kami ingin mendapatkan keadilan. Ini ujian Uan. Ujian Uan ini ujian nasional. Kenapa terjadi kebocoran seperti ini? Dan kenapa guru-gurunya nekat? Mungkin di mushala dekat WC pun.” Kepala SMK 5 Padang Yefrison menyangkal tuduhan itu. Menurut Yefrison yang membuat kegaduhan saat ujian nasional justru guru dan siswa SMK Dhuafa. “Kenapa seakan… Kenapa guru kami tidak dilibatkan panitia Kenapa guru kami tidak boleh Kenapa Kepala Sekolah kami. Menteri Pendidikan Nasional Bambang Sudibyo menyesalkan pemboikotan ujian nasional karena merugikan siswa. Menurut Mendiknas, pelaku

Code

Sgr-1-3-B-no regret for not taking the test Fairness


pembocoran ujian nasional akan dikenai sanksi pidana. “Bisa kerjasama erat. Kalau peraturannnya adalah peraturan pelanggaran” Dugaan kebocoran soal ujian nasional dan kecurangan di SMK 5 Padang kini dalam penyelidikan polisi bersama Dinas pendidikan Provinsi Sumatera Barat. Kontributor TPI Padang dan Jakarta.

8.2.4 Television news 4 Comment

Failure leads to decision making which leads to autonomous learning ( learning facilitation from failure)

Data Ujian nasional tahun ini menjadi hari menegangkan bagi puluhan SMK Fatahillah Sawangan Depok Jawa Barat. Tahun lalu hanya satu siswa SMK Fatahillah yang lulus ujian nasional. 32 dari 67 siswa yang mengikuti ujian di sekolah ini adalah siswa yang gagal dalam ujian nasional tahun lalu. Seorang siswa yang gagal tahun lalu mengatakan lebih memilih mengulang ujian nasional tahun ini dari pada mengikuti ujian persamaan tahun lalu. “Ya saya sayang sekolah 3 tahun. Ikut lagi sih kaya gimane kagak tertera mengulang setahun lagi deh dan paket C ya? Insya Allah mudah-mudahan lulus.” Salah satu siswa kelas 3 SMK Fatahillah ini mengaku tidak berniat pindah untuk sekolah lain meski sekolah ini gagal total meluluskan siswanya. “Nggak, Pak” “Kenapa?” “Ya karena udah terlanjur aja Pak. Udah terlanjur kelas tiga kan?” 6 bulan menjelang ujian, para siswa kelas 3 dan siswa yang gagal ujian tahun lalu mendapatkan pengayaan materi pelajaran Bahasa Indonesia, Bahasa Inggris dan Matematika. Karena itu Kepala Sekolah SMK Fatahillah Saparno optimis angka kelulusan tahun ini jauh lebih baik dari tahun lalu. Hamdan Maliki, Herman Kas, TPI Depok, Jawa Barat melaporkan.

Code

Sgr-3-3-B-take a make-up exam

Sgr-1-3-B-optimistic

Sgr-4-1-B-getting material enrichment 6 months before test implementation

251 8.2.5 Television news 5 Comment Thursday, 19 April 2007. 17.00. Lintas Lima, TPI.

Data “Anak kami secara umum siswa SMK nasional. Karena apa? Karena SMK itu sudah dituntut untuk kompetensinya harus baik. Ada lagi tes produktif, sehingga kalau nilai ujian nasional masih batasan seperti itu jelas itu masih” SMK PGRI 4 adalah sekolah dengan status sekolah nasional dan reputasi pun sudah diakui sebagai sekolah favorit di Ngawi. Menurut Kepala Dinas pendidikan Kabupaten Ngawi Abdullah Zaini tidak seharusnya sekolah ini merasa keberatan dengan standar yang ditetapkan pemerintah. Karena itu sudah diukur, dan menaikkan nilai itu tidak menaikkan. Sudah diperhitungkan secara rinci, secara njlimet istilahnya. Jadi bisakah anak-anak kita. Dan diharapkan memang setiap tahun ada kenaikan. Ha ini. Ha kemungkinan kekhawatiran kepala sekolah semacam itu mestinya tidak boleh terjadi.” Lalu siapakah yang sebenarnya bersalah dalam hal ini? Sayangnya siswa siswi SMK PGRI 4 yang berhadapan langsung dengan sistem pengajaran dan soal-soal ujian enggan menyampaikan pendapat dan perasan mereka. “Sulit nggak?” “Lumayan” “Ya” “Lumayan sulit” “Lumayan sulit?” “Sulit nggak mbak?” Standar kelulusan siswa memang sudah ditetapkan oleh pemerintah, bahkan dinaikkan dari 4.25 tahun lalu menjadi 5.00 pada tahun ini. Namun kasus Ma’mun Effendi

Code

Sgr-3-3-B-reluctant express feelings Sgr-4-2-B- finding rather difficult

to test


menunjukkan bahwa keputusan ini masih menimbulkan sejumlah permasalahan kalangan pendidikan dan siswa berharap pemerintah solusi konkrit supaya tahun mendatang tidak ada lagi masalah yang menyertai pelaksanaan ujian akhir nasional. Dari Ngawi, Arif Wahyu Effendi melaporkan.

252 8.2.6. Television news 6 Comment Thursday, 19 April 2007. 18.00. Liputan 6 Petang. SCTV

Data … dan mengancam guru pengawas ujian nasional di hari terakhir ujian nasional. Mereka kesal, karena guru terlalu ketat ujian berlangsung. Lain lagi di Padang. Sejumlah siswa SMK Dhuafa memboikot Ujian Nasional dan akibatnya mereka terancam tidak lulus. [Suara jeritan, histeria massa ] Para siswa yang tengah diliputi amarah ini melemparkan apa saja ke bangunan sekolah. Mereka pun memecahkan kaca-kaca jendela sekolah. Tidak berhenti sampai di situ, para siswa juga mencari guru pengawas ujian nasional Sri Handayani yang bersembunyi di toilet. Situasi sang guru menenangkan para siswa. Ulah para siswa ini dipicu lantaran guru Sri Handayani dinilai mengawasi secara berlebihan saat pelaksanaan ujian nasional. Di kota Padang ruangan tempat pelaksanaaan ujian nasional pelajar SMK Dhuafa yang berlokasi di SMK 5 Padang kosong. Hanya seorang siswa yang terlihat mengerjakan ujian. Boikot ini adalah kelanjutan dari Rabu kemarin ketika para siswa SMK Dhuafa keluar dari ruangan ujian memprotes tindakan pengawas yang memberi kan jawaban soal kepada sejumlah murid SMK N 5. Hari ini mereka mendatangi Kantor Pendidikan Nasional Provinsi Sumatera Barat. “Ada kemungkinan ujian susulan, Bu? “Dalam peraturan guru-guru ini apakah kepala sekolahnya. Kita pelajari dulu. Data ini baru kita

Code

Sgr-1-2-B


himpun dan kita kumpulkan.” Dinas pendidikan Sumatera Barat belum menentukan langkah atas laporan kecurangan guru pengawas seperti yang dilaporkan siswa. Meski demikian, karena telah meninggalkan bangku ujian Diknas Kota Padang sudah memutuskan tidak ada ujian susulan bagi mereka. Tim liputan 6 melaporkan.

8.2.7 Television news 7 Comment Thursday, 19 2007. 19.00. Headline news, Metro TV.

Data April

… merusak sekolah karena merasa diawasi secara berlebihan saat ujian berlangsung. Hallo, selamat malam, pemirsa. Inilah headline news pukul 19.00 WIB bersama Meutya Hafidz. Pemirsa. Para siswa SMK Dhuafa Nusantara Kota Padang mendatangi Kantor Dinas Pendidikan Provinsi Sumatera Barat. Mereka membeberkan terjadinya kebocoran soal ujian saat pelaksanaan ujian nasional di SMK 5 tempat mereka mengikuti ujian. Kedatangan 83 siswa SMK Dhuafa Nusantara setelah 2 hari memboikot ujian nasional di SMK 5 Jalan Beringin, Kecamatan Padang Utara Kota Padang hari Rabu dan kamis. Kepada wakil Kepala Dinas Pendidikan Sumatera Barat Jefrial, mereka membeberkan seluruh kecurangan di lokasi mereka mengikuti ujian nasional. Diantaranya para pengawas dan guru SMK 5 memberikan jawaban kepada siswanya. Pembeberan itu diwarnai cucuran air mata para siswa. Namun Jefrial belum dapat memberi jawaban apakah mereka akan ikut ujian nasional susulan atau tidak. Untuk sementara waktu, kasus tersebut tengah dipelajari tim independentyang melibatkan perguruan tinggi negeri dan swasta di Sumatera Barat.

Code

Sgr-3-3-B-meeting official of Education Office Sgr-3-3-B-exposing and pointing out leakage

Sgr-3-3-B-pointing out cheating Sgr-3-3-B-showing deep sorrow


Data

Friday, 20 April 2007. 07.00. Metro TV.

He tried to control the situation

8.2.9 Televised news Comment Friday, 20 April 2007. 07.00. Investigasi. Trans TV

Code

. .Exam in East java ended like chaos. Students felt Teachers went overboard in monitoring the exam. One student reportedly said a teacher his class because his suspected student was cheating Other students and the teacher and after the exam windows of the neighboring And we move on Vice President Jusuf Kalla is denying a report of cheating another irregularity in the high school final exam in Jakarta. Kalla is hopeful that at least 92% of the students will pass the test, adding that students are better prepared than ever because there is no make-up test this year. Kalla was responding to reports of leak exam answer keys from school officials and test messages particularly in West Java.

Data

Pemberi penilaian akhir bukanlah guru sekolah, melainkan pihak lain, yaitu Depdiknas. Bisa jadi, sistem pendidikan nasional yang berlaku sekarang tidak menghargai jerih payah pendidikan di sekolah, artinya jerih payah 3 tahun ditentukan hanya dalam waktu 120 menit saja. 120 menit yang bisa jadi membuat siswa bahkan orang tua yang akalnya pendek akan gelap mata,

Code


Making the test considered significantly important.

Concerned about achievement and graduation. It might also be considered as an effort to liberate the students from ‘colonizing’ tests

“..Katanya pengambilan naskah di Diknas. Itu katanya ya, pengambilannya nggak mau dikawal.” “Nggak mau dikawal polisi?” “Itu katanya” “Dia bawa sendiri dengan mobil?” “Ya” Ujian nasional dilaksanakan untuk memperoleh gambaran pelaksanaan pendidikan sesuai standar nasional. Suatu keinginan untuk meningkatkan kualitas pendidikan di negeri ini. Namun sayang, cita-cita tersebut boleh jadi tak terwujud dengan beragam kecurangan yang ada. Tak hanya siswa, oknum pendidik pun terlibat di dalamnya. Seperti terjadi di Ngawi, Jawa Timur. Oknum Kepala Sekolah sebuah SMK ini nekad mencuri naskah ujian akhir nasional. Berharap agar anak didiknya bisa mendapatkan nilai bagus pada ujian nasional dijadikan oknum kepala sekolah ini dalam beraksi. Naskah yang dicuri pelaku masing-masih mata pelajaran Bahasa Indonesia dan satu buah kaset Bahasa Inggris. Kedua barang sangat rahasia tersebut dicuri dari kantor Diknas. Kesempatan mengambil barang sangat rahasia itu terbuka saat pelaku mengambil soal-soal tersbut di Kantor Diknas. Keanehan sebetulnya mulai kelihatan saat ia menolak dikawal petugas. Rupanya kesempatan itu digunakan pelaku untuk mencuri naskah ujian yang dibawanya. Namun aksinya gagal. Pasalnya petugas mulai curiga dengan gerak geriknya. “Katanya pengam dia ternyata dia mengambil apa jenenge naskah Bahasa Indonesia sama kepingan CD. CD Bahasa Inggris. Oknum kepala sekolah ini pun langsung diamankan di kantor polisi untuk diperiksa. Namun dia beruntung karena akhirnya polisi tak menahannya. Kasus ini pun masih ditangguhkan menunggu rampungnya ujian akhir nasional karena pelaku adalah panitia pelaksanaan ujian nasional di sekolahnya. “Murni bahwa kami kan bertanggungjawab secara moral bahwa anak sudah menyelesaikan pendidikan 3 tahun berusaha mengantarkan mereka sampai lulus. Itu yang terpenting. Jadi e benar-benar itu sukarelawan dari guru. Guru-guru yang ingin untuk supaya anak kita semua lulus. Itu karena standar kelulusan kan sangat tinggi.” Ujian akhir nasional UAN untuk para siswa SLTP maupun SLTA telah berakhir. Tentu saja para siswa berharap bisa lulus dengan nilai yang memenuhi syarat kelulusan. Namun syarat nilai kelulusan minimum sebesar 4,26 dianggap cukup memberatkan bagi sebagian siswa. Ini untuk mata pelajaran pokok, yaitu Matematika, Bahasa Inggris dan bahasa Indonesia. Jika tidak lulus berarti usaha siswa dan orangtua mereka selama 3 tahun akan sia-sia. Rupanya standar nilai kelulusan yang dianggap terlalu tinggi ini memberi inspirasi bagi sekolah tertentu untuk membantu para siswanya agar lulus ujian. Sekolah ini bahkan membentuk Tim Sukses agar semua siswa mereka bisa lulus. “Tim sukses itu terdiri dari guru-guru atas dasar instruksi dari jajaran kepala sekolah, ada kepala sekolah, wakil kepala sekolah. Kemudian juga inisiatif guru-guru juga.” Pagi hari sebelum ujian dimulai, lembar soal sudah diambil pihak sekolah Totok. Soal-soal ini dalam bungkusan tersegel. Ini untuk membuktikan bahwa soal tidak bocor. Soal ujian kemudian diserahkan kepada pengawas ujian yang terdiri dari sekolah lain dan pengawas independen. Dalam setiap bundel soal selalu ada sisa 1 soal untuk cadangan. Cadangan inilah yang difotokopi, kemudian dikerjakan oleh para guru tim sukses kelulusan siswa. Saat ujian selesai, lembar jawab siswa dikumpulkan. Setelah dirasa aman dan para siswa pulang, tim sukses ini mulai bekerja. Mereka menghapus jawaban soal yang salah dan menggantinya dengan jawaban yang benar. Mereka mengubah jawaban hingga tiap siswa memperoleh nilai yang memenuhi standar kelulusan. “Sama sekali tidak ada unsur uang di sana murni bahwa kami bertanggungjawab seara moral bahwa anak telah menyelesaikan pendidikan 3 tahu berusaha mengantarkan mereka sampai lulus. Itu yang terpenting. Jadi benar-benar itu sukarelawan dari guru.” Tim sukses kelulusan siswa ini hanya membutuhkan waktu sekitar 40 menit untuk memanipulasi lembar jawaban ujian. Ini karena lembar jawaban harus segera dikembalikan ke Rayon Wilayah masing-masing sekolah. “Sebelum diantar ke Rayon, karena yang mengantar ke rayon adalah panitia sekolah, sebelum ke rayon dibelokin dulu oleh panitia


tersebut, si pengantar soal. Entah di lingkungan sekolah entah di luar sekolah, yang penting dalam waktu sekin menit itu bagaimana mengecek jawaban siswa, berapa salah berapa kurang berapa, agar nilai minimalnya tercapai sehingga kita harus mbenarin, mbenahin dulu. Harus ditambah berapa ini anak biar lulus.” Kami juga menemukan kecurangan ini di SLTP pinggiran Jakarta. Ardi seorang siswa mengaku jika diberikan jawaban soal tanpa membayar. “Suruh datang itu sebelum pengawas datang. Itu mbagiin jawabannya.” Jumlah jawaban yang diberikan guru mereka kepada Ardi dan teman-temannya hampir 50% dari jumlah soal. Pihak Depdiknas sendiri tidak yakin akan timbul kecurangan yang dilakukan antara guru dan murid ini. “Itu rumor, kemudian dipublikasikan, kemudian diblow up. Ha itu yang terjadi kan kita. Orang bilang ada joki ada tim sukses. Siapa yang anak-anak kelompok anak-anak itu yang dikerjakan. Nggak ada juga.” Suyanto juga memastikan bahwa dengan standar nilai yang diberikan saat ini tidak mempengaruhi tingkat kelulusan siswa. Atas dasar ketidaklulusan tahun kemarin, maka soal-soal yang tahun ini sudah diujicobakan ke daerah-daerah sekolah yang merupakan kantong yang merupakan daerah wilayah di mana anak-anak yang tidak lulus itu banyak.” Guru selayaknya memberi contoh yang baik kepada muridnya, seperti kata pepatah “Guru kencing berdiri, murid kencing berlari” Aksi tidak terpuji oknum guru dalam ujian akhir nasional bukan tanpa alasan. “Gak bolehlah kalau menurut saya kemudian dipukul rata hanya ditentukan 3 bidang studi. Nah saya sangat setuju kalau sistem ujian akhir nasional itu tidak lagi menentukan kelulusan.” Mereka berbuat curang karena memang ada tuntutan. Namun dengan alasan apa pun tindakan yang mengotori dunia pendidikan ini seharusnya tidak terjadi. “Sebaiknya sih, menurut saya nggak usah ada ujian ya, karena e buat apa ada ujian tapi setiap sekolah nggak punya apa nilai murni.” Guru atau sistem pendidikan nasional kita yang perlu dibenahi?


Data

Friday, 20 April 2007. 08.00. Good Morning. Trans TV.

…Malaya, Jawa Barat berunjuk rasa di sekolahnya saat ujian nasional berlangsung. Mereka marah dan memprotes tindakan para pengawas yang mengawasi dan mengambil lembar jawaban sebelum waktu ujian selesai. Reportase selengkapnya setelah jeda berikut. Tetaplah bersama kami di Good Morning, Good morning. Terima kasih masih bersama kami di sini. Gimana ujian akhir nasionalnya untuk anda yang mengikuti ujian akhir nasional? Nah. Hari terakhir nih, Tik. [...] Ratusan siswa SMK N 2 Tasik Malaya berunjuk rasa di sekolahnya saat ujian nasional berlangsung. Mereka marah dan memprotes tindakan para pengawas yang mengambil lembar jawaban sebelum waktu ujian habis. Dah, dah, dah, dah. Rebut-rebutan nih. Ujian juga rebut. Selain membakar spanduk, mereka sempat para pengawas di sekolah. Wah. Disandera juga nih. Hmm. Ratusan siswa SMK N 2 Tasik Malaya Jawa Barat ini marah karena merasa diperlakukan tidak adil oleh pihak pengawas ujian nasional. Mereka pun lalu berunjuk rasa di depan gedung sekolahnya Kamis siang. Mereka memprotes tindakan pihak pengawas ujian akhir nasional yang mengambil lembar jawaban para siswa sebelum waktu ujian berakhir. Menurut para siswa, para pengawas telah mengambil lembar jawaban para siswa padahal waktu ujian tinggal 5 menit lagi. Akibatnya mereka memblokir gerbang dan membakar spanduk sekolah, bahkan para pengawas tidak diperbolehkan keluar dari wilayah sekolah mereka. “Mereka tidak memberi kebijaksanaan waktu e apa adanya saja. Yang baru diisi 10 diisi 10, yang baru 15, 15 saja. Udah dikumpulkan.” “Ya, tapi waktunya sudah selesai kan?” “E memang waktunya sudah selesai, tapi kan kami minta kebijaksanaannya.” Beruntung pihak Dinas pendidikan dan Kepolisian Kota Tasik Malaya segera datang dan memberi penjelasan kepada para siswa.

It might mean that allocation of time prevented them to maximize their achievement. Students might feel they were able to finish the test, and

Code Sgr-1-3-Birritated


might think the score would not represent their competencies.

“Saya kira tidak mungkin terjadi. Karena kita sudah punya penetapnya bahwa dari jam 8 sampai 10 gitu. Kalau jam 10 selesai dan itulah waktu yang terakhir. Jadi tidak mungkin pengawas menarik sebelum waktunya selesai.” Pengambilan lembar jawaban yuang dinilai siswa sebelum waktunya tersebut dikhawatirkan akan mengurangi nilai ujian mereka, hingga lebih kecil dari standar kelulusan yaitu sebesar 4,25. Budi Pardiana, Tasik Malaya Jawa Barat. Ya udah dikerjain 10, diambil 10. Kerjain 15 diambil 15. Kasih itu injury time kan skornya. Kaya main sepak bola pertambahan waktu. Ada nggak sih, pertambahan waktu? Hehehe perpanjangan. Perpanjangan waktu. Aduh, yang namanya ujian nasional di mana-mana selalu bikin heboh dan ada saja berita mengenai itu. Tahun lalu, tahun ini. Na, sementara itu ada yang luar biasa nih. Pelaksanan ujian nasional di kota Banjarmasin Kalimantan Selatan justru berlangsung lancer. Namun di sejumlah sekolah diduga juga terjadi kecurangan ini, Fer. Beberapa siswa mengaku mendapat bocoran jawaban melalui SMS. Oh. Serba jaman canggih ya? Iya! Jawaban juga lewat SMS… Emang boleh, ujian bawa handphone, gitu? Harusnya handphonenya diambil dulu ya kan? … Namun pada hari terakhir ada dugaan terjadi kebocoran soal ujian. Di sejumlah sekolah ditemukan adanya siswa peserta ujian yang menggunakan telepon genggam. Bahkan beberapa siswa mengaku mendapatkan jawaban melalui pesan singkat atau SMS. Bocoran jawaban melalui SMS ini antara lain untuk mata ujian Bahasa Inggris yang dinilai sebagian siswa sebagai mata ujian yang cukup sulit. “ Ya kalau benar ya. Udah menyebar yak e teman-teman yang lain? Orangorang nya aja yang tahu. Sendiri tahu nggak? Tahunya dari teman. Dikasih teman? Dikasih. Berarti bisa mencocokkan dong. Bisa.” {...} Nani Hayati, Syaifudin Arjasa, banjarmasin, Kalimantan Selatan.

Sgr-1-2-B-angry with supervisor Sgr-3-3-Bdemonstration

Ya itulah ujian nasional. Jaman kamu dulu. Jaman dulu aku nggak ada Ponsel belum ada. Bocor nggak? Nggak bias bocor karena nggak ada ponsel kan. Jadi pakai pager. Tut tut bunyi terus. Bunyi berisik. Tolong kirimnya dua kali ya, Hahaa..

256 8.3. Printed News 8.3.1 Printed News 1 Comment

Data

Kedaulatan Rakyat, 17 April 2007

HARI INI, DIIKUTI 5 JUTA SISWA Unas Bukan Satu-satunya Ukuran Lulus

It might mean competition among students, teachers, schools, provinces, officials and other stakeholders

Jakarta (KR) – Sebanyak 2,5 juta lebih siswa tingkat SLTA bakal mengikuti Ujian Nasional (Unas) yang akan dimulai secara serentak Selasa (17/4) di seluruh Indonesia. Unas tahun ini menghabiskan dana RP244 miliar. Ketua Badan Standar Nasional Pendidikan (BNSP) M Yunan Yusuf di Jakarta Senin (16/4) mengatakan, koordinasi antara pusat dengan daerah sudah pada tahap akhir dan sejauh ini semua berjalan dengan lancar. Jumlah peserta Unas 2007 diperkirakan sebanyak 2,5 juta lebih siswa SMA/MA, SMA Luar Biasa dan SMK dan 2,2 juta lebih siswa SMP/MTs dan SMP Luar Biasa. Jumlah itu tersebar di 53 ribu sekolah di 33 propinsi. Totalnya menjadi 4,7 lebih siswa SLTA dan SLTP yang akan mengikuti Unas pada April ini. Prinsipnya semua sudah siap, baik panitia penyelenggara, pihak percetakan, tim pengawas maupun pihak keamanan. Alhamdulillah semua berjalan lancar, kata M Yunan Yusuf. Dia mengatakan master naskah soal sudah berada di semua propinsi dan sebagian di antaranya kini mulai pada tahap pencetakan. Terutama untuk daerah terpencil yang sekolahnya sulit dijangkau. Hanya saja, pengiriman ke sekolah baru akan dilakukan beberapa jam menjelang Unas berlangsung untuk menghindari terjadinya kebocoran soal. Menurutnya, penyelenggaraan Unas tahun ini memiliki kekhasan dibanding Unas

Code


tahun sebelumnya. Disamping adanya dua paket soal yang berbeda dalam satu kelas BSNP juga menerjunkan tim pengawas independent yang diambil dari perguruan tinggi setempat. Masing-masing sekolah akan diawasi satu orang pengawas independent dan pada tingkat kabupaten minimal tiga orang independen. Unas juga bukan menjadi satu-satunya penentu kelulusan. Akan tetapi, menjadi salah satu penentu kelulusan, di samping lulus dengan baik Ujian Sekolah, mengikuti seluruh proses pembelajaran di sekolah, baik dalam nilai pendidikan agama dan moral, dan lulus dengan baik dalam ujian akhir per mata pelajaran yang tercermin dalam raport tahunannya.

257 8.3.2 Printed news 2 Comment Kompas, 18 April 2007 Editorial It built public opinion. It contributed to establish seriousness among the test stakeholders

Data Sesudah Ujian Nasional Ujian nasional atau UN hingga tahun 2007 ini masih jadi polemik, Namun, UN jalan terus. Pada 2008 akan diterapkan di tingkat sekolah dasar. Tanggal 17-19 April 2007 UN diselenggarakan untuk tingkat SMA dan sederajat, 24-26 April untuk SMP dan sederajat. Kedua UN itu diperkirakan diikuti sekitar 5 juta siswa, di antaranya sekitar 2 juta siswa tingkat lanjutan atas yang seluruhnya tersebar di 53.000 sekolah. Nilai rata-rata standar kelulusan beberapa mata pelajaran yang di-UN-kan naik dari 4,5 tahun lalu menjadi 5,0. Dengan standar 4,5, baik untuk peserta tingkat menengah atas maupun menengah pertama, persentasi kelulusan UN tercapai di atas 92 persen. Kenyataan itu menjadi dasar menaikkan standar nilai kelulusan dengan asumsi timbal balik antara persentase kelulusan dan tingkat kesulitan soal. Arahnya secara bertahap mutu hasil didik diperbaiki. Hasil UN menjadi tolok ukur prestasi sekolah. Yang terlihat kemudian, dari UN SMA tahun 2006, peringkat pertama jurusan IPA dan IPS dicapai SMA Negeri I Bangil, Jawa Timur. Untuk IPA, misalnya, nilai rata-rata Bahasa Indonesia 9,46, Bahasa Inggris 9,71, Matematika 9,71. Stigma bahwa sekolah-sekolah berprestasi selalu berada di kota-kota besar runtuh . Apa artinya? Terlepas dari kiat-kiat lolos dengan memacu mata pelajaran yang di-UN-kan, UN bisa menjadi salah satu cara menaikkan mutu. Mengujikan beberapa mata pelajaran secara nasional sesuai pula dengan prinsip perlunya standardisasi minimum. Yang di-UN-kan adalah mata-mata

Code


pelajaran yang memberi motivasi belajar (incentive learning) sesuai dengan jurusan setiap peserta Yang perlu dilakukan adalah perbaikan tata cara pelaksanaan UN. Tidak hanya masalah kebocoran soal yang sebaiknya jadi urusan polisi, pelaksanaan UN dan proses penilaian, tetapi juga kesamaan pendapat tentang makna UN sebagai salah satu tolok ukur hasil praksis pendidikan. Benar UN bukan satu-satunya alat ukur. Benar UN masih memerlukan perbaikan sana-sini, selain proses pelaksanaannya juga perbaikan sarana infrastruktur. Namun, dengan prinsip proses belajar harus dievaluasi dengan semangat incentive learning dan diperoleh hasil minimal yang standar, UN rasanya adalah pilihan terbaik kita saat ini. Yang perlu adalah bagaimana setelah UN menjadi keputusan politis, salah satu cara menaikkan mutu perlu dibarengi dengan perbaikan sarana. Sarana pendidikan meliputi gedung, buku, dan guru. Semangat make the best sesudah (pasca) UN hendaknya jadi pintu masuk bersama. Semangat “sesudah UN” tidak lagi memolemikkan UN demi kepentingan politis, melainkan demi mempersiapkan masa depan generasi penerus bangsa ini. Tanpa mengingkari semangat pedagogis anak didik sebagai pusat orientasi, kita pusatkan perhatian pada perbaikan sarana pendidikan, diantaranya realisasi anggaran pendidikan dalam APBN minimal 20 persen

258 Appendix 9 Documents 9.1 Official document 1 DEPARTEMEN PENDIDIKAN NASIONAL

DIREKTORAT JENDERAL MANAJEMEN PENDIDIKAN DASAR DAN MENENGAH Jl. Jenderal Sudirman, Senayan, Jakarta 10270, Gedung E Lantai 12-13 Telp. 5725477 (hunting) 5725466-69, 5725471-74 Fax : (021) 5725469, Website : www. dikmenjur.net SURAT KEPUTUSAN DIREKTUR PEMBINAAN SEKOLAH MENENGAH KEJURUAN NOMOR: 0012a/C5.5/Kep/KP/2007 TENTANG PENERIMA SUBSIDI SERTIFIKASI BAHASA INGGRIS TAHUN 2007

Menimbang

Mengingat

:

:

1.

Bahwa salah satu program Direktorat Pembinaan Sekolah Menengah Kejuruan pada tahun anggaran 2007 adalah Program Sertifikasi Bahasa Inggris Standar TOEIC (Test of English For International Communication)

2.

Bahwa kemampuan berkomunikasi dalam Bahasa Inggris merupakan salah satu persyaratan yang harus dimiliki tamatan SMK agar mampu bersaing untuk memperoleh lapangan kerja di Dunia Usaha/Industri dalam negeri maupun luar negeri.

1.

Nota Kesepakatan (Memorandum of understanding) Direktur Jenderal Manajemen Pendidikan Dasar dan Menengah dengan Chief Executive Officer PT International Test Center tentang Penyelenggaraan Sertifikasi Bahasa Inggris, Nomor 3380/C5/LN/2006, tanggal 28 Nopember 2006.

2. Daftar Isian Pelaksanaan Anggaran (DIPA) Direktorat Jenderal Manajemen Pendidikan Dasar dan Menengah tahun Anggaran 2007 Nomor: 0111.0/023.03.0/-/2007, tanggal 31 Desember 2006. 3.

Usulan Kepala SMK tentang permohonan Subsidi Sertifikasi Bahasa Inggris Standar TOEIC bagi siswa SMK.


MEMUTUSKAN MENETAPKAN : Memberikan Subsidi keikutsertaan siswa dalam sertifikasi Bahasa Inggris Standar TOEIC bagi sejumlah siswa sebagaimana daftar terlampir.

Pertama

:

Kedua

:

Nilai subsidi keikutsertaan siswa dalam sertifikasi TOEIC adalah sebesar Rp. 165.000,- (seratus enam puluh lima ribu rupiah) untuk setiap siswa.

Ketiga

:

Semua pembiayaan sebagai akibat dari keputusan ini dibebankan pada DIPA Kegiatan Peningkatan Mutu Layanan Kesiswaan, Direktorat Pembinaan SMK, Direktorat Jenderal Manajemen Pendidikan Dasar dan Menengah Tahun Anggaran 2007 Nomor: 0111.0/023.03.0/-/2007, Tanggal 31 Desember 2006.

Keenam

:

Hal-hal yang belum diatur dalam keputusan ini akan ditetapkan dalam keputusan tersendiri dengan catatan bahwa bila dikemudian hari ternyata ada kekeliruan dalam penetapan ini akan diadakan perubahan/perbaikan sebagaimana mestinya.

Ketujuh

:

Keputusan ini berlaku sejak ditetapkan sampai berakhirnya seluruh pekerjaan. Ditetapkan di : Jakarta Tanggal : 08 Maret 2007

Direktur Pembinaan Sekolah Menengah Kejuruan

Dr. Joko Sutrisno NIP. 131415680 Tembusan : 1. Direktur Jenderal Manajemen Dikdasmen; 2. Kasubdit di lingkungan Dit. PSMK; 3. Kasubag Tata Usaha Dit. PSMK; 4. Kepala SMK yang bersangkutan; 5. PUM Kegiatan Peningkatan Mutu Layanan Kesiswaan SMK.

This legal, official document meant to increase the level of competitiveness for vocational students by providing financial assistance to take an international high-stakes test. This therefore shows a concerns about quality improvement


9.2 Official Document 2 DEPARTEMEN PENDIDIKAN NASIONAL DIREKTORAT JENDERAL MANAJEMEN PENDIDIKAN DASAR DAN MENENGAH

Jl. Jenderal Sudirman, Senayan, Jakarta 10270, Gedung E Lantai 12-13 Telp (021) 5725477 (hunting), 5725468-9, 5725471-4, 5725466 Fax : 5725469 Nomor Lampiran Hal

: 0352b/C5.5/MN/2007 : 1 (satu) set : Pelaksanaan Sertifikasi Bahasa Inggris Standar TOEIC

06 Maret 2007

Yth Kepala Dinas Pendidikan Provinsi Salah satu program Direktorat Pembinaan Sekolah Menengah Kejuruan dalam rangka meningkatkan mutu tamatan SMK adalah Sertifikasi Bahasa Inggris Standar TOEIC (Test Of English For International Communication) bagi siswa SMK yang diselenggarakan oleh International Test Center. Berkaitan dengan hal tersebut, kami sampaikan informasi sebagai berikut: A. Penyelenggaraan Tes 1. Pada tahun anggaran 2007 Direktorat Pembinaan SMK mengalokasikan subsidi sertifikasi TOEIC bagi 100.000 siswa SMK dengan nilai subsidi sebesar Rp.165.000,- per siswa. 2. Berdasarkan usulan SMK sampai dengan awal Maret 2007, Direktorat Pembinaan SMK menetapkan siswa SMK peserta Sertifikasi Bahasa Inggris Standar TOEIC tahun 2007 dari setiap provinsi seperti Surat Keputusan (SK) terlampir. 3. Sertifikasi TOEIC akan diselenggarakan di 382 Sekolah Penyelenggara Tes (SPT) yang tersebar di seluruh Provinsi antara tanggal 21 sampai dengan 30 Maret 2007. Jadwal pelaksanaan pada setiap SPT seperti terlampir.

This letter would be so B. Mekanisme dan Persiapan Pelaksanaan powerful in arousing 1. Direktorat Pembinaan SMK bekerjasama dengan International Tes Center telah melaksanakan Technical Assistance terhadap effects among 390 guru Bahasa Inggris SMKpsychological yang dicalonkan sebagai SMK Penyelenggara Tes. 2. Sekolah Penyelenggara Tes agar membentuk Tim Pelaksana yang terdiri dari Kepala SMK Penyelenggara sebagai Ketua, Guru as sebagai wellkoordinator, as teachers Bahasa Inggris yang mengikutistudents Technical Assistance Komite Sekolah dan guru lain sebagai anggota. 3. Tim Pelaksana bertugas mempersiapkan ruangan tes, pengawas/proctor, dan mensosialisasikan jadwal pelaksanaan kepada siswa calon peserta tes baik dari SMK yang siswaIt SMK lain yang menggabung. inbersangkutan the testmaupun sites. 4. Pada saat pelaksanaan akan hadir supervisor dari International Test Center di setiap lokasi dan observer dari Direktorat Pembinaan SMK di beberapa lokasi. contributed in setting up 5. Pada hari pelaksanaan, para siswa peserta tes harus menandatangani bukti kehadiran dan penerimaan subsidi keikutsertaan tes sesuai format yang disediakan oleh panitia pusat. testing environment and 6. Setelah tes selesai, Tim Pelaksana harus membuat Berita Acara Pelaksanaan Tes TOEIC yang ditandatangani oleh Kepala SMK SPT. atmosphere 7. SMK Penyelenggara membuat dan mengirimkan laporan pelaksanaan sertifikasi bahasa Inggris ditujukan kepada Direktur 8.

Direktorat Pembinaan SMK dengan tembusan kepada Kepala Dinas Pendidikan Kabupaten/Kota dan Kepala Dinas Pendidikan Provinsi, selambat lambatnya satu minggu setelah pelaksanaan. Beberapa hal yang perlu diperhatikan oleh SMK Penyelenggrara adalah: a. Apabila siswa yang tercantum dalam SK berhalangan hadir, tidak diperkenankan untuk diganti oleh siswa lain dan tidak ada tes susulan. b. SMK tidak diizinkan menambah peserta yang tidak terdaftar dalam SK. c. Untuk menjaga kualitas hasil tes para siswa, Kepala SMK dan guru yang terlibat dalam pelaksanaan tes harus bertanggungjawab atas kerahasiaan soal.

C. Sosialisasi Hasil Tes 1. Hasil tes siswa akan diolah dan dikirimkan oleh International Test Center kepada Direktorat Pembinaan SMK dan kepada SMK Penyelenggara Tes, 14 (empat belas) hari setelah pelaksanaan tes. 2. Direktorat Pembinaan SMK akan mensosialisasikan hasil tes kepada Pemangku Kepentingan baik di tingkat pusat, provinsi maupun kabupaten/kota. 3. Para Kepala SMK Penyelenggara agar mensosialisasikan hasil tes dimaksud kepada para siswa peserta dan orang tua siswa yang bersangkutan. Sehubungan dengan hal tersebut di atas, kami mohon Kepala Dinas Pendidikan Provinsi dapat segera menginformasikan kepada Kepala Dinas Pendidikan Kabupaten/Kota agar masing masing SMK Penyelenggara Tes (SPT) dapat melakukan persiapan sesuai dengan jadwal yang telah ditetapkan. Demikian kami sampaikan, atas perhatiannya diucapkan terimakasih. Direktur Pembinaan Sekolah Menengah Kejuruan, Dr. Joko Sutrisno NIP. 131415680 Tembusan: 1. Dirjen Manajemen Dikdasmen; 2. Kepala Dinas Pendidikan Kabupaten/Kota; 3. Kepala Subdit Kegiatan Kesiswaan; 4. Kepala SMK yang bersangkutan; 5. CEO International Test Center.











ENGLISH TESTING AND LEARNING AT THE VOCATIONAL SCHOOL: STUDENT S LIVED EXPERIENCE

Recommend Documents