1 VIETNAM NATIONAL UNVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST-GRADUATE STUDIES VŨ HOÀNG DUNG DESIGNING AN ENGLISH ACHIEVEMENT TEST FOR THE 12TH FORM OF NON-ENGLISH MAJORS AT LE QUY DON HIGH SCHOOL – DONG DA, HANOI (Thiết kế kiểm tra Tiếng Anh cuối kì cho học sinh lớp 12 không chuyên ngữ trường Trung học phổ thông Lê Quý Đôn – Đống Đa, Hà Nội) M.A Minor Programme Thesis Field: Methodology Code: 60 14 10 Hanoi, August 2010 VIETNAM NATIONAL UNVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST-GRADUATE STUDIES VŨ HOÀNG DUNG DESIGNING AN ENGLISH ACHIEVEMENT TEST FOR THE 12TH FORM OF NON-ENGLISH MAJORS AT LE QUY DON HIGH SCHOOL – DONG DA, HANOI (Thiết kế kiểm tra Tiếng Anh cuối kì cho học sinh lớp 12 không chuyên ngữ trường Trung học phổ thông Lê Quý Đôn – Đống Đa, Hà Nội) M.A Minor Programme Thesis Field: Methodology Code: 60 14 10 Supervisor: Pham Thi Hanh, M.A Hanoi, August 2010 TABLE OF CONTENTS STATEMENT OF AUTHORSHIP i ACKOWLEDGEMENTS ii TABLE OF CONTENTS iii ABSTRACT vi LIST OF ABBREVIATIONS vii LIST OF TABLES AND APPENDICES ………………………………………… viii Part A: INTRODUCTION………………………………………………………… 1 Rationale for the study………………………………………………………… … Aims and scope of the study………………………………………………… …… Methods of the study………………………………………………………… …… Research questions…………………………………………………………… … Design of the study…………………………………………………………… … Part B: LITERATURE REVIEW……………………………………………….… Basic Concepts of Testing…… …………………………………………………… Roles of testing……………… …………………………………………………… Types of Test………………………………………………… …………………… 3.1 Achievement tests………………………………………… ………………… 3.2 Placement tests……………………………………………… ……………… 3.3 Diagnostic tests……………………………………………… ……………… 3.4 Proficiency tests……………………………………………… …………… 3.5 Direct versus indirect testing………………………………… ……………… 3.6 Discrete point versus integrative testing……………………… …………… 11 3.7 Norm-referenced versus criterion-referenced testing………… …………… 11 3.8 Objective versus subjective testing…………………………… …………… 12 3.9 Communicative language testing……………………………… …………… 13 Characteristics of a Good Test……………………………………… …………… 14 4.1 Validity……………………………………………………… ……………… 14 4.1.1 Face validity………………………………………… ……………… 14 4.1.2 Content validity……………………………………… ……………… 15 4.1.3 Construct validity……………………………………… …………… 16 4.2 Reliability……………………………………………………… …………… 17 4.3 Practicality…………………… …………………………………………… 18 4.4 Discrimination…………… ………………………………………………… 19 Test Items……………………… ………………………………………………… 19 5.1 Direct test items…………… ……………………………………………… 19 5.2 Indirect test items………… ………………………………………………… 20 Part C: THE STUDY…………………………………….………………………… 22 The issues influencing current testing situation at LQD-DD High School…… … 22 1.1 The students and their backgrounds…………………………………… …… 22 1.2 The English teaching staff……………………………………………… … 22 1.3 English teaching and learning………………………………………… …… 23 1.4 The objectives of the K-12 course……………………………………… … 24 1.5 Teaching material used for the K-12…………………………………… … 24 1.6 The content and construction required for the achievement test for the K-12 26 The current testing situation………………………………………………… …… 26 The process of testing………………………………………………… ………… 27 3.1 Determining the purpose of the test……………………………… ……… 28 3.2 Planning the test……………………………………………………… …… 28 3.3 Selecting items and tasks…………………………………………… ……… 29 3.4 Administering the test……………………………………………… ……… 29 3.5 Scoring and rating………………………………………………… ……… 30 3.6 Descriptive data from test score…………………………………… ……… 30 3.6.1 Frequency distribution 31 3.6.2 Measures of central tendency 32 3.6.3 Measures of dispersion 33 3.6.4 Item analysis 34 Survey 36 4.1 Data collection procedure 36 4.2 Findings 37 Archiving ………………………………………………… ……………………… 38 Some discussions from test result analysis 38 Part D: CONCLUSION ……………….…………………………………………… 39 Conclusion ………………………………………………………………………… 39 Recommendation …………………………………………………………………… 39 REFERENCES……………………………………………………………………… 40 LIST OF ABBREVIATIONS D: Discrimination Index FV: Facility Value GCSE: General Certificate of Secondary Education K-12: The 12th form L: The number of lower group students who answered the item correctly LQD – DD: Le Quy Don – Dong Da MCQs: Multiple Choice Questions S.D.: Standard Deviation U: The number of upper group students who got the item correct LIST OF TABLES AND APPENDICES Table 1: Objectives and contents of the test Table 2: The frequency of marking scores Table 3: The measures of central tendency Table 4: The measures of dispersion Table 5: Facility value and Discrimination index APPENDIX 1: The contents of the eight units taught in the 1st term of K-12 APPENDIX 2: The structure of English tests for the GCSE national examination APPENDIX 3: Revision outline for the 1st term test APPENDIX 4: The 1st-term English examination for K-12 APPENDIX 5: Keys for the 1st-term English examination for K-12 APPENDIX 6: Students’ performance on test arranged from the highest score to the lowest score APPENDIX 7: The survey questionnaire for students APPENDIX 8: The survey questionnaire for teachers 10 Part A: INTRODUCTION Rationale for the study Assessment plays a significant part in the field of linguistics It allows teachers as well as administrators to make important decisions regarding the proficiency, placement, and achievement of second language learners Among some kinds of assessment, testing is the one that is typically used at the end of a stage of instruction to measure students‟ achievement Henning (1987, p 1) says that the most common purpose of language tests is “to pinpoint strengths and weaknesses in the learned abilities of the student” Heaton (1990, p 9) also shares the idea that it is “to find out how well the students have mastered the language areas and skills which have just been taught” Besides, teachers can use language tests to encourage students Students can show their ability and progress through tests Heaton (1990, p 10) agrees that tests are also “used for the purpose of increasing motivation” Having good results, students will be eager for their coming study With not very good test results, they will know which level of language they are at so as to set a more realistic goal for themselves Being aware of the above things, the author of this study, who has been a teacher of English for about 10 years and has written many tests to evaluate the language abilities of students, has noticed that not all the tests have satisfying results Some tests have bad results, which means that too many students get bad marks or too many students get very good marks Teachers cannot use the results of these tests to categorize the students Therefore, such tests are not reliable Taking the problem into consideration, the author has found out that it is the technique of designing tests that leads to the situation When writing tests, the test writer may only feel that this or that test item is suitable for their students She or he may have never analyze the test designing and the test result carefully in order to find out the causes for the problem On the other hand, a well-designed test is always necessary for all students of every language level, especially those of high school level - the elementary level which aims at acquiring survival English Therefore, getting more knowledge and experience in designing tests is the author‟s desire In this minor thesis, the author would like to base herself on the knowledge of testing and testing situation to design an achievement test for the 12th form of non-English majors at Le Quy Don-Dong Da High School, Ha Noi 11 Aims and scope of the study This study seeks to use the academic knowledge in test design to write a reliable achievement test for the 12th form of non-English majors at Le Quy Don-Dong Da High School Because this is an achievement test given to students after a study period of about four months, the 1st term of the school year, and students are required to use what has recently been taught and practiced to the test, we, the teachers of English at LQD – DD High School, expect our students to score fairly high marks, at least 70% of which should be from mark upwards Therefore, this study specifically seeks to answer such questions as whether the items of the test meet the requirements of the course, whether these test items are suitable for at least 70% of students to get marks from upwards and what recommendations should be given to improve the testing situation so that we can build a bank of reliable tests for the school in the coming years As students of grade 12 are going to take part in the national examination for General Certificate of Secondary Education (GCSE), they should be given to a chance to get used to the structure of this kind of examination Therefore, the study will focus on the existing situation at LQD – DD High School, which means that the author will design a multiple choice written achievement test only on such fields as phonetics, grammar, vocabulary, communicative function, reading and writing skills as directed in the book called “Cấ u trúc đề thi môn Ngữ văn , Lịch sử, Đi ̣a lí , Ngoại ngữ” (i.e “The Test Structures of Literature, History, Geography, Foreign Languages”) by Nguyen (2009, p 104) The study will also provide analyzed data of the test, the teachers‟ and students‟ comments on the test as well as their suggestions for its improvement Methods of the study The main methods employed in this study included test design, test result analysis and teacher and student survey For the purpose of designing tests, the author based on the theory and principle of language testing, major characteristics of a good test, especially a good achievement test and testing situation at LQD-DD High School The test result was analyzed by the software SPSS to present the frequency distribution, central tendency and dispersion of the test score The analysis of test items is carried out manually Finally, the survey enabled the author to collect teachers‟ and students‟ opinions on the reliability and suitability of the test A questionnaire in both English and Vietnamese 12 was given to 50 students and another in English was given to teachers of English at LQD – DD High School The questionnaires include most closed-ended questions because the quantitative data was the major interest and it was easy to carry out the survey There was only one open-ended question in each questionnaire so that more suggestions could be given to improve the quality of test design Research questions This study was intended to answer such research questions as: What test items are appropriate to meet both the requirements of the course and the test writer‟s expectation in the testing situation at LQD-DD High School? What items are the easiest and most difficult of all? What should be done to improve the test? Design of the study This thesis consists of four parts, with a list of references, and appendices Part A, “Introduction”, provides the rationale for the study, the aims, the scope, the methods, the research questions and the design of the study Part B, “Literature Review”, discusses the basic concepts of testing, the roles of testing, the types of test, the characteristics of a good test and the test items Part C, “The Study”, presents five sections The first is issues influencing current testing situation at LQD-DD High School The second is the current testing situation The third is the process of testing, which is ended with the descriptive data from test score Then the study goes on with the teacher and student survey including data collection procedure and some findings The next is about the archive of good test items The last is some discussions from test result analysis Part D, “Conclusion”, gives some brief conclusions on the study and some recommendations for further study 13 Part B: LITERATURE REVIEW Basic Concepts of Testing For the concepts of testing, many linguists have expressed their opinions It is said to refer to “the specific procedures that teachers and examiners employ to try to measure ability in the language, using what learners show they know as an indicator of their ability” (Hedge, 2000, p 378) According to Bachman (1990, p 20), a test is a measurement instrument designed to elicit a specific sample of an individual‟s ehaviour Besides the term “testing”, it is necessary that we should consider some more terms such as “evaluation”, “assessment” and “measurement” as all of them have a close relationship Bachman (1990, p 18) proposes that these terms are often used as synonyms that refer to the same activity However, he also points out that each of them has its own distinctive characteristics Therefore, tests are only one of many different types of measurement Hedge (2000) also agrees that: Assessment is the more inclusive term: it refers to the general process of monitoring or keeping track of the learners‟ progress Testing is one kind of assessment, one which is typically used at the end of a stage of instruction to measure student achievement (Hedge, 2000, p 376) Bachman (1990, p 22) says that “tests in and of themselves are not evaluative” Davies (2000, pp 170-171) shares the ideas that evaluation is a more general concept than testing but tests remain to be the main instruments for evaluation of learning in most teaching situations In summary, “not all measures are tests, not all tests are evaluative, and not all evaluation involves either measurement or tests” (Bachman, 1990, p 24) Roles of testing in teaching and learning The roles of testing have been recognized by many researchers McNamara (2000, p 4) and Heaton (1988, p 7) say that language tests play a powerful part as a tool for teachers to find out which parts of the language program have caused difficulty to the class They act as gateways at important transitional moments in education, in employment, and in moving from one country to another Teaching and testing have an intimate relationship, which is considered as that of partnership (Hughes 1989, p 2), and testing is also essential in learning It provides students with an opportunity to show their ability to perform certain tasks in the language (Davies, 2000, pp 169-170) It helps ... and testing situation to design an achievement test for the 12th form of non -English majors at Le Quy Don- Dong Da High School, Ha Noi 11 Aims and scope of the study This study seeks to use the. .. the academic knowledge in test design to write a reliable achievement test for the 12th form of non -English majors at Le Quy Don- Dong Da High School Because this is an achievement test given to... VIETNAM NATIONAL UNVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST-GRADUATE STUDIES VŨ HOÀNG DUNG DESIGNING AN ENGLISH ACHIEVEMENT TEST FOR THE 12TH FORM OF NON-ENGLISH