Aimed at reporting the result of the examination of current final achievement test for the 12th form students at NQHS in terms of its validity

CHAPTER 1: INTRODUCTION 1.1. RATIONALE A good test can be used as a valuable teaching device. Heaton (1991:5) states that “test may be constructed primarily as devices to reinforce learning and to motivate the student or primarily as a means of assessing the students’ performance in the language.” According to this linguist, the relationship between testing and teaching is “so closely interrelated that it is virtually impossible to work in either field without being constantly concerned with the other”. For proper evaluation and assessment of the English language learning and teaching process, testing, an important tool in educational research and for program evaluation (Lauwerys and Seanlon (1969:2) is employed as an indispensable part of the training program at Ngo Quyen high school (NGHS) in Hai Phong city. However, the designing a good test is not simple. Having been a teacher of English for many years, I have been involved in designing, administering and marking many kinds of English tests such as progress and end-of-term tests and also have often heard teachers and test-takers at NQHS complaining that some of the final achievement tests for 12th form students do not faithfully reflect the real linguistic competence of the test-takers. What is tested is not really taught and the test measures neither the achievement of the course objectives nor the expected linguistic skills and knowledge of the students. Probably, this is because the test writers use the tests which are designed elsewhere and are not suitable for the students. What test writers are concerned with seems to be the reliability of the test rather than its validity. The situation coincides with the comments made by some test researchers as Brown (1994: 373) and Hughes (1989:1) on recent language testing, “a great deal of language testing is of very poor quality. Too often language testing has a harmful effect on teaching and learning and too often they fail to measure accurately whatever it is they are intended to measure”. Another reason is that language testing here has not been paid enough attention to. I have not witnessed either comprehensive or systematic evaluation on the effectiveness and appropriateness of these tests. For the above-mentioned reasons, the author is encouraged to undertake this minor thesis with the aim at investigating the designing final written achievement tests for the 12th form students at NQHS through evaluating a current final achievement test by both students and teachers mainly in terms of its validity. I hope that the result of the study can then help to improve the quality of the final achievement tests for the 12th form students at NGHS. 1.2. SCOPE OF THE STUDY Due to the limitations of time and ability, the scope of the study is limited to research on examining the current final achievement test for the 12th form students at NQHS mainly in terms of its validity. The study provides empirical evidence of the current final achievement test and proposes practical suggestions on the improvement of the final tests for the 12th form students at NQHS in general. 1.3. AIMS OF THE STUDY The study is aimed at reporting the result of the examination of current final achievement test for the 12th form students at NQHS in terms of its validity. It highly emphasizes analyzing the teachers’ and students’ evaluation on the test and their suggestions towards its improvement. The specific aims of the research are: - To investigate the NQHS English teachers’ and the 12th form students’ evaluation of the current final achievement test in terms of its validity. - To find out the differences and similarities (if there are any) in teachers’ and test takers’ evaluation of the test and to suggest reasons why there are such similarities and differences. - To provide some practical recommendations for the improvement of the final achievement tests so as to achieve more accurate measures of students’ English competence. 1.4. RESEARCH QUESTIONS The research questions of the study are as follows: - How is the current final achievement test for the 12th form students at NGHS evaluated by both students and teachers in terms of its validity? - What improvements are recommended by the teachers and students with regard to the validity of the test? 1.5. METHODS OF THE STUDY In order to achieve the above aims, a study has been carried out with the following methodologies. First, the author based herself both on the theory and principles of language testing, major characteristics of a good test (with special focus on test validity), achievement test and practical tips to write it. From her critical reading, many reference materials have been gathered, analyzed, and synthesized to draw out a theoretical basis to evaluate the current final achievement test for the 12th form students at NQHS . Second, qualitative methodologies involving data collected through survey questionnaires were employed. Two questionnaires were administered to the 12th form students and teachers of English at NQHS in order to investigate their evaluative comments on the current final achievement test in terms of its validity and their suggestions for its improvement. Besides, many other methods such as interviews, informal discussion with students, teachers, and classroom testing observation are also used to get more needed information. 1.6. STRUCTURE OF THE STUDY The minor thesis is organized into five major chapters: - Chapter one presents basic information such as the rationales, the aims, the research questions, the methods, and the structure of the study. - Chapter two is about a review of related literature that provides the theoretical basis for evaluating and building a good language test. This review consists of background on language testing, criteria of a good test, theory on the written achievement test such as its two kinds and practical tips for writing achievement tests. - Chapter three, the main part of the study, analyzes the results of the survey including the questionnaires and direct interviews to find out the existing problems in designing the current achievement test in particular and other final achievement tests in general at NGHS. - Chapter four proposes some suggestions on improvement of designing the final achievement tests basing on the mentioned theoretical and practical study. - Chapter five provides a summary and suggestions for further research on the topic, and reference materials as well. CHAPTER 2: LITERATURE REVIEW 2.1. DEFINITION OF TESTING A test is generally defined by Carroll (1968:46) as “A psychological or educational test is a procedure designed to elicit certain behavior from which one can make inferences about certain characteristics of an individual”. Simply put, a test is an instrument designed to elicit a specific sample of an individual’s behavior. Similarly, Davies (1991:13) states that the tests are operational in nature, i.e, they are intended to measure whether or not the candidates can do certain things in English. The “things” they are asked to do are specified at each level and represent authentic tasks of the sort which confront language users in real life. Genesee and John A. Upshur (1996) look at tests as a task that measures one’s ability to perform a particular task. They argue that a test is, first of all, about something. That is, it is about intelligence, or European history, or second language proficiency. In educational terms, tests have subject matter or content. Second, a test is a task or set of tasks that elicits observable behavior from the test taker. The test may consist of only one task, such as writing a composition, or a set of tasks, such as in a lengthy multiple-choice examination in which each question can be thought of as a separate task. Different test tasks represent different methods of eliciting performance. Third, tests yield scores that represent attributes or characteristics of individuals. In order to be meaningful, test scores must have a frame of reference. Test scores along with the frame of reference used to interpret them is referred to as measurement. Thus, tests are a form of measurement. (p.141). In other words, content, methods and measurement are three aspects of tests. The quality of the end-of-year tests depends on whether the content of the test is a good sample of the relevant subject matter. If the content of a test is a poor reflection of what has been taught or what is supposed to be learned, then performance on the test will not provide a good indication of achievement in that subject area. What a test is measuring is a reflection of not only its content but also the method it employs. Tests that employ different methods are measuring somewhat different skills, no matter how similar their content might be. Tests in education measure differences in degree. They describe how proficiently students can read a second language or how appropriately they speak in particular social situations, for example. In the foreign language teaching context, a test can be defined as an educational instrument which is designed to measure what someone can do with the foreign language to serve a particular purpose. (McNamara:11) As an instrument, a test may be responded to differently by testees and test-users. Understanding testees and test-users’ responses to, and perceptions of tests has been a critical issue in foreign language testing. Such understanding is even more important where learner-centredness is promoted as a philosophical orientation in foreign language teaching. Testing, the act of administering a test, is closely related to teaching and learning. This relationship is discussed in the next section (section 2.2) 2.2. RELATIONSHIP BETWEEN TESTING, TEACHING AND LEARNING With regard to the relationship between testing, teaching and learning, there have been two extreme views. In the past, there was a common view that teaching and learning were separated both theoretically and in practice. According to this view, a test is a necessary but unpleasant imposition from outside the classroom: it helps to set standards but uses up valuable class time. But other researchers acknowledge the close link between them. For example, Harrison (1991:7) believes that far from being divorced from each other, testing and teaching are closely interrelated. A test is seen as a natural extension of classroom that can serve each as a basis for improvement. Upshur (1971) adds that, language testing both serves and is served by research in language acquisition and language teaching. Language tests can be valuable sources of information about the effectiveness of learning and teaching. Language teachers regularly use tests to help diagnose student strengths and weaknesses, to assess student progress, and to assist in evaluating student achievement. Language tests are also frequently used as sources of information in evaluating the effectiveness of different approaches to language teaching. As sources of feedback on learning and teaching, language tests can thus provide useful input into the process of language teaching. That kind of feedback is termed “backwash” by Hughes (1989) who defines the term as “the effect of testing on teaching and learning”. He goes on to explain that testing can have either a beneficial or a harmful effect on teaching and learning. “If a test is regarded as important, then preparation for it can come to dominate all teaching and learning activities. And if the test content and testing techniques are at variance with the objectives of the course, then there is likely to be harmful backwash”. (p.1). However, he notes that the relationship between teaching and testing is that of partnership. In other words, we cannot expect testing only to follow teaching; rather a good test is an obedient servant since it follows and apes the teaching (Davies (1968: 5). What we should demand of it, however, is that it should be supportive of good teaching and, where necessary, exert a corrective influence on bad teaching. If testing always had a beneficial backwash on teaching, it would have much better reputation amongst teachers (Hughes:2) Cohen (1994) discusses the effects of backwash more broadly, in terms of “how assessment instruments affect educational practices and beliefs” (p.41). Wall and Alderson (1993), go a little bit farther to argue convincingly on the basis of extensive empirical research, that backwash has potential for affecting not only individuals, but the educational system as well. Read (1983:2) points out: “A test can help both teachers and learners to clarify what the learners really need to know assuming that it is unrealistic to expect them to master everything they are presented with during a particular course.” The result of tests shows teachers not all but part of learners’ ability, which helps teachers to improve ways of teaching or revise knowledge. According to Heaton (1898:7), “a well-constructed classroom test will provide the students with an opportunity to demonstrate their ability to perform certain tasks in the language and the students should be able to learn from their weakness”. Obviously, under the influence of the tests, the students are motivated to use what they have done and avoid the mistakes and errors that they have made. The learners know how far they have achieved the object of the course so that they can upgrade their level or they have to learn more. “A good test can sustain or enhance class morale and aid learning.” (Madsen, (1983:3). Because of the important role a test plays in either supporting or impeding teaching and learning, it is critical that a test must be supportive of good teaching. This raises the necessity to investigate the opinions of the test users, specifically the learners and the teachers. 2.3. TYPES OF ACHIEVEMENT TESTS An achievement test is one of the means available to teachers and students alike of assessing progress. According to Hughes (1990:10), “achievement tests are directly related to language course, their purpose being to establish how successful individual students, groups of students, or the courses themselves have been in achieving objectives”. To make it clearer and to distinguish it from others simultaneously, Harrison (1991) stresses that “an achievement test looks back over a longer period of learning than the diagnostic test” (p.7). He provides a clear distinction between achievement and diagnostic tests in that achievement tests cover a much wider range of material than a diagnostic tests and relate to long-term rather than short-term objectives. Achievement tests are designed to assess the whole course or even a number of courses. Those students who have finished an English course will sit for the test and will be evaluated whether or not they have learnt it well. Their standards and differences are judged in relation with other students in the same stage by test results. On the other hand, diagnostic tests also look back on the previous course for persistent errors for which they from remedial work. It can be referred that diagnostic tests can be used to predict and improve future teaching and learning. Additionally, Heaton- when widening the concept of achievement tests- defined them as the ones “based on what the students are presumed to have learnt- not necessary on what they have actually learnt nor on what has actually been taught” (Heaton, 1991:172). According to the time of administration and designed objectives, achievement tests can be subdivided into two kinds of achievement tests: Progress achievement and final tests. 2.3.1. Progress achievement tests Progress achievement tests are always administered during the course, after a chapter or a term, and often written by the teacher. They are based on teaching program. Hughes (1990:12) claims “these tests are intended to measure the progress that students are making.” Since “progress” in achieving course objectives, these tests should be related to objectives. These should make a clear progression towards the final achievement tests based on course objectives. Then if the syllabus and teaching methods are appropriate to these objectives, progress tests based on short term objectives will fit well with what has been taught. If not, there will be pressure to create a better fit. Progress achievement tests are supposed to help the teacher to judge the degree of success of his or her teaching and help to find out how much students have gained from what have been taught. Accordingly, the teachers can identify the weakness of the learners or diagnose the areas not properly achieved during the course of study. In short, progress achievement tests can be regarded as a useful device that provide the students with a good chance to perform the target language in a positive and effective manner and to gain additional confidence in doing them. This way can be a good preparative and supportive step towards the final achievement test for the students because they will get familiar with the tests and the strategy to do them. 2.3.2. Final achievement tests Final achievement tests, as the name suggest, is usually a formal examination, given at the end of the school year or at the end of the course to measure how far students have achieved the teaching goals (Hughes(1990:10). They may be written and administered by ministries of education, official examining board, or by members of teaching institutions. The content of these tests must be related to the courses with which they are concerned. Hughes (1990:11) suggests two approaches towards designing achievement tests: syllabus-content approach and objective content approach. The syllabus-content approach means that the content of a final achievement tests should be based on a detailed course syllabus or on the books and other material used. The tests designed basing on what the students have already learnt in the course books can be considered fair tests. On the contrary, the badly designed syllabus or badly chosen material which is different from the course objectives may bring about misleading results which are unlikely to show what students have achieved on the other. When this occurs, test results will fail to meet the test validity in terms of course objectives. The syllabus-objective approach is to design the test content directly on the objectives of the course. This approach has some good points. Firstly, it forces course designers to elicit about course objectives. Secondly, this approach can help to work against the poor teaching practice that syllabus content-based tests fail to do. However, this approach has to cope with the problems in testing what the students have neither learned nor prepared. Of the two approaches mentioned, Hughes (1990:11) favors the latter one by arguing that it will provide more accurate information about individual and group achievement, and it is likely to promote a more beneficial backwash effect on teaching. 2.3.3 Roles of achievement tests The roles of achievement tests are clearly shown by McNamara(2000:6): “ Achievement tests accumulate evidence during or at the end of a course of study in order to see whether and where progress has been made in terms of the goals of learning. Achievement tests should support the teaching to which they relate.” That is, achievement tests play an important role in the teaching-learning process. Besides bearing all the characteristics of a normal test, achievement tests can supply more accurate and fuller information because they look back on the course students have been learning. These tests’ backwash effect can show teachers how appropriately or effectively their teaching has been. Furthermore, results obtained from achievement tests enable teachers to become familiar with the of each student and with the progress of the class i