Assignment on elt practice Project- A Study on Test Designing

Contents Acknowledgement Part 1: Introduction Reasons for choosing the topic Description of teaching and learning English at Trung Vuong Secondary School Part 2: Implementation 2.1. Theoretical background 2.1.1. The relationships between teaching, learning and testing 2.1.2. Qualities of a good test 2.1.2.1. Reliability 2.1.2.2 Validity 2.1.2.3. Practicality 2.1.3. Test types and test items 2.1.3.1. Test types 2.1.3.2. Test items 2.2 Procedures of the project 2.2.1. Purpose of the test 2.2.2. Test items 2.2.3. Test specification Grid 2.2.4. Sample test 2.2.5. Teacher comments 2.2.6. The improved test 2.2.7. Marking instruction and marking scale 2.2.7.1. Marking instruction 2.2.7.2. Marking scale 2.2.8. Test result 2.3. Analysis the test result 2.4. Major Finding and Recommendation Part 4: Conclusion References Part 1 Introduction 1.1. Reasons for choosing the topic Test is really important in teaching and learning English, not only with school children but also with all other types of learners It is no double that test is a useful way for teachers to base on and measure their students’ abilities or what they have gained in a period of learning. From the result of the test, the teachers themselves evaluate the effectiveness of their teaching and adjust it, if necessary, to make it more suitable and efficient. Moreover, it is also a good way that shows the learners how much they have achieved and motivates them much in the learning process. Being an essential method to assess but writing a good test is not easy. It is indeed quite a challenge and it requires lots of patience and experience, and it takes a degree of trail and error. In order to design a good test, we have to know much about the theory, the rules of it. Since being aware of the importance of test and the difficulties in designing a good test, I want to spend time researching this topic so that I can understand more about it, then I can apply it effectively in my teaching career. 1.2. Description of teaching, learning and testing English at Trung Vuong Secondary School : The teaching and learning process at Trung Vuong is taken placed in rather well-equipped conditions: The teaching aids, modern equipments are available. Moreover, the teachers are willing to assessed the modern technology and teach in the communicative way. Each class has about 50 students studying haft a day at school. Some classes have chances to learn with foreign teachers, so their learning environment is really good. Testing is also seriously administered. Each teacher has to design their own test before, and then all the others give comments and finally they choose the best ones to carry out. Every test has clear marking instruction and marking scale so teachers can mark the tests quickly and correctly. After each test, teachers and students correct it in further practice periods so that the students know where and how they were wrong, and where they need to improve. In this project, I will carry out test in 2 classes of grade 8. Students in grade 8 have three periods of English a week, three of which are spent on new lessons, the rest two for further practicing ( the practice may follow a particular topic or they may be the grammar practice periods). Part 2 Implementation 2.1. Theoretical background 2.1.1. The role of testing in language teaching and learning As Brown (1994) points out, tests are a way of life in the educational world. In every learning experience there comes a time to pause and take stock, to put our focal process to their best use and to demonstrate-either to self or others- accumulated skills or knowledge. Traditionally, there was a tendency to separate testing from teaching. However, it has widely been recognized that testing and teaching are so closely interrelated that it is virtually impossible to work in either field without constantly concerned with the other. They are so “interwoven and interdependent that it is difficult to tease them apart” (Brown, 1994, p.375). In addition, Hughes (1991) affirms that the relationship between testing and teaching is that of partnership. “Testing should be supportive of good teaching, and where necessary, exert a corrective influence on bad teaching” (p. 2). Heaton (1989) stresses the necessity of tests in the classroom. He insists that it is like doctors having to first diagnose the patient’s illness, it is very important for teachers to diagnose the student’s weaknesses and difficulties. Therefore, a good classroom test may help locate the precise area of difficulty encountered by the class or by the individual student. Moreover, classroom tests can help teachers increase their own teaching by making judgments in the effectiveness of the syllabus, methods and material they are using. For example, the test results may indicate certain areas of the syllabus that have not taken sufficient account of. In such cases, teachers have to deal with these areas in more depth and length of time. In this way, a good test can be used as a valuable teaching tool, enabling students to learn from their weaknesses and benefit more. Furthermore, Madsen (1983) believes that well-made tests of English can help students in at least two ways. Firstly, they enable teachers to create a positive classroom experience by giving the class a sense of accomplishment. Secondly, English tests can help students master the language. To illustrate, when tests are given, learning can be enhanced by students’ growing awareness of the objectives and the areas of emphasis in the course. Naturally, when students are aware of the course objectives and their personal language needs, they can adjust their personal goals, thus capable of making their learning more effective. 2.1.2 Qualities of a good test In order to design a good test, teachers have to take into account a number of factors, such as the purpose of a test, the content of the teaching programme and the students’ background knowledge. In addition to these factors, test characteristics play a very important part in constructing a good test. According to a number of scholars in testing like Bachman (1990), Henning (1987), Harrison (1987), Bachman and Palmer (1994), Harris (1969), Bracey (2000), Morrow (1998), and Heaton (1989) all tests possess four main characteristics: validity, reliability, practicality. Among these qualities, validity and reliability are essential in the development and use of language tests. 2.1.2.1. Validity From Bachman’s point of view (1990), the most important quality to consider in the development, interpretation, and use of language test is validity. The American Psychological Association (1985) defines validity as the “appropriateness, meaningfulness, and usefulness of the specific inferences made from test scores” (as cited in Bachman, 1990, p. 243). On the other hand, Henning (1987) and Heaton (1989) state that a test is said to be valid to the extent that it measures what it is supposed to measure. Likewise, Bracey (2000) asserts that a test has validity if it measures what it says it is measuring. Different types of validity are defined by test experts. Two of four types of validity suggested by While Heaton (1998) are face validity, content validity. Content validity refers to what goes into the test. Specifically, the test content must be decided whether it represents an adequate sample of the syllabus and reflects all the areas to be assessed in proper proposition. On the other hand, face validity is concerned with what teachers and students think of the test. It answers the question if the test appears reasonable or too easy or too difficult. In addition, among various types of validity, content and face validity are considered to be most important for teachers to consider when constructing his own tests (Harrison, 1987). 2.1.2.2. Reliability The reliability of a test is its consistency (Harrison, 1987; Heaton, 1989; Bachman & Palmer, 1994; Bracey, 2000, Brindley, 2001). Henning (1987) proposes that reliability is the obtaining of similar results when measurement is repeated on different occasions or with different instruments or by different persons. In other words, if a group of people took a test on two different occasions, they should get pretty much the same scores both times (assuming that no memory of the first occasion carries over to the second). If people scored high at time one and low at time two, we can’t say that the test is reliable (Bracey, 2000, p. 10) Furthermore, Bachman (1990) states that while validity is the most important of test use, reliability is a necessary condition for validity in the sense that test scores which are not reliable cannot provide a basis for valid interpretation and use (p. 289). On the other hand, a test can be reliable without being valid. For instance, a multiple choice test of vocabulary could be reliable but it is not valid if used to measure students’ ability to use this vocabulary productively. There are four aspects of reliability: Reliability and sire: Reliability is achieved through size: though a large numbers of test items within the test and through piloting the test with large number of candidates. Reliability and test item level of difficulty: Reliable test items ensure that between one or two third of the candidates will get it right, that is, individual test items are not too hard or too easy for everyone. Tests, which have appropriately one third scoring between 45% - 65% and one third scoring between 65% - 100%, are reliable tests are reliable tests. Reliability and marking: A test can be unreliable sometimes because of the way it is marked. The maarks may be affected by the markers’ subjective opinions. Reliability and syllabus: How much the test is based on passages and questions taken directly from the textbook and how much it is based on the syllabus within the textbook, not the book itself, may have some influences on the reliability of the test as well. 2.1.2.3. Practicability The question of practicability is administrative. That is, a test ought to be practical- within the means of financial limitations, time constraints, ease of administration, scoring and interpretation of scores. A test constructor has to take into considerations how long the test will take, what special arrangements have to be made, whether any equipment is needed or how the scoring is carried out. Harrison (1987, p. 13) states that tests should be as economical as possible in time (preparation, sitting and marking) and in cost (materials and hidden costs of time spent) 2.1.3 Test types and test items: 2.1.3.1. Test types: There are four types of test: Proficiency tests: Proficiency tests give a general picture of a student’s knowledge and ability. They are frequently used as stages people have to reach if they want to be admitted to a foreign university, get a job, or obtain some kinds of certificate. These tests have a profound backwash effect since, where they are external exams, students obviously once to pass them, and teachers’ reputations sometimes depend upon how many of them succeed. Diagnostic tests: Diagnostic tests aim to diagnose a particular aspect of a language. It can be used to expose learner difficulties, gaps in their knowledge, and skills deficiencies during a course. Thus, when we know what the problems are, we can do something about them. Placement tests: Placement tests are used to put students into appropriate level of language curriculum. The aim of these tests is to see what students are good at and what they need to improve. . Certain Proficiency tests and diagnostic tests can act in the role of placement tests. Achievement tests: Achievement tests relate directly to classroom lessons, unit or total curriculum. They are given out at the end of the course to see how successful each individual student has been in taking the course. They only work if they contain item types, which the students are familiar with. Achievement tests at the end of a term should reflect progress not failure. They should reinforce the learning that has taken place, not go out of their way to expose weaknesses. They can also help us to decide on changes. 2.1.3.2. Test items: All the test items can be classified into Subjective or objective, recognition or reproduction. Subjective tests: marked by an English expert. The “marking scheme” may be given out but the final decision comes from the marker own head, which makes the marks sometimes unreliable. Oral exam and composition type writing exam are put in Subjective type items. Objective tests: can be marked by a non-English speaker (a clerk) basing on the provided answer keys. That’s why Objective tests can be “clerically” marked. Placement tests in Multiple-choice format and all computer-marked tests are objective in this way. Recognition type test items: Test-taker only has to “choose” or “recognize” the right answer. This kind of test items doesn’t take students much time to do and it is easy to mark. It can test various language items at the same time. However, it is likely to test the memory only; it can help strong students to show their ability in producing the language. Moreover, students can easily discuss and copy the answer from others, so it is occasionally unreliable. Multiple-choice, True or False, Underline the mistake, Circle the unnecessary word, etc. These are all Recognition tests items. Reproduction type test items: require the test- takers to ‘produce” the answer for themselves by filling missing words, answers, sentences, dialogues, paragraph or essays and includes both spoken and written text. This type of test help rank students as it is where strong students can show their language ability, but it may be difficult for weak students and it makes students write a lot. It also takes more time than recognition one to mark and correct. 2.2. Procedures of the project: 2.2.1. The purpose of the test: Requests of Hanoi Education and Training Department for tests: Every 45- minute tests have to be applied : 20% of vocabulary; 20% - 25% of grammar and structure; 20% of reading skill; 20% of writing skill 15%-20% listening skill. Test takers: Students in grade 8 at Trung Vuong secondary school. These classes are at level with about 20% strong students, 70% of the students standing in the middle and 10% weaker ones. Course book (English 8- Educational Publishing House): It is the third book of four books for the students in every secondary school in Vietnam. It was written on Thematic Approach. All the themes in the book are developed in spiral principles; therefore students can consolidate and develop their ability in language skills. It has a clear syllabus: Unit  Topic  Competencies/ vocabulary  Language focus/ grammar   1  My friends Page 10  -Introduce/ Describe people -Respond to introductions -Write about oneself & other people.  - Present simple. - (not) adjective + enough + to- infinitive.   2  Marking arrangements Page 18  - Talk about intentions. - Use the phone to make & confirm arrangements. - Take a telephone massage.  - Future with be going to. - Adverbs of place.   3  At home Page 27  -Describe places / situations. - Ask for and hive reasons. - Write a description of a room.  - Reflexive pronouns. - Modals: have to, ought to - Why- because.   4  4- Our past Page 38  -Talk about past events. - Express feeling. - Distinguish between facts and opinions - Write a short imaginary story.  - Past simple. - Prepositions of time: on, in, at , after, before, between - used to   5  Study habit  - Give instructions and advice. - Express obligation. - Write a letter to a friend.  - Adverbs of manner. - Modal: should. - Commands, requests and advice in reported speech.   6  The young pioneers club  - Talk about the future. - Ask for favors and offer assistance. - Write a letter telling about a future plan.  - Present simple with future meaning. - Gerunds. - Modals: may, can, could.   7  My neighborhood  - Make comparisons. - Ask for information and assistance. - Write a community notice  - Present perfect with for and since. - Comparison with like, (not) as…as, (not) the same as, different from.   8  Country life and city life.  - Talk about differences/ future events and changes. - Write a letter to a friend  - Present progressive to talk about future. - Comparative and superlative adjectives.   9  A first- aid course.  - Make and respond to offers, promises and request. - Give a thank- you note  - Future simple. - In order to, so as to. - Modal will to make requests, offers and promises.   10  Recycling  - Give and respond to instructions. - Talk about feelings. - Write a set of instructions.  - Passive forms. - Adjectives followed by: - An infinitive. - A noun clause.   11  Traveling around Viet Nam  - Express interest. - Make and respond to formal requests. Make suggestions  - -ed and -ing participles. - Requests with : Would /Do you mind if…? Would/ Do you mind+Ving   12  A vacation abroad  - Make plans - Make, accept and decline invitations - Write a postcard  - Past progressive with when, while and always.   13  Festivals  - Ask for explanation of events - Give an account of something - Write a paragraph about what was done  - Passive form - Compound words - Reported speech   14  Wonders of the world  - Seek information - Recognize and correct mistakes - Write a letter to a friend  - Passive form - Indirect questions with if and whether - Question words before to-inf - Verb + to- inf   15  Computer  - Express opinions - Complete a flow chart - Write a set of instructions  - Present perfect with yet and already - Comparison of present perfect and past simple   16  Inventions  - Say what something was like - Talk about processes - Write about a process  - Passive form - Sequence markers   - Testing content: General English around the knowledge of three topics: Festival / Wonders of the world / Computer and four skills of English covered five units 1, 2, 3, 4, 5. - Testing time: At the end of the second semester. - Time allowance: 45 minutes. - Type of test: Achievement test. - Purpose of the test: + To assess students’ achievement at the end of the second semester. + To identify students’ abilities on learning vocabulary, reading skill, writing skill, listening skill, grammar points and structures. + To evaluate the effectiveness of teaching methods currently employed. 2.2.2. Test items: Basing on the above –cited test purpose, I use these test items. The following grid will clarify all items in the test: Test items  Objective  Subjective  Recognition  Reproduction   Choosing the different pronunciation       Gapfilling by the correct word form       Multiple-choice questions       Sentence transformation       Writing sentences with the word cues       Reading and answering questions       Filling information (listening)       2.2.3. Test specification Grid: Focus  Input  Response/ Item type  Marks  Weights   Language elements  Language skills       Pronunciation   4 questions: Syllable stress  Multiple-choice  1pt / 10  10%   Vocabulary   4 questions  Gapfilling  1pt / 10  10%   Grammar   10 qu