There are numerous ways to analyze a modern language test.
1. Identify what different “skills” are being tested such as speaking, listening, reading, writing, culture, vocabulary and grammar. How many points are allocated to each? How many points are there cumulatively in each skill area? What does that show about the test’s priorities? If listening is a total of 20 points but grammar is 50 points, then the test is predominantly a grammar test. If grammar and vocabulary outweight the other skills than the test is definitely not communicative.
2. Identify how each part is scored. For example, if students have to write out an answer and each answer is worth eight points, how many points are given for answering the question (content), for vocabulary, and for grammar? If grammar is five points, content two and vocabulary one, then the test evaluates grammar, not the communication of ideas.
3. How much of the test is contextual or situational based as opposed to discrete unrelated items? Test items such as He _____ (to work), She ______(to cook) are only connected by being verb conjugations. These same items could be used in a realistic conversation in which students complete a conversation by conjugating the verbs, sometimes in statements and sometimes in questions.
4, Do students answer questions that relate to their own lives? Often students have a writing part on a test but do those questions allow them to tell about their own lives? Are stipulations put on the writing such as include three -er verbs, include a place, and include an occupation so that the writing is forced? Are students encouraged to think on their feet by answering questions for which they did not prepare or is the writing part a writing out of a memorized writing?
5. How much of the test involves the students reading and writing items in sections that are not labeled as the reading or writing category? Listening comprehension can be based on a picture with multiple choice short answers such as “How many computers are in the room? A-two B-twenty ….” or listening comprehension can be tested by giving the students four full sentences from which to select. The second method changes the listening comprehension to include a reading component. Students can write out numbers without having to write out a full sentence, especially if points are taken off for grammar writing points. Each category should be as purely that skill as possible without depending on other skills otherwise the teachers can not identify what the results signify. In addition, reading and writing are the least used skills in normal communication while listening and speaking can count for up to 70% of normal communication. Does the test represent that real life percentage?
6. What percentage of the test is in English? How many questions involve translation from English to the modern language? Prompts for writing, in English, such as “tell your age, where you live” cause the students to translate. Likewise, a vocabulary exercise such as ” tall =, ” relies on translation. Is there a modern langage reading passage that students answer in English? The greater the test percentage is in English, the less the students use the target language.
7. Does the test assess the the most common verbs and the most common vocabulary according to the 100 most common verbs and 100 most common words lists? Does the test focus on everyday common use of the language or on specific irregularities, exceptions, or non-critical words? Likewise, does the test evaluate the most common language functions like “I would like…”, “I’m sorry”, and “Really”? If students were in the targe language country, they would use many of these functions each day.
8. What does the test score tell the teachers about the students’ ability to communicate in the language? How closely does the test reflect the NCSSFL-ACTFL Can-Do Proficiency statements? Teachers can label each test section with the specific proficiency statement.
What does your test analysis reveal?
At http://bit.ly/tpthtuttle, I have 30+ ready-to-use activities (about 24 for Spanish and 6 for all Modern Language) to develop student’s spontaneous speaking starting with highly structured or scaffolded speaking. Students work in pairs to communicate and they usually assess each other in a formative assessment manner.
My ebook, Modern Language Proficiency: Can-Do Strategies is available at http://bit.ly/tsmash. It contain many activities to help students advance through the Can-Do statements with half the activities focusing on interpersonal communication/ speaking.
At http://bit.ly/tpthtuttle, there are four modern language culture inquiry activities and one Spanish culture inquiry activity. My ebook, 90 Mobile Learning Modern Language Activities ebook contains many communicative and cultural activities, http://bit.ly/tsmash