Post 13: Assessment

Blog Post 13

November 26, 2012   

Shohamy (2000) pages 201-215

Brown Chapters 23-24

            In Shohamy’s article, she discusses how discourse can affect language testing.  There are various elements of the discourse which can affect a language test. “The issue is not the validity of the discourse elements within communicative language competence, but rather the extent to which these features manifest themselves in testing situations and their effect on test takers’ scores (Shohamy 203).

 First off, I believe that the test writer should consider all of their test takers.  Their prior knowledge needs to be taken into account.  According to the article, prior knowledge could give some test takers advantages. Hock (1990) showed both language proficiency and prior knowledge predicted test takers scores on a cloze test.  Yet Clapham (1994) examined this effect of background knowledge in relation to test performance and found no such effect.  Either way, it should be considered.  Let’s say there is a reading passage about Halloween or Thanksgiving.  Two American holidays.  Some students may not have any background knowledge and others might.  This could in turn alter their test scores.

In Brown’s chapter 23, he discusses the five basic principles for designing effective tests and assessments.  The first is practicality.  A good test needs to be practical in the sense of financial limitations, time constraints, ease of administration and scoring and interpretation.

The second principle is reliability.  A reliable test is consistent and dependable.  There are many things that can cause a test to be unreliable, though.  Some of these are the construction of the test, administration of the test, the test taker, and the scoring.  In my ENG 346 class this is something that we have studied in depth.  There is person-related reliability, instrument related reliability and rater related reliability.

The third principle is validity.  This is a very complex criterion for a good test.  There are three main types of validity: content, face, and construct.  Each needs to be assessed carefully before test administration.  The fourth principle is authenticity.   This is like saying that the test you have created is a task that is likely to be enacted in the real world. According to Brown (451) there are many things that need to be in place to assure this.  For example, the language of the test needs to be as natural as possible.  Items should be contextualized rather than isolated.  Topics and situations are interesting, enjoyable, and humorous.  Some thematic organization to items is provided, such as through a story line or episode.  The tasks should represent, or closely approximate, real-world tasks.

Brown’s chapter 24 discusses classroom-based assessment.  The first topic discussed is the difference between norm-referenced and criterion-referenced tests.  Norm-referenced tests are designed to compare students by dispersing the average student scores along a bell curve.  This can determine students’ individual performance in comparison to others. While developing a norm-referenced test, the publisher must analyze the correlation between parts of the test and reliability measure.  The student is also is placed in percentage among their peers.  As for interpretation of results, a norm-referenced test’s scoring is based on the mean and standard deviation.  The assessor must also consider the standard error of measurement, which is the difference between the score the student receives and the true score, which is what the student actually knows.  Essentially, norm-referenced tests relates one student’s performance to that of other students’ performance.  It does not, however, usually permit you to identify specifically what students have or have no learned.  It does not enable you to isolate aspects of instruction that need additional attention in order to enhance student achievement.

The purpose of criterion-referenced tests is to classify people according to whether or not they are able to perform some task or set of tasks satisfactorily. The tasks are set, and the performances are evaluated. Constructing this test involves describing language objective, selecting appropriate test tasks, devising and assembling test tasks, and devise a scoring system.  Choosing the type or types of tasks to include in a test depends on a combination of factors: instructional objectives, students’ proficiency level, instructional activities, and available testing resources. Tasks can be open-ended or closed-ended. The tests are constructed such that a ‘pass’ is obtained by completing the great majority of the test tasks successfully.  Students take only the test or tests on which they are expected to be successful. This test is also useful for isolating aspects of instruction that need additional attention in order to enhance student achievement.

In the end, assessment helps everyone.  It helps the instructors identify where their student are and what needs to be taught.  They can reflect on their teaching strategies and see if they need to be altered.  Students benefit from assessments from washback, such as self-confidence gain.  Students also learn where they are struggling and can identify where they need help.