Texts for Testing Listening Comprehension

Good Texts for Testing Listening

  • Are well recorded and clear, with limited noise
  • Have the linguistic characteristics of real-world equivalent texts
  • Are structured so they support good items

Good Texts for Evaluating Listening Comprehension

  • Have a main point, topic or gist
  • Have coherent connections between ideas
  • Have an obvious reason for what is being said
  • Provide a clear context

When Analyzing Texts, Consider

  • Is the text type appropriate for the test?
  • Is the language appropriate for such a text or is itin too oral or too literate?
  • Is the language at an appropriate difficulty level for the test?
  • Is the topic appropriate for the test?
  • Is the length appropriate for the test?
  • Is specialized background knowledge necessary for full comprehension?
  • Are visuals or environmental clues necessary for comprehension?
  • Is the speech rate normal, with natural pauses?
  • Are there natural redundancies?
  • Is the vocabulary at an appropriate level?
  • Are uncommon words explained, or does the context make them clear?
  • Will the text support enough items?
  • Will the text support testing of the target skills?
  • Is the recording clear?

Recording Existing Texts

Pre-Recorded Texts

Pre-recorded commercial and broadcast listening materials are available. These include:

  • Listening textbooks
  • Radio and television programs
  • Audio books
  • Educational videos
  • Internet resources

Some of these are expensive, some are of poor quality, all are copyrighted, and all are in the public domain, so are not suitable for secure tests. Legally, you cannot use any copyrighted material without permission. However, in some cases, you may be able to use a small proportion, for non-profit, educational purposes.

Field Recordings

It is not difficult to record from broadcast sources. But:

  • Sound quality may not always be good enough
  • Security is still a concern
  • Copyright is still an issue

You can also record ‘authentic’ situations, by simply recording people as they speak. But:

  • Permission of the speakers must be obtained
  • Authenticity may be lost when a microphone is introduced
  • Sound quality may be poor
  • Content can not be controlled
  • Texts may not make sense when they lose their context
  • With practice, good results can sometimes be obtained

Composing Your Own Texts

The preferred solution is to compose and record your own texts.

Semi-scripted Texts

When people read written scripts, they usually do not produce realistic spoken language; semi-scripted texts work much better. To make these, you determine the content but not the speakers’ words. These can produce fairly realistic, spontaneous texts. Even if not entirely natural, they may be good enough.

Example Semi-script: hotel guest talking to desk staff

  • Guest: (Complaint) Wanted a plaza suite, with the Central Park view. Unhappy with the view.
  • Staff: It has some view.
  • Guest: Contradicts.
  • Staff: Tries to find another suite.
  • Guest: 2nd complaint: only one living area.
  • Staff: Looks at reservations on the 11th floor. Finds a room with a view of the park.
  • Guest: Does it have satellite BBC TV?
  • Staff: It hasn’t.
  • Guest: He asked on the phone. Special match he wants to see. Threatens to move.
  • Staff: (Leaves to speak with the manager, then returns) Offers the presidential suite.
  • Guest: Concerned about the price.
  • Staff: No extra charge.
  • Guest: Wants to check the room.

Making Semi-Scripted Texts More Natural

  • Practice and then wait until the speakers become relaxed
  • Allow speakers to be spontaneous

Monologues

  • Easiest type of text to make
  • Find a topic that interests the speaker, and get them to talk
  • Can be spontaneous or semi-scripted (voice mail message, etc.)
  • Lack of listener feedback (back channeling) makes discourse somewhat unnatural (the person doing the recording can provide visual feedback, but silently)
  • Ending the monologue is unnatural for inexperienced speakers
  • Monologues do not provide good examples of interactive speech
  • Good monologues can test most oral characteristics

Interactive Texts

These are more natural situations, and are often necessary to test language in use. It is important to ensure that the speakers can be easily distinguished: a male and a female voice is standard.

Free discussions

  • Ask two people to discuss a topic
  • Keep the discussion relevant
  • Prepare main points in advance

Interviews

These are a very good option, because:

  • They are spoken texts designed for the benefit of the over-hearer
  • The question and answer format makes them suitable for writing questions
  • Responses are usually short, self-contained, and free standing
  • Even with prepared questions given in advance, answers can be spontaneous

Improvised Role Plays

Describe the situation, and let the participants act it out. These work fairly well when the situation is familiar to the speakers; unfamiliar situations may require semi-scripting or practice.

Role plays are good for testing service interactions, such as shopping or banking.

What Functions Are Important

Interactional language is not usually appropriate for asking questions:

  • It is very culture-dependent
  • It is not intended to be listened to
  • It is more of a social skill than a listening skill

Keep interactional language to a minimum; emphasize transactional language; focus on the exchange of meaningful information.

The Listener in Interactive Texts

Remember, in an interactive text, there is a listener, who should function as the ‘stand in’ for the test taker as listener. Make sure that their role is filled properly. Good listeners should:

  • Evaluate their comprehension in the light of the task requirements
  • Identify any problems, and convey these to the speaker for clarification: i.e. actively cooperate with the speaker to negotiate the meaning
  • Provide appropriate back channeling
  • Follow appropriate turn taking conventions

Using Realistic Texts with Lower Ability Listeners

Low-ability listeners may find realistic speech too difficult. Ideally, use authentic texts that are also simple, but if texts are too difficult, here are some suggestions:

  • Use realistic texts, but use tasks that require only superficial processing
  • Choose simple topics
  • Ask speakers to speak a more slowly than usual and to pause between utterances
  • Ask speakers to use simple vocabulary (avoid slang and infrequent vocabulary)
  • Ask speakers to use simple statements (avoid embedded clauses)
  • Ask speakers to imagine that they are addressing their comments to a foreigner who does not speak the language well
  • Play the text a second time, to make comprehension easier
  • Play the text once, after which show them the test items, then play the text a second time before they reply. This makes the test much easier.

Making Listening Texts Easier or Harder

Because listening comprehension is an interaction between each listener and the text, it is not possible to pre-determine the difficulty level of a text. However, some texts are clearly more challenging than others. Here are some rough guidelines on the variables that affect text difficulty.

Linguistic Variables

All other things being equal:

  • High frequency vocabulary tends to be easier than low frequency vocabulary
  • Simple conjunctive clauses tend to be easier than embedded clauses
  • Slower or normal speech rates tend to be easier than faster speech rates
  • Natural intonation tends to be easier than unnatural intonation
  • Less complex pronoun referencing tends to be easier than complex referencing
  • Longer inter-clausal pauses tend to make texts easier than shorter inter-clausal pauses

Organization

All other things being equal:

  • Events described in temporal order tend to be easier than flashbacks and non-linear structures
  • Stating the point before illustrative examples tends to make texts easier than examples before the point

Explicitness

All other things being equal:

  • More explicit content tends to be easier than more inferential content
  • More redundancy tends to be easier than less redundancy

Content

All other things being equal:

  • More familiar topics tend to be easier than less familiar topics
  • More concrete content tends to be easier than more abstract content
  • Static relationships tend to be easier than more dynamic relationships
  • Visual or other support tends to make texts easier than no support

Context

All other things being equal:

  • A clearly defined context of situation tends to make texts easier than less clear context
  • Extensive accompanying co-text tends to make text easier than less co-text

Techniques for Recording Texts

Here is some practical advice on how to make your own audio recordings.

  • Studio quality recordings are preferable, but not necessary
  • What is necessary are clear recordings with little noise
  • With care and some practice, it is possible to make quite acceptable recordings

The basic idea is very simple: record the voice as loud as possible, and any noise as low as possible.

Equipment

  • Good equipment is far better than cheap equipment: buy, beg, or borrow the best you can
  • Digital is far better that analog: use your computer (Audacity is an open source audio recording and processing program that is free, and will record directly onto you computer (download at http://audacity.sourceforge.net)
  • Learn to use the equipment before hand and, with practice, you will improve

Recording

  • Choose a place that is quiet and get rid of any noise: turn off the fridge, the neon lights, wait until there are no kids around, and put the dog out.
  • Try to choose a place with soft absorbent furnishing: carpets, curtains, etc. not a place with shiny wood or glass (which reflects the sound). If possible, hang up blankets or curtains over reflective surfaces.
  • Set up the microphone about 6 inches from the source: the speaker’s mouth (not too near, otherwise their plosives such as /p/ will make a popping sound).
  • Turn up the recording level as high as possible without getting distortion.
  • Monitor and adjust the recording level while you are recording.

Managing the Speaker

  • Do not let your speaker make unnecessary movements: fidgeting should stop, chairs should remain still, pencils should not be tapped, bracelets should not jangle, etc.
  • Give the speaker some time to relax: expect to make more recordings than you will use.
  • Be patient, expect a number of false starts and mistakes: it may not be perfect, but it may be good enough.
  • Do not expect to simply put a recorder in the middle of a room and get useable recordings. You will have to manage the recording session.

Editing the Recordings

  • With Audacity (or any other audio editing software) you can edit the recording. Just open it, and cut and paste the wave diagram, just as you would cut and paste a paragraph in a word processor.
  • Be prepared to use your audio editing program. Even if speakers keep making mistakes, you may be able to edit the various parts into one coherent discourse.
  • Check to see whether there are any strange noises on the recording—coughs, heavy breathing, mouth clicks, lip smacks, false teeth rattling—are common. Cut them out. If you cannot cut them out, then highlight them and reduce the volume.

If you read the manual you will be able to do far more. You can even:

  • Slow down the speech
  • Speed it up
  • Compress it
  • Normalize it
  • Equalize it

Using audio processing software is as easy as using a word processor.