Sound Lab

A soft melody, a harsh tone, a frightened voice, a sharp cry, a beep, a screech, a ring, a buzz...

Index

The Power of Sound ¦ When Sound Is a Barrier ¦ Activity ¦ Tips on Presenting Alternatives to Sound

Note: sound files are provided in *.au format and should be playable on any machine with sound capability. Netscape on Windows machines gives an error message when you try to load sound files, but the file should load and play in spite of that. Movie files are in QuickTime format and require the QuickTime viewer available free from Apple.

The Power of Sound

We know that sound (narration, music, effects) can be used in many creative ways to inform, instruct and entertain. Be it a science fiction film, an instructional video clip or a computer game, sound can enrich experiences and help engage learners. For example,

Sound creates high energy impact.

• Hear a window pane breaking (43k).
• Listen to a motorcycle race by (89k).

Sound sets the mood and pace of a scene or event.

Music in the background of movies sets the mood, and conveys the pace. Most movies would be flat without it.

• Hear a piece of music (168k) that can set the mood and pace. Consider the feelings you have while listening to this, and what qualities of the music bring up those feelings. If this were playing in the background while you were reading this page, it would probably impact your overall experience of the page.
• Listen to another piece of music (168k). This would lead to a very different experience of this page. Maybe it's more appropriate, maybe less--but either way, would be an integral part of the site.

Sound conveys affect and emotion.

• Read the following sentence:
  "Forget what he said. What does he know?"

  Think about the way you heard that in your head. What is the speaker's mood--angry, puzzled, disgusted, or something else? What cues led you to that conclusion?

• Now listen to a recording of those words (270k). Depending on your interpretation from the reading, this may or may not have sounded as you had expected. Consider the aural cues that make it different from just a straight reading of the text.

Sound provides feedback to an action on a computer screen.

• Hear an auditory response to a button click (32k) on the screen. A sound such as this can confirm that you clicked a button, and different sounds can represent different actions. Combined with visual effects, such as the button appearing to "depress" when clicked, sound effects can make the user interface much more intuitive.

Sound in voice-over narrations provides additional information.

d

Click here to play a video clip (422k), which has no sound. What's it about? (Use the Back button on your browser to return to this page.)

Now click here to play the same movie with sound (518k). Probably you got a very different interpretation of the intent of that movie.

Back to index

When Sound Is a Barrier

Sound provides a wealth of information. More than a medium of verbal communication, sound is a primary means by which the richness of human experience can be expressed. It is so pervasive that those of us who hear it take it for granted. Sound is taking that role on the computer as well. But for those with hearing impairments, for those whose computers don't have sound capability, and for people in contexts where other noise impairs the audibility of sound or where sound would be intrusive, sound becomes a barrier rather than a means of access and support. We need to convey auditory information in other ways to these people. And even when sound is accessible to someone, redundancy of media can reinforce or clarify ideas for all learners.

Back to index

Activity: Creating Alternative Representations for Sound

Creating alternative representations for sound poses many challenges. Using captions to transcribe speech seems straightforward. Conventions of punctuation convey some aspects of intonation and emotion. Yet many subtle cues even in dialogue may not be fully captured by punctuated text. Sound effects can be described by text (a crash, soft footfalls, a cat's meow) but the impact they carry may be lost. Sounds may be woven into a subtle background texture the effect of which could be spoiled by detailed enumeration and description. Music evokes different responses in different people and an interpretation is both subjective and limiting for the viewer who is then deprived of the opportunity to create his or her own interpretation. Then there is the problem of how to represent the feelings or information conveyed by the music. Departures from accepted conventions leave ambiguity about meaning, and decrease the utility of the alternative representation.

How does one provide alternative representations for sound? Activities designed to raise issues, to encourage Web site visitors to consider and work with them, and to present some of CAST's ideas on this question will soon be posted here.

Back to index

Tips on Presenting Alternatives to Sound

Some of the alternative ways in which spoken dialogues can be represented include:

• a text transcript
• Captioning, which is a transcript presented in synch with the speech, usually located at the bottom of the video screen
• Captioning using cartoon bubbles within the video clip
• Using newer captioning technology, it is possible to include images of the speakers alongside the verbatim captioning to clarify who the caption applies to
• Likewise, you can associate symbols and images that help enhance the meaning of the text
• Color codes can help convey affect
• Change in the size and shape of the captioned text can denote variation in tone of voice, e.g., a louder tone can be shown in bold

Some of the alternate ways in which sound effects can be represented include:

• Text equivalents of the sound ("beep-beep", "rrrrrring", etc.)
• Images along with the text, such as a picture of a telephone along with the "rrrrrring" text

Some of the alternate ways in which music can be represented include:

• Musical notations in an alternate track
• Images to convey the affect present in the music

Note:

• Not everyone shares the same symbolic language or codes, so be careful when using this means of access.
• The nature and extent of the information we provide is dependent on purpose and context. A simple click sound does not need an extensive description, but an emotional dialogue may need both a transcript/caption and additional cues to convey the emotion.
• Be as non-interpretative as possible when providing emotional cues. Different people can have surprisingly different interpretations, so you should try to describe the affect cues themselves (e.g., "speaking louder and quicker" may indicate anger or excitement), not provide the meaning of the cue.

Back to index