Objective

By using different kinds of real-time captioning tools in the market, we aim to enhance the learning accessibility of students with different types of Special Education Needs (SEN), and reduce their barriers to classroom participation.

What is real-time captioning?

Real-time captions is a synchronous textual representation of the audio within media , which is generated by the engine through automatic speech recognition. This kind of system can enhance information accessibility of people with hearing impairment or people who experience difficulties in perceiving speech information in class, no matter in a face-to-face lecture or an online lesson.

Why captions?

In the academic year 2020-21, with the support of TDLEG, The Centre for Sign Linguistics & Deaf Studies (CSLDS) has conducted a 1-year pilot project to investigate whether real-time captioning is beneficial to students with special education needs. Six students with different kinds of special needs participated in the trial of real-time captioning in the lessons they chose and gave feedback on the service provided after trials. According to the feedback received, all of them found the system helpful in enhancing learning accessibility in class though the captions produced were not 100% correct. The results showed that not only students with hearing impairment (especially those with mild to moderate hearing loss), but also students with other types of SEN such as students with attention deficit benefitted from the real-time captioning service in the lecture. Students replied that they would like to continue using this service in the future and recommend other students with similar special educational needs to try using the system in the lessons.

Benefits of real-time captions:

• Enable students to track back information which they have missed in the real time learning process.
• Improved classroom engagement due to reduced information loss during the lessons, resulting in better online and offline classroom experience.
• Improved information retention and better concentration of students in class when both visual (captions) and oral mode of instructions are provided.

Limitations of real-time captions:

• Accuracy of real-time captions done by stenographers can reach almost 100% accuracy, but it is costly and hard to find service providers locally.
• Real-time captions provided in the service are generated through automatic speech recognition, which is much cheaper, but no 100% accuracy can be expected.
• Mainly acts as a supporting tool for deaf and hard-of-hearing students. However, other supporting services including sign language interpretation and note-taking service are still necessary for students with severe to profound hearing impairment, to reduce information loss in the classrooms.
• Each real-time captioning tool/ mobile application has its limitations (e.g. language use, display method, control system, etc.)

How to get captioning support for your lessons?

There are different methods of captioning used for lessons with different modes of delivery: face-to-face or online lessons (ZOOM).

1) Online lessons

Auto-captioning as a built-in function can be found in most of the online meeting applications such as Google Meet, Microsoft Teams, and ZOOM and the operation is simple. As ZOOM is a popular online teaching system. The procedure to switch on the auto-captioning function of ZOOM is shown in the short video below.

Once the function is on, all participants in the ZOOM meeting room (i.e. students) can see the captions generated automatically from the system.

2) Face-to-face lessons

With some funding support this year, The Centre for Sign Linguistics and Deaf Studies will continue to provide real-time captioning to students with special education needs in order to alleviate their difficulties in class. The Centre continues to have a close contact with the Office of Student Affairs. In case students would like to receive real-time auto-captioning, they can contact the Centre or the Office of Student Affairs. We will meet with the students to understand the learning situation or difficulties first. With the consent of the course teacher, we make arrangements for the students to have a trial of the system in class. The students can borrow our wireless microphone (see Figure 1) and tablet for the captioning support if necessary. The use of wireless microphone is to make sure that the speech of course teachers received is intelligible enough, to increase the accuracy of the captions.

The wireless microphone will be kept by the student his/herself. He or she is responsible to pass the microphone to the course teacher every lesson. Course teachers can simply wear the wireless microphone in the lectures for sound input. On the other hand, the students have to connect the receiver of the microphone to their electronic device (e.g. mobile phone or tablet). Teachers’ speech will be transcribed to written text and generated through the system for the students. The students can further contact the Office of Student Affairs for long-term usage if they are satisfied with the quality of the captions.

Fig. 1 Wireless Microphone Set

Fig 2 Process of Applying and Using Real-time Captioning Service

A set of microphone with two transmitters is recommended to use in class. One transmitter is to pick up the speech of the course instructor during the lecture and the other one can be passed among students when there are Q&A sessions or group discussions.

Fig 3 Example Scenario of Using Two Transmitters

In face-to-face lessons, the setting is different from online lessons. There are many different auto-captioning applications in the market that students can choose to use as the tool for auto-captioning. Here is the information some of the applications we suggested to use:

Table 1 List of Real-time Auto-captioning Applications

Application Method

For more information, please contact us. (Email: cslds@arts.cuhk.edu.hk Phone: 3943 4178)