Reports from Program Research Studies

Understanding user perspective of the speed/accuracy/delay tradeoff for captioning fast-paced media

Accessibility Standards Canada: Summary Report

Closed captioning (CC) in Canada has made great strides since the 1990s but there remains much to do to meet the benchmarks of equity and inclusion set out in the Accessible Canada Act. In this project, the main goal is to examine the tradeoffs between speed, accuracy and delay that have been identified by people who are Deaf or Hard of Hearing (D/HoH) as their top three concerns for live broadcasts. Specifically, in this project, we focus on fast-paced sports as this content is particularly problematic because there are few opportunities for correcting errors. In addition, the advent of more reliable and accurate speaker independent Speech-to-Text (STT) and artificial intelligence (AI) may offer methods to resolve some of these issues. We report on three user studies 1) play-by-play (PBP) versus commentary-only (CO) for two fast-paced sports; 2) captioner workload; 3) user study of PAVOCAT, one software development tool resulting from these studies. Two additional user studies that are a continuation of the original four studies remain in progress as master’s theses.

The main findings are that: 1) captioner workload for fast-paced live events could be considered high and should be reduced; 2) captioning the CO portions of a live broadcast fast-paced sport instead the PBP announcing and the commentary seems promising as one solution to speed and accuracy issues for CC those games; 3) PAVOCAT that uses an AI system to generate live CC with captioner supervision seems to offer one solution to the delay and speed issues that exist with conventional CC although addition research and development is required to ensure that the system is robust and usable; and 4) captioners do not trust AI captioning.

The two remaining studies will investigate the acceptability of commentary-only CC over the long term, and the Deaf perspective on trustworthiness, believability, quality and comprehension of AI-generated CC compared with conventional CC for live, fast-paced broadcast content.

Download the report in English (PDF)

Download the report in French (PDF)