PaperlayerView exact revision on arXiv

Source-linked AI summary

Online Sign Language Interpretation System

Christophe Ponsard, Jean-François Molderez

arXiv:2610.07872v1cs.CYcs.SE

TL;DR

The study addresses how to improve deaf people's access to administrations and public services by comparing alternative human-and-technology solutions. It treats remote human interpretation by videoconference as feasible, while full automation remains infeasible with the technology then available.

  • Problem

    The study asks how to improve deaf people's access to administrations and public services through combinations of human resources and technology.

    Source: §1 · p. 6

  • Method

    The study reviewed similar remote-interpretation services abroad, built a prototype, and tested technical and organizational requirements while examining alternative solution classes.

    Source: §1 · p. 6, §2 · p. 9

  • Results

    Remote videoconference interpretation was considered feasible with specifications, whereas complete automation was not feasible with current technology, though partial systems could fit specific circumstances.

    Source: §1 · p. 6, §2 · p. 9

  • Takeaways & Limitations

    Remote videoconference interpretation could improve interpreter availability by saving travel time and make emergency interpretation possible.

    Source: §2 · pp. 8–9

    Complete automation of interpretation is not feasible with current technology, although individual subsystems may work in specific circumstances.

    Source: §2 · p. 9

Abstract

from arXiv · show

The Walloon Region commissioned this study to improve deaf people's access to public services for the new millennium. It focuses on sign language users, about 25,000 adults in the French-speaking Community, for whom written French is close to a foreign language. The core problem is a severe shortage of interpreters: only about twenty work in French-speaking Belgium, so appointments take weeks to arrange and emergencies cannot be covered. The study assessed three options. The recommended one, remote video interpretation, could be deployed at the time of writing: a professional interpreter joins the deaf person and the employee by videoconference, saving travel time and enabling an emergency service. Similar services already ran in Sweden, Finland, the Netherlands and France. Prototype tests in May 2003 were a clear success and showed that interpretation needs at least CIF video above 256 kbps, ideally 384 kbps. The system suits simple administrative tasks better than emotional, medical or legal situations. The second option, fully automated interpretation, was not yet feasible, although partial systems using speech recognition and signing avatars could already serve kiosks, websites and routine procedures. The third option looks further ahead: UMTS and WiFi could eventually support mobile signed video calls and remote interpretation, provided quality of service is guaranteed. The main recommendation is a first pilot in real conditions before any wide rollout, supported by high-bandwidth links as part of e-government modernisation. The system is meant to relieve interpreters, not replace them, so their training, status and numbers should improve. The authors also call for digital training for deaf people, awareness among public-service staff, more sign language content online, and a multidisciplinary avatar project with Belgian, French and European partners.

1 Introduction

This section examines how to improve signing deaf people’s access to public services, outlining communication barriers and the linguistic status of sign language alongside its history and structure. It assesses remote interpretation, automated systems, and future mobile delivery, then considers implementation recommendations.

1 Introduction

  • The introduction frames the report around three options: feasible human videoconference interpretation, not-yet-feasible full automation, and prospective mobile services using high-speed wireless technologies.The report concludes with recommendations for short-term implementation.Source: §1 · p. 6

3 Domain Study

  • The domain review is representative rather than exhaustive, framing deafness around communication needs and the study’s focus on signing deaf people’s interactions with public services.It considers communication and stakeholder organisations to inform suitable solutions.Source: §2 · p. 7, §3 · p. 10

2 Problem Context and Methodology

  • With about twenty interpreters in French-speaking Belgium, signing deaf people can face weeks-long waits and uncovered emergencies; videoconference interpretation could save travel time and improve availability.The study identifies interpreter availability as its main problem and presents remote interpretation as a possible response.Source: §2 · pp. 8–9
  • The study uses KAOS goal-oriented analysis to derive technical and organisational requirements, and tests a prototype to refine them.It also examines alternative solutions; full automation is not feasible with the technology available, though some automated subsystems may work in specific circumstances.Source: §2 · pp. 8–9

3.1 Medical Aspects of Deafness

  • Medical aids can reduce hearing impairment, but none restores intact hearing, making communication support essential to consider.The chapter treats the deaf person’s existing means of communication as the focus rather than assuming medical solutions resolve the problem.Source: §3.1 · pp. 12–13

3.2 Consequences of Deafness

  • Deafness creates communication difficulties that can complicate daily activities and personal development, with impacts depending partly on when it occurs.The report estimates that about 25,000 people in the French Community are concerned by sign-language use.Source: §3.2 · p. 13

3.2.1 Consequences of Prelingual Deafness

  • Prelingual deafness can lead to language impairment, delayed cognitive development, social withdrawal, emotional disorders, and longer-term social and cultural exclusion.For people who are prelingually deaf, sign language is a natural language when they have the opportunity to acquire it.Source: §3.2.1 · p. 14
  • Researchers show that early sign-language learning supports later intellectual learning and French mastery, making French/sign bilingualism essential; many deaf adults remain functionally illiterate after oralist education.The passage estimates that about 95% of deaf children are born into hearing families not equipped to pass on sign language.Source: §3.2.1 · p. 14

3.2.2 Consequences for Late-Deafened People

  • For late-deafened people, losing the ability to use the telephone marks a functional boundary of deafness, and lip-reading is ambiguous and depends on visual access to the speaker.Communication difficulties can affect work, safety, radio listening, and telephone use, and may contribute to job loss or depression.Source: §3.2.2 · p. 15

3.2.3 Deafness and Disability

  • The recommended minimum on Belgium’s assistance index is level 2—written main information plus diagrams or illustrations—because signing deaf people may struggle with French.Deafness may be concealed or denied, complicating recognition of support needs.Source: §3.2.3 · pp. 15–16

3.3 Sign Language (SL)

  • Sign language is a full, non-universal language with regional variants and registers; it is rapidly evolving and needs reunification and enrichment.The passage notes that French Belgian Sign Language and French Sign Language are related, and that signers can adapt to one another’s languages for everyday conversation.Source: §3.3 · p. 17

3.3.1 Brief History

  • After a nearly century-long ban that impoverished and fragmented sign language, a revival followed the cultural-diversity movement after 1968.The passage traces the ban to oralist schooling and the Milan congress of 1878.Source: §3.3.1 · p. 18

3.3.2 Syntactic and Grammatical Components of Sign Language

  • Signs combine handshape, orientation, location, movement, and facial expression in parallel, while signers use space and gaze to convey grammatical relations.The passage also notes that sign order is less important than visual arrangement in space, with time and place adverbials generally preceding the verb.Source: §3.3.2 · pp. 18–19

3.3.3 Sign Language vs Other Forms of Gestural Communication

  • Sign language should not be confused with other forms of gestural communication.Source: §3.3.3 · p. 19

A. Fingerspelling

Fingerspelling spells alphabet letters through simple, static handshapes.

A. Fingerspelling

B. Signed oral language

This section distinguishes sign-language communication from French-ordered signing and other visual supports, then surveys broadcast and physical media. Each medium offers different access or learning benefits, alongside specific constraints.

B. Signed oral language

  • Signed French maps signs to French words and word order, often omitting spatial and facial nuance and making sentences long, visually overloaded, and tiring.Source: B. Signed oral language · p. 20

C. Co-verbal gestures

  • LPC and AKA add phonetic visual information to speech but carry no meaning of their own; sign language can communicate without another modality.Source: C. Co-verbal gestures · p. 20

A. Broadcast media

  • Television sign-language access includes interpretation in an inset and programs made specifically for deaf people; subtitles also face the obstacle of written oral language.Source: A. Broadcast media · p. 21

B. Physical media

  • Sign language lacks a generally used written equivalent, while video and disc media offer access and navigation with limits in viewing, storage, quality, or dictionary coverage.Avatars promise compact descriptions and variable viewing angles and speed, but rendering facial expressions and specific signs remains a restriction.Source: B. Physical media · pp. 22–23

A. Communication with a hearing person

Communication with hearing people may require a bilingual intermediary, while remote signed communication was not operationally available in French-speaking Belgium. The study situates these needs among local organizations and defines a focused public-service scope.

A. Communication with a hearing person

  • When deaf and hearing people lack a common language, a professional interpreter offers neutrality, accuracy, and confidentiality, but only about twenty interpreters serve French-speaking Belgium.Interpretation is mentally exhausting and quality degrades on average after about twenty minutes, so long interactions require interpreters to take turns.Source: A. Communication with a hearing person · p. 24

B. Remote communication

  • French-speaking Belgium had no operational telephone equivalent for remote sign-language communication, and no relay service, though SMS, fax, and other text options partly address access.Videoconferencing was not yet widely operational, and diverse communication tools complicate service organization while volatile Internet software may discourage investment.Source: B. Remote communication · pp. 24–25

3.4.1 Overview of the Deaf World in French-speaking Belgium

  • French-speaking Belgium has about 80 relatively independent deaf associations, making common advocacy and global projects difficult; the FFSB coordinates links among organizations.Source: §3.4.1 · pp. 25–26

3.4.2 Stakeholders Consulted

  • The study describes the bodies consulted and their skills, including named organizations and contacts in the stakeholder list.Source: §3.4.2 · pp. 26–27

3.5 Scope of the Study

  • The study focuses on adult signing deaf people and public-service accessibility, excluding education-specific issues, medical aspects, people mastering neither French nor sign language, and other communication-affecting disabilities.Source: §3.5 · p. 28

4 Requirements Analysis

The chapter uses KAOS goal analysis to organize needs and frame a public-service interpretation scenario. It examines interpreter scarcity and travel as barriers to timely service, and assesses remote human interpretation, partial automation, and mobile access in terms of their benefits, limitations, and requirements.

4.1.1 General Goals

  • Well-being is refined into education, autonomy, and access to culture and information, while integration goals include sign-language recognition, communication, and employment.The employment goal includes addressing deaf adults’ lack of qualifications through training.Source: §4.1.1 · pp. 30–31

4.1.3 Goals for Improving the Education of Deaf People

  • Education goals span early care and bilingual language development, school-age education with multidisciplinary support, and adult French-language and employment training.The education goal is outside this study’s scope, but is included for information; mainstream employment education with interpreter support is followed by only a small percentage of deaf adults.Source: §4.1.1 · p. 31, §4.1.3 · pp. 32–34

4.1.2 Communication Goals

  • Communication is identified as a major problem, especially with hearing people; interpretation and social services exist, but a relay service is absent, while remote sign-language communication lacks adequate means.Videoconferencing is proposed for remote interpretation and potentially wider use among deaf people; sign-language content dissemination is also poor.Source: §4.1.2 · pp. 31–32

4.1.4 Goals for Access to Culture and Media

  • Access to culture and media requires adapted content and dissemination, but subtitling can be costly and poorly suited to signing deaf people; online video with synthetic characters is proposed, and deaf people should help validate or produce physical-media content.The cited workload for subtitling is estimated at 40 times broadcast time, and cost constraints prioritize important subjects such as health information.Source: §4.1.4 · pp. 35, 37
  • For the CFLS, online tools could support more efficient sign-language unification meetings and community participation, while an online lexicon faces expression, access, and pricing obstacles.Virtual characters and Internet distribution are presented as alternatives for disseminating sign-language content.Source: §4.1.4 · pp. 35–36

4.1.5 Non-functional Requirements

  • Implementation requirements include personalisation, real-time performance, affordability, reliability, interoperability, and user anonymity and privacy.Interoperability favors standards that avoid manufacturer lock-in and help equipment work together and remain durable.Source: §4.1.5 · p. 37

4.2.1 Communication Situations

  • Communication situations range from administrative and legal to medical, cultural, professional, and private; relying on relatives can risk inaccurate or altered messages, poor confidentiality, and reduced autonomy, motivating professional interpretation.Professional interpreters are described as ensuring accuracy, neutrality, and confidentiality.Source: §4.2.1 · p. 38
  • Deaf users receive reimbursed interpretation-hour tickets, but the allowance for professional needs is often insufficient and users may exchange tickets.The communication scenario is represented as a sequence of interactions among system users.Source: §4.2.1 · pp. 40–41, Figure 26 · p. 40

4.2.2 Analysis of an Interpretation System

  • About twenty interpreters serve French-speaking Belgium, creating waits of several weeks; travel, urgent requests, and regional shortages make emergencies impossible to handle.The analysis identifies interpreter training and improved working conditions, including recognition of their status, as needed improvements.Source: §4.2.2 · p. 41

4.2.3 Solution 1: Remote Sign Language Interpretation System

  • Remote interpretation could reduce waiting by eliminating interpreter travel, improve working conditions, and enable an on-call service for emergencies.The system captures and transmits images and speech between the deaf person, interpreter, and hearing person.Source: §4.2.3 · p. 42
  • Remote interpretation raises concerns about facing the hearing person alone, image quality, technology-related intimidation, and camera confidentiality; it is best suited to routine, neutral tasks.It is not recommended for crisis, emotional, medical, or legal situations, although urgency may justify its use.Source: §4.2.3 · p. 42

4.2.4 Solution 2: Automated Interpretation System

  • Fully automated interpretation is not yet capable of replacing an interpreter, requiring speech and gesture recognition, translation, synthetic signing, and speech synthesis.The component techniques vary in maturity, and oral-language translation systems are also far from effective.Source: §4.2.4 · p. 43
  • A phrase-based signing avatar has been validated for simple post-office procedures, but it lengthens transactions and mainly serves as a fallback when improvised communication fails.Source: §4.2.4 · p. 43

4.2.5 Solution 3: Extension to Mobility

  • Mobile interpretation could let deaf people contact interpreters from any WiFi HotSpot using their own portable camera-equipped terminals, provided the infrastructure guarantees suitable quality.The proposed terminals include PDAs or tablet PCs, and suitable infrastructure could reduce communication and equipment costs.Source: §4.2.5 · p. 44

4.2.6 Summary of the Operational Alternatives Considered

  • The three alternatives draw different boundaries between automation and human actors, combining different agents and technologies to meet remote interpretation needs.Source: §4.2.6 · p. 44, Figure 28 · p. 45

5 Sign Language Video Interpretation

This chapter specifies a sign-language videoconferencing system and reviews the technical constraints that shape its design. It proposes requirements and describes prototype tests to validate them.

5.1 General Principles

  • Remote interpretation keeps the deaf and hearing participants together while linking the deaf person to a remote interpreter by video and the hearing person by audio.The video must preserve hand movements and facial expressions.Source: §5.1 · p. 46
  • Video relay extends this setup by contacting the hearing person by telephone; the system can also serve oral interpreters and direct calls between signing deaf people.It transmits auditory and visual information without interacting with its content, so it can support different oral and signed languages.Source: §5.1 · pp. 46–47

5.2 Videoconferencing

  • Videoconferencing transports voice and moving colour images in real time between people who are far apart.Source: §5.2.1 · p. 47

5.2.1 General Presentation

  • Each site needs cameras, microphones and a codec to digitize, compress, multiplex and decode signals; this study restricts its design to point-to-point calls.Multipoint videoconferencing requires a bridge to exchange streams between sites.Source: §5.2.1 · p. 48

5.2.2 Digital Networks

  • Symmetric bandwidth and guaranteed quality of service matter because weak upstream limits video quality, while drops and interruptions can undermine use and acceptance.Source: §5.2.2 · p. 49

5.2.3 Videoconferencing Standards

  • H.320 targets ISDN and quality-of-service networks up to 2 Mbps, H.323 reasonable-rate Internet, and H.324 56K modem links; these umbrella standards support interoperability.Source: §5.2.3 · p. 50

A. Audio quality (G.7xx)

The section reviews videoconferencing codecs, terminals and software, then surveys existing remote-interpretation systems and their technical requirements. Available services and evaluations illustrate both workable arrangements and constraints on video quality and use.

A. Audio quality (G.7xx)

  • 12 frames per second is the stated minimum for interpretation, while image quality must preserve small features such as fingers and gaze.H.261 and H.263 trade image quality against frame rate as bandwidth varies; CIF is 352x288, whereas QCIF is 176x144.Source: B. Video quality (H.261 and H.263) · pp. 50–51

5.2.4 Videoconferencing Terminals

  • Small-screen all-in-one terminals are poorly suited to remote interpretation, and existing H.324 products cannot guarantee good-quality video over narrow telephone-line bandwidth.The cited Beamer system offered QCIF at an average of 2 to 5 frames per second.Source: §5.2.4 · pp. 51–52

5.2.5 Videoconferencing Software

  • Videoconferencing software is generally H.323-oriented, and many packages also support T.120 collaboration features such as chat, whiteboards and file transfer.Some packages can use H.320 with ISDN cards; software designed for deaf users is discussed alongside existing services.Source: §5.2.5 · p. 53

5.3.1 Existing Systems

  • The review describes remote services or projects in Sweden, Finland, France and the Netherlands, including a public Swedish service, a Finnish deployment, France’s eSourds service and Dutch experiments.Sweden’s service handled about 400 calls per month (1998 figures); the review aims to give a representative view, not an exhaustive one.Source: §5.3 · p. 53, §5.3.1 · pp. 54–55

5.3 Existing Systems and External Evaluations

  • Existing evaluations describe a quality trade-off: single ISDN can be tiring, while triple ISDN offers better conditions at higher cost and setup complexity.Remote interpretation is used for public-service contacts, but this review excludes other application areas beyond the study’s scope.Source: §5.3.1 · pp. 55–56

A. ACE study (Australia)

The ACE study found broad support for remote interpretation, while also identifying a preference for in-person service providers and recommending protocols to support effective three-way interaction.

A. ACE study (Australia)

  • All participants supported remote interpretation, particularly because it enabled more natural interaction than text relay, including emotional expression.The evaluation involved 19 deaf participants, 6 Auslan interpreters and 26 hearing service-provider representatives.Source: A. ACE study (Australia) · p. 57
  • Participants preferred the service provider’s physical presence for conversation pacing and human contact, but saw video relay as useful where providers are absent.The preference was especially evident for medical interventions.Source: A. ACE study (Australia) · p. 57
  • The study recommended protocols for pacing, interpreter conduct, seating and physical environment, plus introductions and guidance for service providers.Recommendations also covered a warm-up exchange and interpreter replacement if either party wished to stop.Source: A. ACE study (Australia) · p. 57

B. "SignWorks" evaluation, Bristol (UK)

Evaluations report workable but effortful videophone interpretation, with image quality and interaction arrangements shaping its use. The accompanying design requirements address welcoming, premises, interpreter organization and communication quality.

B. "SignWorks" evaluation, Bristol (UK)

  • SignWorks found that sign-language conversations were possible but took about 50% longer, and fingerspelling made small-screen all-in-one videophones problematic.The hearing and deaf person together with a remote interpreter was the most popular of the tested configurations and was selected for this study.Source: B. "SignWorks" evaluation, Bristol (UK) · pp. 57–58

C. Evaluation by the State of Pennsylvania (USA)

  • The Pennsylvania evaluation found traditional telephone links inadequate, single ISDN acceptable and triple ISDN more comfortable.Medical-field implementation was envisaged.Source: C. Evaluation by the State of… · p. 58

D. Omnitor evaluation (Sweden)

  • For medium-shot framing, fingers and eyes imply CIF (352x288); QCIF (176x144) distinguishes fingers but not gaze. Fingerspelling at sometimes 7 signs per second, with 3 frames per sign, requires 21 frames per second.Lower rates cause signers to slow down and lead to frustration; 12 frames per second is the minimum.Source: D. Omnitor evaluation (Sweden) · p. 58
  • Omnitor identifies motion blur as a barrier to perceiving handshapes and reports that delays above 1.2 seconds become irritating.Good cameras and lighting help avoid blur; delays below 0.5 seconds are not really noticeable.Source: D. Omnitor evaluation (Sweden) · p. 58

5.4 Requirements for the Design of a Remote Interpretation System; A. Welcoming the deaf person; B. Familiar interpreter; C. Preliminary exchange

5.4.2 Organisation of the Premises in the Public Service

  • Public-service counters can hinder video and audio transmission, privacy and three-way seating, so a private room with a triangular arrangement is preferable.Lighting, sound insulation and a seated arrangement around a table are among the suggested room characteristics.Source: §5.4.2 · p. 60

5.4.3 Organisation of the Interpretation Centre

  • An interpretation centre can scale from one station, but scheduling and call redirection may be needed as demand or simultaneous calls increase.Emergency interpretation is described as a possible extension requiring further examination with potential partners and outside the study’s scope.Source: §5.4.3 · pp. 60–61

5.4.4 Communication Quality

  • Remote interpretation inevitably degrades communication, adding noise that makes interpreters’ simultaneous tasks harder and accelerates fatigue.The design therefore seeks to minimize these communication-quality losses.Source: §5.4.4 · p. 61

A. Communication infrastructure

The design specifies communication, camera, image, display, audio and collaboration requirements for remote interpretation. Validation used controlled prototype tests with throttled bandwidth, but did not include real-network trials.

A. Communication infrastructure

  • Triple ISDN is recommended for bandwidth and quality of service, while broadband Internet is considered if providers guarantee more than 200 Kbps.The study describes the triple link as the price of interpreter comfort.Source: A. Communication infrastructure · p. 61

B. Use of the camera

  • A fixed camera can suffice if the parties choose the viewing angle and framing together at the session’s start.A slightly oblique view supports sign observation, while a frontal view can simulate eye contact; showing the employee is optional.Source: B. Use of the camera · pp. 61–62

C. Image

  • The image requirements specify a 12-frames-per-second minimum and QCIF minimum resolution, with CIF recommended and delay capped at 1.2 seconds.A delay below 0.5 seconds is recommended; lighting, contrast and a plain background also support clear images.Source: C. Image · p. 62

D. Screen size and type

  • A small all-in-one screen is insufficient; a 17-inch LCD offers a restful image, while televisions are economical but can be more tiring.Video projectors are expensive and require subdued lighting that conflicts with camera needs.Source: D. Screen size and type · pp. 62–63

E. Audio transmission

  • Open audio at the public service suits employees without headsets and is needed when several hearing people participate or the deaf person has residual hearing.Open microphones can cause echo or feedback; an interpreter-side headset microphone is preferred for listening comfort and isolation.Source: E. Audio transmission · p. 63

F. Additional means of communication

  • Additional communication tools can support spelling names, drawing maps and sharing forms, and a deaf-friendly total-communication interface is recommended.The interoperability of optional collaboration functions should be checked; a special camera is recommended for displaying documents.Source: F. Additional means of communication · p. 63

5.5 Validation Tests; 5.5.2 Test Scenarios

  • Validation tests placed deaf, hearing and interpreting participants in typical administrative scenarios using a prototype with controlled parameters.The planned scenarios involved issuing a passport and a recruitment discussion; tests took place at the FFSB premises in May 2003.Source: §5.5 · p. 64, §5.5.2 · p. 66

5.5.1 Hardware Infrastructure Used

  • Tests used laptop stations and bandwidth throttling on a private 100 Mbps network, but planned trials over available cable and ADSL connections were not conducted.The laptops used cameras exceeding one million pixels, and could simulate set-top-box and all-in-one terminals through their screens and TV output.Source: §5.5.1 · pp. 64–66

A. Scenario 1: passport application

Prototype videoconferencing supported realistic administrative and job-interview scenarios, but usable interpretation required high-quality video and bandwidth. The paper contrasts this direct human-mediated approach with automated translation, whose language-processing components were substantially more complex.

C. Implementation

5.5.4 Lessons Learned

  • All test scenarios were completed without immediate rejection or insurmountable difficulties, and participants judged them fairly realistic.The test used passport-application and job-interview scenarios.Source: A. General overview · pp. 67–68

A. General overview

D. Minimum quality required

  • CIF video above 256 Kbps, preferably 384 Kbps, was required for interpretation; 128 Kbps QCIF was not recommended.Attempts to reduce image size or bandwidth quickly caused communication problems.Source: D. Minimum quality required · p. 69

6.1 Deaf ↔ Hearing Communication Chain

  • Automated communication must translate meaning between different language modalities, whereas videoconferencing transmits the raw messages with minimal degradation.Hearing-to-deaf translation may require speech recognition, sentence-level semantic analysis, and visual sign animation; the reverse direction requires sign recognition and translation into French.Source: §6.1 · p. 71

6.2.1 Synthesis of Sign Language Content

  • Video-based sign output sequences recorded elementary signs, but needs a same-signer dictionary and cannot flexibly reproduce natural sign-language variation.The paper describes it as more usable for signed oral language; avatar control commands, by contrast, require far less transmitted data than corresponding video.Source: §6.2.1 · pp. 71–72
  • Sign-language animation requires real-time characters with fine hand movements and facial expressions, which many existing avatars lack.Motion capture reproduces recorded movements, while inverse kinematics compactly describes articulated motion; facial displacement nodes can simulate expressions.Source: §6.2.1 · pp. 72–73

6.2.2 Speech Processing Technologies

  • Speech processing includes recognition, synthesis, and coding, with only recognition and synthesis treated as relevant to the study.The paper distinguishes recognisers by speaker and utterance type, while synthesisers produce artificial speech from digital or symbolic input.Source: §6.2.2 · p. 75

A. Speech synthesis

The review finds that speech synthesis and constrained speech recognition had practical uses, while sign recognition and machine translation remained major obstacles. Partial systems could serve stable-speaker settings by restricting the communication task or avoiding full translation.

A. Speech synthesis

  • Speech synthesis from large speech databases had advanced to voices described as almost indistinguishable from human speech.The paper names BrightSpeech from BabelTech as an example of unit-selection synthesis.Source: A. Speech synthesis · p. 75

B. Speech recognition

  • Single-speaker word recognition could achieve almost perfect results, but large-vocabulary, multi-speaker recognition remained error-prone and vulnerable to noise.These systems use statistical acoustic models, including hidden Markov models or artificial neural networks.Source: B. Speech recognition · p. 76

6.2.3 Sign Language Recognition

  • Gesture recognition was not fine enough for sign-language recognition, despite progress in interactive gesture-based applications.The paper also describes real-time facial-expression recognition as emerging technology that could convey visual communication over narrow bandwidths.Source: §6.2.3 · pp. 76–77

6.2.4 Machine Translation

  • Machine translation remained largely unsolved, and sign language was especially difficult because it differs from oral languages and was poorly described.The paper identifies signed French and pre-encoded standard sentences as ways to avoid full translation.Source: §6.2.4 · pp. 77–78

6.3.1 General Possibilities

  • Deaf-to-hearing visual recognition and translation were not feasible, while hearing-to-deaf systems could rely on stable speakers, signed French, or pre-encoded phrases.The study treats signed French as a second-best solution that deaf people nevertheless accepted.Source: §6.3.1 · p. 78

6.3.2 Partial Interactive Communication Systems

  • Partial interactive systems assume the return channel is unproblematic or less important, and speaker-dependent recognition requires repeatedly encountered, trained speakers.The paper gives education and routine administrative procedures as settings where the same teachers or employees recur.Source: §6.3.2 · pp. 78–79

A. TESSA: Text and Sign Support Assistant (ViSiCAST project)

TESSA uses speech recognition and a signing avatar to support predictable post-office transactions, while iCommunicator offers speech-to-text support rather than sign-language visualization in the tested prototype. Both examples illustrate the constraints of partial, task-specific systems.

A. TESSA: Text and Sign Support Assistant (ViSiCAST project)

  • TESSA supports simple, predictable post-office transactions by converting a clerk's speech into an avatar animation and accompanying text.The computer recognizes a sentence, assembles its corresponding animation, and displays it to the customer.Source: A. TESSA: Text and Sign Support… · pp. 79–80
  • TESSA completed transactions but doubled the time required compared with leaving the deaf person without an interpreter; it was still judged useful as a fallback.Its limited key-sentence set makes interactions rigid and vulnerable to unforeseen events.Source: A. TESSA: Text and Sign Support… · p. 80

B. iCommunicator

  • The iCommunicator prototype displayed recognized speech as text, but the tested version had no sign-language visualization and was unsuitable for signing deaf people.Recognition quality degraded quickly when the favorable test conditions were not met.Source: B. iCommunicator · p. 82

6.3.3 Information Systems in Sign Language

  • Sign-language information systems include dictionaries, content editors, and real-time broadcasting, but require tools to generate sign-language content easily.The paper identifies this content-rendering chain as posing no major problem in itself.Source: §6.3.3 · p. 83

A. Sign language dictionaries

The reviewed applications include sign dictionaries and content systems, avatar-based broadcasting, lip animation, and assisted subtitling. For public services, the authors identify sign-language explanations and limited routine interactions as practical uses.

A. Sign language dictionaries

  • The Sign Smith Illustrated Dictionary uses a synthetic character, while other cited dictionaries rely on video.Its interface includes controls for signing speed and viewing angle.Source: A. Sign language dictionaries · p. 83

B. Sign language content management systems

  • LinkIt synchronizes text and sign-language video for web-like content, but uses a proprietary format requiring a special viewer.The paper notes that it was intended more for content distributed on CD-ROM.Source: B. Sign language content management… · p. 83

C. Real-time sign language broadcasting system

  • The Visia television concept would convert subtitles into avatar animation, but its translation component was not considered realistic for generating sign language.A proposed first phase instead captures a real interpreter's movements, with about 15 Kbps of data to transfer.Source: C. Real-time sign language… · p. 85

A. Automatic lip synthesis

  • SynFace phoneme recognition was reported to improve communication by 20%, while remaining 18% below a human face.The system used two recognition methods, HMM and ANN, and did not appear to add information for distinguishing ambiguous phonemes.Source: A. Automatic lip synthesis · p. 86

B. Semi-automatic subtitling

  • Live subtitling at VRT uses a trained-voice recognizer, with a second person checking and correcting the text before transmission.The system had an acceptable error rate of a few percent and was described as useful for news, debates, and football matches.Source: B. Semi-automatic subtitling · p. 87

6.4 Possible Uses in Public Services

  • Public services could offer sign-language explanations about procedures, required documents, and forms through local kiosks or online access.A system similar to TESSA could also support familiar, simple procedures.Source: §6.4 · p. 87
  • The CFLS database contains about 9,000 unified signs and could support online or CD-ROM content and collaborative sign unification.The authors also suggest enriching the linguistic knowledge base toward sign-language search tools.Source: §6.4 · p. 88

7 Towards Greater Mobility

The chapter reviews mobile network technologies, terminals, and services that could support sign-language communication. It distinguishes mobile real-time calls from deferred messages and information access, with service quality and terminal constraints shaping what is practical.

7.1 Infrastructure for Mobile Communications

  • UMTS could support real-time video telephony through its circuit domain at 384 Kbps, while WiFi currently offers no quality-of-service guarantee for conversational applications.UMTS deployment is described as delayed, and its coverage model may provide lower bit rates in rural areas and has an inside-building coverage problem.Source: §7.1.1 · pp. 89–90, §7.1.3 · p. 92

7.1.2 WiFi

  • WiFi is designed for nomadic access at hotspots, but Internet quality-of-service limits and support for simultaneous connections constrain conversational use.Indoor coverage works well, although short ranges may require several access points.Source: §7.1.2 · p. 91

7.1.3 Comparison

  • UMTS and WiFi are complementary rather than competing standards, serving different coverage niches through wide-area and local wireless networks.The comparison passage identifies WLAN with WiFi and WWAN with UMTS.Source: §7.1.3 · p. 91, Figure 65 · p. 92, Table 8 · p. 92

7.2 Portable Terminals

  • Mobile terminals could display real-time or deferred sign-language video, but signing requires a suitable screen and stable camera positioning that permits both hands to be used.The Nokia 3650's QCIF video test had a fairly low estimated frame rate of 10–15 frames per second.Source: §7.2 · pp. 93–94

7.3 Service Offering

  • Mobile sign-language services could include real-time calls, deferred signed messages, information servers, and remote interpretation or emergency relay access.Real-time calls depend on UMTS meeting its quality-of-service specification; streaming or download is sufficient for deferred messages and information access.Source: §7.3 · p. 94
  • The WISDOM project considered sign-language input for navigation, and a system already recognized continuous sentences from a restricted 152-sign vocabulary with 91.1% accuracy.The 152-sign recognition system is reported as current research, not as a deployed WISDOM service.Source: §7.3 · p. 95

8 Conclusions and Recommendations

The conclusions recommend combining human remote interpretation with selective automated services and longer-term mobile options. They also call for a real-world pilot, stronger support for interpreters and users, more sign-language content, and collaborative avatar research.

Remote interpretation

  • Remote videoconference interpretation is the most realistic option considered and can improve public-service access, reduce interpreter travel, and support emergency service.The study cautions that the system is not suited to every situation.Source: §8 · p. 96

Automated interpretation

  • Full machine translation was judged unrealistic as a whole, although partial systems combining speech recognition and synthetic signing could serve selected public-service tasks.Suggested uses include explanatory information, known employee communications, and simple familiar procedures; these systems can complement rather than replace remote interpretation.Source: §8 · p. 96

General recommendations

  • Deaf people should receive support and training to use videoconferencing terminals and multimedia websites, including possible financial help for equipment and Internet access.The recommendations suggest working through deaf associations.Source: §8 · p. 97
  • Public-service staff and other hearing people should be made more aware of deaf people's communication needs and trained to use available communication options.The recommendations also call for continuing public-awareness actions.Source: §8 · p. 97

Remote interpretation

  • Remote interpretation is intended to relieve interpreters rather than replace them, so the authors recommend improved training and status recognition and a significant increase in their numbers.The proposed system reduces tiring travel, addressing interpreter workload and availability.Source: §8 · p. 97
  • A small-scale pilot under real conditions should precede costly broad deployment, with high-speed links developed as part of wider infrastructure or e-government modernization.The pilot is intended to validate and refine the proposed organization; remote interpretation is not applicable in all situations.Source: §8 · pp. 97–98

Sign-language content

  • Websites for deaf people should offer more sign-language material, and the French WebSourd project should be followed for possible collaboration.The source describes WebSourd as expected to become operational soon.Source: §8 · p. 98

Avatar development

  • The authors recommend a multidisciplinary avatar project because current sign-language avatars have limitations in expressiveness and content creation and make too little use of sign-language structure.The proposed team would combine computing, linguistics, and deaf-association expertise.Source: §8 · p. 98

10 Appendices

The appendices include a glossary of technical and sign-language terms and a survey question about which public services should receive equipment. The supplied material also identifies the survey's response limit.

10.1 Glossary and Acronyms

  • The glossary covers communication and video technologies, including FPS, GPRS, H.323, WiFi, and UMTS.It also gives definitions of transmission-rate and video standards.Source: §10.1 · pp. 103–105
  • The glossary defines sign-language and hearing-related terms, including sign language, kineme, deaf person, and hard-of-hearing person.It notes that sign language has its own grammar and syntax, varying between countries and regions.Source: §10.1 · pp. 104–105

10.2 Results of the Survey Conducted by the FFSB

  • The survey asks which listed administrative or public services should be equipped with the system, allowing up to three choices.Source: §10.2 · p. 106

B. Absence of the interpreter:

The supplied appendix material describes a companion CD-ROM containing study resources and explains how to open it. It lists the main file formats used for the included materials.

B. Absence of the interpreter:

  • The companion CD-ROM contains cited videos, reports, research articles, demo software, and an electronic copy of the report.Source: §10.3 · p. 107
  • The CD-ROM can be opened in a web browser by opening its root index.html file, and it should start automatically on Windows.Source: §10.3 · p. 107
  • The listed media formats are RealVideo for videos and PDF for articles and reports.Source: §10.3 · p. 107
Loading 2610.07872v1…