Intelligible encoding of ASL image sequences at extremely low information rates

https://doi.org/10.1016/0734-189X(85)90034-9Get rights and content

Abstract

American Sign Language (ASL) is a gestural language used by the hearing impaired. This paper describes experimental tests with deaf subjects that compared the most effective known methods of creating extremely compressed ASL images. The minimum requirements for intelligibility were determined for three basically different kinds of transformations: (1) gray-scale transformations that subsample the images in space and time; (2) two-level intensity quantization that converts the gray scale image into a black-and-white approximation; (3) transformations that convert the images into black and white outline drawings (cartoons). In Experiment 1, five subjects made quality ratings of 81 kinds of images that varied in spatial resolution, frame rate, and type of transformation. The most promising image size was 96 × 64 pixels (height × width). The 17 most promising image transformations were selected for formal intelligibility testing: 38 deaf subjects viewed 87 ASL sequences 1–2 s long of each transformation. The most effective code for gray-scale images is an analog raster code, which can produce images with 0.86 normalized intelligibility (I) at a bandwidth of 2,880 Hz and therefore is transmittable on ordinary 3 KHz telephone circuits. For the binary images, a number of coding schemes are described and compared, the most efficient being an extension of the quadtree method, here termed binquad coding which yielded I = 0.68 at 7,500 bits per second (bps). For cartoons, an even more efficient polygonal transformation with a victorgraph code yielding, for connected straight line segments, is proposed, together with a vectorgraph code yielding, for example, I = 0.56 at 3,900 bps and I = 0.70 at 6,000 bps. Polygonally transformed cartoons offer the possibility of telephonic ASL communication at 4,800 bps. Several combinations of binary image transformations and encoding schemes offer I > 80% at 9,600 bps.

References (81)

  • M.S Landy et al.

    HIPS: A Unix-based image processing system

    Comput. Vision Graphics Image Process.

    (1984)
  • M.S Landy et al.

    Vectograph coding: Efficient coding of line drawings

    Comput. Vision Graphics Image Process.

    (1985)
  • T Sakai et al.

    Extraction of invariant picture substructures by computer

    Comput. Graphics Image Process.

    (1972)
  • P Suetens et al.

    A semiautomatic digitization method and the use of spline functions in processing line drawings

    Comput. Graphics Image Process.

    (1981)
  • T Pavlidis

    Technique for optimal compaction of pictures and maps

    Comput. Graphics Image Process.

    (1974)
  • D Proffitt et al.

    Metrication errors and coding efficiency of chain-encoding schemes for the representation of lines and edges

    Comput. Graphics Image Process.

    (1979)
  • T.H Morrin

    Chain-link compression of arbitrary black-white images

    Comput. Graphics Image Process

    (1976)
  • U Ramer

    An iterative procedure for the polygonal approximation of plane curves

    Comput. Graphics Image Process

    (1972)
  • D Meagher

    Geometric modeling using octree encoding

    Comput. Graphics Image Process

    (1982)
  • Technical Staff, Bell Labs

    Engineering and Operations in the Bell System

    (1977)
  • E Klima et al.

    The Signs of Language

    (1979)
  • E.L Cohen et al.

    A New Dictionary of Sign Language

    (1977)
  • W.C Stokoe et al.

    A Dictionary of American Sign Language on Linguistic Principles

    (1965)
  • H.G Furth

    Thinking without Language: Psychological Implications of Deafness

    (1966)
  • J Jeffers et al.

    Speechreading (Lipreading)

    (1971)
  • G Sperling

    Video transmission of American Sign Language and finger spelling: Present and projected bandwidth requirements

    IEEE Trans. Commun.

    (1981)
  • D.E Pearson

    An experimental visual telephone for the deaf

    Television-J. Roy. TV Soc.

    (1978)
  • G Sperling

    Bandwidth requirements for video transmission of American Sign Language and finger spelling

    Science (Washington, D.C.)

    (1980)
  • D.E Pearson et al.

    An experimental visual telephone system for the deaf

    J. Roy. Television Soc.

    (1976)
  • D.E Pearson

    Visual communication systems for the deaf

    IEEE Trans. Commun.

    (1981)
  • H Poizner et al.

    Motion analysis of grammatical processes in a visual gestural language

  • G Johansson

    Visual analysis of biological motion and a model for its analysis

    Percept. Psychophys.

    (1973)
  • G Johansson

    Visual motion perception

    Sci. Amer.

    (1975)
  • H Poizner et al.

    Perception of American Sign Language in dynamic point-light displays

    J. Exp. Psychol.: Human Percept. Perform.

    (1981)
  • V.C Tartter et al.

    Perceiving sign language from an array of 27 moving spots

    Nature

    (1981)
  • D.E Pearson et al.

    Low data-rate moving-image transmission for deaf communication

  • J.F Abramatic et al.

    A narrow-band video communication system for the transmission of sign language over ordinary telephone lines

  • G Sperling et al.

    Image processing in perception and cognition

  • O.R Mitchell et al.

    Multilevel graphics representation using block truncation coding

  • D Marr et al.

    Theory of edge detection

  • Cited by (0)

    George Sperling, and Michael Landy are with the Human Information Processing Laboratory, Department of Psychology, New York University, 6 Washington Place, Room 980, New York, NY 10003. M. Pavel is with the Department of Psychology, Stanford University, Stanford, CA 94305. Yoav Cohen is with the National Institute for Testing and Evaluation, Jerusalem, Israel.

    View full text