Math & Science Working Symposium
May 12 & 13, 1994
Conference Update -- George Kerscher, RFB
This update is to let you know about the developments of the
conference. There has been a very good response by those initially
invited. Many people have informed me about other researchers and
developers that should be invited and these invitations have gone
out. Some people have not responded as of yet, but we are trying
to determine who will be attending.
Update Table of Contents
Part 1 - Technical Considerations
Part 2 - Future Considerations
Part 3 - Thoughts as to an Agenda
Part 4 - List of confirmed persons attending
Part 5 - Conference and Accommodation Confirmation
Part 6 - Attachments
**** Part 1 Technical Considerations ****
I want to set a high, computer science, technical tone for the
conference. A traditional conference has presenters, and everybody
leaves the meeting aware of significant work in the field; the CSUN
conference is a fine example of this activity. The Math & Science
Working Symposium will build on the presentations of CSUN, but I
assure you that we intend this conference to be a working meeting
intended to launch the development of new systems with many groups
working together. We want to face the challenge of access to math
and science together, capitalising on everybody's strengths.
Dr. T.V. Raman's work at Cornell has demonstrated "Audio System for
Technical Readings" (AsTeR). The data structures he developed are
rich and powerful. He represents math in a n-ary tree. Each node
may have up to six additional dimensions, which may be a n-ary tree
itself. This system allows the user to have the links spoken
interactively. Following these links allows the user to hear the
math in ways never experienced before. We all must congratulate
Dr. Raman.
At the end of this update is an article from Siam and the abstract
of Dr. Raman's thesis. In addition, RFB would be pleased to send
you a cassette tape demonstrating his work. RFB will make this
tape available to you, but there is one major reservation. AsTeR
is a completely interactive system. The user controls how and what
is to be spoken. Much of the power comes from this interaction.
Of course, the cassette recording "freezes" this spoken math and,
you must understand, that the interaction is lost. We know the
limitations of recordings, but we feel it would be helpful to have
been exposed to Raman's work as background to the conference. (Send
E-mail to cbfb_gwk@selway.umt.edu or call Vicki at 406/728-7201 to
request the two track stereo recording.)
We believe that the types of data structures defined in Raman's
thesis can enable accurate transformations. Braille in any form,
Dotsplus, correct visual representations and spoken math can all be
extracted from the tree. A software program can be developed that
provides the underlying structure to modules that produce
simultaneous outputs. We want this conference to explore ways to
extend Raman's work.
We want the technical computer science oriented researchers, to
contribute to the discussions that are sure to arise. We are
looking for expertise necessary to make AsTeR and related systems
easily available on diverse computer platforms. Some people have
indicated that they want to observe and that their backgrounds will
not allow them to participate in the technical discussions. For
those lawmakers, teachers, and program directors in the field who
want to observe this conference we have made provisions for them to
observe Friday afternoon. We also will make the conference
proceedings available upon request.
**** Part 2 - Future Considerations ****
We want to investigate the feasibility of a flexible interactive
reading and writing system for math that incorporates braille,
correct visual layout and spoken math. Such a system should
support new developments such as a tactile refreshable tablet.
Investigation in this direction should concern itself with
practical matters of electronic file manufacturing, availability,
copyright issues, and delivery systems. This direction may serve
to bring a wide variety of groups together in a development effort.
**** Part 3 - Thoughts as to an Agenda ****
Thursday
Introductions Several minutes with each researcher
explaining their affiliations and brief
summary of their work. (3 minutes each)
AsTeR
Presentation Dr. Raman will explain AsTeR
Extending Raman's Work
Panel discussion Dr. Rich Cox, Chris Brooks, George
Kerscher, Steve Edwards, Dr. Raman. This
discussion will focus on: technical
considerations, problems, kinds of
activity, skills, and try to put
dimensions around what is to be done
Lunch
The Big Picture
Round Table All researchers and developers will be
asked to share in how their work:
overlaps, compliments, extends and ties
in with extending access to math and
science. Individuals should be prepared
to answer detailed questions from the
participants. We are very interested to
hear about the activities of the European
Maths project.
Friday
Brainstorming
Round table What is worth doing? How should it be
done? What would I like to do? Next
steps!
Lunch
Observers Those observers now joining the
conference at this point will be
introduced.
Remarks by Ritchie Geisel, President of RFB
Summary Description of our conclusion of what
needs to be done. Raman and selected
others
Follow up discussion Organization development, action items.
**** Part 4 - List of confirmed persons attending ****
Dr. T.V. Raman, DEC Research Analyst
Dr. Caryn Navy, Raised Dot Computing
Mr. David Holladay, Raised Dot Computing
Mr. Joseph Sullivan, Duxbury Systems, Inc.
Dr. Norberto Salinas, University of Kansas
Prof. John Gardner, Oregon State University
Dr. Abraham Nemeth, Professor Emeritus, University of Detroit
Dr. James Thatcher, IBM Special Needs
Dr. Albert Blank, The College of Staten Island
Dr. Michael E. Kress, The College of Staten Island
Dr. Karen Luxton Gourgey, Center for the Visually Impaired, Baruch
College, City University of New York
Dr. William B. Woolf, American Mathematical Society
Ms. Barbara Freeze, Canadian National Institute for the Blind
Mr. Paul Clarke, Canadian National Institute for the Blind
Mr. Venkatesh R. Chari, Technology For Independence, Inc.
Mr. John Cookson, National Library Service for the Blind and
Physically Handicapped, Library of Congress
Dr. William Barry, Oregon State University
Dr. Gerhard Weber, University of Stuttgart
Mr. Lar Kaufman, Polymedia Services
Ms. Lynne M. Schmelz, Harvard University
Dr. Charles Halperin-Hamu, InfoDesign Corporation
Prof. Norman Coombs, Rochester Institute of Technology
RFB Participating Staff
Ritchie Geisel, RFB President
Christopher Brooks, RFB VP Information Technology
John Kelly, RFB VP Library Services
Dr. Richard Cox, RFB Technical Advisory Committee
Mr. Robert Finger, RFB Technical Advisory Committee
Mr. James I. Magid, RFB Technical Advisory Committee
George Kerscher, RFB Director R&D (RFB R&D team)
Steve Edwards, RFB Research Analyst (RFB R&D team)
Bill Robinson, RFB R&D team
Linden Clarke, RFB E-Text Mfg. team
Sarah Johns, RFB E-Text Mfg. team
Rhona Ethe', RFB IT Management team
Cindi Krnac, RFB IT support
**** Part 5 Conference and Accommodation Confirmation ****
Recording for the Blind
20 Roszel Road
Princeton, NJ 08540
Conference and Accommodation Confirmation
"Math and Science Working Symposium"
May 12 & 13, 1994
Guest:
Name: Dir.ir. J. J. Engelen
Title: Professor of Applied Electronics and Optics
Company: Katholieke Universiteit Leuven
Street Address: Kardinaal Mercierlaan 94
City, State, Zip: B-3030 Leuven - Heverlee BELGIUM
Phone: +(32) 16 22 09 31
Toll Free:
Fax:
E-mail: fheda02@cc1.kuleuven.ac.be
Conference Location: Castles's Room, Recording for the Blind, 20
Roszel Road, Princeton, New Jersey 08540
Presenters are responsible for providing their own handouts.
Hotel Information:
Novotel Princeton
100 Independence Way
Princeton, NJ 08540
Phone: 609-520-1200
Fax: 609-520-0594
Single or Double Room:
Preferred Accommodation (Non Smoking, etc.):
Room Rate (May 11 & 12 only): $39.00/night
Arrival Date:
Departure Date:
Confirmation #: Vicki
Please communicate any changes regarding the above conference
information or accommodations to Vicki Norbury. If you have not
confirmed your lodging, please notify Vicki Norbury by
April 15, 1994.
Vicki Lee Norbury, Adm. Asst.
Recording for the Blind
Research and Product Development
127 N. Higgins Avenue, Suite 202
P. O. Box 7068
Missoula, MT 59807
Phone: 406-728-7201
Fax: 406-728-6331
InterNet: cbfb_gwk@selway.umt.edu
**** Part 6 Attachments ****
The following is the abstract of Dr. Raman's thesis
Topic: Audio System for Technical Readings
Time: 13:15
Date: 17 January, 1994
Place: 5130 Upson
-------------------------------------------------------------------
The advent of electronic documents makes information available in
more than its visual form ---electronic information can now be
display-independent.
We describe a computing system, AsTeR, that
{\em audio formats\/} electronic documents to produce
audio documents.
AsTeR can speak both literary texts and highly technical documents
(presently in La)TeX) that contain complex mathematics.
Visual communication is characterized by the eye's ability to
actively access parts of a two-dimensional display. The reader is
active, while the display is passive. This active-passive role
is reversed by the temporal nature of oral communication:
information flows actively past a passive listener. This prohibits
multiple views ---it is impossible to first obtain a high-level
view and then "look" at details. These shortcomings
become severe when presenting complex mathematics orally.
Audio formatting, which renders information structure
in a manner attuned to an auditory display, overcomes these
problems. AsTeR is interactive, and the ability to browse
information structure and obtain multiple views enables
{\em active\ /} listening.
--Raman
********
The following article appeared in the March 1994 issue of SIAM
News, the monthly news journal of the Society for Applied and
Industrial Mathematics, pages 7-16. Reprinted with permission from
the publisher. Copyright 1994 by the Society for Industrial and
Applied Mathematics, all rights reserved.
THESE INNOVATIVE TOOLS HAVE EYES FOR MATHEMATICS
By Rena F. Bloom
The visually impaired will be finding it easier to approximate
the way sighted people read and write equations and other
mathematical language, thanks to remarkable tools recently
developed by three scientists. Two of the three are visually
impaired themselves; their advances are a result of extraordinary
ingenuity and persistence born partly out of their own frustration.
It took a great deal of effort, for instance, for T.V. Raman
to pursue a mathematical education. Born in India, he was partially
sighted until the age of 14, enough to learn to read and write, but
after that he was dependent on readers, recordings, and Braille.
However, neither Braille textbooks nor the standard Braille
mathematics codes were available to him, and pre-recorded textbooks
were scarce. By 1988, the last year he studied in India, 13 readers
were each spending three hours a week recording texts for Raman.
Although talking computers (which read text in a synthesized
voice) had been developed by that time, he did not have access to
them, and when he began to study computing for his master's degree
at the Indian Institute of Technology in Bombay, someone had to
stand behind him and tell him what was on the visual display.
Audio System for Technical Readings (AsTeR)
The lack of adaptive equipment in India spurred Raman's
decision to leave his homeland in 1989 and join the PhD program in
applied mathematics at Cornell University. There he acquired his
first guide dog, Aster, and his first talking computer. Both, he
says, have enriched his life.
Now Raman is poised to enrich the lives of other visually
impaired scientists. For his PhD dissertation, he has developed
AsTeR, a computing system that converts electronic documents
written in TeX or LaTeX into an audio format through a speech
synthesizer. One way AsTeR can be used is to produce --
electronically and speedily -- audiotapes comparable to those made
by skilled readers of technical material, such as those who read
for the Princeton-based Recording for the Blind (RFB).
The rigorous control over sound available through the voice
synthesizer permits such innovations as the use of a higher pitch
to indicate superscripts and a lower pitch for subscripts. In fact,
one advantage of AsTeR-generated tapes is "absolute consistency" in
the enunciation of mathematical expressions, says Chuck Romine, a
mathematician at Oak Ridge National Laboratory and one of the 30 or
so at ORNL who are regular RFB volunteers.
However, AsTeR's greatest value lies in the interactive
features it offers, Raman believes. Because the system is
programmed to recognize the markup language of the TeX family, it
can comprehend "the logical structure of a document," Raman says,
and convert aspects of that structure to an audio format by
applying "rendering rules." AsTeR thus enables users to take an
active role that is for the most part denied to listeners who are
tied to the linearity of an audiotape and whose participation is
limited to pausing, rewinding, or forwarding the tape.
Like sighted readers who rely on what Raman calls "multiple
views" to aid comprehension, AsTeR users can browse headings to
obtain an overview of the content of a text, skip from equation to
equation, mark a passage to be returned to later, skip passages or
sections, call up a prior reference, and choose the level of detail
they want.
AsTeR's cross-referencing capability is particularly useful
for mathematical material. "If a proof such as 'By theorem 2.1 and
lemma 3.5 we get equation 8 and hence the result' looks abstruse in
print, it sounds meaningless in audio," Raman says. In his system,
when a numbered theorem or equation is encountered, AsTeR allows
the user the option of labeling it with a name (for example,
"Fermat's theorem"). Later, when a numbered cross-reference comes
up, AsTeR uses the name instead of the number. In addition,
pressing a predetermined key will recall the theorem or equation in
full detail.
AsTeR also enables users to preview an equation's "top-level"
structure. When sighted readers see a complex equation, Raman
explains, they may first "view it as an equation with a double
summation on the left-hand side and a double integral on the
right-hand side, and only then attempt to read the equation in full
detail." Actually, most visual mathematical notation represents an
attempt to increase the reader's understanding by "grouping
subexpressions together in a meaningful manner," he says.
To compensate for the relentless linearity of audio
renderings, Raman has built into AsTeR a feature that recognizes
and labels the top-level structure and groupings in expressions.
When this feature is activated, the top-level description of the
equation is spoken first, using such grouping labels as "lower
constraint 1," before the full details of each group are spoken
(Figure 1).
While a skilled reader of equations will provide some
description when recording texts by saying, for example, "the
fraction whose numerator is . . . ," most such readers use a basic
left-to-right formula rather than AsTeR's top-down rendering,
according to Chuck Romine.
Romine is in a good position to make comparisons. To test his
system, Raman sent six printed pages of complex mathematics
(including the equation in Figure 1) to Recording for the Blind.
Romine spent more than an hour recording the material under RFB's
careful, closely monitored procedure. Sometime later, he had the
chance to review an AsTeR-generated tape of the same material that
had taken 15 minutes to produce. He calls the system "an amazing
tool." RFB, also impressed, is making Raman's work part of a major
effort to improve accessibility to mathematics and the sciences.
Making Braille Work for Mathematics
Physicist John A. Gardner of Oregon State University is
another scientist who remembers what it was like to be sighted. He
has also come up with a revolutionary tool, in this case a
refinement of the Braille system, to make mathematics more
accessible to the visually impaired.
When he lost his eyesight about five years ago as a
complication of surgery, Gardner searched for the resources to
enable him to continue his teaching and National Science
Foundation-funded research on high- temperature superconducting
materials. He discovered that talking computers were useful for
keeping up with research in his field, but he found Braille, even
special Braille mathematics codes, complicated and basically
unsuited to the mathematical expressions that are the language of
science. (In Braille, for instance, the nine digits are the same as
the first nine letters of the alphabet and must be distinguished
from them by being preceded by a special "number" character.) Not
unsurprisingly, there appeared to be few advanced mathematical and
science texts in Braille.
Determined to keep up with his research and to make the
language of science accessible to others who share his disability,
Gardner invented Dotsplus, a modified Braille tactile system that,
like AsTeR, introduces a spatial format into a heretofore linear
method. Dotsplus retains the common visual mathematical symbols,
such as plus, minus, summation, and integral signs, enlarging them
and printing them as raised images. An expanded Braille dot system
(using eight-dot "cells" instead of the standard six-dot ones) is
used for letters and numbers, eliminating the need for special
symbols for numbers or capitalization. The placement of characters
on the page is the same as it would be in print for the sighted,
with subscripts below and superscripts above, and numerators of
fractions placed above denominators, thus enabling the Braille user
to approximate the sighted reader's multidimensional view of
equations (see Figure 1, right, page 7).
Needed: A Better Tactile Printer
Finding a printer that can make raised tactile images other
than dots is the biggest stumbling block in making Dotsplus widely
available, says Gardner's OSU physicist colleague, William Barry.
Braille printers basically work by punching holes through paper and
can't reproduce the lines and other graphic images integral to
Dotsplus. Currently Gardner and Barry print Dotsplus on a wax jet
printer that is no longer manufactured. They hope that commercial
companies such as those attending a conference this month on
technology for the disabled at California State University,
Northridge, will take up the challenge of creating a suitable
printer for Dotsplus.
The technology that reduces challenges to the visually
impaired need not be dazzlingly electronic. For years, those who
used manual Braille writers were forced to write from right to
left, because the stylus produced its raised dots on the back of
the paper. Even worse, according to Raman, who recalls trying to
take notes with these devices, was that the user could not "read"
what had been written without removing the paper and turning it
over.
Now, Larry Hawk, an ORNL engineer who is also a Recording for
the Blind volunteer, has invented, along with colleague Joe Turner,
a new hand-held Braille writer that eliminates these problems (see
illustration at left).
Still, as Raman, Gardner, and Barry emphasize in a joint paper
to be presented at the CSUN conference, one aspect of modern
technology has been crucial to their advances. Precise mathematical
markup languages like TeX have enabled computers to translate
mathematical expressions into the tactile symbols of Dotsplus as
well as render them through AsTeR's talking computer.
Last month Raman completed his PhD work at Cornell and left
for Cambridge, Massachusetts, where he has accepted a research
position with Digital Equipment Corporation. Gardner is on
sabbatical leave, pursuing his research at the University of
Konstanz in Germany. Both hope that their advances will soon open
new possibilities for the visually impaired everywhere who are
trying to do mathematics.
Raman (raman@cs.cornell.edu) has made his thesis available from
ftp.cs.cornell.edu in directory /pub/raman as aster-thesis.dvi and
aster-thesis.ps. It will also soon be available from RFB. Gardner
(gardner@zircon.physics. orst.edu) and Barry (barryw@ucs.orst.edu)
can answer questions about Dotsplus. Direct Manual Braille Slates
are available from the American Institute for Life Technologies in
Malibu, California (fax: (818) 889-9826).
Caption for Figure 1: Figure 1. Faa DeBruno's formula for the
generalization of the chain rule to the nth derivative, as
"labeled" in T.V. Raman's AsTeR system (left) and rendered in the
Braille Dotsplus system invented by physicist John Gardner (right).
If so instructed, AsTeR would first summarize the equation: "Nth
derivative with respect to x of w equals summation over 0 less than
or equal to j less than or equal to n summation over lower
constraint 1 of jth derviative of u with respect to x times the
fraction numerator 1 over denominator 1." Then the details of the
labeled parts would be spoken: "Where lower constraint 1 is . . .
, numerator 1 is . . . , denominator 1 is . . . ." In the formula
as represented in Dotsplus (at twice the size shown here),
mathematical symbols as well as dots are raised images, and
placement on the page approximates what a sighted reader would see.
Caption for photo: The Direct Manual Braille Slate, invented by
ORNL's Larry Hawk. The paper is placed between the guide plate and
the slate with its bed of pins. A hollow stylus produces dots on
the top of the paper, enabling users to write from left to right
and to read what they have written.
Caption for photo: T.V. Raman with his two "stars": Aster, his
guide dog, and the computing equipment that he uses for AsTeR, the
system that renders TeX documents in an audio format.