Emacspeak
Direct Speech Access
T. V. Raman
Senior Computer Scientist
Adobe Systems
c
#1996 Adobe Systems Incorporated.All rights reserved.
1

Outline
# Overview of speech applications.
# Emacspeak --Architecture.
# Emacspeak --The user experience.
c
#1996 Adobe Systems Incorporated.All rights reserved.
2

Screen Access
User explores visual display to:
# Construct a mental model of the interface.
# And interpret intent of the UI.
Aural output --Derived from visual display.
c
#1996 Adobe Systems Incorporated.All rights reserved.
3

Screen Access Design
# Encapsulate display in an off-screen model.
# Present this model aurally.
# Enable navigation of this model.
Aural feedback is application independent.
c
#1996 Adobe Systems Incorporated.All rights reserved.
4

Available Screen Access
Character-based interfaces :::GUI.
Richer layout of GUI
l
Harder to build OSM
l
But richer OSM
l
Enables possibly richer feedback
c
#1996 Adobe Systems Incorporated.All rights reserved.
5

Screen Access Innovations
# Speak specific areas of the screen.
# Attach meaningful labels to icons.
# Navigate window hierarchy.
Access separate from application.
c
#1996 Adobe Systems Incorporated.All rights reserved.
6

What Is UI?
1. Obtain user input
2. Compute on the information
3. Display the results
UI = Input+Output
c
#1996 Adobe Systems Incorporated.All rights reserved.
7

Speech Enabling Applications
# Treat speech as a first class medium.
# Application produces its own feedback.
# Exploit features of the spoken medium.
Audio output independent of visual display.
c
#1996 Adobe Systems Incorporated.All rights reserved.
8

Contrasting Approaches
Read Screen Speech Enable
Speak display Speak information
No context Full context
Independent Integrated
Global scope Local context
Uses display Uses environment
c
#1996 Adobe Systems Incorporated.All rights reserved.
9

Advantages
# Produce intuitive feedback.
# Provide a simpler user model.
# Reduces users cognitive load.
User works with one --not two-- applications.
c
#1996 Adobe Systems Incorporated.All rights reserved.
10

Example
March 1996
Sun Mon Tue Wed Thu Fri Sat
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31
Easy to see relevant information.
c
#1996 Adobe Systems Incorporated.All rights reserved.
11

EMACS
A customizable, extensible user environment:
# Editing and browsing electronic information.
# Software development.
Unified interface!Cooperating applications
c
#1996 Adobe Systems Incorporated.All rights reserved.
12

Emacspeak
# Device independent speech interface.
# Core modules for basic spoken feedback.
# Extensions for various applications.
Does not modify Emacs code-base.
c
#1996 Adobe Systems Incorporated.All rights reserved.
13

Advice Overview
BEFORE Called before advised function.
AFTER Called after advised function.
AROUND Called instead of advised function.
Advice enables code re-use.
c
#1996 Adobe Systems Incorporated.All rights reserved.
14

Advice
Let x ! f ! y.
BEFORE AROUND AFTER
Result in function f 0 that computes y 0 .
x ! f
--- #z #
f 0
! y 0
c
#1996 Adobe Systems Incorporated.All rights reserved.
15

The User Experience
# Succinct contextual speech feedback.
# Auditory icons augment interaction.
# User action flushes prior speech immediately.
User focuses on task at hand.
c
#1996 Adobe Systems Incorporated.All rights reserved.
16

Demonstration --Editing
# Simple editing, search and replace.
# Completion and spell checking.
# Syntax coloring using voice-lock mode.
Intuitive interface enables fluent interaction.
c
#1996 Adobe Systems Incorporated.All rights reserved.
17

Demonstration --Browsing Information
# Browsing the file system.
# Reading and responding to email and news.
# Browsing the WWW.
Window to digital information.
c
#1996 Adobe Systems Incorporated.All rights reserved.
18

WWW --Speech Style Sheets
# Voice properties,
# Auditory icons,
# Sound cues for document elements.
Generate richly formatted audio documents.
c
#1996 Adobe Systems Incorporated.All rights reserved.
19

Demonstration --System Tasks
# Running a shell.
# Running terminal based applications.
Behaves like a traditional screen-reader.
c
#1996 Adobe Systems Incorporated.All rights reserved.
20

Device Independent Speech
# Dectalk Express and MultiVoice
# Software Dectalk --DEC-ALPHA
# DECFACE --A Talking Face
Supporting other devices is straightforward
c
#1996 Adobe Systems Incorporated.All rights reserved.
21

Core Modules
Speech enables
# Navigation, editing, and prompting.
Sufficient to implement a working system.
c
#1996 Adobe Systems Incorporated.All rights reserved.
22

Application Specific Extensions
The icing on the cake!
# WWW browser, Mail and News reader
# Tools: Calendar, Spell checker, :::
Code-size is less than 1% of original.
c
#1996 Adobe Systems Incorporated.All rights reserved.
23

Advice Statistics
Package Code Ext % Advice
Core 217,295 3,152 1.45% 188
W3 17,384 515 2.96% 12
Mail 17,943 270 1.50% 14
GNUS 35,528 501 1.41% 37
Emacspeak --A model of code re-use.
c
#1996 Adobe Systems Incorporated.All rights reserved.
24

Conclusion
# Treat speech I/O as a first-class medium.
# Separates UI and computation.
# Enables rich, well-structured interfaces.
c
#1996 Adobe Systems Incorporated.All rights reserved.
25