Generating Language from BlissSymbols Using Semantic Authoring

Yael Netzer , Ofer Biller, Michael Elhadad and Yoav Goldberg

Department of Computer Science, Ben Gurion University, Beer Sheva, Israel

{yaeln | billero | elhadad | yoavg}@cs.bgu.ac.il

This work presents a new Hebrew-English-Bliss online lexicon, and a new approach to generate messages in an AAC display in Hebrew and English, in the context of Natural Language Generation; the main motivation is to provide the user with a communication tool that enables high rate of communication alongside with as wide expressive power as possible.

Motivation

Natural Language Generation (NLG) is a subfield in Natural Language Processing (NLP). The term NLG refers to the process of generating utterances in a spoken language from another representation of data, based on linguistic resources. For all applications, the generated text can be in various languages, leading to applications of multilingual generation (MLG). Multilingual generation (MLG) aims to generate text in several languages from one source of information, without using translation.

Previous works on NLG-AAC systems ([5], [2] [4] for example) have adopted a technique of first parsing a telegraphic sequence, then re-generating a full sentence in natural language. The initial message is of telegraphic nature because it lacks the main cues of morphological and syntactic structure that exist in natural language. As a consequence, reconstruction of the intended meaning is made difficult, and deep semantic and lexical knowledge sources are required to recover it. Such resources are not readily available in general, and as a result, systems with only a reduced vocabulary have been demonstrated. Moreover, as an AAC device is used not only for typing text, but also for real-time conversations, the interpretation of the utterance relies to a large extent on pragmatics -- such as the time of a mentioned event, omitting syntactic roles and making reference to the immediate environment. The need to recover such pragmatic clues makes the semantic parsing of telegraphic style even harder.

Objectives

The main question we address is whether generation is possible not through the process of parsing and regeneration, but through a controlled process of authoring, where each step in the selection of symbols is controlled by the input specification defined for the linguistic realizer.

We have chosen Bliss symbols as the input language of the communication board. Bliss is composed of a narrow set of atomic meaning-carrying symbols. The rest of the symbols are a combination of them. This compositionality is a very important characteristic of Bliss as a language and we designed a lexicon which captures the strong connection between the meaning and the form of the symbols

A practical objective of our work is to provide Bliss tools for Hebrew speakers. When Bliss was adopted for use in Israel, a decision was taken to write Bliss symbols from right to left as the Hebrew writing system, and consequently to inverse the display of the symbols (or at least of most of them). As a result, most software developed in the world for Bliss (either commercial or experimental) could not be used by Hebrew-speaking users.

The Hebrew-English Bliss Lexicon

We have designed and implemented a lexicon of Bliss-Hebrew-English words that takes into account this information: the lexicon can be searched by keyword doctor, or by semantic/graphic component: searching all words in the lexicon that contain both person and medical returns the symbols aiding tool, artificial insemination, dentist, doctor, nurse etc.

The lexicon also allows searches through semantic components (e.g., "all symbols that contain a wheel"), or by forms (e.g., "all symbols that contain a circle").

The lexicon can be used either as a stand-alone application for reference usage or as part of an application (such as a dynamic display or an editor). The design of the lexicon takes advantage of the unique properties of the language. Technically, only a set of atomic shapes is physically drawn while combined symbols are generated automatically, following the symbol's entry in a database that was constructed from the Hebrew and English Bliss Dictionaries.

The design of the lexicon enables easy manipulation of the symbols (graphical editing, adding new synonyms) and an easy way to insert new symbols (by combining existing symbols or by drawing a new one). It contains both Hebrew and English words and adjusts the representation according to the language.

AAC Dynamic Display

The core of the processing machinery of the AAC message generation system is based on SAUT [שגיאה! מקור ההפניה לא נמצא.]- an authoring system for logical forms encoded with conceptual graphs (CG). The system belongs to the family of WYSIWYM (What You See Is What You Mean) text generation systems: logical forms are entered interactively and the corresponding linguistic realization of the expressions is generated in several languages. The system maintains a model of the discourse context corresponding to the authored documents.

We use SAUT as the underlying engine of a virtual dynamic display that serves as a communication board. SAUT is a textual system and follows conventions that are common in computer language editors, such as Intellisense in Microsoft Visual Studio application. Since a Bliss communication board is not a textual system, the conventions were adjusted to the symbols set.

The display is divided into four main areas: a list of participants and of defaults; buttons (as will be elaborated below); a text pane where the chosen symbols are displayed as the sequence entered) and the (possibly partial) sentence generated; the generated text in context.

The display contains a set of buttons (or keys) which are further divided into three types: (1) function keys (2) hyperlink to other displays and (3) symbols.

The system appears to the user as a dynamic communication board, which shows symbols in Bliss and produces fluent output in Bliss, English or Hebrew. As the user selects new symbols, the communication board is re-organized to ease the selection of further symbols. The sequence of symbols entered by the user is "translated'' into fluent language incrementally -- so that at each stage, output text appears on the screen.

The main display is initialized in three possible manners: (1) most frequently used words; (2) most frequent shapes/symbols (e.g., person, house, activity); (3) possible scenarios (e.g. school, home, family). The board is initialized with a selection of candidate Bliss symbols and of references to entities that are likely to be useful in the interaction (we call these "participants''). Entities mentioned in previous utterances are shown as well, and in subsequent utterances, these may be matched to complete an utterance if it has missing entities, using selectional restriction constraints. In addition, for different settings, defaults can be specified. Defaults provide values for attributes such as the tense or the mood of the clauses. Defaults need not be repeated for each sentence, but can be specified only once at the beginning of the session. They can be overridden in specific sentences (but at the cost of extra typing). Defaults can be changed while editing.

The symbols displayed on the screen at each step depend on the context of the previously entered symbols (or the pre-defined initial display). For example, if the previous symbol denotes a verb which requires an instrumental theme, only symbols that can function as instruments are presented on the current display. The general context of each utterance or conversation can be determined by the user, therefore narrowing the diversity of symbols displayed.

Conclusions and Future Work

We have designed and implemented a Bliss lexicon for both Hebrew and English, which can either be used a stand-alone lexicon for reference usage or as a part of an application. In this work, it is used for representing symbols in a communication board.

The design of our system is inspired by both the semantic authoring technique as implemented in SAUT as well as from dynamic displays.The symbols that are displayed on the screen in each step of insertion depends on the context of symbols as previously seen.

Evaluation shows good coverage of the semantic authoring approach and usability is exemplified in the quick learning curve. Efficiency is achieved in the reduced number of symbols that can be selected in each step of the message generation.

Future work includes expansion of the underlying lexicon for Hebrew generation; combining pre-stored sentences in the current application; adding voice output and applying morphological transformations on words for derivation (according to the indicator of a Bliss symbol)

Bibliography

  1. O. Biller, M. Elhadad, Y. Netzer, Interactive Authoring of Logical Forms for Multilingual Generation. In Procs. Of European Workshop on NLG (ENLG-05), Aberdeen, Scotland, August 2005.
  2. A. Copestake, Applying Natural Language Processing Techniques to Speech Prostheses, Working Notes of the 1996 AAAI Fall Symposium on Developing Assistive Technology for People with Disabilities, 1996.
  3. P.W. Demasco and K.F. Mccoy, Generating Text From Compressed Input: An Intelligent Interface for People with Severe Motor Impairments. Communications of the ACM, 5:35, May 1992.
  4. K.F. McCoy, Simple NLP Techiques for Expanding Telegraphic Sentences, In Procs. of NLP for Communication Aids ACL/EACL, Madrid, July 1997.
  5. P. Vaillant and M. Checler, Intelligent Voice Prothesis, Computation and Language E-print Archive, 1995 (