[University of Cambridge] [Dept of Engineering]

Spoken Conversational Interaction for Language Learning (SCILL)


SCILL is a collaborative research project between the MIT CSAIL Spoken Language Systems Group and the Cambridge Machine Intelligence Laboratory. The project is funded within the framework of the Pervasive Computing Knowledge Integration Community (KIC). The aim of this project is to develop a language learning system with four major components:

  1. an intelligent agent capable of maintaining an interactive dialogue with the student in the target language.This agent would act as the "native speaker" versed on some topic selected from a pre-defined set.
  2. a second intelligent agent capable of translating within the topic domain between the student's native language and the target language. This agent will act as an on-line "tutor" giving advice on how to say required words and phrases.
  3. an assessment component which provides a post-mortem analysis of the conversation and gives feedback on errors and areas to improve.
  4. a set of tools which allow second-language teachers to author specific dialogues and scenarios.

In practice, the target language will be Mandarin Chinese and the native language will be US or UK English. However, the system design will as far as possible be language independent.

The system will enable a student to participate in a dialogue with the system in Mandarin whilst simultaneously having access to a "tutor" that could tell them how to say certain phrases. For example, the topic might be about the weather in a particular city, and the bi-lingual "tutor" would provide the student with helpful hints on how to communicate in Mandarin with the system. This allows the student to engage in practice conversation in a non-threatening environment. The student would be able to gauge their success by the degree to which the system understood their Mandarin queries, and the number of times they needed to consult the "tutor" for translation advice.

A later off-line interaction using the assessment component would allow the student to re-examine their speech. The system could provide feedback on their overall pronunciation quality, as well as identifying words that were poorly enunciated, allowing them to compare their pronunciation with a standard. The system could also conceivably repair the tone production, while preserving the overall quality of the student's voice.

The key personnel associated with this project are:

Stephanie Seneff (co-PI at MIT),

James Glass (co-PI at MIT),

Chao Wang (RA at MIT),

Steve Young (PI at Cambridge)

Yulan He (PhD at Cambridge)

Hui YE (RA at Cambridge)

[University of Cambridge | CUED | MI LAB ]