Development of Virtual Humans - Early Years

Domain: Mobile Guides

After the first 1 and 1/2 years of my M.Phil work, I finally found heaven. The area of mobile guides is a relatively simple domain but with much to offer. Tourism in several countries (e.g., Greece) is considered a vital part of the economy. In 2005 I begun working on a system that a) provides navigation instructions in a specific area (i.e., a medieval castle in Greece) b) provides information on selected locations of the castle (e.g., historical information). The system was authored in Macromedia Director MX 2004 multimedia environment and used:

a) The AIMLPad ActiveX control

b) The Maddy character from DAGroupPlc.

c) A location-sensing algorithm that triggered specific scripts at specific coordinates in the castle.

d) A speaker independent ASR engine provided by Babel technologies ().

The system accepted input through a menu of static English phrases and buttons. It generated no output apart from the minor movement of the character's body on the screen. After field testing the system, I realised that a) the GPS component because the rocky environment of the castle was not working. b) The ASR engine could not distinguish the user's voice from the noise in the environment. These two components were removed and a formal evaluation took place in November 2005. The goal of the evaluation was to investigate how such an agent would affect the accessibility and usability of a mobile guide system.

 

In parallel with the developments of the prototype I also had the opportunity to investigate Research-Cyc the most advanced Knowledge-Base on the planet. In parallel with the developments of the prototype I also had the opportunity to mess around with Research-Cyc, the most advanced Knowledge-Base on the planet. The design of the system is shown below.

 

 

The idea was to built a KB that would contain all the castle knowledge and integrated with the prototype system.Back then, this was an ideal solution for me, as with a simple fact in KB (Yannis Ritsos is a poet) the system could answer questions like:

 

“Who is Yannis Ritsos”

“What is the profession of Yannis Ritsos” (poet is a profession)

“Name a poet of Greece” (Greece is a nation and Greeks its inhabitants)

I used the CynD AIML Interpreter. The system uses specially modified AIML to access the KB NL facilites, query the KB and then return the results. For example:

Given the below AIML code and the sentence:

WHAT COUNTRIES BORDER *

(#$isa #$Country )

Consulting cyc returns: What countries border ?

 

The system calls the ResearchCyc parser (through the custom cycquestion tag), translate the input into its CycL (the language in which the KB is written) representations:

((#$and

(#$isa ?WHAT-ONE-1 #$Country)

(#$politiesBorderEachOther #$Greece ?WHAT-ONE-1))

(#$and

(#$bordersOn #$Greece ?WHAT-ONE-1)

(#$isa ?WHAT-ONE-1 #$Country))

(#$and

(#$isa ?COUNTRY #$IndependentCountry)

(#$politiesBorderEachOther #$Greece ?COUNTRY))

(#$and

(#$bordersOn #$Greece ?COUNTRY)

(#$isa ?COUNTRY #$IndependentCountry)))

And then iterate through the list of parses, and try to query the KB to get a possible answer. In the particular example the system returns:

 

Consulting Cyc returns:

0 (((?COUNTRY . Bulgaria))((?COUNTRY . Macedonia))((?COUNTRY . Albania))((?COUNTRY . Turkey))).

1. (((?COUNTRY . Bulgaria))((?COUNTRY . Macedonia))((?COUNTRY . Albania))((?COUNTRY . Turkey)))

2. (((?WHAT-ONE-1 . Bulgaria))((?WHAT-ONE-1 . Macedonia)) ((?WHAT-ONE-1 . Albania))((?WHAT-ONE-1 . Turkey)))

3.(((?WHAT-ONE-1 . Bulgaria))((?WHAT-ONE-1 . Macedonia))((?WHAT-ONE-1 . Albania))((?WHAT-ONE-1. Turkey)))

Finally it calls the NL generator of the KB and translate the results back to English:

 

Consulting Cyc returns: 0 Bulgaria Macedonia Albania Turkey. 1 Bulgaria Macedonia Albania Turkey. 2 . Bulgaria Macedonia Albania Turkey 3. Bulgaria Macedonia Albania Turkey

Problems:

1) CynD will try to query the KB for every single interpretation of the user's question. In the above example, the KB returned the same answer for all parser interpretations. Some kind of tool was needed that would automatically generate AIML categories incorporating all possible parser interpretations, allow the system's author to select the one that produced the answer he expected and make the system's answer more natural. There was no time for that.

 

2) In an open Q-A dialogue with a system in my domain (mobile guides), the user could ask questions about random objects that he may encounter in his path-something like "What is that?" for a church or a particular building. The disambiguation of "that" to the actual object name and location in the physical environment is an extremely difficult problem -- one of the many that NLP has to face in the future. At the moment, there is no location technology that can allow a computer to distinguish between two objects located at a close distance in a psychical environment.

 

3) My work focuses on animated agents. If the dialogue with the system is unknown... how could I possibly generate the proper animations for each output of the CyND engine?

 

5) Hardware resources. Such a system demands massive hardware resources... resources in a M.Phil project are always scarce.

Lessons learned:

1) The pilot evaluation suggested a) several improvements in the experimental design b) several design improvements of the prototype.

2) The need to produce more user-insights. This could only be accomplished, by developing more prototypes varying different attributes of virtual humans (e.g., competence, modality of communication, etc.).

3) The need for technological differentiation. Its impossible to continue the simplistic approach of this prototype, as the research will lead me nowhere. The domain of VH is extremely comparative area with groups of researchers that produce massive contributions to both technology and knowledge every single month.

4) My rather bumpy experience in developing the prototype also suggested the need for an authoring tool that would enable non-programmers to rapidly develop complex Virtual Human systems.