Cognitive Walkthrough - ICT Virtual Human Toolkit

As part of the MGUIDE project, I had to complete the cognitive walkthrough of the ICT Virtual Human Toolkit. This toolkit is a collection of the state-of-the-art-technologies including: speech recognition, automatic gesture generation, text to speech synthesis, 3D interfaces, dialogue model creation to name but a few. Current users of the toolkit include, CSI/UCB Vision Group at UC Berkeley, Component Analysis Lab at Carnegie Mellon University, Affective Computing Research group at MIT Media Lab and Microsoft Research.

An assemble of some of the characters created by the toolkit.

Source: University of Southern California Institute for Creative Technologies

The main idea behind the evaluation, was to provide usability insights on what is perhaps the most advanced platform for multimodal creation on the planet today. The process was completed successfully with 2 experts, and revealed a number of insights that were documented carefully. These insights will be fed to the design of the Talos Toolkit – among the MGUIDE deliverables was an authoring toolkit to aid the rapid prototyping of multimodal applications with virtual humans. Talos is currently just an architecture (see here), but the walkthrough of the ICT toolkit provided some valuable insights that should guide its actual design. However the MGUIDE project was completed, with the development of Talos set for the future goals of the project.

I applied the cognitive walkthrough, exactly as I would applied it in any other project. I performed a task analysis first (i.e., i established the tasks I wanted to perform with the toolkit I broke them into actions) and then, I asked the following questions at each step:

1) Will the customer realistically be trying to do this action?

2) Is the control for the action visible?

3) Is there a strong link between the control and the action?

4) Is feedback appropriate?   

Complex System Architecture 2

Below is the final architecture of the Talos toolkit  - my authoring toolkit for rapid prototyping of Virtual guide systems and research. The design is complete with a number of modules that need detail explanation.  Some of these ideas were implemented in MGUIDE, but implementing the full toolkit is a task best suited for a team.

 

The only other toolkit in existence (free of charge for research) is the  ICT Virtual Human Toolkit. I have performed a cognitive walkthrough of the ICT toolkit, and fed the results into the design of Talos. I discuss my findings on a paper that will be published soon.

 

Please note: For obvious reasons I can not provide any documentation on the workflow of the Talos toolkit. The purpose of the diagram is ONLY to illustrate the complexity of my work.

 

 

 

Current Developments - Talos Authoring tool

Talos: A Virtual Human Authoring tool

Talos is an authoring environment designed to enable content developers to create Virtual Human systems for the domain of mobile guidance. Although the system has been designed with the paper guide book creators in mind, it can also be used in other application domains as well.The final design of Talos is very complex and includes several different modules. I think that it will take a programming team of 5 people 1 year of intense work to complete.

Talos come as an idea when I realised that there are virtually no tools in the market that would enable me to rapidly create the prototypes I needed for my research work. From the final design of Talos, I was only able to implement some of its ideas, those that needed to implement my final prototype systems.

1 ) A script parser to automatically create Haptek scripts. A screen-shot of the parser is show below:

The tool is described in more detail in the topic "Current Developments - Haptek characters". The script parser of the full Talos environment follows a similar approach, but in a fully automated and real-time fashion. Talos will accept pure text as input, and generate full character performances as an output. Of course the problem is this approach, is how to make the character pointing correctly to objects in its background. This requires the character to have knowledge of its environment. My design in this area is incomplete. A possible solution, would be to divide the background of the character into segments and associate each segment with a keyword. I need to do more research on this area.
2) A simple UI (User Interface tool) that enabled me to create without too much effort, the AIML KBs needed for my final prototype systems. In more detail the UI includes:
a) An AIML\XML KB creator.
b) An AIML creator for existing question-sets (e.g., questions that you may already have in XML format)
c) A translator for both AIML/XML KBs
d) A scene character modifier where you can modify various scene and character settings (e.g., probs, backgrounds, etc)
e) A GPS integration module, where you can assign Long and Lat coordinates to various scripts and tests them in real-time
f) A speech recognition module where you can load a grammar and test it
g) A Cyc-creator module. This module was supposed to automate the creation of CYC queries suitable for insertion into AIML scripts. However, the API of ResearchCyc is extremely tough to crack (perhaps because it is pure Java) and I ended up with just a simulation.
From the above I only used "a","b",and "c" in the creation of my final prototypes. The rest of the features were not of any value to the actual development, but they lead to several improvement in the final Talos design. The UI tool is available for free. If you want to experiment with Virtual Humans, AIML and XML KB's, making location-sensitive scripts for your characters that's the best way to start.

A screenshot of the UI tool