User Research – General Setup

User research in the MGUIDE project can be divided into the following stages:

• Requirements Gathering and specification: A prototype system was constructed based on recommendations from the literature on animated agents. The prototype was evaluated in the actual castle of Monemvasia with a number of participants. You can find more details here.  The lessons learned were presented during the M.Phil presentation and influenced the design of the final five (5) prototypes.

 

• Design and Prototypes: Based on requirements gathered from the pilot in the castle of Monemvasia, five(5) prototypes were developed. A number of novelties were achieved during the developed of the prototypes (e.g., an algorithm for natural language understanding, the design of Talos – a toolkit for system prototyping and research, and others). Although the initial idea was to continue the evaluation in the castle, due to lack of resources and time it was decided to simulate the conditions in the lab. There is an on-going debate on whether mobile applications should be tested in labs or fields. For instance,  in 2009 70% of the developed systems were evaluated under lab conditions using a variety of techniques.

 

• Evaluation: The evaluation carried out in Greece and the UK, followed the same setup.  I used detailed panoramic photography and high definition video-clips to represent in high detailed all locations and attractions of the castle. The lab was a simple room in which each user participated individually. In general the approach was successful, as participants could follow the same routes and watch the same presentations about the locations of the castle, from the comfort of their chair.

Examples of the panoramas used in the evaluations is shown below:

 

View Full Album

 

 

User Research Stage

I recently completed the evaluation stage of all five (5) prototypes of the MGUIDE project. The purpose of the studies was to provide accessibility and usability insights into the design process of animated agents for mobile applications, as well as, to support some understanding of the psychology of the users of such systems. One hundred (100) real-world users participated. My role included the following:

  Design and apply various user-centred design methods (UCD) for research including: structured questionnaires, retention tests, interviews, think aloud protocols, observations, performance measures, scenario-based usability testing, to name but a few.

 

Note: For a complete list of the research techniques I excel at, see here (part A) and here(part B)

 

Design, organize and manage five user research studies (with 100 real – world users) across two countries (Greece and the UK)

 

Continuously refine of the prototypes based on feedback collected by the users

 

Regularly update all project sponsors and stakeholders of the progress made

 

Statistically analyze the quantitative data using various statistical tests

 

Rapidly analyze the qualitative data

 

Develop recommendation/requirements to impact the design of animated agents for mobile applications

 

Produce documentation and presentation of findings

 

Write papers for publication in relevant conference proceedings and academic journals.

 

As no publications have been made yet, I can not provide any further details on my experimental work. Once the publication process is completed I will post further details on this very important stage of my work.

 

I want to thank all the students/family/friends/other people that honoured my work with their presence. Their contribution is valuable.

Complex System Architecture 2

Below is the final architecture of the Talos toolkit  - my authoring toolkit for rapid prototyping of Virtual guide systems and research. The design is complete with a number of modules that need detail explanation.  Some of these ideas were implemented in MGUIDE, but implementing the full toolkit is a task best suited for a team.

 

The only other toolkit in existence (free of charge for research) is the  ICT Virtual Human Toolkit. I have performed a cognitive walkthrough of the ICT toolkit, and fed the results into the design of Talos. I discuss my findings on a paper that will be published soon.

 

Please note: For obvious reasons I can not provide any documentation on the workflow of the Talos toolkit. The purpose of the diagram is ONLY to illustrate the complexity of my work.

 

 

 

Complex System Architecture

Below, you will find some sample from the complex system and subsystem architecture I did for the MGUIDE project. In particular,  a) the general system overview of all prototypes b) is the workflow of the MGUIDE’s Natural Language Understanding (NLU) module. For obvious reasons I can not provide any documentation on the workflow of the two architectures below. The purpose of the diagrams are ONLY to illustrating the complexity of my work.

 

 

 

Apart from the semantic analyser (shown in the first diagram as experimental) everything else was fully prototyped and evaluated in the user-research stage of the project.

Rapid Prototyping, Which tool?

Below is an experiment to simulate one of the MGUIDE prototypes using Axure RP Pro. I wanted to see if Axure can be used in such applications, and how fast I could implement a working prototype.The dialogue window is clickable, as well as the buttons. All standard GUI (Graphical User Interface) elements of the MGUIDE interface can be quickly and easily implemented using this tool. The environment is drag drop, supporting adding interactions using visual elements (e.g., If then else statements with a few clicks). However, the absence of an internal scripting language means that I am limited by the number of interactional elements the program has already build in. I can’t build my own interactions, like for example the dialogue branching I easily implemented using VB.NET. Neither can I import my 3D avatar, for lets say a formal evaluation. In conclusion, Axure appears to be a tool to be used very early in the product cycle and mostly for low-fidelity prototypes. Later on, you you will need to build something complex in order to communicate your ideas more effectively. This is where I come in!  I can built highly complex prototypes, in the same amount of time using VB.NET/Adobe Director with just a few lines of code. However, if a a stakeholder is happy with this tool, I can be happy as well. The tool can be used virtually by anybody, let alone myself. 

System Preferences Wireframe

Main Screen Wireframe