Assisted Hybrid Structured 3D Virtual Environments

Abstract

The blending of digital technologies, such as artificial intelligence, interactive systems, 3D interfaces and the Internet is enabling new services for users. In particular, Hybrid Structured 3D Virtual Environments (VE) provide users with a collaborative space not only for entertainment and socialization but also for developing “serious” applications such as e-learning, e-government and e-commerce.

This thesis focuses on Hybrid Structured 3D VE, which are persistent multi­user systems where participants (both human users and software agents) develop “serious” activities. In these systems the 3D interface graphically represents the system and facilitates human participation, and an Organisation Centred Multi-Agent System (OCMAS) structures participants’ interactions. To do so, the OCMAS specifies the roles that participants can enact, the activities where complex tasks can be accomplished, and communication protocols that enable the prosecution of such tasks.

Nevertheless, participating in these systems is not a straight-forward process. Specifically, when the system speci.cation is complex, participants have to perform intricate reasoning processes to understand their applicable regulations at current system state; and they do not have access to information about what happened before they entered the system, neither can further process this information. Moreover, software agents speak a computer-based language, which is usually hard to use by human users. Then, as human users interact with staff agents (software agents devoted to support the system activities) to complete tasks, human-agent interaction style becomes a key issue. In order to overcome these limitations, this work proposes Assisted Hybrid Structured 3D Virtual Environments, where both human users and software agents participation in the system is improved by both assistance and human­agent interaction mechanisms. The system is formalised as a two layered infrastructure. The Organisational Layer structures the interactions of participants, and the Assistance Layer is populated by a set of Personal Assistants in charge of providing with a set of Assistance services to a system participant. There are four types of Assistance services: i) an Information service that processes data about the organisation specification, the participant current state, and the organisational historical execution states; ii) a Justi.cation service that can be triggered once a participant tries to execute a non-valid or prohibited action; iii) an Estimation service that processes whether an action can be performed at current state prior to its execution or not and, if it is actually the case, then it also provides the next system state; and iv) an Advice service, which provides participants with a sequence of actions (i.e. plans) to achieve their goals.

Moreover, this work implements and evaluates v-mWater, a virtual market based on trading water, modelled as an Assisted Hybrid Structured 3D Virtual Environment. The usability evaluation results of v-mWater show that it is per­ceived as a useful and powerful application that could facilitate everyday tasks in the future. Users like its learnability, its immersiveness, and how scenario settings facilitate task accomplishment. In general, users completed the proposed task well and they were able to go to the right destination in the scenario. After doing the test, users improved their opinion about 3D virtual environments. In addition, the overall opinion of the human-agent interaction was positive. Nevertheless, those users less familiar with new technologies experimented problems when using a command-based system to interact with staff agents.

To support assistance services in the system, this work designs and evaluates an Assistance Architecture where the Information service is implemented for software agents; and the Justi.cation, Estimation and the planning Advice service for human users. Nonetheless, these four services could be offered interchangeably for both humans and software agents, since they all simplify the reasoning process as well as the cognitive load required to participate in these complex structured systems.

Speci.cally, the Information service has been extended to help sellers to set the price in their transactions. The tests performed compare the values that different agent satisfaction parameters and system goals take when agents request for different information services, using as a base-line a con.guration without enabling assistance services. The experiments show that system performance and agent satisfaction (and thus, the quality of assistance service) increase with the addition of the information service. Furthermore, individual agents following alternative strategies can request di.erent information as a useful decision sup­port tool. The planning Advice service is the most sophisticated one and makes use of the rest of services to provide a plan that has into account other partici­pants actions and, executed at current system state, will lead to the user’s goal. It is implemented as an extension of A*, namely Plan-eA. Evaluation results indicate that assistance impacts positively in usability measures of efficiency, efficacy and satisfaction.

Related to human-agent interaction, this work integrates a new conversational mechanism within VIXEE, an execution infrastructure for Hybrid Struc­tured 3D Virtual Environments. This new mechanism includes a task-oriented conversational system, which allows staff agents to dialogue with human users using natural language conversations. To do so, this work proposes an extension of the well-known AIML language, namely Task-Oriented AIML, for dealing with task-oriented conversations, which are based on activities’ specification and current system state. Test results give good usability measures of efficiency, efficacy and user satisfaction for the conversational approach.