An Intelligent Group Decision Support System for Urban Tourists Development and evaluation of a well-structured group decisionmaking process
Master’s thesis By K.S. Ngai June 18, 2010
Author details Candidate:
Koh San Ngai
Student number:
1154087
Email:
[email protected]
Graduation Committee Chair:
Prof. Dr. C. M. Jonker Man-Machine Interaction group, EEMCS, Delft University of Technology
Supervisor:
Dr. K.V. Hindriks Man-Machine Interaction group, EEMCS, Delft University of Technology
Committee member:
Msc. A.Pommeranz Man-Machine Interaction group, EEMCS, Delft University of Technology
Committee member:
Dr. G. Kolfschoten Man-Machine Interaction group, TPM, Delft University of Technology
Copyright © 2010 by K.S. Ngai
All rights reserved. No part of the material protected by this copyright notice may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system, without the prior permission of the author.
2
Abstract When a group decides to plan and organize a vacation, many researchers mention that group decision making within the travel planning problem often leads to suboptimal decisions. This can be explained by the fact that the process of travel group decision making is typically ineffective. To overcome some of the problems, we propose an intelligent Group Decision Support System named Trip.Easy that creates synergy between human and machine intelligence in order to improve group decision making. The objective of this study is to develop a prototype of the Trip.Easy GDSS that combines a well-structured decision process with domain knowledge and an intelligent recommendation mechanism that facilitates reaching a consensus for the group trip planning problem. As a result, a well-designed group decision process is provided that (i) facilitates users and makes them aware of all interesting outcomes by providing intelligent recommendations, (ii) supports collaboration at a distance, (iii) minimizes irrational acts due to various influences and (iv) facilitates effective communication by means of a clear and fair process that converges to an outcome that satisfies all group members. Subsequently, a structured experiment has been designed and conducted to empirically acquire measurements of users’ satisfaction for the designed group decision process. A total of 120 participants, divided into 30 groups, were invited for the experiment. Each group was instructed to organize a city trip while using the Trip.Easy GDSS. After each session, the participants were asked to fill in a questionnaire. Analysis of the data showed that users were satisfied with the decision process of Trip.Easy GDSS. Users perceived the interaction through the graphical user interface with the Trip.Easy GDSS during the decision process as user-friendly. Furthermore, users valued the process as fair. Based upon these findings, we may conclude that the proposed group decision process that is integrated in the Trip.Easy GDSS prototype is able to facilitate users to converge towards a satisfying travel destination.
3
Preface It all started approximately one year ago when I and my business partner Kenneth Touw Ngie Tjouw came up with the idea of an online decision support platform for travel consumers. The idea was the result of the frustration I had with planning a trip to Italy with a couple of friends. The decision process was very chaotic, due to a lot of miscommunications and the lack of information. Finally, after months of discussing destinations in Italy we ended up in Croatia. The vacation was great, but organizing it caused a lot of stress. After the vacation I was re-energized and together with Kenneth we started looking for a solution that could simplify the process of planning a vacation with friends. We found out that there were no solutions at all. Thus, we started to think of creating our own solution. We thought of it as being a negotiation problem. It is this that brought us to prof. Catholijne Jonker and Koen Hindriks. They have been conducting a lot of research on artificial negotiation systems. Next we proposed a plan for a thesis project. As Koen was willing to support us, we were able to start. After months of hard working we were able to launch an operational prototype. The many enthusiastic reactions from the participants during the experiments encouraged us to explore the commercial potential of Trip.Easy GDSS. After nights of preparation we were ready to present our concept (of course without revealing too many crucial details) to Vakantiepunten.nl and a seed investor company named 1&12biz. Apparently they also see a huge commercial potential. Currently, they are supporting us in order to write a proper business plan. At the same time, Koen, Kenneth and I worked out a funding application and together we presented our plans to the Dutch Valorization Grant Committee. This fund supports the commercialization of scientific concepts. At this moment we do not know yet whether the fund is granted to us or not. 25th of June 2010 will be the BIG day! I want to take this opportunity to thank all people who have supported this project one way or another, starting with Koen Hindriks for being a great and tough advisor. His honesty and enthusiasm made it possible to realize a complex project like this. I would like to kindly thank Gwendolyn Kolfschoten for sharing her valuable knowledge about Collaboration Engineering with me. Furthermore, I’d like to thank prof. Catholijn Jonker of the Human-Computer Interaction Group for approving this project to be a graduation project. Also I want to thank my lovely girlfriend Lai Lai Tang for supporting me and taking care of much tiny and mighty stuff. Many thanks to my parents Chi Keung and Rong Ben Ngai for their unlimited support. Thanks to my brother Koh He Ngai for taking care of our parents when I’m not home. I would like to apologize to all my friends I have neglected during this project. Last but not least I would like to thank my friend and business partner Kenneth Touw Ngie Tjouw. Together we experienced a lot of adventures and with a bit of luck, that may forever remain. Currently we are on our way to the Silicon Valley adventure in the United Stated of America to further explore the commercial opportunities and to get acquainted with potential investors. The finalization of this thesis took place in a motel in the middle of the Yosemite wilderness. Koh San Ngai @ Yosemite National Park, 18th of June, 2010
4
Table of contents
Abstract ......................................................................................................................................................... 3 Preface .......................................................................................................................................................... 4
INTRODUCTION 1
Introduction ........................................................................................................................................ 13 1.1
Problem definition ...................................................................................................................... 13
1.2
Research objectives .................................................................................................................... 15
1.3
Methodology............................................................................................................................... 15
1.4
Outline of this thesis ................................................................................................................... 17
PART I BACKGROUND 2
Introduction ........................................................................................................................................ 19
3
Decision making in the travel domain ................................................................................................ 20 3.1
Scenario: planning a trip in a group ............................................................................................ 20
3.2
Group travel planning problem .................................................................................................. 22
3.3
Theories and models of decision making by travelers................................................................ 22
3.3.1
Consumer decision making ................................................................................................. 22
3.3.2
Instantiation of the consumer behavior model .................................................................. 24
3.3.3
Travel group decision-making ............................................................................................. 25
3.4 4
5
Summary ..................................................................................................................................... 27
Collaboration Engineering................................................................................................................... 28 4.1
Collaboration............................................................................................................................... 28
4.2
Collaboration patterns ................................................................................................................ 30
4.3
Design approach ......................................................................................................................... 35
4.4
Summary ..................................................................................................................................... 37
Introduction ........................................................................................................................................ 39
PART II DESIGN AND IMPLEMENTATION 6
Scope Trip.Easy Prototype .................................................................................................................. 40 6.1
Requirements .............................................................................................................................. 40
5
7
6.1.1
Functional requirements..................................................................................................... 40
6.1.2
Non Functional requirements ............................................................................................. 42
Design.................................................................................................................................................. 43 7.1
Domain model............................................................................................................................. 43
7.2
City model ................................................................................................................................... 44
7.3
Preference elicitation.................................................................................................................. 46
7.3.1
Short introduction into preference elicitation.................................................................... 46
7.3.2
Preference aggregation....................................................................................................... 48
7.4
Preference model........................................................................................................................ 49
7.5
Trip.Easy Session: Group Decision-Making process .................................................................... 50
7.5.1
Task decomposition ............................................................................................................ 50
7.5.2
Agenda building .................................................................................................................. 52
7.6
8
Graphical User Interface ............................................................................................................. 54
7.6.1
Interface for the Generate step .......................................................................................... 55
7.6.2
Interface for the Waiting status(Screen 4, 6, 8).................................................................. 56
7.6.3
Interface for the Reduce step ............................................................................................. 57
7.6.4
Interface for the Clarify step .............................................................................................. 60
7.6.5
Interface for the Evaluate step ........................................................................................... 62
Implementation .................................................................................................................................. 64 8.1
Web application .......................................................................................................................... 64
8.2
Abstract architecture .................................................................................................................. 65
8.3
Technology .................................................................................................................................. 66
8.4
Development approach .............................................................................................................. 67
9
Conclusion ........................................................................................................................................... 67
PART III EVALUATION 10
Experimental design........................................................................................................................ 69
10.1
Research objective ...................................................................................................................... 69
10.2
Hypotheses ................................................................................................................................. 70
10.3
Measures..................................................................................................................................... 72
10.3.1
Satisfaction of the decision process.................................................................................... 72
10.3.2
Usability............................................................................................................................... 74
6
10.3.3
Fairness ............................................................................................................................... 79
10.4
Protocol ....................................................................................................................................... 82
10.5
Subjects ....................................................................................................................................... 83
10.6
Equipments and setting .............................................................................................................. 84
10.7
Destinations Database ................................................................................................................ 84
10.7.1
List of cities.......................................................................................................................... 85
10.7.2
Resources ............................................................................................................................ 85
10.7.3
Mapping .............................................................................................................................. 85
10.8 11
Pilot test ...................................................................................................................................... 86 Results ............................................................................................................................................. 87
11.1
Satisfaction of decision process .................................................................................................. 87
11.1.1
Construct: Satisfaction Process (SP).................................................................................... 87
11.1.2
Conclusion ........................................................................................................................... 88
11.2
Exploratory evaluation of the meeting process per step ........................................................... 89
11.2.1
Construct: Perceived Gained Net Attainment (PGNA)........................................................ 90
11.2.2
Evaluation of step 1: Generate ........................................................................................... 91
11.2.3
Evaluation of Step 2: Reduce - Filtering .............................................................................. 96
11.2.4
Evaluation of Step 3: Clarify – Building shared understanding ......................................... 103
11.2.5
Evaluation of Step 4: Evaluate – Communication of Preference ...................................... 109
11.2.6
Evaluation of Trip.Easy GDSS decision process in general ................................................ 116
12
Discussion and Conclusion of the Evaluation ............................................................................... 122
12.1
Part 1 of the evaluation ............................................................................................................ 122
12.2
Part 2 of the evaluation ............................................................................................................ 123
12.2.1
Usability............................................................................................................................. 123
12.2.2
Fairness ............................................................................................................................. 124
12.3
Final conclusion......................................................................................................................... 126
FUTURE WORK and Discussion 13
Future Work and Discussion ......................................................................................................... 127
13.1
Decision-making process .......................................................................................................... 128
13.2
Conflict resolution..................................................................................................................... 128
13.3
Types of trips ............................................................................................................................. 128
7
13.4
Travel features .......................................................................................................................... 128
Bibliography 14
Bibliography .................................................................................................................................. 129
Appendix 15
Appendix 1: Field research - Group Decision Support by a Travel agency ................................... 133
16
Appendix 2: Field research – Interview with domain expert ........................................................ 136
17
Appendix 3: Use Cases .................................................................................................................. 138
18
Appendix 4: City Database ............................................................................................................ 141
19
Appendix 5: Survey ....................................................................................................................... 144
19.1
Introduction .............................................................................................................................. 144
19.2
Questionnaire generate ............................................................................................................ 145
19.3
Questionnaire reduce ............................................................................................................... 146
19.4
Questionnaire clarify ................................................................................................................. 147
19.5
Questionnaire evaluate............................................................................................................. 148
19.6
Questionnaire general evaluation ............................................................................................ 149
19.7
Questionnaire Evaluation PGA .................................................................................................. 150
19.8
Questionnaire Evaluation SP+SO .............................................................................................. 151
19.9
Evaluation misc. ........................................................................................................................ 152
20
Appendix 6: Survey processing ..................................................................................................... 153
20.1
Gender ...................................................................................................................................... 153
20.2
Age ............................................................................................................................................ 153
20.3
Satisfaction Process (SP) ........................................................................................................... 153
20.4
Perceived Goal Net Attainment (PGNA) ................................................................................... 155
20.5
One sample t-test SP + PGA ...................................................................................................... 156
20.6
Evaluation Step “Generate”: ..................................................................................................... 158
20.7
Evaluation Step “Reduce”: ........................................................................................................ 163
20.8
Evaluation Step “Clarify”: .......................................................................................................... 169
20.9
Evaluation Step “Evaluate”: ...................................................................................................... 174
20.10
Evaluation Trip.Easy GDSS in general: .................................................................................. 179
20.11
One Sample t-test ................................................................................................................. 193
8
20.11.1
Generate step ............................................................................................................... 193
20.11.2
Reduce step................................................................................................................... 194
20.11.3
Clarify step .................................................................................................................... 195
20.11.4
Evaluation step.............................................................................................................. 196
20.11.5
Trip.Easy in General ...................................................................................................... 197
9
Table of Figures Figure 1: General consumer behavior model ............................................................................................ 23 Figure 2: Schmoll’s Travel Decision Model ................................................................................................. 24 Figure 3: General model of a collaboration problem ................................................................................. 29 Figure 4: The six general patterns of collaboration with their sub-patterns .............................................. 31 Figure 5: Example of a Facilitation Process Model ..................................................................................... 36 Figure 6: Overview of functionalities of the Trip.Easy GDSS prototype ..................................................... 41 Figure 7: Abstract Domain Model of Trip.Easy GDSS .................................................................................. 43 Figure 8: Representation of a city based on Reutsche (2006) .................................................................... 45 Figure 9: Preference model......................................................................................................................... 49 Figure 10: Facilitation Process Model ......................................................................................................... 53 Figure 11: Basic Trip.Easy GUI ..................................................................................................................... 54 Figure 12: Interface for the Generate step ................................................................................................. 55 Figure 13: Interface during the waiting state ............................................................................................. 57 Figure 14: Interface for the Reduce step .................................................................................................... 58 Figure 15: Interface for the Clarify step ..................................................................................................... 60 Figure 16: Interface for the Evaluate step .................................................................................................. 62 Figure 17: Abstract Infra structure Web-application .................................................................................. 64 Figure 18: Abstract architecture of Trip.Easy GDSS .................................................................................... 65 Figure 19: Abstract conceptual model ........................................................................................................ 72 Figure 20: Meeting Satisfaction as a function of Perceived Goal Attainment ............................................ 73 Figure 21: Schematic outline of the various steps of the experiment ........................................................ 82 Figure 22: Test room arrangement ............................................................................................................. 84 Figure 23: Pilot test ..................................................................................................................................... 86 Figure 24: Exploratory evaluation of meeting process per step ................................................................. 89 Figure 25: Measures with 95% CI regarding the usability concept in the Generate step .......................... 91 Figure 26: Measures with 95% CI regarding the fairness concept in the Generate step ........................... 95 Figure 27: Measures with 95% CI regarding the usability concept in the Reduce step.............................. 97 Figure 28: Measures with 95% CI regarding the fairness concept in the Reduce step ............................ 101 Figure 29: Measures with 95% CI regarding the usability concept in the Clarify step ............................. 104 Figure 30: Measures with 95% CI regarding the fairness concept in the Clarify step .............................. 107 Figure 31: Measures with 95% CI regarding the usability concept in the Evaluate step .......................... 110 Figure 32: Measures with 95% CI regarding the fairness concept in the Evaluate step........................... 114 Figure 33: Measures with 95% CI regarding the usability concept of Trip.Easy in general ...................... 116 Figure 34: Measures with 95% CI regarding the usability concept of Trip.Easy in general ...................... 120 Figure 35: Representation of the actors’ profile....................................................................................... 134
10
Table of Tables Table 1: Overview of the preferences concerning holiday features........................................................... 20 Table 2: Overview of the found holidays .................................................................................................... 21 Table 3: A possible agenda for the travel decision-making problem ......................................................... 35 Table 4: Overview of aggregation methods vs. our criteria ....................................................................... 48 Table 5: Agenda .......................................................................................................................................... 53 Table 6: Satisfaction (SP) ............................................................................................................................. 88 Table 7: PNGA ............................................................................................................................................. 90 Table 8: Overview Usability one-sample t-test Step Generate................................................................... 94 Table 9: Overview Fairness one-sample t-test Step Generate ................................................................... 95 Table 10: Overview Usability one-sample t-test Step Reduce .................................................................. 100 Table 11: Overview Fairness one-sample t-test Step Reduce................................................................... 102 Table 12: Overview Usability one-sample t-test Step Clarify ................................................................... 106 Table 13: Overview Fairness one-sample t-test Step Clarify .................................................................... 108 Table 14: Overview Usability one-sample t-test Step Evaluate ................................................................ 113 Table 15: Overview Fairness one-sample t-test Step Evaluate................................................................. 115 Table 16: Overview Usability one-sample t-test Trip.Easy in general ...................................................... 119 Table 17: Overview Fairness one-sample t-test Trip.Easy in general ....................................................... 121
11
Introduction
12
1 Introduction This study describes and analyzes an intelligent Group Decision Support System (GDSS) that creates synergy between human and machine intelligence in order to improve group decision-making. An intelligent GDSS named Trip.Easy has been developed for the travel domain. Trip.Easy GDSS supports a group of travel consumers with choosing a city trip that represents a best-fit to the group. Trip.Easy GDSS provides users a well-designed decision process and intelligent algorithms for deriving group preferences using available rich sources of travel information that facilitates groups to converge on a consensus. In the first part of this chapter, the problem and current situation in the domain of decision support in the travel industry is explored. The second part presents the objective for this thesis project. The applied methodology within this study will then be described in the third part of this chapter. Finally an overview of the structure of this thesis is given.
1.1 Problem definition Tourism is travel for recreational, leisure or business purposes and has become a popular global leisure activity. According to the United Nations World Travel Organization (UNWTO)1, over 922 million international tourist arrivals were established in 2009. Evidently, before travelling to the picked destination of choice, most of the tourists went through some decision-making process. Substantial consumer decision-making research in tourism behavior has grown exponentially during the past three decades. This resulted in many decision-making models and processes that support tourists in finding suitable tourists’ destinations based on their personal preferences (Sirakaya & Woodside, 2005). Additionally, a majority of those decision models and processes are focused on an individual as the decision-maker. Consequently, living in an internet era, where online decision support is increasing fast (Shim, Warkentin, Courtney, Power, Sharda, & Carlsson, 2002), more and more decision support for individual decision-making for the travel domain is applied by travel agents and independent websites. Those applications’ functions are referred to as decision support systems (DSS). However, tourism mostly occurs in social situations. It is a social activity that involves family, relatives or friends. Group decision-making within a group of people often leads to suboptimal decisions. Imagine a group of friends, colleagues, or family that wants to plan a trip but has not yet decided on where to go. Any group like this needs to: • • • • • •
organize a group discussion, and set up a (virtual) meeting; identify group activity opportunities (possible trips); discuss pros and cons of the alternatives that are available; rank alternatives based on individual and group preferences; converge on a group consensus by selecting one alternative, and organize the group activity.
1
UNWTO is a United Nations agency dealing with questions relating to tourism. The World Tourism Organization is a significant global body, concerned with the collection and collation of statistical information on international tourism.
13
In (Mansfeld, 1992), (Nichols, 1988) and (R.Gitelson, 1995) it was found out that members of a travel party are often not very satisfied with certain parts of the vacation that was chosen by the group. This could be explained by the fact that the process of travel group decision-making is typically ineffective. Some of the most common problems which may cause by an unstructured decision process are: • • •
•
people overlook alternatives (trip destinations in this case) that would fit best with what the group wants most; people are Influenced by a multitude of factors, which may constraint or motivate them to act irrationally, people are not able to communicate effectively to find an satisfying outcome for everyone, especially when meeting(s) needs to be organized between participants who reside at different locations, and during the decision process it is hard to fairly involve every group member in establishing a consensus.
Although many online DSS’s for travel as mentioned earlier and groupware solutions have become available to provide some support, both are limited. The current functioning online decision support systems in the travel industries are just websites with a lot of travel packages (comprehensive catalog) with some filter functions or some user generated content like ratings and remarks. Groupware solutions typically only provide the means to structure and record communication within a group in order to facilitate group interaction. Solutions that add value by providing intelligent advice to a group in order to converge on a group consensus through a well-designed group decision process are currently not available. During this thesis project we had the ambition to find a solution that overcomes these issues by building an online intelligent Group Decision Support System that is useful for practical purposes. As a result, we developed and evaluated a prototype that we named Trip.Easy. Trip.Easy is an intelligent Group Decision Support System that facilitates users with all steps identified above to reach a group consensus on a city trip. In other words, it organizes a group to decide on a trip by facilitating them with a well-designed group decision support process incorporating: • • • • •
creating a group (travel session) by inviting friends, family, colleagues, etc; eliciting individual travel preferences to ensure equal and fair involvement of group members; providing consensus stimulating information to create awareness and build understanding ; supporting users in building motivations and preferences for travel destinations by providing relevant information; and finally converging on a group consensus by structured, decision-making.
Many scientific challenges had to be addressed in order to develop the Trip.Easy GDSS. The expertise in Artificial Intelligence, Human-Machine interaction and Collaboration Engineering has been combined. This appeared to be a large and complex project and therefore we have divided it into two sub-projects. The division is based on the fact that a collaborative meeting can be decomposed into two subjects: a meeting outcome and a meeting process. More details about the collaborative meeting will be explained
14
in the background section. As a result, one sub-project is about the design, development and evaluation of the meeting outcome. This consists of preference aggregation and the recommendation engine of Trip.Easy, which is covered in the thesis of Touw (2010). This thesis covers the design, development and evaluation of the meeting process. This consists of the collaborative decision process including the preference elicitation and user interaction of Trip.Easy.
1.2 Research objectives The aim of the current study is twofold. First, the goal of the Trip.Easy GDSS is to support a group of travel consumers with choosing a city trip that represents a best-fit to the group. Therefore, to realize Trip.Easy GDSS we will deal with challenges such as the design and implementation of: • • • • • •
an extendable and modular GDSS architecture; a domain model for city trips; a preference model for travel consumers; a preference elicitation procedure; a format of the data representation in order to integrate with intelligent aggregation algorithms2; and finally a well-structured collaborative decision process.
Consequently, the objective to establish a prototype of the Trip.Easy intelligent group decision support system could be reflected in the following first main research question:
Can we develop a Group Decision Support System that combines a well-structured decision process with domain knowledge and an intelligent recommendation mechanism that facilitates reaching a consensus for a group trip planning problem? The second aim of this study is to explore whether the Trip.Easy GDSS is capable of supporting users satisfactorily when making a consensus based decision about their “city” trip. If people were to dislike a meeting (Trip.Easy session meeting) because of the technology used to conduct meetings, they would be less likely to use that technology in the future. Even if it were to help them to produce better results (George, Easton, & Nunamaker, 1990). As described in the previous paragraph, this thesis research will cover the meeting process part of the Trip.Easy GDSS. Therefore, this part of the study will focus on the evaluation of the designed meeting process (decision process), which leads us to the second main research question:
How do Trip.Easy GDSS users evaluate the decision process for establishing a consensus?
1.3 Methodology To achieve the Trip.Easy GDSS, a successful integration and application of ideas and results of various fields of research is required. This includes: 2
This will be the focus in the thesis of Touw (2010)
15
• • • • •
marketing (consumer behavior and decision making); tourism (domain knowledge); artificial intelligence (preference elicitation, aggregation and recommendation mechanism); Human-Machine Interaction (usable graphical user interfaces) and finally; and collaboration engineering (collaborative decision process).
This thesis reports on the tasks that resulted in the full working prototype Trip.Easy GDSS. The tasks are described below in chronological order. Decision-making tourism survey - We start the research by examining the need and the current support for group decision-making in the travel domain by conducting a field research. Travel agents, regular people and tourism experts were interviewed. The interviews resulted in a greater understanding of how decision-making in the travel domain works. Furthermore, it shows that currently no usable system (neither machine, nor human) exists that can widely support a group of users with planning a trip on a consensus-based way. (Group) Decision Support Systems survey - Since the rise of the internet, different web applications have been developed in which travelers are supported individually in deciding between different travel products. The current decision support systems within the travel domain are focused on individuals instead of groups. The survey shows us what is currently available for the travel domain and what is available concerning group decision support systems in other domains. Literature survey - Given the need of group decision-making in the travel domain and the list of currently available group decision support systems in different domains, we conducted a thorough literature research of the travel domain. Moreover, the following topics are also detailed: background theories on consumer decision-making, collaborative decision processes and technical issues like preference elicitation. Field research for obtaining a group decision process - Several field researches were conducted to collect a representative set of tasks within a group travel planning problem. First, an interview among travelers and a tourism scholar were held and analyzed to determine, adjust and negotiate about the requirements and constraints on the tasks to be performed during the group travel planning. Second, two groups of 8 people were observed how they collaborate on distance (by email and chat) for selecting a consensus based travel destination. Finally, a field research was conducted on several travel agencies to analyse the tasks during the group decision process. Without any announcements of our intention we (two ‘actors’) walked in a preselected travel agency and started to chat with the travel. A fictional pre defined group travel case was submitted to them and we observed how the travel consultants supported us in picking a consensus-based travel destination. Design and implementation of the system - Based on the knowledge gained from the literature survey and the field researches, the Trip.Easy GDSS prototype was developed. The requirements and the scope of the Trip.Easy GDSS were set against the technical possibilities and limitations identified in the literature survey. By applying software and collaboration engineering, a design is developed that consists of: 16
• • • • • • •
a modular and scalable web architecture; an extendible group decision support framework; a domain model (knowledge); a (group) preference model; a preference elicitation procedure; a collaborative decision-making process; and a graphical user interface including interaction design.
Finally, the design was implemented into a fully working prototype of the Trip.Easy GDSS. Testing and Evaluation - Evaluation was needed in order to assess and test whether the decision-making process of Trip.Easy GDSS behaves as we expect and meets the requirements. To test and evaluate the decision-making process we are in need of data of groups of people who are planning to go on a short city trip using the Trip Easy group decision support system. Therefore an experiment was designed and conducted to gather empirical data, which was processed by statistical tests. By analyzing the available data we were able to determine whether users are satisfied by using the Trip.Easy GDSS technology as a support for consensus-based travel destination picking.
1.4 Outline of this thesis The thesis is divided into three parts. Part I contains an overview of the problem domain providing the background and the context for the research. The design of a well-structured decision-making process and the design of the Trip.Easy GDSS prototype will be presented in part II. Part III presents the experiments and the results of the evaluation. Part I – is constituted of chapter 2 to chapter 4. This part starts with an introduction and the motivation leading to this research, which is followed by a detailed summary of relevant theories that form the starting point of this research. The foundation of decision-making theory in the travel domain will be presented in chapter 3. Chapter 4 provides the theoretical background of how to design a useful and satisfying group decision process. Part II – describes the development and implementation of the Trip.Easy GDSS. Chapter 5 gives an introduction of the design and development approach. In chapter 6, the scope and the requirements of the intended intelligent Group Decision Support System will be defined. In chapter 7 the design of the fundamental components like the domain model, city model and preference model will be presented. Furthermore, the design of the group decision-making process and its Graphical User will also be depicted. Finally, the implementation of the Trip.Easy GDSS is covered in chapter 8. Part III – is the final part of this thesis which presents the experiments and the results of the evaluation. Chapter 10 describes the experiment setup including the research objectives, hypotheses, measures and protocol in order to conduct the experiment. The data obtained from the experiment will be analyzed in chapter 11. Conclusions of the findings will be presented in chapter 12. Chapter 13 finishes with an overall discussion and suggestions for future work.
17
Part I
Background
18
2 Introduction In this first part of the thesis we cover the motivation and theory related to the present study. The present study aims to design and implement the Trip.Easy intelligent group decision support system (GDSS) and to evaluate the performance of this system. First, chapter 3 discusses decision-making theory in the travel domain. A concrete group travel planning problem is used to illustrate the decision making procedure of a group of friends. Thereafter, we analyze this example travel planning within a group in more detail. From there on we will take a close look to how groups of people go about making decisions, starting with the earliest and most influential “grand model” of consumer behavior. This model is further adapted by tourism scholars as a starting point for explaining the process used to purchase tourism services. A summary of some interesting travel decision making models is given and some of the benefits and limitations are discussed. This discussion will illustrate that the existing models do not take into account the group interactions and roles that may influence what product/destination will be chosen. Chapter 4 provides the theoretical background of how to design a useful and satisfying group decision process while taken into account the decision making issues of travelers mentioned in chapter 3. We show that a group decision problem in fact can be seen as a collaboration problem. Achieving consensus by effective collaboration between peoples remains a challenge. The Collaboration Engineering discipline is an approach to design and deploy high value recurring collaborative work practices that can be executed by practitioners by themselves without ongoing support from professional facilitators. This approach is very suitable for developing our intelligent group decision support system Trip.Easy.
19
3 Decision making in the travel domain 3.1 Scenario: planning a trip in a group John, Sam, Melody and Eva have been friends for a few years now. They decided that it would be great to go on holiday together. In order to plan a trip together they organized a meeting at John’s place. First they discussed everyone’s ideas about the holiday in terms of expectations, destination and activities. They all like sunny weather and want to relax a little. But more concrete information about possibilities had to be collected. They decided that each of them should have a look on the internet and visit travel agencies to pick up some travel magazines to find out what’s available and what they individually prefer. After that, new appointments (always a cumbersome activity) were made and they met again to discuss the findings. This time they were better informed and thus more holiday features were taken into account, such as: accommodation, transportation, duration, destination, budget, distance and the type of attraction. During the meeting which was unstructured and informal, as in most cases, they shared their concrete preferences and findings. An overview of the preferences concerning the holiday features mentioned in the previous paragraph is showed in Table 1. Not all of the features are filled in yet; e.g., distance and destination are still missing. They discovered that they have different preferences concerning certain features and that this could lead to conflicts. In Table 1 we see that John, Sam, Eva and Melody all have different preferences about the features. As said before, they all want to relax, but apparently John finds ‘hiking through the mountains’ a relaxing activity while for Sam the only way to relax is ‘laying on the beach’. Name
Accommodation
Duration (weeks)
Transportation
Budget
Type of attraction
John
Camping
2
Car
750
Sam
Hostel/Hotel (3 star rating) No preferences Hostel/Hotel (2 star rating)
2
Plane
1200
1 3
No preferences Train
1000 850
Hiking through mountains Laying on a sunny beach Shopping Surfing
Eva Melody
Table 1: Overview of the preferences concerning holiday features
While concessions are expected to be inevitable, the next step was to find a holiday arrangement or travel plan which matches the preferences of our friends as much as possible. As they scroll through magazines and websites they find some destinations that might be interesting. In Table 2 the outcomes are listed.
20
Preferred by
Accommodation
Duration (weeks)
Transport
Budget
Eva, Sam
Hotel (4 star rating)
1
Plane
1000
John, Melody
Camping
3
Hitchhiking/train /bus
1000
Sam, Eva
Hotel (3 star rating)
2
Plane
1200
Type of attraction
Destination
Distance
Surf facilities, Mountains, shopping centre, beach Surf facilities, Beach, Shopping, Mountains Surf facilities, Beach and shopping centre
Al Garve, South- Portugal
Near
South-WestFrance
Near
Miami, SouthEast USA
Far
Table 2: Overview of the found holidays
The following step is to build consensus. John and Melody find that the travel destination Al Garve is no option because of the costs of 1000 euro per person per week. Though camping isn’t something Eva had in mind when she thought of a relaxing holiday, John tries to convince her that a holiday is not about luxury but about adventure. Sam raises that Miami is the best option because for just 1200 euro you are able to go to a far destination with excellent shopping facilities, surfing facilities and of course Miami beach. Sam is very skilled in convincing people and manages to convince the group that Miami is the best deal. Even though John cannot go hiking through the mountains and needs to borrow extra money to be able to go on holiday, he doesn’t want to spoil it for the rest of the group. Our friends are tired of discussing and don’t want to look any further, so they decide to go to Miami. To summarize, we learned from this scenario that our friends did not want to spend too much effort in planning a trip. First of all they only considered South-West-Europe and the US. Since there are so many options, it’s impossible to consider them all and most of the time the search is based on an awarenessset that people have. What does a person think of when he or she thinks of a sunny destination? Some think Spain other South-East Asia. In addition, we clearly see the influences that certain group members have. Sam actually managed to convince the others to do what he preferred most. These influences are always present in group decision processing. Moreover, communicating effectively with each other is not trivial, because formulating or motivating well why some preferences or choices fit best to the group is not always that easy. Finally, due the unstructured decision process it is hard to involve everyone in establishing a consensus. As we have seen, the group was tired of the endless discussion due the countless possibilities, the confusing state about everyone’s preferences and what is best for the group, the lack of conflict resolution solution and the lack of a facilitator to structure everything and to lead the decision process.
21
3.2 Group travel planning problem In the previous section the scenario showed that planning a vacation is not trivial. We have seen that when a group of friends, colleagues, or family wants to plan a trip but has not yet decided on where to go then any group like this needs to: • • • • • •
Organize a group discussion, and set up a (virtual) meeting, Identify group activity opportunities (possible trips), Discuss pros and cons of the alternatives that are available, Rank alternatives based on individual and group preferences, Converge on a group consensus by selecting one alternative, and Organize the group activity.
In Mansfeld (1992), Nichols (1988) and Gitelson (1995) it was found that members of a travel party are often not very satisfied with certain parts of the vacation that were chosen by the group. This could be explained by the fact that the process of travel group decision making is typically ineffective and challenging for people. Some of the most common problems are that people: • • • •
•
At different locations need to organize themselves to arrange a discussion, Tend to overlook opportunities and selection criteria for settling on an agreement, Influenced by a multitude of factors, which may constraint or motivate them to act irrationally, Not able to communicate effectively to find an outcome that is satisfying to everyone, especially when communication needs to be organized between participants that resides at different locations and, Usually hard to fairly involve every group member in establishing a consensus.
These issues were also illustrated in our scenario and occurred as a result of an unstructured group decision making process which often leads to suboptimal decisions (travel destination here).
3.3 Theories and models of decision making by travelers In order to solve the travel group planning problem we need to understand the general process of the consumer decision making process first.
3.3.1 Consumer decision making Scholars from a variety of social science disciplines focus on how individuals go about making decisions (Solomon, 2006), (A. Pizam, 1999) and (Sirakaya & Woodside, 2005). In the field of marketing a substantial body of decision-making literature was built since the 1950s. In the literature, we found pioneering decision making models which are the earliest and most influential models provided by Nicosia (1966), Engel, Kollat and Blackwell (1968) and Howard and Sheth (1969). These three models are considered the “grand models” of consumer behavior. These models describe the decision making process of a consumer. A consumer’s purchase reflects his or her need recognition concerning the purchased product or service. At a certain moment the consumer realizes the need for a certain good or service and from that moment he or she goes through a series of steps and stages in order to make the
22
right purchase. It has been widely held among researches in the consumer behavior field that the consumer’s decision to purchase is a multi-staged process, based on the so called informationprocessing theory which is central to all consumer behavior models. The information-processing theory, drawn from consumer behavior literature, as outlined by Solomon (2006) and Sirakaya & Woodside (2005), identified that consumers transit the stages of: postpurchase evaluation
need recognition
choice of product or sevice
information search
evaluation of alternatives Figure 1: General consumer behavior model
Need recognition - Need recognition is the first stage in the buyer decision process in which the consumer recognizes a problem or a need. Information search - Information search is the stage in the process where the consumer is aroused to search for more information. This state of arousal may encourage the consumer to go in active search of information or may heighten their attention to relevant information sources including advertising. Evaluation of alternatives - During this stage different solutions of the problem (need) are being evaluated. The evaluation process is complex, because it varies per situation. However, in general terms the consumer will examine the attributes of the product, assign different levels of importance to such attributes, determine the likely level of overall satisfaction with each alternative and derive an attitude toward the different solutions/brands. Choice of product or service - The consumer purchases the product or service that it considers to be the best solution for its need (problem). Post purchase evaluation - In the evaluation stage, consumers rank brands and form purchase intentions for future. Evidently, this decision-making process is influenced by both psychological (internal variables), for example, attitudes, motivation, beliefs and intentions, and non-psychological (external variables) (e.g. time, pull factors and marketing mix) (Sirakaya & Woodside, 2005). Given the centrality of the selection
23
decision process to tourists’ behavior, a clear understanding of the complexities and interrelationships of these variables is important. In the next section we pay attention to what this model looks like in the travel domain.
3.3.2 Instantiation of the consumer behavior model In A.Pizam (1999) and Sirakaya & Woodside (2005) it is claimed that the ‘tourist purchase decision’ is also based on the stages as presented in the consumer behavior model from the previous paragraph. According to (A. Pizam, 1999) and (Sirakaya & Woodside, 2005) all decision making goes through the same sequential process. The travel decision models therefore are seen as an instantiation of the consumer behavior model (Figure 1) by tourism scholars (Sirakaya & Woodside, 2005). The tourism literature shows multiple conceptual and empirical decision theories to describe tourists’ destination choice. These models provide us insight into which variables and factors will influence the decision making of the final travel destination for an individual. Also, it can help us to design our group decision support system to support the travel consumers to make optimal decisions. 3.3.2.1 Schmoll’s Tourist Decision-Making Process model The Travel decision Process model of Schmoll (1977) drew much attention. This model is based on the Howard and Sheth (1969) and Nicosia (1966), the “grand models” of consumer behavior). He believes that all decision making goes through the same process, which may be instantaneous or take years, but which goes through the same steps. Schmoll’s decision-making process model is the only model that pays attention to constraints and their impact on the decision-making process (A. Pizam, 1999). It explicitly specifies the relationships between various components and shows which factors have influence on choice decision. Furthermore, this model relates theoretical concepts to real world.
Figure 2: Schmoll’s Travel Decision Model
24
The model consists of four components: 1. Travel Stimuli – External stimuli such as trade publications 2. Personal and social determinants of travel behavior – Travel needs and desires determined by personality, socio-economic factors, attitudes, and values 3. External variables – External variables such as confidence in the travel agent, destination image, previous experience, and cost and time constraints 4. Characteristics and features of service distribution – Destination- or service-related characteristics that have a bearing on the decision process and its outcome The ultimate choice of a destination will depend on the nature of interaction among these variables. The assumptions throughout the model are that decision-makers exhibit rationalistic behavior in their choices among alternative destinations (Sirakaya & Woodside, 2005). They will select a destination, which offers the greatest utility subject to the individual constraints. The selection process is a funnellike one, in that travelers narrow down choices among alternatives and are influenced both by sociopsychological factors and non-psychological. With this model of Schmoll’s we have a comprehensive overview of how the individual travel decisionmaking process works. Therefore we think this model is worth elaborating on. This supports us in gaining more insight in how to design and develop a group decision process.
3.3.3 Travel group decision-making In the previous section we described the individual decision making process. We learned that different factors have influence on the decision behavior of the individual travel consumer, such as the image of destination or the cost of the accommodation. However, people usually go on vacation within a group, it is a social activity that involves family, relatives, friends and other, where every member participates in the decision making process. The treatment of an individual decision maker, as if they were in a vacuum, is common to all decisions making models (Sirakaya & Woodside, 2005), including the model of Schmoll (1977). These models accept that other individuals affect the decision-maker but do not address active interaction with other individuals or sources along the decision-making process. The existing models lack the integration of these issues into a single unique model that is theoretically sound, complex, and still useful for practical purposes. Ten decision-making models in tourism were evaluated in E. Sirakaya and A.G. Woodside (2005). Based upon this work it appears that there are no models that describe the travel decision process of social groups like friends and colleagues. This lack, however, has been admitted by most of the tourism behavior models mentioned. The level of communication, the type of decision making (consensus, bargaining, voting, dictatorship), and the result of confrontation (agreement versus conflict) were mentioned as important determinants of these group decision processes in Mansfeld (1992) and Sirakaya and A.G. Woodside (2005).
25
In A. Pizam et al. (1999) it was observed that the smallest group affecting an individual’s decision is a family. Prior studies mainly centered on who “gives the orders” on family travel. There are primarily three types of decision-making modes (in terms of how the decision making process is being influenced within this group): • • •
husband-dominant wife-dominant joint decision between husband and wife
In most cases the decision generally results from joint decision-making (A. Pizam, 1999). Most of the time a family does not only exist of a husband and wife but also children. Studies like (A. Pizam, 1999), (Wang, 2004) and (Nichols, 1988) also took the influence of children into account in the travel decision process. Richard Gitelson and Deborah Kersteller (Gitelson & Kerstetter, 1995) did a first step to determine the extent to which friends and/or relatives influence the travel decision-making process beyond the role of information provider. It appears that during decision making in some social setting like friends or colleagues each individual automatically plays some specific role. Each role has a particular influential power associated with it. The following influential categories of roles are distinguished: • • • • •
Sole decision maker, i.e., 100% responsible for the decision Dominant influence, i.e., the greatest percentage allocation of any decision maker Shared influence, i.e., equal influence with at least one other decision maker and more than any other decision maker Lesser role, i.e., a percentage of influence less than some other decision maker, or No influence.
Now that we have described the influences within groups, we pay attention to the consequences of those influences on the travel decision-making process. Regarding the influential categories of a participant, we notice that in all categories the influences aren’t equal; one or some participants of a group have greater influence on the decision making process than others. These influences cause suboptimal outcomes from travel group decision making, because the influences are affecting all stages of the consumer behavior model depicted in Figure 1. For instance the choice of product is done without taking all the participants’ preferences into account, but only those of some participants.
26
3.4 Summary In summary, the group travel planning problem is challenging. So far no single unique group decisionmaking model or process that is theoretically sound and still useful for practical purposes has been developed (Sirakaya & Woodside, 2005) by tourism scholars. However, the literature (e.g. Mansfeld (1992), Nichols (1988), Pizam (1998), Sirakaya & Woodside (2005) and Gitelson (1995)) has demonstrated some of the limitations of the current decision-making models. The models that consider the individual as the decision-making entity remain limited because tourism is a social activity and thus involves joint decision-making processes. Family, friends and other social settings impact an individual’s evaluation (weighing) of alternative attributes, thus the preference for some travel destination. Due to the lack of a structured joint decision-making process, the ineffective way of travel group decision making causes suboptimal decisions. Therefore, leading a travel party to a satisfied and optimal decision, a structured group decision-making process should take the following factors into account: active interaction with other individuals, level of communication, type of decision making, result of confrontation and the influential power of roles. For more background information about consumer behavior in travel and tourism please consult the literature study (Ngai K. , 2008).
27
4 Collaboration Engineering In the previous chapter several issues were mentioned that affect the behavior of each individual during group travel decision-making (i.e. information overload, dominant peoples, bad interaction within the group, no conflict resolution mechanism, etc). In order to develop a group decision support system that can deal with those issues, we took some fundamental ideas from the Collaboration Engineering discipline that can help us to define useful and satisfactorily joint decision processes. The relevant questions are: Which steps and activities have to be taken by each individual to reach the goal? What is the sequence of actions in this collaboration process (e.g. retrieve an exact preference model for each individual member before recommending a travel plan versus recommend possible travel plans first and let users provide feedback to determine their preference model)? “The Collaboration Engineering discipline is an approach to design and deploy high value recurring collaborative work practices that can be executed by practitioners by themselves without ongoing support from professionals” (Briggs R. , 2003). Collaboration engineering can be a starting point in designing a group travel decision making process, in which the goal of the group is to work together on a travel plan that will be accepted by all members. We can see this group travel decision-making problem as a collaboration problem. To clarify this, we start with explaining what collaboration actually is in the next section.
4.1 Collaboration “The term collaboration comes from the Latin word collaborare. Collaborare means “work with”, which is derived from com, meaning “with” + labore, meaning “to work”. Thus, collaborative efforts are joint, rather than individual. If collaborative efforts are joint, then they must be directed towards a group goal. A goal is a desired state or outcome. Thus, collaboration involves multiple individuals who combine their efforts to achieve mutually desired outcomes. Therefore we define collaboration as joint effort towards a group goal” (Briggs, 2006). Note that this definition does not require that members concur on the merits of a group goal, nor that they necessarily feel happy about the group goal; it only requires that members for whatever reason make effort to achieve the group goal. So collaboration can be seen as a goal- or outcome-directed interaction between two or more people (teams, groups). To solve a collaboration problem some process must exists. Through interacting within a series of action steps, it utilizes a set of resources (people, technology) to transform the group’s present problem state into its desired future state (accomplishing specific meeting outcomes). Figure 3: General model of a collaboration problem illustrates a model of a typical collaboration problem.
28
Figure 3: General model of a collaboration problem
Action steps can be described in terms of a core set of generic activities3. A sequence of action steps organized in a well planned time schedule is called an agenda. For example, to accomplish a particular topic, a group might generate information; organize the information into alternatives; evaluate and select alternatives; and discuss (build consensus) their actions. These generic activities can be used to describe an agenda for any decision task (Bostrom, 2002). The general model of a collaboration problem (Figure 3) also depicts two general types of outcomes of a collaboration process: task and relational. From a task outcome perspective, a collaboration process brings together a set of resources (primarily people) to accomplish a task (Bostrom, 2002). The task provides the “content” or “what people will be interacting about” in a collaboration process. There are many tasks that can be accomplished in a collaboration process: creating a travel plan, solving a problem, negotiate about the price, resolving a dispute, sharing information, making decisions etcetera.
3
Generic activities are steps of activities decomposed from tasks. The activities will be performed by users in a collaboration.
29
From a relational outcome perspective, a collaboration process is a relationship between people (Bostrom, 2002). The relational outcome of a collaboration process is to create and maintain positive emotions that will lead to constructive relationships that promote working together effectively. Thus, in an effective collaboration process, negative affect is not avoided but instead refocused in a positive direction. Based primarily on how participants feel about or react to certain aspects of the meeting, this relationship can be broken into the following four sub-relationships: 1. Content of interactions (task). Each participant has various ways of reacting to the content, like; travel plans, decisions, information, and so on, being created. 2. The feelings group members have toward each other (interpersonal); these feelings are often reflected in the amount rapport, openness, trust, and cohesiveness in the group. 3. The ongoing interactions (process), for example, how participants react to the agenda and activities. 4. Reactions of participants to themselves and their contributors (self). How a person reacts to himself or herself can affect that person’s self-esteem or self-efficacy and can produce certain feelings (Bostrom, 2002). Each of these sub-relationships (task, interpersonal, process, and self) provides a source of emotions in a collaboration problem that influence the development and quality of these sub-relationships. In most collaboration problems, there is some combination of both task and relational outcomes to achieve. Even in cases in which task outcomes are strongly emphasized, good relationships need to be developed and maintained in order for the group to work effectively.
4.2 Collaboration patterns In the previous sub-section it was shown that achieving effective team collaboration is a challenge as it requires achieving a number of challenging goals. Collaboration processes need to be explicitly designed, structured and managed to maximize the outcome given the effort. As people work together toward a goal, they move through some process in which they must combine their expertise, their insights, and their resources. The interaction between people must be done in a way that accommodates the needs and interests of each individual stakeholder. Because of the limitations of the human mind, many collaborative efforts result in suboptimal results (de Vreede & Briggs, 2005). A well-structured design of a collaboration process should improve the collaboration. In Collaboration Engineering (CE), design patterns are used to support the design collaboration processes (Briggs R. , 2003). Design patterns are descriptions of known, reusable solutions to recurring problems. To successfully design collaboration processes with predictable outcomes, we must know how the implementation of collaborative patterns will produce specific outcomes (Kolfschoten, Lowry, Dean, & Kamal, 2007). By predictable group intervention we mean an understanding of how a specific intervention – which evokes specific patterns of collaboration – will result in specific outcomes.
30
CE defines general patterns of collaboration to classify group activities. Patterns of collaboration characterize a group activity as the members move from an initial state to a next state (Briggs, 2006). The six general patterns of collaboration including their sub-patterns and the intervention are listed in Figure 4.
Generate
Reduce
Creativity
Filtering
Clarify
Organize
Categorizing
Evaluate
Build consensus
Choice: Social or Rational
Sense making
Choice
Outlining
Gathering
Summarizing
Sequencing
Building shared understanding Reflecting
Communication of preference
Building agreement / visualizing divergence of preferences
Qualitative evaluation
Building commitment
Building causal decomposition
Abstracting Modeling
Figure 4: The six general patterns of collaboration with their sub-patterns
Details about the patterns are listed below and are taken from the (Kolfschoten G. , 2007). Generate: Move from having fewer to having more concepts in the pool of concepts shared the by group. Sub-patterns Creativity
Gathering
Reflecting
Pattern definition Move from having fewer to having more new concepts in the pool of concepts shared by the group Move from having fewer to having more complete and relevant information shared by the group Move from having fewer to having more valid information, shared and understood by the group
Intervention Participants add ideas within the scope of the topic Participants add information to describe the topic of interest Participants add comments to concepts under reflection, challenge the group questioning assumptions and intended qualities of the information
31
Reduce: Move from having many concepts to a focus on fewer concepts that the group deems worthy of further attention. Sub-patterns Filtering
Summarizing
Abstracting
Pattern definition Move from having many concepts to fewer concepts that meet a specific criteria according to the group Move from having many concepts to having a focus on fewer concepts that represent the knowledge shared by group members Move from having many detailed concepts to fewer concepts that reduce the need to attend to details
Intervention Derive criteria and let participant choose concepts from the list that meet these criteria Distil the key concepts that represent the information generated or remove information that overlaps or that is redundant Identify more abstract concepts that are useful for the group’s purposes. Test whether these encompasses the information under consideration and meet the objectives of the group
Clarify: Move from having less to having more shared understanding of concepts and of the words and phrases used to express them. Sub-patterns Sense making
Building shared understanding
Pattern definition Move from having less to having more shared meaning of context, and possible actions in order to support principled, informed action Move from having less to more shared understanding of the concepts shared by the group and the words and phrases used to express them
Intervention Discuss the context and possible actions that can meet the group objective Discuss different meanings of concepts and words to achieve shared meaning
Organize: Move from less to more understanding of the relationships among concepts the group is considering. Sub-patterns Categorizing
Outlining
Sequencing
Building causal decomposition
Pattern definition Move from less to more understanding of the categorical relationships among concepts the group is considering Move from less to more understanding of the logical connections among concepts the group is considering Move from less to more understanding of the sequential relationships among concepts the group is considering Move from less to more
Intervention Participants identify categories and categorize concepts among the categories or let participants cluster concepts and label the clusters Participants identify and label logical connections among concepts in discussion Participants discuss to determine the sequential order of concepts and visualize the resulting sequence Participants identify and visualize
32
Modeling
understanding of the causal relationships among concepts the group is considering Move from a more to less complex overview of the concepts and relations the group is considering
causal relations among the concepts
Participants visualize relations among concepts and challenge the consistency of the resulting model
Evaluate: Move from less to more understanding of the relative value of the concepts under consideration. Sub-patterns Choice: Social or Rational
Pattern definition Move from less to more understanding of the concept(s) most preferred by the group
Communication of preference
Move from less to more understanding of the preference of participants with respect to the concepts the group is considering
Qualitative evaluation
Move from less to more understanding of the perspective of participants with respect to the preference of concepts the group is considering
Intervention Participants express their preference and aggregate a group vision, then discuss to build consensus or to determine the rational best choice Participants express their preference and discuss differences, then discuss to gain mutual understanding of preferences and disagreement Participants express their preference and perspectives on the concepts under evaluation
Build consensus: Move from having fewer to having more group members who are willing to commit to a proposal. Sub-patterns Choice
Building agreement / Visualizing divergence of preference
Building commitment
Pattern definition Move from less to more understanding of the concept(s) most preferred by the group Move from less to more understanding of the difference in preference among participants with respect to concepts the group is considering Move from less to more understanding of the willingness to commit of participants with respect to proposals the group is considering
Intervention Participants express their preference and visualize the group result Participants express their preference and assess agreement and disagreements among the group, then resolve disagreements with respect to the outcomes Participants express their willingness to commit to a proposal, negotiate, modify the proposal or argue to increase commitment for a (modified) proposal
These six fundamental collaborative patterns and their sub-patterns serve as high-level building blocks for collaboration process design. Frequently, a single group activity will produce more than one of these patterns. For example, for planning a city trip a group of friends can first generate a list of possibilities
33
and preferences in a brainstorming activity, and then reduce the list to eliminate redundancy, and then evaluate the likelihood and satisfaction of each possibility.
34
4.3 Design approach Collaboration Engineering (CE) has developed a step-by-step design approach to create a high quality collaboration process (Kolfschoten & de Vreede, 2007). It starts with analyzing the collaboration problem by performing a task diagnosis. To do this e.g. a conversation with the stakeholders involved in the collaboration problem has to be performed. The aim is to collect the goal and the requirements of the collaboration. When the goal and requirements are sufficiently clear, the collaboration process needs to be determined. This is done by decomposing the tasks from a collaboration problem into steps of activities that can be performed with the collaboration patterns. Patterns of collaboration characterize a group activity as the users move from an initial state to the next state. The activities from the analyzed collaboration problem can be directly matched to a pattern of collaboration. Subsequently, the agenda (a sequence of patterns) can be built. An illustration of an agenda for the travel decision-making problem could be represented as follows: Acti vity
Description
Question/Assignment
Deliverable
Collaboration pattern
Time
1
Collect and share known travel preferences from individual group members
Provide as much as you can your individual travel preferences
Gather
0.15
2
Identify groups preferences
Try merge the individual preference model into a group preference model
Reduce/Select
0.15
3
Gather
2.00
4
Search for a set of travel plans by using the group preference model Group the travel plans by similarity
Organize
0.15
5
Rank the travel plans
Evaluate/rank
0.30
6
Select one travel plan which the group members like most
List with individual preference model First set of group preferences A set of travel plans A set of grouped travel plans A order set of grouped travel plans Consensus based travel plan
Build consensus
1.00
Identify the order of the preferred travel plans
Table 3: A possible agenda for the travel decision-making problem
The first column is to identify and number the activities. The second column describes the task. An example of an activity is “give critiques to proposed travel plan” or “rank travel plans”. The next column is reserved for the specific questions or assignments to the group. The next column describes the deliverable: a specification of the expected output or a more general output like “ranking of the results”, “categorization of the travel plans”. The fifth column indicates the collaboration pattern it aims to evoke. The last column lists the estimated time needed for each activity. Based on the information in the agenda, the flow of the collaboration process can be graphically represented using a Facilitation Process Model (FPM). An example is illustrated in Figure 55. A FPM focuses attention on the logics that control the flow of the process from activity to activity. Each activity is represented as a rectangle that has three fields. The top left field gives the activity number, corresponding with the agenda. The top center field gives the name of the sub-pattern. The top right
35
field lists the time required for the activity. The largest field contains a description of what the team is supposed to do in the activity. Finally, the field on the left names the primary pattern of collaboration to be created in the activity. Decision points in the process flow are represented as circles and the decision outcomes are indicated along the flow arrows.
Figure 5: Example of a Facilitation Process Model
By following this agenda, participants should be able to collaborate towards a desired state (in this case a consensus based travel plan). Unfortunately, various researches have shown that in most cases, people are not capable to work effectively together, even with a pre defined agenda (Bostrom, 2002). Here comes the role of a facilitator in the picture. Facilitators can often move and guide a group through a collaboration process far more effectively and efficiently than would be possible if the group were left to its own devices (de Vreede & Briggs, 2005). Facilitators monitor the needs of the group and the requirements of the task, and when necessary, devise new interventions on-the-fly to make the group more productive. However, invoking a facilitator is an expensive opportunity that is often used for high-value and risk business purposes. They are typically not available to support collaboration problems like group travel decision processes. Nonetheless, their support could substantially improve the outcome (task of relational) of a group of peoples who want to go on vacation together. This is supported in our field research (Appendix 1: Field research - Group Decision Support by a Travel agency) and an interview with a domain expert from the travel industry (Appendix 2: Field research – Interview with domain expert).
36
4.4 Summary Group decision making can be seen as a collaboration problem. Interaction between two or more people (team, groups, travel party) collaborating towards a goal or outcome. To solve a collaboration problem some process must exists wherein an interaction that utilizes a set of resources (people, technology) to transform the group’s present problem state into its desired future state. Collaboration Engineering can be invoked to design efficient, effective and satisfying collaboration processes that can be executed by peoples through i.e. Group Decision Support Technology. The six fundamental collaborative patterns, namely generate, reduce, clarify, organize, evaluate and build consensus make up most if not all collaborative work. Using these (sub) patterns as building blocks, designing an agenda like in (Table 3) and a Facilitation Process Model (Figure 5), a group decision making process can be developed. By following the agenda (built from the collaboration patterns), participants should be able to collaborate towards a desired state. Unfortunately, various researches have shown that in most cases, people are not capable to work effectively together, even with a pre-defined agenda. Here comes the role of a facilitator in the picture. Facilitators monitor the needs of the group and the requirements of the task, and when necessary, devise new interventions on-the-fly to make the group more productive. In practice using a facilitator to facilitate a group decision-making process for a trip planning is not feasible. Our aim therefore is to develop a well-structured decision process for the Trip.Easy GDSS which facilitates users to collaborate towards a consensus based outcome. People may use this Trip.Easy GDSS anywhere and anytime without the need of a professional facilitator.
37
Part II
Design and Implementation
38
5 Introduction This part deals with the first objective of this thesis, namely investigating the research question:
Can we develop a Group Decision Support System that combines a well-structured decision process with domain knowledge and an intelligent recommendation mechanism that facilitates reaching a consensus for group trip planning problem? The background information and theory discussed in the previous section forms the foundation for the approach of this problem. To answer the research question we will invoke practices from Artificial Intelligence, Human-Computer Interaction and Collaboration Engineering. First, in chapter 6 the scope of the group decision support system for the group travel planning domain will be strictly defined in order to focus on the core functionalities. This is necessary because the travel domain is extensive. However, the system has to be able to facilitate realistic uses cases. Finally, the functional and non-functional requirements that have to be taken into account while designing the Trip.Easy GDSS will be discussed. The system design is detailed in chapter 7. First the domain model for the Trip.Easy GDSS is described. Subsequently, a city model is needed which will be used as an outcome of a ‘simplified’ travel planning session. Aiming for a consensus-based outcome, a preference model has to be established. Therefore, an applicable preference model has to be designed which will be used by preference elicitation and aggregation algorithms. Finally, based on the design approach from Collaboration Engineering the wellstructured group decision process is designed. Furthermore, a Graphical User Interface is proposed in order to let users interact with each other and guide them through the decision process. Furthermore, in chapter 8 the technology and implementation approaches detail how the components are merged together based on the designed architecture. The architecture and components are realized using an enterprise web-application framework. Finally in chapter 9, a conclusion will be given whether the designed and implementation of the Trip.Easy GDSS is successfully accomplished.
39
6 Scope Trip.Easy Prototype Developing a GDSS that can operate with the comprehensive group travel decision-making domain was not feasible in this thesis research. Consequently, the domain has been strictly defined giving us the opportunity to focus on the core functionalities. However, the system has to be able to facilitate realistic use cases. This was important because the aim of the present study is to build a group decision support system that can be used in real-world situations. Therefore, the following parts of the group travel decision-making domain are adjusted: Type of travel: A city trip is a form of a travel which occurs frequently in social situations. It is a social activity that involves family, relatives or friends which is common for most people. Therefore, the choice of a domain has been set on city trips. As a result the Trip.Easy GDSS prototype will be a group decision support system that supports a group with organizing a city trip. Outcome of the decision making: By organizing a city trip it is meant that users collaborate together towards a consensus based outcome (travel plan). Instead of a comprehensive travel plan which usually contains: the travel date, travel duration, several travel destinations, several transportation methods (air, sea or road), accommodations and the total price, the outcome of the collaborative work will be: • One final travel destination (city), • An indication of price range of the city trip composed by an accommodation and a two way flight, and • A sense of satisfaction towards the picked travel destination with every member of the travel party (consensus). This reduced travel domain has been recommended and reviewed by Marjolein Visser a Tourist Marketing expert and lecturer at NHTV Breda University. The less complex travel domain still could be seen as a good approach of a real-time situation because of the characteristics of the travel consumer behavior (see Appendix 2: Field research – Interview with domain expert for more information). The outcome (travel plan) of Trip.Easy GDSS has taken into account the two important decision components of a trip namely the travel destination and the budget (A. Pizam, 1999).
6.1 Requirements Based on the scope of the present study, the requirements of the Trip.Easy GDSS prototype were defined.
6.1.1 Functional requirements The following functional requirements must be met by the prototype: Request for friendship - Users should be able to request other users to be their friend. If the other user accepts the request, the two are connected with each other. Initiate a Trip.Easy Session - A Trip.Easy Session facilitates the decision-making process for a group of users. Every user of Trip.Easy can start a session. Invite friend - As soon as a user has initiated a Trip.Easy Session, they can invite their friends to join in.
40
Participating in a Trip.Easy Session (decision-making process) - Within the group decision-making process users should be able to: • • • • •
Insert their personal preferences, View the recommended cities that fit these preferences, Have awareness of the preferences other travel members, Share feedback on these cities and, Make a final decision.
Registration - New users need to fill-out the registration form. Login - When users are registered they can login by using their account. In Figure 6 an overview of the functionalities are presented in a UML diagram. Use-Cases of each component can be found in Appendix 3: Use Cases.
Figure 6: Overview of functionalities of the Trip.Easy GDSS prototype
41
6.1.2 Non Functional requirements Beside the functional requirements, a set of non-functional requirements were defined in order to have a fully working Trip.Easy GDSS prototype. Ease of use - The Graphical User Interface (GUI) should be intuitive. User should know exactly what and when to perform some operations. Active facilitation - Instead of a passive tool, the intelligent GDSS must act like a professional facilitator. It must be able to guide the users through the collaborative process by acting passive e.g. when users do not respond well, the system should actively monitor the situation and react appropriately. Time and space independence - Users should be able to use Trip.Easy GDSS anytime (not bounded to business hours) and anywhere (i.e. at home or work). Starting or participating into this collaborative travel decision making process should be in a time and space independent environment. In Group Decision Support System field it is also called a synchronous or asynchronous setting. Lead to consensus - The outcome (travel plan) of the group decision-making process should be consensus based. The aim is to have a satisfying affection by all the members of the travel party after using the Trip.Easy GDSS.
42
7 Design In this chapter we elaborate on the design of the Trip.Easy GDSS. The design is based on the scope and the requirements discussed in the previous section.
7.1 Domain model The domain model provides a structured view of the domain. The design of Trip.Easy relies on the domain model that describes the various entities in the system, constraints and their relationships. In Figure 7 an abstract overview of the designed domain model is presented. This domain model is obtained from a thoroughly analysis of the scope, requirements and the conducted field researches and interviews with domain experts (Appendix 1, Appendix 2). The concepts: user, friends, travel party, travel session, preferences, travel plan, city and budget forms the fundament of the Trip.Easy group decision support system.
Figure 7: Abstract Domain Model of Trip.Easy GDSS
User: A user is someone who uses Trip.Easy GDSS to plan his/her city trip. The user object contains static personal information like name, address, email, gender, age, etc. Furthermore, it can be used for identification in the GDSS. Friends: A relationship between one user and another. It will connect users with each other by the “friends” relationships. No friendships means no collaboration in this system. Travel party: A group of connected (by friendship) users that want to travel together. Trip.Easy travel session: The travel session is the core of the decision-making process. A travel session consists of user preferences, group preferences, a travel plan and the state of the decision making process. The travel session facilitates a well-structured decision-making process for users. Within the session users are able to collaborate towards a consensus based travel plan. Preferences: The users’ preferences will be captured and updated during the decision process. Furthermore, based on the individual preferences the group preferences will be calculated during the decision process. The preference models will be further explained in section 7.4. Travel plan: A travel plan will be generated and recommended by the GDSS based on the elicited user preferences after an aggregation process. A travel plan exists of a destination and an estimation of a budget.
43
Budget: The budget of the city trip is based on the price of a 3 star hotel and a two-way flight. This simplified budget calculation will be assigned manually for each city. City: The city is a complex concept. Users must be able to create an (positive) image about the possible travel destination based on the descriptions of a city. On the other hand, the system must be able to compare and recommend cities based on users preferences. Therefore, users preferences must be properly aligned with the properties of the cities. Additional descriptions about the city models will be given in 7.2.
7.2 City model As mentioned previously in chapter 6, the scope of Trip.Easy will be city trips. Its aim is to support travel consumers to decide and agree on which city they will visit. In order to extract and aggregate the preferences and subsequently generated recommendations during a Trip.Easy session, it is essential to have a consistent general representation (model) of a city. Therefore, to provide advice (recommendations) to the users, it is crucial that the system is able to recognize the groups’ preferences and match it with a certain city. Describing a city can be done in several ways. A common approach is to describe cities in a qualitative way which can be found everywhere like in travel guides, websites, brochures, etc. Describing a city like Washington DC in the United States is presented in the website TripAdvisor like: “Make sure you bring cash to Washington, D.C. Not that it’s expensive (actually, many museums are free) or that no one accepts plastic (they do)—it’s just an awful lot of fun to hold up a $5 bill next to the actual Lincoln Memorial, or a $20 in front of the White House. In between touring monuments and historical sights, check out the quirky International Spy Museum, watch pandas at the National Zoo, or catch a military band playing an outdoor concert on Capitol Hill.” A description of Rome in Italy is presented like: “It’s nicknamed the Eternal City for a reason. In Rome, you can drink from a street fountain fed by an ancient aqueduct. Or see the same profile on a statue in the Capitoline Museum and the guy making your cappuccino. (Which, of course, you know never to order after 11 am.) Rome is also a city of contrasts—what other place on earth could be home to both the Vatican and La Dolce Vita?”
However, according to the information-seeking in recreational planning from B.Brigham and J. Perron (2004) it is very hard for humans to compare why Washington DC suits better to a person or travel party than Rome based on the qualitative descriptions. Moreover, it is even harder for computers to recommend the person or travel party based on the qualitative descriptions. No standard format/structure is defined between publishers of travel contents. As a result, every publisher developed his or her own way to describe the attractiveness of a city. Another way to describe cities and make them comparable is to make use of features. Examples of city features are; museums, ancient monuments, restaurants, parks, etc. Users often find certain features more important than others, thus preferring one feature over the other. This behavior makes it possible to distinguish one city from another. As a result, the Trip.Easy GDSS can use these preferences to recommend cities that might fit best to the users. Reutsche (2006) studied what the specific elements of
44
the urban tourism product are that determines the attractiveness of a city for visitors. It resulted in a model that is presented in Figure 8. In Reutsche (2006) it was explained that a city could be described in terms of certain elements of a city’s tourism resource. According to Reutsche (2006) these elements provide the main reasons why tourists visit a particular city. These elements (features) are grouped by categories represented by the rectangles in Figure 8. Using in total 40 features to describe cities makes it possible to compare the cities for the system. Furthermore, users can express their preferences according to this model. For this study we therefore decided to use Reutsche’s model for the description of the cities. It provides a good fundament and structure to let Trip.Easy GDSS capture the preferences of each user. The individual preference models will be used to form a group preference model and make recommendations based on the preferences. Liveliness Language Customs Cultural Heritage Friendliness Security Political Stability
Parks Buildings Historical street pattern Ancient monuments and statues Waterfront Natural attractions
Atmosphere
Environment
Shopping Boutiques Malls Markets Streets
Culinary International restaurants Local restaurants Lunch facilities Cooking courses
City Amusement Night Clubs Casinos Excursions Festivals Zoo
Cultural Museums Art galleries Theaters Cinemas Concert Halls Convention Centers
Sport Walking tours Cycling tours Adventure Outdoor activities
Figure 8: Representation of a city based on Reutsche (2006)
Recommendations can be made when the distances between the preference model and the city models can be calculated. In order to make that kind of computations (distance calculations like Spearmans footrule) the representation of the cities based on the model from Reutsche (2006) has to be quantified and saved into a database. The structure of the city database is designed according to this model. Subsequently, a weight between 0 and 1 is assigned to each of the 40 features. As a result, the database will contain data like:
45
Paris = { feature_parks(0.82), feature_buildings(0.35), feature_museums(0.77) … feature_40(0.2) } New York = { feature_parks(0.95), feature_buildings(0.52), feature_museums(0.66) … feature_40(0.75) } City (x) = { feature1(subjective quantified value) ….. feature40(subjective quantified value) } Consequently, the presence of a certain element could be expressed for each city (i.e. Paris is a very popular destination for shopping, thus it gets a high weight value). The weights were based on qualitative information from content providers (such as TripAdvisory.com). By subjectively evaluating the information, the features were quantified as a weight and assigned manually via the administration system of Trip.Easy GDSS. Consequently, the system is able to interpret the preferences of the user by matching it with the cities in the database. The matching and interpretation of the cities based on utility functions are covered in the thesis of Touw (2010).
7.3 Preference elicitation Until now we have discussed city model which can be used for preference elicitation in a travel planning problem. Users are faced with the problem of choosing the most preferred outcome from a large set of possibilities. As people are unable to browse and filter through them manually, decision support systems are often used to automatically find the optimal solution. A crucial requirement for such a system is to have an accurate model of the users’ preferences. In order to form a preference model a good representation of the user’s preferences is needed. However, the structure and representation of the model heavily depends on the solution that is chosen to elicit accurately the preferences from the users. Furthermore, we want to find a solution for taking everyone’s preferences into account. Therefore preference aggregation is applied. Preference aggregation is the process of combining the preferences of several individuals in order to get a representation of the preferences of the group of these individuals (Thomas L. Saaty, Jen S.Shang, 2005). The highly cohesive relationship between the preference elicitation, preference model and preference aggregation makes the design complex. Therefore, the design phase had to be carefully aligned with the research of Touw Ngie Tjouw. As a result we were able to design a preference elicitation method and designed a representation for the preference model which is applicable within the scope. Before elaborating on the resulted preference model, we present a short introduction of the theory of preference elicitation (More information about preference elicitation can be consulted in our literature research Ngai (2008)) and an overview of the design choices that were made for the preference aggregation component which is detailed the thesis research Touw (2010).
7.3.1 Short introduction into preference elicitation An essential component is the preference elicitation function. This function is responsible for facilitating user preference specification and is required to have a high level of usability. Since the goal of a decision support system is to assist users with making decisions, it is especially important for the system to have an accurate model of the users’ preferences. The goal of preference elicitation is to devise algorithmic techniques that will guide a user through an appropriate sequence of queries or interactions and determine enough preference information to make a good or optimal decision (Braziunus, 2006). In our literature research, Ngai (2008), a comprehensive description about preference elicitation can be found.
46
The theoretical basis of user preference models can be found in decision and utility theory (Braziunus, 2006). Decision theory lies at the intersection of many academic disciplines – statistics, economics, psychology, game theory, operations research, and others. Assuming a set of axioms for rational behavior, it provides a theory for modeling user preferences and making optimal decisions based on these preferences (Braziunus, 2006). In the basic formulation, a decision maker has to select a single alternative from the set of available alternatives (or actions). An outcome/consequence of the chosen alternative depends on the state of the world Θ. The consequence function Θ maps each action and world stat into an outcome. User preferences can be expressed by a value, or utility, function that measures desirability of outcomes. The goal is to select an alternative/action that leads to best outcomes. If the world state is known, the set of outcomes is equivalent to the set of alternatives. When uncertainty over world states is quantified probabilistically, utility theory prescribes an action that leads to the highest expected value (Braziunus, 2006). In short within decision theory we can distinguish three different situations for preference modeling: • • •
Preferences under certainty Preferences under uncertainty Multi-attribute outcomes
Preference under certainty means that all actions and related outcomes are known. Preferences over outcomes determine the most optimal action: A decision maker would choose the action which results in the most preferred outcome. Preferences under uncertainty, concerns the situation where the consequences of actions are not clear, but can be modeled by a probability model (utility theory). In case of multi-attribute outcomes, we observe that the outcome of an action exists of different attributes, so it is endowed with a multidimensional structure. This is the case in most real life situations, mostly combined with an uncertainty about the outcome. The main concept of the multiattribute utility theory (MAUT) is that outcomes are defined by the assignment of values to a set of attribute variables , … , . Attribute variables are either discrete or continuous. The outcome space composed of the space of all possible outcomes is the Cartesian product of Ω … . The set of outcomes considered for a decision problem is contained by Ω. It is common for to be very large. In order to make decisions based on , a decision maker often needs a ranking of all outcomes determined by preferences. In real life situation, the best way to describe travel preferences is probably the Multi-Attribute Utility Theory. However, within the scope of this study where the outcome is not more than a travel destination, it is sufficient to treat the travel preferences as preferences under uncertainty. Consequently, a utility function has been designed for Trip.Easy GDSS and verified by a tourism scholar. The design of this utility function is outside the scope of this the study, for more information about the utility function the thesis of Touw (2010) can be consulted. Given this fundamental choice we further elaborate on the next component, the preference aggregation.
47
7.3.2 Preference aggregation Within group decision support systems, preference elicitation is often succeeded by preference aggregation. Preference aggregation is the process of combining the preferences of several individuals in order to get a representation of the preferences of the group of these individuals (Thomas L. Saaty, Jen S.Shang, 2005). Every aggregation approach demands an aggregation function, which is a function that outputs a collective preference relation or utility function (Thomas L. Saaty, Jen S.Shang, 2005) based on the individual preferences. Given the fact that for the Trip.Easy GDSS the preferences can be seen as preferences under uncertainty, an aggregation method has to be selected or designed in order to establish a group preference model. Five best-practice aggregation approaches are discussed and evaluated for applicability within the Trip.Easy GDSS (consult the thesis of Touw (2010) for more information). In order to make a selection we have defined certain criteria: •
•
• • •
Non-manipulability: This means that users are not able to manipulate the recommendation mechanism. It prevents certain members from having more influence on the outcome than others. Usability: Preference aggregation methods are effective if they have the right preference information about users. However such information can get very complex and cumbersome. This leads to a preference elicitation process that is too exhaustive for the users and thus unusable. Feasible aggregation: Preference aggregation is an optimization problem. Solving such problems can be quite difficult and depends on the preference representation. Empirical validation: The method must be tested in real world situations. Based on that data we can predict the performance of the method and thus of our aggregation mechanism. Representation of preferences: The method must allow an effective representation of the preferences of users. It is essential that users can entirely express their preferences.
The evaluation of the five aggregation methods can be viewed in the following table:
Classic voting AHP CP-NETS Kemeny rule Spearman’s Footrule
Usability
Feasible aggregation function
Non-manipulability
Empirically validated
Representation of preferences
Table 4: Overview of aggregation methods vs. our criteria
Based on the results the Spearman Footrule has been chosen. Spearman’s Footrule meets all the defined criteria. In comparison with the Kemeny rule (rank aggregation by using Kendall’s Tau), Spearman is computationally less complex, which is why we prefer Spearman’s Footrule over Kendall’s Tau. A direct consequence is that the preferences will be represented by rankings of city features.
48
7.4 Preference model Finally, after well alignment with the research of Touw (2010), the preference model is designed. A preference model can be seen as a representation of the preferences of users concerning a set of items. Therefore, in this case the preference model is dependent on the city model, since the city model determines the items (features) of a city. In the previous paragraph it was shown that for aggregation the Spearman’s Footrule will be used and as consequence a ranking of the features is needed. This means that a preference model can be established by collecting the preference ordering on the features of the city. Therefore we propose that users will express their preference by ranking the different features based on its importance. In Figure 9 an abstract representation of a preference model based on the city model is illustrated.
Figure 9: Preference model
The seven orange elements in Figure 9 represent the categories of the city model. Below each category the features of the city model are showed. The user can rank the categories as well as the features belonging to that category. The numbers represent the rank of a category or a feature. The highest rank is 1 and the lowest is N or K (where N in case of categories; is equal to the amount of categories while in case of features; K is equal to the amount of features belonging to a category). The preference model requires that the user specifies his or her preferences by ranking the (N) categories and (K) features per category according to what he or she finds important. Every user is requested to establish such a preference model. A typical representation for instance looks like this: John = {2(2,1,4,3), 4(4,3,1,2), 5(3,2,7,6,5,1,4) … }. This means that the user John prefer shopping more than sport and sport more than atmosphere etc. Furthermore, within the shopping category, John prefers malls over shopping streets and shopping streets over markets etc.
49
The next step is to design a group decision process which integrates the need that is provided by the preference aggregation, elicitation and model.
7.5 Trip.Easy Session: Group Decision-Making process As mentioned in section 7.1 the core of the present study is constructed in this Trip.Easy session component. As shown in the domain model in Figure 7, it can be seen that the Trip.Easy session is formed by a group decision-making process called an agenda. Users of the travel party participate together in this session to collaborate towards a consensus based travel plan. The group travel planning problem described in section 3.2 showed that the unstructured group decision making during a collaboration session leads to suboptimal decisions. Therefore the aim is to design a well-structured group decision-making process which can be used in the Trip.Easy session that facilitates reaching a consensus for group trip planning problem. The well-structured decision process must enable users to: • • • • • •
Organize or participate into the session at different locations and different moments (space and time independent), Eliminate overlooking opportunities due the lack of domain knowledge or information overload, Act rationally by eliminating influential factors like dominant members during a discussion or not objective information provided by other members, Having their voice heard by other members by communicate effectively and finally, Fairly involve every group member in establishing a consensus, Converge on an outcome that is satisfying to all group members.
The challenge is to design a group decision process regarding to the requirements and the scope mentioned previously. Therefore this study invokes the Collaboration Engineering approach to create a high quality collaboration process.
7.5.1 Task decomposition To identify the activities to be performed by the group, the collaboration problem has been analyzed by performing a task diagnosis. Several field researches were conducted to collect a representative set of tasks within a group travel planning problem. First, an interview among travelers and several tour operators were held and analyzed to determine, adjust and negotiate about the requirements and constraints on the tasks. Second, two groups of 8 people were observed how they collaborate on distance (by email and chat) for selecting a consensus based travel destination4. Both parties in total sent and forwarded on average 53 emails to each other and it took more than 3 weeks to pick a final destination. This clearly shows how inefficient this process is. Finally, a field research was conducted on several travel agencies to analyses the tasks during the group decision process. Without any announcements of our intention we (two ‘actors’) walked in a pre selected travel agency and started to chat with the travel consultants (in this case also functioning as facilitators). A fictional pre defined group travel case (see Appendix 1: Field research – group Decision Support by a Travel Agency) was
4
Both cases were real-life situations. Both travel party were planning a vacation for the summer 2009.
50
submitted to them and we observed how the travel consultants supported them in picking a consensus based travel destination. All together, a collection of tasks and activities were collected from the field researches and the literature. Guided by the patterns from the Collaboration Engineering which characterizes a group activity, the activities of the process were matched to a pattern of collaboration regarding to the requirements. As a result, the basic decision process for Trip.Easy GDSS is constructed from five fundamental collaboration patterns e.g. Generate, Reduce, Clarify, Evaluate and Build Consensus. Generate: Applying the Generate collaboration pattern it moves from having fewer to having more concepts in the pool of concepts shared by the group. In our domain, a concept is regarded as a possible city to visit. This means that users should be able to suggest travel destinations into the group. Clearly, Trip.Easy GDSS should facilitate this by determining the preferences (preference elicitation) for each member. To support users in finding what their preferred travel destinations are, Trip.Easy GDSS uses the domain (city trips) knowledge to match city characteristics (features) into to cities. Furthermore this will eliminate the chance of overlooking travel opportunities. Based on the group preferences (preference aggregation), a selection of best-fit travel destination should be generated by Trip.Easy GDSS and recommended (recommendation mechanism) to the group. Reduce: Applying the Reduce collaboration pattern it moves from having many concepts to a focus on fewer concepts that the group deems worthy of further attention. In this case the recommended set of possible travel destinations should be reduced taking into account the consensus goal. Therefore each user will select a set of cities that they find most interesting. Detailed information about the characteristics of a city should be provided during the selection. Trip.Easy GDSS gathers the preferred set of cities from each user. Then it will find a reduced set of cities which is fitting the group best. Clarify: Applying the Clarify collaboration pattern it moves from having less to having more shared understanding of concepts and of the words and phrases used to express them. In other words, the Clarify pattern facilitates users to create awareness and understanding among group members by sharing their opinions and intensify their preferred concepts (travel destinations). Trip.Easy GDSS will collect the information and structure it in order to let users have a clear overview about each other’s preferences. Based on the input, Trip.Easy GDSS should be able to suggest which travel destination fits best to the group. Evaluate: Applying the Evaluate collaboration pattern it moves from less to more understanding of the relative value of the concepts under consideration. Based on the result of the previous Clarify step of the decision-making process, users are supposed to make a final choice among the remaining choices. To support users converging towards a final choice by helping them creating commitment, Trip.Easy GDSS provide users relevant consensus stimulating information. Build Consensus: Applying the Build Consensus collaboration pattern it moves from having fewer to having more group members who are willing to commit a proposal. This pattern is used continuously within the decision process. Trip.Easy GDSS provides consensus stimulating information to create awareness and understanding within the travel party.
51
7.5.2 Agenda building The sequence of actions identified during the task decomposition could be executed in many different ways. Therefore the next step is to concretize the sequence and define the conditions of execution of the actions which are matched with the collaboration patterns. To concretize the sequence, we (i) studied the observed processes during the field research (Appendix 1: Field research – Group Decision Support by a Travel Agency), (2) interviewed a domain expert (Appendix 2: Field research – Interview with domain expert) and (3) studied many decision making theories (Chapter 3). As a result, we propose a possible structured decision process that facilitates the users to establishing a consensus-based travel destination. This group decision process is based on the general consumer behavior cycle and takes in to account the identified determinants of a good group decision processes (see section 3.3.3) which are: • • • • •
active interaction with other individuals level of communication type of decision making result of confrontation the influential power of roles
This agenda has been reviewed by experts from the collaboration engineering and tourism discipline. In Table 5 the agenda can be found. The first column is to identify and number the activities. All activities like waiting for other members also are included. The second column describes the tasks which is altogether the decision making process. The third column describes the actions to complete the task. Finally, the last column shows the applied collaboration pattern. Acti vity
Description
Assignment
Deliverable
Collaboration pattern
1
Start decision making process
A travel party
Other
2
Introduce the focus of the collaboration effort (Trip.Easy session) Generate of a set of possible travel destinations from which to choose
Create travel party and invite the members Inform the users how a Trip.Session works and what the aim of this session is Formulation of objectives for the city trip by ranking the features of a city trip When all users inserted their preferences, Trip.Easy GDSS aggregates the individual preference models and searches for cities that matches the group preferences Indicate the travel worthy destinations by raking the 20 cities from highly preferred to less preferred. When raking the cities, the travel budget and interest should be taken into account When all users submitted their preferred set of travel destinations, Trip.Easy GDSS aggregates the results Rate the cities and provide comments and opinions why a city is preferred or not When all users provided the
Objective of the collaboration process
Other
An individual preference model
Generate
A group preference model is first calculated by Trip.Easy GDSS. Then a set of 20 best-fit travel destinations are matched and recommended to the group
-
An ordered list of preferred travel destination.
Reduce
A best-fit set of 6 cities is calculated and recommended to the group by Trip.Easy GDSS
-
A set of comments, opinions and ratings of the 6 cities is collected
Clarify
A clear and structured overview of
-
3
4
Wait
5
Create a smaller set of possible travel destinations for focus
6
Wait
7
Create understanding why a city is worth visiting
8
Wait
52
arguments, Trip.Easy GDSS aggregates the results 9
Make a final decision and select the travel destination
Start a discussion based on the available information and work towards a consensus based travel destination
the comments and opinions of the 4 remaining highly rated cities by the group A consensus based travel destination
Evaluate
Table 5: Agenda
Based on the information in the agenda, the flow of the collaboration process is graphically illustrated in Figure 10. The Facilitation Process Model (FPM) depicts the logics of the flow of the process from activity to activity. One of the aims of this study was to design and develop a decision process that users can follow without a professional facilitator. The FPM can therefore be used to integrate the decision process into the Trip.Easy GDSS.
Figure 10: Facilitation Process Model
53
7.6 Graphical User Interface In the previous sections the group decision process has been proposed and developed. The next step is to develop a user friendly Graphical User Interface (GUI) to let users interact with the Trip.Easy GDSS the decision process (a Trip.Easy session). Throughout the GUI design process attention is paid to the usability factor because it is directly related to the user’s experience of the decision process. In Figure 11 the basic design of the Trip.Easy GDSS is depicted. The user interface is a common setup for online applications. It exists of three sections: a header, body and footer.
Figure 11: Basic Trip.Easy GUI
Header section: The header (currently masked with a blue layer) placed at the top of the screen contains the menu and functions of the Trip.Easy GDSS. Body section: Most of the GUI design is focused on in the body section (currently masked with an orange layer) which is placed in the middle on the screen. This section will always have a status bar which shows the current status of the decision process. Each step in the process is different, consequently each screen requires a different implementation of the interface in order to support users to complete the required actions for that specific task. Footer section: The footer (currently masked with a blue layer) placed at the bottom of the screen contains additional information about the Trip.Easy GDSS. In the coming sections, we will elaborate on the design of the screens starting with the screen of the Generate step.
54
7.6.1 Interface for the Generate step The agenda in section 7.5.2 showed the goal of this step: Acti vity
Description
Assignment
Deliverable
Collaboration pattern
3
Generate of a set of possible travel destinations from which to choose
Formulation of goals and objectives for the city trip by ranking the features of the city trip
An individual preference model
Generate
The assignment for the users in the Generate step is to formulate the goals for the city trip. Using a featured based preference elicitation procedure mentioned in section 7.3 users can insert their preferences by ranking the different features of the city. The features are based on a city model (section 7.2) of relevant features for city trips. Figure 12 shows the user interface we designed that facilitates easy completion of the required actions for the Generate step.
Figure 12: Interface for the Generate step
During the design of this interface we took the following design issues into account:
55
Clearness of the objective: An information box at the top of the body section is presented every time when a user reaches a new step of the decision process. The information box contains user instructions for the interface for this specific step. A user can hide or retrieve the instructions whenever appropriate. Ranking method: The preference elicitation method that is chosen (section 7.3) requires from the user to insert their preferences by ranking different features of the cities. Ranking can be implemented in many ways. Our criteria were that the ranking procedure should be easy and a low cognitive load should be required. A drag & drop interaction mechanism is therefore applied for ranking the city features. By dragging and dropping the features to the left side (highly important) or to right side (not important) of the screen, an ordering from the level of importance could be captured. This ordering of the features forms the individual preference model. Moreover, each element that can be dragged and dropped is marked with an “arrow cross” to increase the affordance of the drag & drop possibility. Ranking of order: It must be clear how to express the ranking using the drag & drop interaction. Therefore we suggest using a color bar with warm colors representing the (highly) preferred features and cold colors representing the less preferred features. The ordering goes from very preferred features on the left side to less preferred features on the right side of the interface. Grouping of the city features: The features are based on a detailed city model of relevant features for city trips. Seven categories with in total 40 features should be ranked using the drag & drop interaction. Using a color scheme, the categories are separated from each other for easy distinction. The associated sub-features are colored with a lighter version of color of their parent feature.
7.6.2 Interface for the Waiting status(Screen 4, 6, 8) While others are still busy with a certain step, users that already finished this step must wait until all users are ready. Moreover, members of a travel session may complete the tasks of a certain step on a different moment due the asynchronous property of our decision process. Therefore, Trip.Easy GDSS will actively monitor the progress by sending messages to members who are slowing the progress down, just like a facilitator. Additionally, members should be able to monitor the progress of other members as well to stimulate active participation. When all members completed the specific step, the next step of the decision process can be started. The Facilitation Process Model in section 7.5.2 showed the three waiting moments. Figure 13 illustrates the designed interface that facilitates easy completion of the required actions for this step. During the design of this interface we took the following design issues under consideration: Clearness of the objective: Every time when the user signed in the session, Trip.Easy GDSS will notify the users about the waiting state and thus no actions are required to take. Furthermore, the status bar shows the current stage of the decision process. Stimulation of active participation: An information section implemented in the interface visualizes the progress state of the decision process. It should stimulate the members to participate actively through the transparency of each other status.
56
Figure 13: Interface during the waiting state
7.6.3 Interface for the Reduce step The agenda in section 7.5.2 showed the goal of this step: Acti vity
Description
Assignment
Deliverable
Collaboration pattern
5
Create a smaller set of possible travel destinations for focus
Indicate the travel worthy destinations by raking the 20 cities from highly preferred to less preferred. When raking the cities, the travel budget and interest should be taken into account
An ordered list of preferred travel destination.
Reduce
The assignment for the users in the Reduce step is to provide their preferred travel destination by ranking cities from the recommended set. To support users in the selection procedure additional city information, which illustrates the characteristics of a city, can be retrieved. Furthermore, to encourage consensus building, Trip.Easy GDSS should provides information like: (1) preferences of other members, (2) a best-fit score calculated5 by Trip.Easy GDSS which says how well a city suits the travel party for 5
More details about the calculation of the score, the Touw (2010) can be consulted
57
travel and, (3) explanation why a specific city fit well for the group. Figure 14 illustrates the designed interface that facilitates easy completion of the required actions and requirements for this step.
Figure 14: Interface for the Reduce step
During the design of this interface we took the following essential design issues under consideration: Clearness of the objective: An information box at the top of the body section is presented every time when user enters into a new step of the decision process. The information box contains instructions for
58
use of the interface for this specific step. A user may hide or easily retrieve the instructions whenever appropriate. Ranking method: Trip.Easy ranks city trips based on the preference input of the group and suggests a list of city trips ordered from highly fit to less fit for traveling. Then users should re-rank the city trips according through their preferences to progress further with the decision process. The system will automatically update the user’s preferences model. To rank the cities we have chosen for the drag & drop interaction which the users are familiar with from the previous step. The whole element containing the best-fit score, destination image, name of city, country name and the price forms a block which can be dragged and dropped. Again, using the “arrow cross”, the drag & drop possibility is visualized. Ranking of order: The city with the highest rank is most preferred; from thereon it is descending towards the city with the lowest rank, which is least preferred. A recognizable element is used to represent the ranking order. A color bar is used with warm colors representing (highly) preferred cities and cold colors representing theless preferred cities. Visualization of preferences of the travel party: Creating awareness among members and stimulate consensus, transparency of the preferences was needed. To present the preferences to the users in a clear and understandable way, a column chart is used to visualize the preferences of each member. Each column represents a member. The height of the column gives the likeliness of that specific preference of the user. Thus, a user can have insight in the preferences of the members by reading the chart. Furthermore, a (blue) line which is the calculated group preferences is also visualized in the chart. Visualization of city characteristics: City characteristics are visualized to support users in choosing their preferred travel destinations. The city model that is used for describing the cities is presented with their value. The seven categories and their features are visualized through a percentage bar which colors from red (low value) to green (high value). A summary of the city is also visualized through a yellow line in the chart in order to have quick reference for the users. Best-fit scores: Trip.Easy provides information to the user of its own feature ranking and the ranking of those features by the group. This information provides a group member with a clear overview of individual and group preferences to refine the currently presented ranking. Two types of best-fit scores are used in this screen to explain why a city suits the travel party well. First, the score for a city which is calculated as the best-fit for the travel party is visualized near the destination image in the drag & drop element. Second, a best-fit score for a certain city for each member is also presented. This set of scores illustrates how well the city suits for each individual. For instance, Las Vegas has a best-fit score of 79% for the group, but 1 member has a score of nearly 55% while other has much higher scores. This information can therefore be considered before chose for a certain city for travel.
59
7.6.4 Interface for the Clarify step The agenda in section 7.5.2 showed the goal of this step: Acti vity
Description
Assignment
Deliverable
Collaboration pattern
7
Create understanding why a city is worth visiting
Rate the cities and provide comments and opinions why a city is preferred or not
A set of comments, opinions and ratings of the 6 cities is collected
Clarify
The process is designed to converge on an outcome that is satisfying to all group members. Therefore, based on the rankings by the group obtained in the previous step, a shortlist of the most highly ranked city trips is presented. This time the user is asked to rate the remaining cities and share comments and opinions in order to make other group members aware of their interests. The top-4 average highest rated cities will be selected for the next step. The comments and/or opinions will be used in the next step to ease the making of the final decision. Figure 6 illustrates the designed interface that facilitates easy completion of the required actions and requirements for this step.
Figure 15: Interface for the Clarify step
60
During the design of this interface we took the following essential design issues under consideration: Clearness of the objective: An information box at the top of the body section is presented every time when user enters into a new step of the decision process. The information box contains instructions for use of the interface for this specific step. A user may hide or easily retrieve the instructions whenever appropriate. Rating method: Using stars to express the likeliness of a certain case/object is very common for internet applications. Therefore, this time a rating interaction mechanism has been applied. The ratings provided by the user’s forms actually a voting system which will be used to converge to a consensus based outcome. Comments and opinions: To support the rating, users may provide comments and opinions. These will be transparently presented to other members in the next stage of the decision process. When a user rated a city low, explanation can be given by checking the predefined arguments. Additional comments to share with can be provided by the user through the text box. Decision support information: To support users with their rating, the previous presented city and consensus stimulated information may be retrieved by pressing the “+” symbol.
61
7.6.5 Interface for the Evaluate step The agenda in section 7.5.2 showed the goal of this step: Acti vity
Description
Assignment
Deliverable
Collaboration pattern
9
Make a final decision and select the travel destination
Start a discussion based on the available information and work towards a consensus based travel destination
A consensus based travel destination
Evaluate
In the final step of the decision process a small set of cities remains for final decision making. Using the input obtained in the previous step, Trip.Easy GDSS suggests a final group consensus outcome that is a “best fit” for the group. The assignment for the users in the Evaluate step is to discuss and negotiate about the final choice to be made. To support them in this final stage of the decision making process Trip.Easy GDSS structures the available information that can be retrieved easily by the users. Furthermore, Trip.Easy GDSS also presents few other highly rated city trips to allow users to discuss and negotiate about these alternatives and deviate from the outcome proposed by Trip.Easy GDSS. Figure 16 illustrates the designed interface that facilitates easy completion of the required actions and requirements for this step.
Figure 16: Interface for the Evaluate step
62
During the design of this interface we took the following essential design issues under consideration: Clearness of the objective: An information box at the top of the body section is presented every time when user enters into a new step of the decision process. The information box contains instructions for use of the interface for this specific step. A user may hide or easily retrieve the instructions whenever appropriate. Structuring of information: Based on these ratings the scores are calculated. Subsequently, the list is ordered by the scores from high to low. A high score means that on average, the specific city is rated as a worthy travel destination by most of the users. Consequently, they are committed to that specific city when the city is chosen as a final decision. Furthermore, all the comments and opinions provided by the users in the previous step is structured and grouped together. Clicking on a city will retrieve all these information in order to support the users during the discussion. Decision support information: To support users with their final decision making, the previous presented city and consensus stimulated information may be retrieved by clicking on “show city details”.
63
8 Implementation In this chapter we explain how the Trip.Easy intelligent group decision support system is implemented.
8.1 Web application The aim was to design an asynchronous group decision support system where the users were not bounded to place (i.e. home or work) and time (i.e. not bounded to business hours). Therefore the Trip.Easy GDSS is built on the internet technology. The Trip.Easy prototype is a web application that runs on a web-server and is accessed over the internet. No advance hardware is needed by users because complex calculations and logics are processed centrally through the server. It is sufficient for a user to communicate with the Trip.Easy GDSS server using a supported web browser. Trip.Easy GDSS is currently compatible with the Google Chrome, Safari and Mozilla Firefox web-browsers. Furthermore, webapplications make the maintenance and updates of the Trip.Easy GDSS application on the server possible. The distribution of new versions happens automatically, no installments on all the different computers are required anymore. Consequently, more efficiently could be worked during the testing and experimental phase. Finally, internet applications support cross-platform compatibility. Different devices, such as (smart) mobile phones, tablet pads, notebooks, desktops and multi-touch tables can be used to access the Trip.Easy GDSS application. As long as a device has the supported web browser installed. Figure 17 illustrates how web-applications work in a simplified way.
Figure 17: Abstract Infra structure Web-application
64
8.2 Abstract architecture Based on the domain model and scope the architecture of the Trip.Easy GDSS is designed. The software architecture refers to the abstract representation of the Trip.Easy GDSS. Architecture is concerned with making sure the system will meet the requirements of the product, as well as ensuring that future requirements can be addressed. The architecture also addresses interfaces and relations between the components. In order to easily extend or replace the components, the architecture of the Trip.Easy GDSS is developed as modular as possible. An abstract overview of the architecture of Trip.Easy GDSS is presented in Figure 18. Each component of the architecture will be detailed below.
Figure 18: Abstract architecture of Trip.Easy GDSS
Graphical User Interface - The GUI layer makes it possible for the users to interact with the system. A user-friendly GUI is required in order to achieve a satisfying affection by users. Because a seemingly amount of actions has to be performed during the decision making process, the interaction design forms a very important element of the GUI. Facilitator - The facilitator component facilitates the group decision process which is defined in the agenda sub-component. The task is to monitor the progress of the decision process and provide consensus stimulating information to the group. The facilitator determines which component (i.e.
65
Preference aggregation or city matcher, etc) has to be invoked and communicates with the GUI about what to present to the user. Preference Model Constructor - To model the user preferences, preference elicitation techniques were applied. Many different preference elicitation methods can accurately elicit user’s preferences. However not every preference elicitation approach will take into account the issues like user’s cognitive and emotional limitations of information processing. In addition, the elicitation methods differ from each other on the ability i.e. to let users: discover hidden preferences, avoid preference reversals and making tradeoffs when confronting with competing objectives. This component is responsible for enabling users to express their preferences in a user friendly but effective way by interacting with the GUI. Thereafter, it will construct a preference model out of the input from the GUI. Preference Model - A preference model represents the preferences of an individual. The preference model may change overtime during the decision process; therefore the model can be updated or incrementally constructed. Recommendation Mechanism - This component is covered in the thesis study of Touw (2010) named: The intelligence of the GDSS comes from the recommendation engine. Using the preference aggregator, utility calculator and the city matcher sub-components, intelligent suggestions that is best-fit for the travel party can be calculated by Trip.Easy GDSS. It starts with aggregating the individual preferences which is prepared by the preference elicitation component. Then it uses diver’s intelligent aggregation or utility algorithm from the Artificial Intelligence practice to calculate the best-fit for the group. The algorithms are modified in order to be applicable in the travel domain. Mapper / Retriever – This is a layer that communicates with the databases. Components have access to the data by invoking this layer. No difficult queries are needed in order to retrieve or save complex data structures like preference models or feature-based city descriptions. ` Databases - The database of Trip.Easy GDSS can be globally divided into three types of data, starting with ‘cities’. TripEasy GDSS contains information about 150 cities world-wide (Appendix 4: City database). Each city is described by the categories and features defined by the city model (section 7.2). Additional information such as name, country and price has been added. Furthermore, the database has information about the users. It has data about individual preference models and general information such as name, age, username, password, etc. Subsequently, the database contains information about the Trip.Easy GDSS sessions conducted by a group of users. The data exists of group preference models, the recommendations generated by Trip.Easy GDSS per process step and the final outcomes of groups.
8.3 Technology To implement the Trip.Easy GDSS prototype as a web-application a powerful and modular webframework is used. GRAILS is an open source web application framework which leverages the Groovy programming languages (which in turn is based on the JAVA platform). Following the “code by convention” paradigm, many professionals acknowledge it as a high-productivity framework. It provides a stand-alone development environment hiding much configuration details from the developer which
66
makes the prototyping much more efficient. Furthermore, this framework makes it possible to strictly implement the application using a Model View Controller approach. As a result, modifications and replacements of components can be easily done for future developments.
8.4 Development approach The development of the Trip.Easy prototype required an agile approach. This means that we designed and developed parts of the system. Starting with the design, implementation and testing of the decionmaking process and the GUI’s described in section 7.5 and 7.6. Based on that integrated these components with a web framework depicted in section 8.2. Thereafter we designed and build the database. The GDSS fundamental functionalities described in Appendix 3 – Uses Cases were implemented on top of the framework. Finally we integrated the user interfaces and the database with the framework and the components. As a result the full-functioning Trip.Easy prototype was realized.
9 Conclusion In this part of the research our first goal was to design and implement a well-structured decision process which is integrated with domain knowledge and an intelligent recommendation mechanism. The objective was therefore to investigating the following research question: Can we develop a Group Decision Support System that combines a well-structured decision process with domain knowledge and an intelligent recommendation mechanism that facilitates reaching a consensus for group trip planning problem? In order to realize the depicted intelligent Group Decision support System during the design and implementation phases we successfully dealt with the challenges such as: • • • • • •
an extendable and modular GDSS architecture; a domain model for city trips; a preference model for travel consumers; a preference elicitation procedure; a format of the data representation in order to integrate with the diverse intelligent aggregation algorithms6 and finally; a well-structured collaborative decision process.
We have successfully shown that the knowledge in the background- and theory section, combining together with the practices in Artificial Intelligence, Human-Machine Interaction and Collaboration Engineering makes it possible to develop the intended intelligent group decision support system prototype named Trip.Easy. In the next chapters we will discuss the evaluation of this system.
6
This will be the focus in the thesis of Touw (2010)
67
Part III
Evaluation
68
10 Experimental design In the previous section we have discussed the design and development of the Trip.Easy GDSS which resulted in a working functional prototype. We however start an evaluation in order to assess and test whether the Trip.Easy GDSS behaves as we expect and meets the requirements. An experiment was designed and conducted to gather empirical data, which were processed by statistical tests. In this chapter we present the experiment that determines whether users are satisfied by the Trip.Easy GDSS technology used as a support for consensus based travel destination picking. This chapter describes the experimental design that includes the following: 1. The research objective and the motivation of the main research question for this evaluation part of the research; 2. The formulation of the hypotheses in order to answer the research questions; 3. The variables to be measured, as well as how they were measured in order to answer the hypotheses; 4. The protocol that each experimental session followed; 5. The sample size and subjects and how the participants were selected; 6. The equipment and setting that was used; 7. The construction of the city database by mapping detailed city information into our database model; 8. Finally, the findings of a pilot test helped in checking assumptions and finalize the experimental design.
10.1 Research objective The goal of the Trip.Easy GDSS is to support a group of travel consumers with choosing a city trip that represents a best-fit to the group. Therefore a well-structured decision process (we call this a Trip.Easy session) has been designed in combination with intelligent technology that provides smart recommendations. If people were to dislike a meeting (Trip.Easy session) because of the technology used to conduct meetings, they would be less likely to use that technology in the future, even if it were to help them to produce better results (George, Easton, & Nunamaker, 1990). In the current state of the development of this innovative Trip.Easy GDSS concept, it is interesting to know whether there is a chance that the users will accept and use the Trip.Easy GDSS. A study has therefore been performed to find out whether the Trip.Easy GDSS is capable of supporting users satisfactorily with making a consensus based decision about their “city” trip. As described previously in section 1.1, the research and development of this Trip.Easy GDSS concept is divided into a outcome part7 and a process part.
7
The research and development of the meeting outcome of the Trip.Easy GDSS can be found in the Master’s Thesis Research from Touw (2010)
69
The focus of this study is on the evaluation of the designed meeting process (cf. decision process) which leads us to the following main research question for this experiment:
How do Trip.Easy GDSS users evaluate the decision process for establishing a consensus? In order to answer this research question, the concept satisfaction of the meeting process has been investigated. Satisfying experiences with collaborative technology, which Trip.Easy GDSS is, should fuel ongoing acceptance (Briggs & Vreede, 1997). In other words, if users are satisfied with the meeting process, which is an important aspect in the Trip.Easy GDSS, satisfaction towards the Trip.Easy GDSS in general will likely manifest. Although the measure of the satisfaction of the decision process has been shown to be useful for evaluating a collaborative technology such as Trip.Easy GDSS (Briggs & Vreede, 1997), additional insight is needed in order to evaluate the meeting process in more detail. There are two concepts in particular that will influence the users’ attitude positively or negatively towards the meeting process. First, the concept usability has been used because it is directly related to the quality of a user’s experience. By using the graphical user interface (GUI) users are supposed to interact and walk through the meeting process. It is believed that a positive user experience is a key element that will influence the meeting (Briggs & Vreede, 1997). Hence, if users perceive the interaction within each step of the meeting process as e.g. hard to use, confusing, bad navigational support which hinders the attainment of their goal for establishing consensus, then it is likely that it will affect the users attitude towards the meeting process because of the dissatisfied user experience. Second, members of a meeting care about the process used to reach a decision in addition to the decision itself (Kim & Mauborgne, 1995). Team members value process fairness, such as having their input considered and having influence over the final decision (Korsgaard, Schiweiger, & Sapienza, 1995). Hence in Trip.Easy GDSS, users should perceive the process to be fair due to i.e. the open/transparent setup of the GUI where their individual preference models are shown to other trip session members. Therefore, the concept fairness was included in the present study. Thus, the principal interests are satisfaction of the meeting process, usability and fairness, which together will support us in determining how users evaluate the meeting process in order to establish a satisfactory travel destination.
10.2 Hypotheses Hypotheses were formulated related to each of the concepts. The first concept studied here concerns satisfaction of the decision process. It was expected that users would satisfied with the designed group decision process which should be user friendly in usage and fair in order to allow for collaboration and selection of the travel destination. This lead to the following hypothesis: H1: Users experience the decision process implemented in Trip.Easy GDSS as satisfying
70
To explore and gain more insight why users have satisfying user experiences for each step of the decision process, hypotheses for the usability and fairness concept were formulated. The second concept of the study concerns the usability of Trip.Easy GDSS. Users make use of the graphical user interface (GUI) designed for Trip.Easy GDSS in order to attain their goal. Therefore, each screen of the GUI forms a step of the decision process that the users need to complete. Consequently, if users are satisfied with the usability for each step of the GUI, they will likely find the decision process satisfactory. Hence, we are interested in knowing whether users evaluate the usability of steps positively. Each step of the decision process has distinct usability focuses due to the steps differing in their goals. For instance, in the generate step the goal is to capture an individual’s travel preferences. The users therefore should be able to easily provide their input into Trip.Easy GDSS. To evaluate the design choices that have been made for each step, the following hypotheses were defined: H2: Users positively evaluate the usability in the “Generate” step H3: Users positively evaluate the usability in the “Reduce” step H4: Users positively evaluate the usability in the “Clarify” step H5: Users positively evaluate the usability in the “Evaluate” step H6: Users positively evaluate the overall usability of the Trip.Easy GDSS The last concept of the study is the fairness. It was expected that team members perceive the Trip.Easy GDSS meeting decision process to be fair due to the design choices that have been made (Part II of this thesis) like showing the preferences of all members of the session. In several steps in the decision process the concept fairness has been taken into account, which leads us to the following hypotheses: H7: The “Generate” step of Trip.Easy contributes to the perception of having a fair process H8: The “Reduce” step Trip.Easy contributes to the perception of having a fair process H9: The “Clarify” step Trip.Easy contributes to the perception of having a fair process H10: The “Evaluate” Trip.Easy step contributes to perception of having a fair process H11: Users perceive the overall process of the Trip.Easy GDSS to be fair An overview of the relations between the research question and the concepts is given in Figure 19.
71
Figure 19: Abstract conceptual model
By testing the hypotheses, we will obtain a good indication about the attitude towards each concept that influences how users evaluate the decision process in order establish consensus. Consequently, we should be able to answer the main research question. Moreover, it will provide us with new insights, data, ideas and hypotheses to improve or enhance the Trip.Easy GDSS.
10.3 Measures The present study is aimed at determining the attitude of users towards a group decision support system in the travel domain, in this case the Trip.Easy GDSS. In the previous section the research objective was presented and the research question and hypotheses were formed. In order to answer the research question and test the hypotheses, in this section the designed study will be detailed wherein the measured variables are motivated as well as how to measure them. The three concepts: satisfaction of decision process, usability and fairness will be operationalized from an abstract construct to a measure.
10.3.1 Satisfaction of the decision process Collaborative technologies such as group support systems (GSS) are often developed to improve the effectiveness and efficiency of teams. Similar to these systems the purpose of the Trip.Easy GDSS is to improve the decision process of a group of urban tourists. Consequently, we are interested in whether users are satisfied with this system. Different well-known evaluation models like TAM (Technology Acceptance Model) and AMIS (Assessment Model of Internet Systems) exist to capture the attitudes of users towards a system. Both models could be applied in this study, but for group decision support systems we know that the satisfaction users have with the processes and outcomes of the teamwork itself often determines the ultimate adoption and sustained use of collaborative technologies (Reinig, 2003). Therefore we were looking for a more suitable evaluation model for GDSS. Satisfaction with the process and outcomes of meetings is referred collectively as meeting satisfaction (Reinig, 2003). To determine meeting satisfaction the Goal Attainment Model for meeting satisfaction has been applied. This survey instrument developed by (Briggs & Vreede, 1997) measures the outcome and the process of a meeting as affective responses.
72
10.3.1.1 Reinig’s Goal-Attainment Model of Meeting Satisfaction Reinig’s Goal-Attainment Model decomposes meeting satisfaction into two constructs: satisfaction with meeting process (SP) and satisfaction with meeting outcome (SO). SP is defined as an affective arousal on the part of a participant with respect to the procedures and tools used in a meeting (Briggs, Reinig, & Vreede, 2006). SO is defined as an affective arousal on the part of a participant with respect to that which was created or achieved in a meeting. SO and SP are both positive functions of the Perceived Goal-Attainment (PGA) (Figure 20). PGA is defined as the degree to which one perceives that some object of satisfaction either advances or hinders the attainment of one’s salient individual goals.
Figure 20: Meeting Satisfaction as a function of Perceived Goal Attainment
The Goal Attainment Model assumes that individuals hold multiple goals and that, during the course of a meeting, some goals may be advanced (resulting in positive value appraisal) while others may be hindered (resulting in negative value appraisal). The model, with foundations in Locke and Latham’s (1990, 2002; Locke, 1969) goal-setting theory, posits a cognitive mechanism that automatically and subconsciously aggregates the advances and hindrances associated with an object of satisfaction to arrive at a net value, which in turn gives rise to satisfaction if the net value is positive and to dissatisfaction if the net value is negative. The model also recognizes that one important goal people usually hold for a meeting process is to produce a satisfactory outcome. Meeting processes that produce satisfactory outcomes are more likely to be satisfying than processes that give rise to dissatisfactory outcomes. Thus, SO should also account for some portion of the variance in SP, although this relationship would be expected to have less strength than the relationship between PGA and SP because SO would be only one of many possible goals. For more detailed information about the Reinig’s GoalAttainment Model we refer to the paper (Briggs, Reinig, & Vreede, 2006).
73
10.3.1.2 Operationalization The focus of this study is the attitude towards the Trip.Easy session decision process. The construct PGA and SP is useful for this experiment. We used the survey instrument to measure SP as affective responses, using terms such as “I feel satisfied with …” and “I feel good about …”. PGA is a judgment of the net value of the gains and losses with respect to individual goal attainment, in other words, participants have to evaluate costs and benefits with respect to that object over the duration of the meeting. When considering costs, one considers giving up something of value. When one considers benefit, one considers acquiring something of value. By using the collected data from the two constructs we can accept or reject the hypothesis: H1: Users experience the process implemented in Trip.Easy GDSS as satisfying The survey instrument contains 4 questions for the SP and 4 for questions for the PGA construct (see Appendix 5: Survey). All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. 10.3.1.3 Reliability and Validity The questionnaire items to determine the SP and PNGA were empirically validated in (Briggs, Reinig, & Vreede, 2006). Moreover, an expert on group decision-making measurement conducted an initial review to establish face validity. The Cronbach’s coefficients were calculated to test the reliability. According to Field (2009) a cut-off value of 0.70 was considered acceptable due to the research has an exploratory nature. The reliability analyses indicated that the measurements of the SP and PGA are reliable. The Cronbach’s for the SP = 0.88 and the Cronbach’s for the PGA = 0.848.
10.3.2 Usability The ISO’s 9241-11:1998 standard defines usability as ‘the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use’ (Brinkman, Haakma, & Bouwhuis, 2009). Effectiveness concerned the accuracy and completeness with which the participants achieved their goals. Efficiency concerned the resources used to achieve the goal in relation to its accuracy and completeness. Satisfaction concerned the freedom from discomfort and positive attitudes towards the use of the product. These three constructs will be used in order to measure the usability of Trip.Easy GDSS. The focus of the study was at satisfaction, because the current stage of the development was to test whether Trip.Easy GDSS can be used for establishing a consensus based travel destination rather than simply improve effectiveness or efficiency. 10.3.2.1 Operationalization As stated before in section 10.3.1.1, the Perceived Goal Attainment (PGA) from the Goal-Attainment Model is used to recognize that it takes effort to fulfill goals. Implicit to the desire to fulfill goals is to do so in a manner that the benefit of the goal exceeds the cost incurred by fulfilling the goal. It is believed that good interaction experiences from a system like Trip.Easy GDSS will result in a positive evaluation of the usability, which in turn will result in a more positive PGA. The aim is to try to get an understanding of the user experience concerning the decision process of the system. Therefore we want to capture wellformed beliefs about the usability of the system.
74
Breaking down the decision process into a sequence of steps, the evaluation of the interaction can be more specific. Using the heuristics from the heuristic evaluation approach which is a common technique in Human Computer Interaction (HCI) evaluations (Dix, Finlay, Abowd, & Russell, 1997), we examined each step of the decision process using self-reported measures. This way of usability evaluation is like the component-specific way of usability evaluation from (Brinkman, Haakma, & Bouwhuis, 2009). In the Trip.Easy GDSS case the components described by Brinkman et al. are in fact steps in the decision process. In the next sub-section we describe how we evaluated the steps in the GUI in order to accept or reject the hypotheses. For each step we selected the heuristics that we think it is important for that specific step. The result survey for usability is shown in Appendix 5: Survey. The process of converting the survey into usable data is described in Appendix 6: Survey processing. 10.3.2.1.1 Step: Generate H2: Users positively evaluate the usability in the “Generate” step Users’ attitude towards the usability of this “Generate” step of the decision process is based on their interaction experience when operating with it. In section 7.5.2 we explained that the goal of the “Generate” step is to capture the preferences of the city trip from the user. Therefore in this part of the GUI it is important that users can express their preferences when interacting with it. The ordering method applied in the form of “drag and drop” the features of a city trip should help them attaining that goal. Hence, from an evaluation point of view, it seems essential that we measure their attitudes towards the interaction concerning this specific step. The following set of questions were derived and adapted from the heuristics evaluation approach based on what is relevant for this specific screen: 1. 2. 3. 4. 5.
Are users satisfied with the ordering method in order to express their preferences? Do users like the drag and drop interaction to specify their preferences? Was it difficult to execute the process? Was it clear to the users what is expected from them? Was the presentation and structure of the components in the GUI clear enough to the users in order to execute the process successfully?
All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. In addition, usability data were also collected in the experiment through an open question by asking them to provide comments about the usability for this specific step in the decision process. Altogether it will reflect how users evaluate the usability of this step and thus accept or reject the hypothesis H2.
75
10.3.2.1.2 Step: Reduce H3: Users are satisfied with the usability in the “Reduce” step Users’ attitude towards the usability of this “Reduce” step of the decision process is based on their interaction experience when operating with it. In section 7.5.2 we explained that the goal of the “Reduce” step is to reduce the size of the list with cities. The composition of that list is the result of the aggregation of the preferences from each participant in the collaboration process. In order to reduce the list with opportunities (possible city trips here) users need to express their preferences for the cities they like, but eventually with respect to the group. Therefore it is important to know whether or not the GUI is providing enough information and good interaction for the user to make a decision. Hence, from an evaluation point of view, it seems therefore essential that we measure their attitudes towards the interaction and information provision concerning this specific step. The following set of questions were derived and adapted from the heuristics evaluation approach based on what is relevant for this specific screen: 1. Are users satisfied with the ordering method in order to express their preferences for the cities? 2. Do users like the drag and drop interaction to specify their preferences? 3. Was it difficult to execute the process? 4. Was the goal of this “generate” step clear to the users? 5. Was the presentation and structure of the components in the GUI clear enough to the users in order to execute the process successfully? 6. Do users think that the graph is a good way to summarize the (group) preferences? 7. Do users think that the current representation of city characteristics provides a good subjective view for a city? All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. In addition, usability data were also collected in the experiment through an open question by asking them to provide comments about the usability for this specific step in the decision process. Altogether it will reflect how users evaluate the usability of this step and thus accept or reject the hypothesis H3. 10.3.2.1.3 Step: Clarify H4: Users are satisfied with the usability in the “Clarify” step Users’ attitude towards the usability of this “Clarify” step of the decision process is based on their interaction experience when operating with it. In section 7.5.2 we explained that the goal of the “Clarify” step is to let users express the pros and cons of the alternatives (cities in the aggregated list) to the group. A meaning is very subjective and could be anything. In order to make it comparable and more structured, we need the user interface to incorporate these factors. Therefore it is important to know whether or not the GUI is providing enough information and good interaction for the user to complete this step. Hence, from an evaluation point of view, it seems therefore essential that we measure their attitudes towards the interaction and information concerning this specific step. The
76
following set of questions were derived and adapted from the heuristics evaluation approach based on what is relevant for this specific screen: 1. Are users satisfied with the rating method in order to express their preferences for the cities? 2. Do users like using stars to rate their preferences? 3. Was it difficult to execute the process? 4. Was the goal of this “clarify” step clear to the users? 5. Was the presentation and structure of the components in the GUI clear enough to the users in order to execute the process successfully? 6. Do users perceive it as useful to have an option to retrieve group preferences and city information? All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. In addition, usability data were also collected in the experiment through an open question by asking to provide comments about the usability for this specific step in the decision process. Altogether it will reflect how users evaluate the usability of this step and thus accept or reject the hypothesis H4. 10.3.2.1.4 Step: Evaluate H5: Users are satisfied with the usability in the “Evaluate” step Users’ attitude towards the usability of this “Evaluate” step of the decision process is based on their interaction experience when operating with it. In section 7.5.2 we explained that the goal of the “Evaluate” step is to start a discussion negotiation in the group which should converge on a group consensus by selecting one alternative (travel destination). Therefore it is important to know whether or not the user interface is providing enough information for the group to start a discussion and to make a consensus based decision. Hence, from an evaluation point of view, it seems therefore essential that we measure their attitudes towards the interaction and information provision concerning this specific step. The following set of questions were derived and adapted from the heuristics evaluation approach based on what is relevant for this specific screen: 1. Are users satisfied with the discussion method in order to express their preferences for the cities? 2. Do users like to discuss to make a final decision for the travel destination? 3. Was it difficult to execute the process? 4. Was the goal of this “evaluate” step clear to the users? 5. Was the presentation and structure of the components in the GUI clear enough to the users in order to execute the process successfully? 6. Does the interface provide enough options and information in order to express their preferences to a specific city to other users?
77
7. Do users perceive it as useful to have an option to retrieve group preferences and city information? 8. Do users like the new set of recommendations by Trip.Easy in the current stage of the decision process? 9. Does the interface provide enough options and information in order to start a good discussion? All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. In addition, usability data were also collected in the experiment through an open question by asking to provide comments about the usability for this specific step in the decision process. Altogether it will reflect how users evaluate the usability of this step and thus accept or reject the hypothesis H5. 10.3.2.1.5 Trip.Easy GDSS in general H6: Users are satisfied with the overall usability of the Trip.Easy GDSS In the end we want to measure the interaction experience for the TripEasy GDSS in general. The main goal of Trip.Easy GDSS in general is to provide users an intelligent tool in order to start a group decision making process. The result of a well-defined collaborative decision making process plus intelligent aggregation algorithms should lead to a satisfied consensus based travel destination for all members of the travel party. The aim is to explore what might influence the satisfaction level of the decision process during operating with Trip.Easy GDSS. Therefore we want to capture the usability experiences about the interaction and information provision from the users. The following set of questions were derived and adapted from the heuristics evaluation approach based on what is relevant for the overall GUI: 1. Do users perceive the effort to operate Trip.Easy GDSS as high? 2. Were the activities that needed to be performed in each step of the decision process clear to the users? 3. Do users find Trip.Easy GDSS difficult to operate? 4. Do users find the Trip.Easy GDSS handy in use in order to achieve a consensus based travel destination? 5. Do users perceive that the decision process is more time-saving (efficient) when using Trip.Easy GDSS? 6. Do users perceive the usage of Trip.Easy GDSS as an improvement (effective) of the group decision process? All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. In addition, usability data were also collected in the experiment through an open question by asking them to provide comments about the usability for this specific step in the decision process. Altogether it will reflect how users evaluate the usability of Trip.Easy GDSS in general and thus accept or reject the hypothesis H6.
78
10.3.2.2 Reliability and Validity The questionnaire items to determine the usability concept has an explorative nature and therefore are not statistically validated. However, an expert on HCI research conducted an initial review to establish face validity. As reference for the questionnaire, we used (Brinkman, Haakma, & Bouwhuis, 2009) and (Dix, Finlay, Abowd, & Russell, 1997) and derived the questions from.
10.3.3 Fairness Since the objective of the meeting process from the Trip.Easy GDSS is to establish a consensus between the group members, it is believed that the fairness concept will have positive or negative influences on the attitude towards the decision process. This is also supported by (Briggs, Reinig, & Vreede, 2006) where is showed that team members value process fairness, such as having their input considered and having influence over the final decision. Thus, if a team member perceives a process to be fair (e.g., having one’s voice heard and considered), then, ceteris paribus, such a perception should instantiate positive goal attainment and cause satisfaction with the meeting process. Furthermore, if a team member believes his or her interests have been taken into consideration, then he or she is more likely to believe that those interests are reflected in the final outcome, and consequently he or she should exhibit more positive attitudes toward the final decision as well. 10.3.3.1 Operationalization In order to examine the fairness on an exploratory way, a set of questions were developed for each step in the meeting process of the Trip.Easy GDSS using self-reported measures. Hence, breaking down the meeting process into a sequence of steps, the valuation of the process fairness can be more specific. Below we describe how we evaluated the process fairness for each step in the collaboration process given the hypotheses in section 10.1. This study will use self-reported measures by using the survey method. The resulted survey for fairness is shown in Appendix 5: Survey. The process of converting the survey into usable data is described in Appendix 6: Survey processing. 10.3.3.1.1 Step: Generate H7: The “Generate” step of Trip.Easy contributes to the perception of having a fair process In section 7.5.2 we explained that the goal of the “Generate” step is to capture the preferred characteristics of cities the user might like to visit. In respect to the fairness concept, it is therefore important in this step of the meeting process that users having the feeling that their voice is heard. This means that they must be able to tell Trip.Easy GDSS what their preferences are. We analyzed the city trip domain and translated it into a set of features which Trip.Easy GDSS for instance use to capture the preferences from the users about their preferences for the city trip. By ordering the predefined set of features, we are interested whether users have the feeling that their preferences are taken into account by Trip.Easy GDSS.In order to find that out, we measure the following aspect: 1. Can users identify with the given features in order to capture their preferences for a city trip?
79
This item was measured by the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, is treated as interval data. In addition, fairness data were also collected in the experiment through an open question by asking them to provide comments about the fairness for this specific step in the decision process. Altogether it will reflect how users evaluate the fairness of this step and thus accept or reject the hypothesis H7. 10.3.3.1.2 Step: Reduce H8: The “Reduce” step of Trip.Easy contributes to the perception of having a fair process In section 7.5.2 we explained that the goal of the “Reduce” step is to reduce the size of the list with cities. The composition of that list is the result of the aggregation of the preferences from each participant during the collaboration process. So users need to express their preferences for the cities they like, but eventually with respect to the group. To stimulate the fairness of the meeting process, in this step we believe, that not only the voice of the user needs to be taken under consideration but users should also be aware of the preferences of other travel members. An ordering technique was applied in this step to let the users express their preferred travel destination. In order to stimulate consensus, awareness about the preferences of other team members is created through a graph component in the interface. Also incorporated in the interface are detailed city characteristics which tell how well the specific city suits the travel party. Therefore, the following set of questions is used in order to measure the fairness concept for this step: 1. Do users like to have insight about the preferences of the group and team members? 2. Do users use the information of the group and other member’s preferences for making their decision? All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. In addition, fairness data were also collected in the experiment through an open question by asking them to provide comments about the fairness for this specific step in the decision process. Altogether it will reflect how users evaluate the fairness of this step and thus accept or reject the hypothesis Hd. 10.3.3.1.3 Step: Clarify H9: The “Clarify” step of Trip.Easy contributes to the perception of having a fair process In section 7.5.2 we explained that the goal of the “Clarify” step is to let the user express their opinions and remarks to the group for each city in the aggregated list. In other words, it’s all about building shared understanding. Which means that somehow within this step the user needs the possibility to express selves but, importantly also consider the preferences of the other travel members in order to establish a consensus. A rating method has been applied in this step to let users express their likeliness for a specific travel destination. Users may also provide comments why they like or dislike a particular city and eventually specify using a predefined set of features why they rated the city low. In order to create awareness and to stimulate for a consensus, the GUI provides the users the possibility to retrieve
80
the preferences of the group and team members. Therefore, the following set of questions is used in order to measure the fairness concept for this step: 1. Do users perceive assigning points (rating with stars) as fair in order to express their preferences? 2. Do users perceive have enough options to express their preferences? 3. Do users use the information of the group preferences for making their decision? All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. In addition, fairness data were also collected in the experiment through an open question by asking them to provide comments about the fairness for this specific step in the decision process. Altogether it will reflect how users evaluate the fairness of this step and thus accept or reject the hypothesis H9. 10.3.3.1.4 Step: Evaluate H10: The “Evaluate” step of Trip.Easy contributes to the perception of having a fair process In section 7.5.2 we explained that the goal of the “Evaluate” step is to start a discussion and negotiation in the group which ends with a consensus based decision. To stimulate the fairness of the meeting process, in this step we believe that the method that is chosen, namely discussion and negotiation with structured and clear information of the current situation, will lead to a satisfied consensus. The GUI provided the last aggregated set of possible travel destinations based on the previous steps of the decision process. With this information the travel party is expected to discuss and negotiate about their final consensus based decision. Respecting the fairness concept, we are wondering whether users perceive the implementation of this step to be fair in contributing to the decision making process. The following set of questions was used in order to measure the process fairness for this step: 1. Does transparently showing the preferences and comments from each user of each city help in reaching consensus? 2. Do users perceive the discussion method in this step as fair in order to express their preferences? All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. In addition, fairness data were also collected in the experiment through an open question by asking them to provide comments about the fairness for this specific step in the decision process. Altogether it will reflect how users evaluate the fairness of this step and thus accept or reject the hypothesis H10.
81
10.3.3.1.5 Trip.Easy GDSS in general H11: Users perceives the overall meeting process of the Trip.Easy GDSS to be fair In the end we want to measure the fairness of the process perceived by the users for the TripEasy GDSS in general. The main goal of Trip.Easy GDSS is to support a travel party in selecting a consensus based travel destination. Trip.Easy GDSS should therefore at least stimulate a situation where users have a fair and satisfied feeling about the group decision process. Therefore we measured the fairness of the process in general using the following questions: 1. Do users perceive the decision process in general as fair in order to achieve consensus? 2. Do users have the feeling that their preferences had influence on the process? All measures use the following 7-point Likert-type scale: 1 = strongly disagree, 4 = neutral, 7 = strongly agree, and so, were treated as interval data. In addition, fairness data were also collected in the experiment through an open question by asking them to provide comments about the fairness for this specific step in the decision process. Altogether it will reflect how users evaluate the fairness of Trip.Easy GDSS in general and thus accept or reject the hypothesis H11.
10.3.3.2 Reliability and Validity The questionnaire items to determine the fairness concept has an explorative nature are therefore not statistically validated. However, an expert on group decision-making measurement conducted an initial review to establish face validity. As reference for the questionnaire, we used (Brinkman, Haakma, & Bouwhuis, 2009) and (Dix, Finlay, Abowd, & Russell, 1997) and derived the questions from.
10.4 Protocol In order to answer the research question and test the hypotheses as stated in the previous section this exploratory study inquired a flexible design strategy using case studies. Participants, a group of friends, have the objective to collaborate and select a travel destination using the Trip.Easy GDSS. They were asked to carry out a series of tasks so that the needed measures have been captured. The whole process of the experiment is schematically depicted in Figure 21.
Figure 21: Schematic outline of the various steps of the experiment
During the introduction the participants were informed about the aim of the study and what they can expect from the experiment. To motivate the participants we announced that they automatically took part in a lottery, they can win a city trip8. A second stimulus to motivate them was introduced.
8
Sponsored by a private investor (Maarten van Dijk)
82
Participants will get the possibility to join “vakantiepunten.nl” and if they do, they will get extra “vakantiepunten9” which are points that they can trade against discounts on future trips. Like in a real situation someone has to take the initiative and suggest for instance a group of friends to go on a trip. Therefore in each experimental session one of the participants was assigned to be the initiator and had to invite other participants to join the collaboration process using the Trip.Easy GDSS. When participants accept the invitation, they need to walk through the whole collaboration process and end with making a decision about the destination. During the collaborative process users were not allowed to communicate with each other in any form except with the Trip.Easy application. This was done in order to simulate the usage in a place and time independent situation just as Trip.Easy GDSS was designed for. After they have successfully finished the session, participants took an individual online survey to evaluate each step of the Trip.Easy GDSS. The survey was composed of eight sections that were completed sequentially; four sections that evaluate the steps in the meeting process, one section for the evaluation of the system in general, two sections to measure the possible acceptance future usage and the final section for further improvements. Each section contains an open question where the participants were asked to provide remarks. The survey can be found in Appendix 5: Survey. We developed a survey system to collect all quantitative and qualitative data that easily can export the data into other applications like SPSS for statistical processing.
10.5 Subjects Before we improve the efficiency and effectiveness of the Trip.Easy GDSS in the future, it is important for us to have a representative indication about users’ satisfaction with the Trip.Easy GDSS concept. To gain representative results we focused on two issues: (i) choice of the participants and (ii) the sample size. The choice of subjects was vital to the experiment. In evaluation experiments subjects should be chosen to match the expected user population as closely as possible. Our target audiences for the Trip.Easy GDSS are users who are familiar with internet and social media. This means that the demographic diversity may vary greatly. As mentioned earlier in the theory (chapter 2), another important factor is that most of the time a travel party consists of members who know each other, like a group of friends, colleagues or family. Therefore subjects of our experiments were asked to invite three more members that they know to participate in their travel party. To compare the results within our experiment we decided that each travel party has the size of four members.
9
Sponsored by vakantiepunten.nl
83
The second issue related to the subject set is the sample size to be chosen. With our small amount of resources, the availability of subjects was limited. To have the sample size large enough to be considered as representative we used statistics to determine the necessary size of the sample. As a result our sample size of n=120 participants is based on a population of 6.400.00010, with a margin of error of 10%, a confidence level of 95% and a spread of 50%. According to (Bartlett, 1977) a sample size of minimal n=96 is representative. With our questionnaire design which makes use of a 7 point Likert scale, according to (Brinkman, Questionaire design) a sample size of minimal n=119 will be acceptable. To summarize the selection of our subjects for the experiment: we randomly choose 30 subjects who in turn invite 3 more participants to form a travel party of 4 members in total. As a result our sample size of n=120 is large enough to be considered as representative for this experiment.
10.6 Equipments and setting The experiment took place at the office of Studio Kenneth&Koh. In a room containing 4 computers, we can test one group at the time. The duration of the experiment was approximately 40 minutes.
Figure 22: Test room arrangement
Each computer has the same configuration and LCD monitor with a screen resolution 1280x1024. To run the Trip.Easy GDSS application an internet connection was needed and a XHTML compatible browser. For this experiment, the participants used the Google Chrome browser.
10.7 Destinations Database To accommodate the experimental work, representative urban destination data was needed. As mentioned in chapter x, the current scope of Trip.Easy GDSS is urban traveling. Consequently, a mapping of detailed city information into our database model has to be performed. 10
Volume and intensity of short holidays of the Dutch (length 2-8 days) in 2008: CBS Toerisme en recreatie in cijfers 2008
84
The procedure to fill the database essentially consisted of three parts: (i) determine the city to be included, (ii) select trustworthy resources, and (iii) map the information from different resources into the database. The result list of cities can be found in Appendix 4: City Database.
10.7.1 List of cities To let users experience a real life scenario as much as possible when selecting destinations for vacation, the following points were taken into account: •
•
•
In the list of sovereign states 201011, a total of 193 widely recognized countries/sovereign states are listed. Consequently, if we want to select one city for each country, as a result the database will directly have 193 entries. So we have taken into account that not all countries/cities in the world are travel worthy; The database must be filled up with sufficient choices. In practice a set of one city for each country is not representative. Travelers want to have several choices within a country like, San Francisco, New York, Washington or Miami for the United States; The last element would be the diverse price range, which will have substantial influence on the choice.
As a result we carefully selected 47 countries with a developed tourism industry from the following continents: Europe, North America, South America and Asia. This resulted in a total of 157 cities in our database.
10.7.2 Resources The next challenge is to collect the needed information from trust worthy resources like travel websites and travel magazines. The city information needs to be processed into another format in the next step so internet websites were preferred sources of information due to their digital contents. TripAdvisor12 is a user generated content website where travelers and experts can provide review and opinions about the trip destinations. With their comprehensive description of the cities and over 30 million trusted reviews, we choose to use this website as our resource to fill up our database.
10.7.3 Mapping According to our city model, which is specified in section 7.2, cities are described in the database using features. The rating of the features for each city is done in a very subjective way. By analyzing the contents on TripAdvisor and by using traveler reviews we were able to express the weights of the features (see section 7.2 for mapping details).
11
http://en.wikipedia.org/wiki/List_of_sovereign_states: overview of states around the world with information on the status and recognition of their sovereignty. 203 entries exist, divided in two parts. The first part lists all 193 widely recognized sovereign states, including all member states of the United Nations and Vatican City. The second part lists states which are de facto sovereign, but which are not widely recognized by other states. 12 http://www.tripadvisor.com
85
10.8 Pilot test Pilot tests were performed to evaluate and fine-tune the experimental protocol. Also we wanted to make sure that there are no critical malfunctions or bugs left in the system. In this section the process, findings and results of this pilot test are briefly described. Sixteen subjects participated in the pilot test. We selected 4 subjects and asked them to bring 3 more friends to join in order to have a travel party of 4 participants each. These 4 groups will not participate in the final experiment, in order to prevent biases. The subjects went through the experimental protocol and filled in the questionnaire. The pilot test indicated that several modifications to the Trip.Easy GDSS were needed because of malfunctioning or performance problems. For instance, in several steps in the decision process, wrong data were presented, furthermore aggregation of the 4 group preference model at the same time failed due the limited computational power of our test machine. Also, the long discussions held by the participants in the “Evaluate” step resulted in an expired session time, so proceeding further with the decision process was not possible anymore. Based on the findings discovered from this pilot test, we improved the Trip.Easy GDSS prototype and created a version that was used for the actual experiment.
Figure 23: Pilot test
Additionally, we improved the organization of the experiment by providing the subjects a good introduction and explanation. The duration of the experiment was set to a maximum of 30 minutes for the Trip.Easy GDSS session and an additional 30 minutes was allocated for the completion of the survey. After the adjustments to the system, questionnaire and experiment protocol, we were finally ready to conduct the actual experiment.
86
11 Results The experiment was conducted to evaluate the Trip.Easy GDSS decision support process by measuring the satisfaction level, usability and fairness concerning the meeting process. A total of one hundred twenty participants, 72 male and 48 female, between the ages of 18 and 55 (M = 28.1, SD = 7.75) 13 years old participated in the experiment. They were asked to operate the Trip.Easy GDSS that was developed to support them to pick a consensus based travel destination. Subsequently, the participants filled in the survey where their perceived satisfaction, usability and fairness of the Trip.Easy GDSS decision process were captured. In this chapter, we analyze the measurements for the satisfaction level towards the decision process as a whole in order to accept or reject hypothesis h1. Subsequently we go in more detail by analyzing the measurements for each step in the decision process in order to accept or reject the hypotheses with regard to the usability (hypotheses h2 – h6) and fairness (hypotheses h7 – h11).
11.1 Satisfaction of decision process In this section we describe how the concept “satisfaction of decision process” has been measured. With this concept we wish to evaluate the users attitude towards the Trip.Easy GDSS and explore if it is capable to support users with making a consensus based decision about their “city” trip on a satisfactorily way. By analyzing the construct SP we are able to accept or reject the hypothesis H1: Users experience the decision process implemented in Trip.Easy GDSS as satisfying.
11.1.1 Construct: Satisfaction Process (SP) The most important measure of this research is the construct SP from Reinigs Goal-Attainment Model. By adding the measures (question 5,6,7,8 category 7 Appendix 5: Survey) and then calculate the average for each subject the SP can provide us a good indication whether users were having a positive attitude towards the decision process of Trip.Easy GDSS. The results show that the overall satisfaction of the decision process was 5.89 (SD=0.76, CI.95 = 5.75, 6.02) on a seven-point scale, this indicates that on average the users had a satisfied feeling. Almost 91% of the subjects perceived the decision process as somewhat satisfying till very satisfying. Details of the statistical results can be found in Appendix 6: Survey Processing.
13
Appendix 9 for more detailed information
87
Satisfaction Very unsatisfied Unsatisfied Little unsatisfied Neutral Somewhat satisfied Satisfied Very satisfied Total
Percent (%) 0 0 1.6 7.5 25 51.7 14.2 100 % Table 6: Satisfaction (SP)
In order to find out whether we can accept hypothesis H1 we performed a ‘one sample t-test’ with a test value of 5.0, which is the break-even point between ‘satisfied’ and ‘somewhat satisfied’. To be able to perform the ‘one sample t-test’ we have to meet the assumption of normality. In Appendix 19.5: Survey Processing we show that the satisfaction of the process (SP) is normally distributed in the QQ-plot. Furthermore, according to Field (2009) a sample size greater than 100 is assumed to be normal. The sample size of our study is 120. Consequently, we assume that the data of SP is normal. The results of the ‘one sample t-test’ showed that on average, participants perceived a significantly greater satisfaction to the process 5.89 (SD=0.76, CI.95 = 5.75, 6.02) than the break-even point of 5.0, (t (119) =12.750, p < .01).
11.1.2 Conclusion The promising result suggests that the majority of the users are having a highly positive affection for our group decision support system for the urban tourism domain. According to Reinig’s model of meeting satisfaction, a ‘satisfied’ satisfaction level may indicate that users perceive the decision process as effective, efficient and fair for decision-making. Based on the result of the ‘one sample t-test’ we therefore accept hypothesis H1: “Users experience the decision process implemented in Trip.Easy GDSS as satisfying” may be accepted. Next, we will go to the explorative questions which may help us to gain more insight into why and what explains the satisfactory feeling of the users.
88
11.2 Exploratory evaluation of the meeting process per step We elaborate on the meeting process in order to explore how the usability and fairness concept affects the satisfaction response. First, the construct Perceived net Goal Attainment (PGA) from Reinig’s meeting satisfaction model will be analyzed. As a reminder, PGA in Trip.Easy GDSS is the perceived benefits less the perceived costs of attempting to decide on a satisfying travel destination. In other words, it takes effort to fulfill goals. The PGA causes the SP and can therefore be used to indicate on a high level why users are satisfied with the meeting process in general. Not only are we interested in the high level explanation, but we also want to evaluate each step of the meeting process to gain more insight. To this end the obtained qualitative and quantitative data from the usability and fairness construct will be analyzed for each step of the meeting process. Figure 24 presents an overview of this exploratory evaluation: Trip.Easy GDSS in General
Quantitative data: PGA
Step 1: Generate
Quantitative data: Usability
Quantitative data: Fairness
Qualitative data: Usability
Qualitative data: Fairness
Quantitative data: Usability
Quantitative data: Fairness
Qualitative data: Usability
Qualitative data: Fairness
Quantitative data: Usability
Quantitative data: Fairness
Qualitative data: Usability
Qualitative data: Fairness
Quantitative data: Usability
Quantitative data: Fairness
Qualitative data: Usability
Qualitative data: Fairness
Quantitative data: Usability
Quantitative data: Fairness
Qualitative data: Usability
Qualitative data: Fairness
Step 2: Reduce
Step 3: Clarify
Step 4: Evaluate
Trip.Easy GDSS in General
Figure 24: Exploratory evaluation of meeting process per step
89
11.2.1 Construct: Perceived Gained Net Attainment (PGNA) The Perceived Goal Net Attainment (PGA) is a measure that reflects the desire to fulfill goals in a manner that the benefit of the goal exceeds the cost incurred by fulfilling the goal. When Trip.Easy GDSS users for instant experience the interaction as hard to use or the decision process as inefficient in order to establishing a consensus based travel destination, then probably the PGA will result in a low value. As effect it will negatively affect the attitude towards the meeting process. The PGA was measured by four questions (question 1,2,3,4 category 6 Appendix 5: Survey). The four questions measures the same construct, therefore we can calculate the mean of the measures and obtain the PGA value of each subjects. As a result the overall satisfaction of the PGA has an average of 5.93 (SD=0.71, CI.95 = 5.80, 6.06) on a seven-point Likert scale. On average the users had a satisfied feeling. Almost 94% of the subjects perceived the PNGA as somewhat satisfying till very satisfying. Details of the statistical results can be found in Appendix 6: Survey Processing. Satisfaction Very unsatisfied Unsatisfied Little unsatisfied Neutral Somewhat satisfied Satisfied Very satisfied Total
Percent (%) 0 0.8 0 5.9 26.7 55 11.7 100 % Table 7: PNGA
In order to find out the significance of this measure we performed a ‘one sample t-test’ with a test value of 5.0, which is the break-even point between ‘satisfied’ and ‘somewhat satisfied’. To be able to perform the ‘one sample t-test’ we have to meet the assumption of normality. In Appendix 19.5: Survey Processing we show that the Perceived net Goal-Attainment (PGA) is normally distributed in the QQplot. Furthermore, according to Field (2009) a sample size greater than 100 is assumed to be normal. The sample size of our study is 120. Consequently, we assume that the data of SP is normal. The results of the ‘one sample t-test’ showed that on average, participants perceived a significantly greater satisfaction to the process 5.93 (SD=0.71, CI.95 = 5.80, 6.06) than the break-even point of 5.0, (t (119) =14.278, p < .01). 11.2.1.1 Conclusion The positive measured value for the PGA implies that the users perceive the realization of their goal worth the effort. Apparently, users perceive the benefit of the goal is exceeding the cost that is incurred by fulfilling the goal. This indicates that users have a positive attitude towards the interaction with the decision process of Trip.Easy GDSS to establishing a consensus based travel destination. This corresponds well with Reinigs’ goal attainment model theory, which says that a positive PGA should result in a positive amount of meeting satisfaction. Apparently this also applies for Trip.Easy GDSS, we observed a high positive amount of PGA and as a consequence a satisfied meeting process was found. 90
11.2.2 Evaluation of step 1: Generate The objective of this step is to identify the preferences of the city trip from the users so that Trip.Easy GDSS can establish a group preference model. Based on this group preference model Trip.Easy GDSS proposes a list of cities which will be used in the next step of the decision process. Two hypotheses were formed in order to evaluate this specific step, which should help the user to achieve the goal of this “generate” step. The two hypotheses are: • •
H2: Users positively evaluate the usability in the “Generate” step H7: The “Generate” step of Trip.Easy contributes to the perception of having a fair process
The hypotheses will be verified through analyzing the qualitative and quantitative data from the usability and fairness construct. All 120 participants completed the quantitative questions; only 43 participants provided useful qualitative data Appendix 19.6. A screenshot from the GUI for this step can be found in section 7.6.1. 11.2.2.1 Construct: Usability As pointed out earlier, the focus of the evaluation of the usability concept will be on the satisfaction component. We selected the heuristics from the heuristic evaluation method which are appropriate to apply to this end. Since users are supposed to provide their travel preferences into the system, it is important that users can express their preferences when interacting with it. By dragging and dropping from the list of general city features defined by the city model users should complete the “Generate” step. Five questions were formulated in paragraph 10.3.2.1.1 which was analyzed using a standard mean value calculation and one sample t-tests. Figure 25 presents the average level of satisfaction concerning the usability from the users.
Figure 25: Measures with 95% CI regarding the usability concept in the generate step
91
Considering the 95% confidence interval and the value 4 as the neutral value of the scale the results suggest that the majority of the users leaned towards a satisfied attitude towards the usability of the “Generate” step from the decision process. Below we will explain the results by using the quantitative data. We will use the qualitative data to explain the rationale behind these positions and provide additionally insight. Expressiveness - The expressive power of the ordering method. In order to find out whether users are satisfied with the expressive power of the ordering method, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the expressive power and ‘satisfied’ with the expressive power. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on the expressiveness aspect with a mean of 5.53 (SD=1.13) than the break-even point of 5.0, (t (119) =5.172, p < 0.01). This implies that users are more than ‘somewhat satisfied’ with the expressive power by ordering the features of city characteristics. Moreover, 60% of the users have a satisfied or very satisfied satisfaction level. This is an important indication because users’ preferences should be heard in order to provide good recommendations. This could be supported by the remarks that users mentioned, i.e. “Playing with the predefined set of features will let me think about my real preferences about a city trip. Normally I just don’t know what kind of cities suits me or my friends. The fact that I don’t have to know much about topography but just say what I want to see and what I want to do really attracts me”. Method - Dragging & dropping which is used to implement the ordering method. In order to find out whether users are satisfied with the dragging & dropping which is used to implement the ordering method, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with drag&drop interaction and ‘satisfied’ with the drag&drop interaction. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 6.06 (SD=1.03) than the break-even point of 5.0, (t (119) =11.24, p <0 .01). This implies a satisfaction level of ‘satisfied’. The results even shows that 75,9% of the users has a satisfied or very satisfied satisfaction level. This means that users has a satisfactorily interaction experience using the drag and drop method for ordering the features which will contribute to a positive attitude towards this step. From the qualitative data it becomes apparent that users like to use the drag and drop interaction because of its intuitive way of usage for ordering things. Several times it is mentioned that dragging and dropping interaction works very smooth. Using the colorful grouping of the features makes it easy to switch the focus between them. Effort - The effort used to execute the step. In order to find out whether users are satisfied with the effort to execute the step, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the effort used and ‘satisfied’ with the effort used. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.54 (SD=1.42) than the break-even point of 5.0, (t (119) =4.16, p <0.01). This implies that users may perceive the effort to be used to complete the “generate” step is almost “not much”. This may contribute to an acceptable usability of this step. The easy use of drag and drop interaction has been mentioned several times in the remarks, but the quantity of the features to drag and drop is not always accepted by the users. One user said “When I order the items, I just care only
92
about the first two, maybe three items. I don’t care about the order for item six, seven or eight. It takes too much unnecessary effort to order the features I already know that I don’t prefer”. Objective - The clearness of the objective of the “generate” step. Users should know what is expected from them in order to capture their travel preferences. Instructions and intuitive icons should support the user to interact with the interface. In order to find out whether users are satisfied with clearness of the objective, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the clearness of the objective and ‘satisfied’ with the clearness of the objective. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.49 (SD=1.40) than the break-even point of 5.0, (t (119) =3.84, p < 0.01). This implies a satisfaction level of a large “somewhat satisfied” value with the clearness of the objective. To be more precise, 61.7% of them had a satisfied or a very satisfied feeling. We think that the satisfaction level could be increased by optimizing the instructions. The remarks show strikingly that users do know what to do in the very first moment. They know that they can provide their preferences, but apparently not know how! There is often mentioned that a short fancy video instruction or some subtle movements of the features can make clear to the user that the colorful blocks (cf. features) could be dragged and dropped. Presentation - The presentation of components for the GUI. To advance the usability a consistent, structured and recognizable division of the components must be taken into account. The status of the decision process is permanent on the top of the screen; the instructions is framed in a blue box which the user may hide; followed by a section where users may order the features from low to high (preference input); finally, a recognizable orange button is used to let users process further with the process when ready. . In order to find out whether users are satisfied with the presentation, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the presentation and ‘satisfied’ with the presentation. The results of the ‘one sample t-test’ showed that on average, participants do not have a significantly greater score on the presentation aspect with a mean of 5.17 (SD=1.49) than the break-even point of 5.0, (t (119) =1.22, p > 0.05). The results showed that the measure is not significant. Thus, it cannot be claimed based on the results whether users are satisfied or not. However, the frequency chart shows that 51.7% of the users have a satisfied or very satisfied attitude towards the presentation. Analyzing the remarks we think that the presentation of the sub-features is not clear in the situation there is too much of them. The sub-features will then be divided into several lines which is confusing when ordering. Many users had problems with the interpretation about the priority levels. Questions arise like “I can prioritize my ordering from left (highly important) to right (not important), but what does it mean with the sub-features on the second line? I think it would be much clearer if we can order from the top to the bottom instead of left to right. The space problem could then be eliminated”. 11.2.2.1.1 Conclusion usability The five self-reported attitude variables measure the attitudes towards the interaction thus the usability concerning this specific screen. Only one of the five self-reported attitude variables showed an insignificant result with the ‘one sample t-test’. The means of the values were all tested against the 5.0 break-even point. An overview of the result:
93
Variable Expressiveness Method Effort Objective Presentation
Mean 5.53 (SD=1.13) 6.06 (SD=1.03) 5.54 (SD=1.42) 5.49 (SD=1.40) 5.17 (SD=1.49)
One sample t-test with test value 5.0 t (119) =5.172, p < .01 t (119) =11.24, p < .01 t (119) =4.16, p < .01 t (119) =3.84, p < .01 t (119) =1.22, p > 0.05
Table 8: Overview Usability one-sample t-test Step Generate
Based on these measures, we believe the majority of the users perceive the usability for this “generate” step is close to a “satisfied” satisfaction level. This means that the users positively evaluate the usability. Therefore, the hypothesis H2: Users positively evaluate the usability in the “Generate” step is accepted. The acceptance of H2 implies that the objective to capture users’ travel preferences using the implemented GUI and interaction is acceptable and usable. Helpful suggestions and critiques are gathered from the remarks which offer us a good insight what may have influenced the satisfaction level. Apparently the “drag and drop” interaction for ordering the travel features is received well to temper a positive attitude toward this specific step. Possibly, this positive attitude in turn should have a positive influence on the overall decision process. 11.2.2.2 Construct: Fairness As stated earlier, fairness is an important factor in a group decision support system. In order to gain trust from the user it is important in this “generate” step to listen to the user. A user must have the feeling that their voice will be heard, which means that they must be able to tell Trip.Easy GDSS what their preferences are. One question is formulated in paragraph 10.3.3.1.1 which is analyzed using a standard mean value calculation and one sample t-test. Figure 26 presents the average level of satisfaction of the fairness from the users. Features - The features describe the characteristics of a city trip. By ordering the set of features from highly preferred to not preferred, Trip.Easy GDSS gathers the travel preferences from each participant of the travel party. Using this information an individual preference model will be created to which an aggregation algorithm is applied so that a group preference model can be formed. This will be used to find a set of cities that fits the group best. Considering the 95% confidence interval an average score of 5.31 (SD=1.31) suggest that the majority of the users experienced that the defined set of features could be used to express their travel preferences. The results of the ‘one sample t-test’ support this too, because on average, participants have a significantly greater score on the feature aspect with a mean of 5.31 (SD=1.31) than the break-even point of 5.0, (t (119) =3.40, p = 0.01 p < .05). However, several users emphasized in their comments that sometimes the features are a little bit ambiguous, like the feature “environment” vs. “atmosphere”. Moreover, it is said that based on the textual representation of the features, it is hard for users to create an imaginary image of the feature. For example, how would you imagine the “environment” feature? Therefore, users suggested using some images in combination with the textual representation of the feature to take care of this.
94
Figure 26: Measures with 95% CI regarding the fairness concept in the generate step
11.2.2.2.1 Conclusion fairness The self-reported attitude variable measures the attitude towards the fairness concept concerning this specific step. Significant result is showed using the ‘one sample t-test’ applied on the ‘feature’ variable. An overview of the result: Variable Feature
Mean 5.31 (SD=1.31)
One sample t-test with test value 5.0 t (119) =3.40, p < 0.01
Table 9: Overview Fairness one-sample t-test Step Generate
Based on the measure, we believe the majority of the users perceive the fairness for this “generate” step is close to a “satisfied” satisfaction level. This means that the users positively evaluate the fairness. Therefore the hypothesis H7: The “Generate” step of Trip.Easy contributes to the perception of having a fair process is accepted. The acceptance of H7 implies that this positive attitude created in this step will at least partly contribute to the perception of having a fair process. Users are able to tell Trip.Easy GDSS what their preferences are by ordering the features from highly important to not important. Because the decision process takes care of their travel desires, the users perceived this property as fair which means that they have feeling that their voices are heard. 11.2.2.3 Conclusion “generate” step We designed and implemented a GUI for the “generate” step with the objective to facilitate users to insert their travel preferences. In order to have more insight why users perceive the decision process as satisfying, we explore how the usability and fairness concept affects the satisfaction response for this “generate” step. The survey study shows that users are satisfied with the usability and the fairness of the “generate” step of the decision process. Both hypotheses H2 for the usability concept and H7 for the fairness concept have been accepted. These results suggest that the attitude towards the “generate” step is at least partly based on the attitude towards the usability and fairness concept. Thus this implies that this step positively contributes towards the decision process in general.
95
11.2.3 Evaluation of Step 2: Reduce - Filtering Next we will evaluate the ‘Reduce’ step where users will proceed to select their preferred cities with respect to the set that has been proposed by Trip.Easy during the “Generate” step. Using the elicited group preference model, Trip.Easy GDSS calculated14 an optimal set of cities which should suit the travel party well. The objective of this step is to let users point out their most preferred cities from the list of cities that have been proposed by Trip.Easy GDSS. As mentioned in the design part of this study, we implemented this step of the decision process using the “reduce” collaborative pattern (the sub-pattern “filtering” from the “reduce” pattern is actually applied). It resulted in a Graphical User Interface (GUI) where users may rank the set of cities from highly preferred to not preferred. Elements of the GUI should stimulate the decision-making of the users to take the preferences of the travel party into account. Two hypotheses were formed in order to evaluate this specific step, which should help the user to achieve the goal of this step. The two hypotheses are: • •
H3: Users positively evaluate the usability in the “Reduce” step H8: The “Reduce” step of Trip.Easy contributes to the perception of having a fair process
The hypotheses will be tested through analyzing the qualitative and quantitative data from the usability and fairness construct. All 120 participants completed the quantitative questions; only 50 participants provided useful qualitative data Appendix 19.7. A screenshot from the GUI for this step can be found in section 7.6.3. 11.2.3.1 Construct: Usability Users are supposed to provide their preferred set of cities into the system and eventually take into account with preferences of other travel members. It is therefore important that the GUI supports the users to achieve this goal when interacting with it. By dragging and dropping the cities, the users can rank the set of cities in order to complete the “Reduce” step. Seven questions were formulated in paragraph 10.3.2.1.2 which are analyzed using a standard mean value calculation. Figure 27 shows the average level of satisfaction of the usability variables from the users. Considering the 95% confidence interval and the value 4 as the neutral value of the scale the results suggest that the majority of the users leaned towards a satisfied attitude about the usability of the “Reduce” step from the decision process. One exception appeared in the measures, namely the “City descriptions” variable where the measured value is somewhat neutral.
14
Detailed information about the aggregation algorithms can be found in the thesis from Touw (2010)
96
Figure 27: Measures with 95% CI regarding the usability concept in the Reduce step
Below we will explain the results by using the quantitative data. We will use the qualitative data to explain the rationale behind these positions and to provide additionally insight. Expressiveness – The expressive power of the ranking method. In order to find out whether users are satisfied with the expressive power of the ordering method, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the expressive power and ‘satisfied’ with the expressive power. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on the expressiveness aspect with a mean of 5.69 (SD=1.14) than the break-even point of 5.0, (t (119) =6.63, p < 0.01). This implies that users are close to a ‘satisfied’ satisfaction level with the expressive power by ranking the set of cities from highly preferred to totally not preferred. Moreover, 72.5% of the users have a satisfied or very satisfied satisfaction level. The high satisfaction level suggests that users think that their preferences are heard and understood by Trip.Easy GDSS which is very important. From the remarks we could see that users are surprisingly enthusiastic with the set of cities proposed by Trip.Easy GDSS based on their actions in the previous step. They mentioned for instance “By ranking the cities I could easily tell my friends what I like! Usually I just come up with one or two cities, but with an ordered list I have the feeling that Trip.Easy will do something to reach an optimal result for the group “. Method - Dragging & dropping which is used to implement the ranking method. In order to find out whether users are satisfied with the drag&drop interaction which is implemented for the ranking method, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the drag&drop interaction and ‘satisfied’ with the drag&drop interaction. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.91 (SD=1.12) than the break-even point of 5.0, (t (119) =8.86, p < 0.01). This implies that users perceive this aspect with an almost ‘satisfied’ satisfaction level. The results even show that 72.5% of the users have a satisfied or very satisfied satisfaction level. This means that
97
users have a satisfactory interaction experience using the drag and drop method for ordering their preferred set of cities. From the remarks it becomes apparent that users like to use the drag and drop interaction because of its intuitive way of interaction for ranking the cities. Multiple times it is mentioned that it is fun to play with the ranking by dragging and dropping the cities on different positions. Effort - The effort to execute the step. In order to find out whether users are satisfied with the effort to execute the step, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the breakeven point between ‘not satisfied’ with the effort to execute the step and ‘satisfied’ with the effort to execute the step. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.38 (SD=1.40) than the break-even point of 5.0, (t (119) =2.933, p = 0.01). This implies that users perceive the effort to be used to complete the “reduce” step as “somewhat satisfying”. The relative high standard deviation indicates that users have a diverse view about this measure. It can be observed that 45.1% of the users think that the effort needed to complete this step range from medium till high. When analyzing the qualitative data we could clearly see what the cause is for this measure. A lot of users think that the list of 30 cities to be ordered is too long. They don’t care about the rankings in the centre of the list. One of the users said: “The first 5 till 10 cities are still ok to rank. It is also fine to explicitly drag the not preferred cities to the end of the list. But I think it’s useless to rank the cities in the middle of the list. It just takes unnecessary effort to think about something that is not interesting”. Objective - The clearness of the objective of the “reduce” step. In order to find out whether users are satisfied with the clearness of the objective, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the clearness of the objective and ‘satisfied’ with the clearness of the objective. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.71 (SD=1.25) than the break-even point of 5.0, (t (119) =6.19, p < 0.01). This implies that users are near a ‘satisfied’ feeling about the clearness of objective. Even though this value is high and thus indicates that users know what was expected from them and how to interact with the GUI in order to complete this step, users mentioned that it was not clear in the beginning that the cities could be dragged and dropped. Even though it was mentioned in the instructions, it took some time for the users to recognize the drag&drop interaction. Some subtle animation of the city blocks could help users recognize the draggable and droppable feature. Presentation - The presentation of the components of the GUI. Two important components define the GUI in this “reduce” step. First, the rankable list of cities proposed by Trip.Easy GDSS can be found in the GUI. The second component in the GUI is an information section that contains information about the cities and explanations why the cities suit the group well. In order to find out whether users are satisfied with the presentation of the components of the GUI, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the presentation of the components and ‘satisfied’ with the presentation of the components. The results of the ‘one sample ttest’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.4 (SD=1.29) than the break-even point of 5.0, (t (119) =3.41, p = 0.01). This implies that users
98
are ‘somewhat satisfied’ with this setup. Only 55% of the users are satisfied or very satisfied with the presentation. Analyzing the remarks we think that the arrangement of the screen in general is fine. Users can easily ranking the cities; have more insight about the group or members preferences; retrieving extra information about the cities is also possible and alternating between the cities. But the users have some problems with understanding all the information. Some users even mentioned that all the information together is very intimidating. Graph - A graphical representation of consensus stimulating information in the form of a graph which explains why a city suits the group. In order to find out whether users are satisfied with the graphical representation of the consensus stimulating information, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the graphical representation of the consensus stimulating information and ‘satisfied’ with the consensus stimulating information. The results of the ‘one sample t-test’ showed that on average, participants do not have a significantly greater score on the graph aspect with a mean of 5.1 (SD=1.42) than the break-even point of 5.0, (t (119) =0.77, p = 0.44). The results showed that the measure is not significant. Thus, it cannot be claimed based on the results whether users are satisfied or not. However, the frequency chart (appendix X) shows that 71.7% of the users have a satisfied attitude between the “somewhat satisfied” level till the “very satisfied” level. Analyzing the remarks we see users perceives the information as is really useful but the problem is that the graph is hard to understand what the graph depicts. The problem could be the graph itself. Some suggest a pie chart or just explain it textually. The difference between the line of the city and the line of the group preference is not clear to the users. Apparently, users found it difficult to interpret the histogram, but notable is that users also say that “once knowing how to read the histogram and graph, it is very interesting. I take more the preferences of other members into account when making a decision”. City descriptions - Information which purpose is to describe the city in more detail. The extra information provided by Trip.Easy GDSS about the city has the purpose to support users with their decision making. Features of the cities were presented in a form of a column chart to support users in creating a mental picture about the city. In order to find out whether users are satisfied with the city information which purpose is to describe the city in more detail, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with city descriptions and ‘satisfied’ with city information. The results of the ‘one sample t-test’ showed that on average, participants do not have a significantly greater score on this measure with a mean of 4.41 (SD=1.42) than the break-even point of 5.0, (t (119) =-4.57, p = .000). The results showed that the measure is significant lower than the break-even point. Thus, it can be claimed that users are not satisfied with this aspect. When we analyze the positive as well negative comments were provided. A notable observation was that users found it difficult to create a mental imagine of a city based on the scores for each feature. Several times it was mentioned that using pictures to describe the features would be appreciated. 11.2.3.1.1 Conclusion usability The seven self-reported attitude variables measure the attitudes towards the interaction thus the usability concerning this specific screen. The graph variable showed an insignificant result with the ‘one sample t-test’. A significantly not satisfying attitude was found for the city descriptions. The means of
99
the values were all tested against the 5.0 break-even point. An overview of the result can be found in Table 10. Variable Expressiveness Method Effort Objective Presentation Graph City descriptions
Mean 5.69 (SD=1.14) 5.91 (SD=1.12) 5.38 (SD=1.40) 6.71 (SD=1.25) 5.40 (SD=1.28) 5.10 (SD=1.42) 4.41 (SD=1.41)
One sample t-test with test value 5.0 t (119) =6.62, p < .001 t (119) =8.86, p < .001 t (119) =2.92, p < .005 t (119) =6.19, p < .001 t (119) =3.40, p < .005 t (119) =0.77, p > .05 t (119) =-4.57, p = .000
Table 10: Overview Usability one-sample t-test Step Reduce
Based on these measures, we still believe the majority of the users perceive the usability for this “reduce” step is close to a “satisfied” satisfaction level. This means that the users positively evaluate the usability. Therefore, the hypothesis H3: Users positively evaluate the usability in the “Reduce” step is accepted. The acceptance of H3 implies that the objective to capture users’ preferred travel destinations using the implemented GUI and interaction is acceptable and usable. More research is needed for the representation of the consensus stimulating data (Graph) and information about cities (city descriptions) in future researches. Helpful suggestions and critiques are gathered from the remarks which offer us a good insight what may have influenced the satisfaction level. Apparently, users are pleased with the ranking method and the extra information to stimulate for a consensus. 11.2.3.2 Construct: Fairness In this step of the decision process perceived fairness has two aspects. First, users are supposed to have insight why a city suits the group for travelling. A transparent and open system should stimulate the perception of having a fair process. After all, users may reveal their preferences to other travel members. The second issue concerns the decision making. We are wondering if the transparent and open system had any influence on the decision making. Two questions were formulated in paragraph 10.3.3.1.2 which were analyzed using a standard mean value calculation. Figure 28 presents the average level of satisfaction of the fairness from the users. Considering the 95% confidence interval and the value 4 as the neutral value of the scale the results show an interesting situation about the attitude towards the fairness of the “Reduce step. Below we will explain the results by using the quantitative data. We will use the qualitative data to explain the rationale behind these positions and provides additionally insight. Reveal - Having insight of the group travel preferences and the preferences of each member. In order to find out whether users are satisfied with possibility to have insight of the group travel preferences and the preferences of each member, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the insight of the preferences and ‘satisfied’ with the insight of the preferences. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 6.13 (SD=1.1) than the break-even point of 5.0, (t (119) =11.29, p < 0.01). This implies that users are ‘satisfied’ with the
100
possibility to have insight about the travel preferences of other travel members and the group preferences as a whole. Preferably 79.1% of the users have the satisfied or very satisfied attitude towards this variable. Users stated in the remarks that that it was fun to explore the preferences of other members. One user said “I never thought that Jenny would prefer culture over culinary. She only talks about food!”.
Figure 28: Measures with 95% CI regarding the fairness concept in the reduce step
Decision – Making use of the consensus stimulating information for decision making. The transparent and open system has the effect that users are aware about the preferences of each other. But the interesting part is whether users will take the information in consideration when ranking the cities. In order to that find out, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with city descriptions and ‘satisfied’ with city information. The results of the ‘one sample t-test’ showed that on average, participants do not have a significantly greater score on this measure with a mean of 3.49 (SD=2.06) than the break-even point of 5.0, (t (119) =8.01, p = .000). The measure is significant lower than the break-even point. Thus, it can be claimed that users are not satisfied with this aspect. This implies that users did not use the consensus stimulating information for their decision making. This effect could possibly be explained by the qualitative data from the users. Users mentioned very often that they find it difficult to interpret the preference information let alone using the information. This effect may have been caused by a suboptimal implementation to present the preference information. This finding is supported by the data from the usability concept. An interesting observation from the qualitative data is that users do like and use the consensus indication (the percentage value which indicates how well the city suites the group).
101
11.2.3.2.1 Conclusion fairness The two self-reported attitude variables measure the attitude towards the fairness concept concerning this specific step. An overview of the result: Variable Reveal Decision
Mean 6.19 (SD=1.1) 3.49 (SD=2.06)
One sample t-test with test value 5.0 t (119) =11.29, p < .01 t (119) =-8.01, p = .00
Table 11: Overview Fairness one-sample t-test Step Reduce
Based on these statistical measures, it cannot be clearly claimed that users perceive the fairness attitude for this “Reduce” step as satisfying due to the unsatisfying attitude towards the “decision” variable. However, when we base the result on the “Reveal” variable and the remarks, we still strongly believe the majority of the users perceive the fairness for this “Reduce” step as “satisfied”. Therefore the hypothesis H8: The “Reduce” step of Trip.Easy contributes to the perception of having a fair process is accepted. The acceptance of H7 implies that the satisfying possibility to express their preferred travel destinations to the group contributed positively towards the fairness concept. Also the possibility to have insight about the preferences of other members and the support from Trip.Easy GDSS with the consensus indication has contributed to the feeling to have a fair decision process. Though, it seems that users did not use the consensus stimulating information optimally because of the difficult interpretation of the information. 11.2.3.3 Conclusion “Reduce” step The objective of the “Reduce” step of the decision process is to let users reduce the proposed set of possible travel destinations to a shorter list. The aggregated set will result in a short list containing 5 cities which will be used in following step of the decision process. The design of the GUI and the implementation of the interactions should support the users to complete the “Reduce” step with a satisfied feeling. Measures for the usability concept indicate that the GUI actually can be used for capturing the preferred travel destinations for each user. The hypothesis H3 is therefore accepted. Measures for the fairness concept indicates that users do appreciate the consensus objective because of the possibility to have insight about others preferences. However, users did not use the consensus stimulating information for their decision making due to the complexity the information. Due to the positive remarks and the high score for the Reveal variable we still accepted the hypotheses H7. We believe that the satisfied attitude for both concepts results in a satisfied attitude towards this “Reduce” step. Though, more research is probably needed for the fairness concept. In the end we believe this result will positively contribute towards the decision process in general.
102
11.2.4 Evaluation of Step 3: Clarify – Building shared understanding Next we will evaluate the ‘Clarify’ step where users have to motivate why they want to go or not to go to one of the proposed travel destination. The third step of the decision process is based on the fundamental “Clarify” pattern of collaboration. We applied the subtype “Building shared understanding” pattern for the actual implementation. The result of the previous step is a set of 5 cities which is calculated15 as the best-fit for the group. Purpose of this step in the decision process is to create more shared understanding of the possible choices among the travel members. Users achieve shared understanding in this step if the group comes to a common understanding of each other’s motivations for the possible travel destination. Therefore, the developed GUI must provide some functionalities and options to let users achieve that goal taking the consensus idea into account. Two hypotheses were formed in order to evaluate this specific step, which should help the user to achieve the goal of this “clarify” step. The two hypotheses are: • •
H4: Users positively evaluate the usability in the “Clarify” step H9: The “Clarify” step of Trip.Easy contributes to the perception of having a fair process
The hypotheses will be tested through analyzing the qualitative and quantitative data from the usability and fairness construct. All 120 participants completed the quantitative questions; only 42 participants provided useful qualitative data Appendix 19.8. A screenshot from the GUI for this step can be found in section 7.6.4. 11.2.4.1 Construct: Usability The aggregated set of 5 best-fit travel destinations for the group is presented in the GUI. In order to build a shared understanding users have to interact with the following components: value a city by a rating component, motivate the valuation by selecting the appropriate criteria’s from the critique component and eventually provide some remarks through the text-area component. Six questions were formulated in paragraph 10.3.2.1.3 which is analyzed using a standard mean value calculation.
15
Detailed information about this aggregation algorithm can be found in the theses of Touw (2010)
103
Figure 29: Measures with 95% CI regarding the usability concept in the clarify step
Figure 29 presents the average level of satisfaction of the usability variables from the users. Considering the 95% confidence interval and the value 4 as the neutral value of the scale the results suggest that the majority of the users leaned towards a satisfied attitude about the usability of the “Clarify” step from the decision process. Below we will explain the results by using the quantitative data. We will use the qualitative data to explain the rationale behind these positions and provides additionally insight. Expressiveness - The expressive power of the rating method. In order to find out whether users are satisfied with the expressive power of the ordering method, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the expressive power and ‘satisfied’ with the expressive power. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on the expressiveness aspect with a mean of 5.64 (SD=1.17) than the break-even point of 5.0, (t (119) =6.03, p < 0.01). This implies that users are close to a ‘satisfied’ satisfaction level with the expressive power by rating each of the five cities one by one. Moreover, 65% of the users have a satisfied or very satisfied satisfaction level. Although users seem to like the expressive power of this rating method, some users were smart enough to use this power to manipulate the results. Some users mentioned the flaw of this rating system. One of the users said the following: “Because I really wanted to visit Barcelona, I rated it with the maximum of 5 stars. Although I wouldn’t mind to go to the other recommended options, I just manipulated the results by rating the other 4 cities with the minimum of 1. With this action I just hoped that the average rating will end up with a high score “. Method - Rating method that use stars to value the cities. In order to find out whether users are satisfied with the drag&drop interaction which is implemented for the rating method, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the drag&drop interaction and ‘satisfied’ with the drag&drop interaction. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure
104
with a mean of 5.79 (SD=1.19) than the break-even point of 5.0, (t (119) =7.30, p < 0.01). This implies that users perceive this rating method almost as ‘satisfying’ for valuing the cities. The qualitative data shows that users have seen this rating interaction in many other applications on the internet. Apparently, using stars to rate things is very common in practice. Effort - The effort to execute this step. In order to find out whether users are satisfied with the effort to execute the step, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the breakeven point between ‘not satisfied’ with the effort to execute the step and ‘satisfied’ with the effort to execute the step. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.76 (SD=1.28) than the break-even point of 5.0, (t (119) =6.473, p < 0.01). This implies that users are satisfied with the effort used to execute this step. Almost 70% of the users perceived the effort used to complete this step as low. The straightforward actions requested from the users and the familiar interaction style must have contributed positively towards this effort variable. Objective - The clearness of the objective of the “clarify” step. In order to find out whether users are satisfied with the clearness of the objective, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the clearness of the objective and ‘satisfied’ with the clearness of the objective. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.65 (SD=1.33) than the break-even point of 5.0, (t (119) =5.34, p < 0.01). This implies that the majority of the users have a satisfaction level of ‘satisfied. We can say that 66.7% of the users have a satisfied or very satisfied feeling. The instructions given by Trip.Easy GDSS and the familiar rating with the stars ensured the objective to create awareness among members is clear to the users. Presentation - The presentation of the components for the GUI. In order to find out whether users are satisfied with the presentation of the components of the GUI, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the presentation of the components and ‘satisfied’ with the presentation of the components. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.75 (SD=1.09) than the break-even point of 5.0, (t (119) =7.56, p < 0.01). This implies that users are close to a ‘satisfied’ satisfaction level. Three important rating components are included for each city of the ordered list; the ratings through stars, the selections of motivations through check-boxes and the comments through a text-area. Although users like the grouping of the components, the analyzed quantitative data clearly shows that the amount of choices for the motivation component is not sufficient. Many users remarked that the two options ‘too expensive’ and ‘not fun’ are not enough. Extra choices like ‘I’ve been there’ are be desirable. Awareness - The possibility to have more information about (group) preferences and cities. In order to find out whether users are satisfied with the possibility to have consensus stimulating information within this step, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the breakeven point between ‘not satisfied’ with the ability to have consensus stimulating information and ‘satisfied’ with the possibility with the consensus stimulating information. The results of the ‘one sample
105
t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.36 (SD=1.44) than the break-even point of 5.0, (t (119) =2.72, p < 0.01). This implies that have a ‘satisfied’ satisfaction level. Merely, 55.8% of the users have a ‘satisfied’ or ‘very satisfied’ attitude towards this variable. The remarks show us that users rather have more detailed city descriptions than repeated awareness information. The score based on the ranking is enough awareness for the users to pay attention to other member’s choices. 11.2.4.1.1 Conclusion usability The six self-reported attitude variables measure the attitudes towards the interaction thus the usability concerning this specific screen. The means of the values were all tested against the 5.0 break-even point. An overview of the result can be found in Table 12. Variable Expressiveness Method Effort Objective Presentation Graph City descriptions
Mean 5.69 (SD=1.14) 5.91 (SD=1.12) 5.38 (SD=1.40) 6.71 (SD=1.25) 5.40 (SD=1.28) 5.10 (SD=1.42) 4.41 (SD=1.41)
One sample t-test with test value 5.0 t (119) =6.62, p < .001 t (119) =8.86, p < .001 t (119) =2.92, p < .005 t (119) =6.19, p < .001 t (119) =3.40, p < .005 t (119) =0.77, p > .05 t (119) =-4.57, p = .00
Table 12: Overview Usability one-sample t-test Step Clarify
Based on these measures, we believe the majority of the users perceive the usability for this “clarify” step to a “satisfied” satisfaction level. This means that the users positively evaluate the usability. Therefore, the hypothesis H4: Users positively evaluate the usability in the “Clarify” step is accepted. The acceptance of H4 implies that the objective to building shared understanding using the implemented GUI and interaction is acceptable and usable. Users knew that if they want to convince other members to travel to one of the cities from the proposed list, then this is the moment to make it clear to them. The GUI obviously supports the user well in order to clarify their choices. 11.2.4.2 Construct: Fairness To achieve the feeling of participating in a fair decision process we designed the ‘clarify’ step with the following aspects taken into account: first, will users have the feeling that they can express and reveal their preferences to other members? With the underlying thought of whether other members will consider the preferences of the user. Second, because of the manipulative property of the rating method, will users perceive the rating method to value the preferred city as fair? Finally, we were wondering if the consensus stimulating information like the “best-fit” percentage has any influence in the decision making of the user. Questions were formulated in paragraph 10.3.3.1.3 which were analyzed using a standard mean value calculation. Figure 30 presents the average level of satisfaction of the fairness from the users.
106
Figure 30: Measures with 95% CI regarding the fairness concept in the clarify step
Considering the 95% confidence interval and the value 4 as the neutral value of the scale the results show an interesting situation about the attitude towards the fairness of the “clarify” step. Apparently consensus stimulating information does not have a big influence on the decision making of the users. Below we will explain the results by using the quantitative data. We will use the qualitative data to explain the rationale behind these positions and provides additionally insight. Reveal - The ability to share users’ preferred travel destination to others. In order to find out whether users are satisfied with possibility to have insight of the group travel preferences and the preferences of each member, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the insight of the preferences and ‘satisfied’ with the insight of the preferences. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.86 (SD=1.00) than the break-even point of 5.0, (t (119) =9.34, p < 0.01). This implies that users had a ‘satisfied’ feeling. By revealing the preferences, it will somehow influence the decision making of each individual. A total of 72.5% of them were satisfied or very satisfied with this function. This is supported by the qualitative data where a participant mentioned: “It makes the decision to choose a consensus based travel destination a lot easier because of this shared understanding. I can quietly motivate why I like a specific city more than the other one. Normally I would not have that chance to do so. Firstly, because of the lack of information (now I can look for more information on the internet) and secondly, because it is too cumbersome and inefficient when everyone is participating in a group discussion”. Voting - The fairness attitude towards the voting mechanism. In order to find out whether users are satisfied the fairness attitude towards the voting mechanism; we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the attitude towards the voting mechanism and ‘satisfied’ with the attitude towards the voting mechanism. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this
107
measure with a mean of 5.65 (SD=1.08) than the break-even point of 5.0, (t (119) =6.58, p < 0.01). This implies that users were ‘satisfied’ with satisfaction level towards the fairness of this voting mechanism. A remarkable observation was found when analyzing the qualitative results. Although 64.2% of the users have a satisfied or very satisfied feeling about the fairness of voting, many times it is mentioned that manipulation of the results can be performed. Many examples are given by users like “If I purposely rate 4 cities low and rate my favorite city high, I think I will have a great chance that my preferred city will end up high in the list!”. Decision - Making use of the consensus stimulating information for decision making. The transparent and open system has the effect that users are aware about the preferences of each other. But the interesting part is whether users will take the information in consideration when ranking the cities. In order to that find out, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the consensus stimulating information and ‘satisfied’ with the consensus stimulating inforation. The results of the ‘one sample t-test’ showed that on average, participants do not have a significantly greater score on this measure with a mean of 4.14 (SD=1.85) than the break-even point of 5.0, (t (119) =-5.09, p = .000). The measure is significant lower than the break-even point. Thus, it can be claimed that users are not satisfied with this aspect. This implies that users did not use the consensus stimulating information for their decision making. This can be explained by the result found in the previous “reduce” step. The additional consensus stimulating information is hard to interpret for many users. Although they find it very useful for forming an image of the member’s preferences, using that information in this current form is too much. On the other hand, the ranking positions of each city influenced the rating and valuation for most of the users. 11.2.4.2.1 Conclusion fairness The three self-reported attitude variables measure the attitude towards the fairness concept concerning this specific step. An overview of the result: Variable Reveal Voting Decision
Mean 5.86 (SD=1.00) 5.65 (SD=1.08) 4.14 (SD=1.85)
One sample t-test with test value 5.0 t (119) =9.34, p < .01 t (119) =6.58, p < .01 t (119) =-5.09, p = .00
Table 13: Overview Fairness one-sample t-test Step Clarify
Based on these statistical measures, it cannot be clearly claimed that users perceive the fairness attitude for this “Reduce” step as satisfying due to the unsatisfying attitude towards the “decision” variable. However, when we base the result on the “Reveal” and “Voting” variable and the qualitative data, we still strongly believe the majority of the users perceive the fairness for this “Clarify” step as “satisfied”. Therefore the hypothesis H9: The “Clarify” step of Trip.Easy contributes to the perception of having a fair process is accepted. The acceptance of H9 implies the objective to build shared understanding within the group is perceived as necessary and wanted. The possibility to reveal users preferences plus their motivation for the travel destination is well received. To let users make use of a voting mechanism to input their preference for a city was perceived to be fair. Though, manipulation of the results is possible. Users may strategically vote to increase the chance of their preferred travel destination. Finally, users base their decisions in this step not only on their own preferences but also on the
108
consensus stimulation information. The ranking of the cities from the list given by the best-fit percentage have significant influence on the decision making. 11.2.4.3 Conclusion “clarify” step The objective of the “clarify” step of the decision process is to create awareness among the members of each others’ preferences about the proposed set of cities. The GUI and the possible interactions are designed in a way that members can openly share their opinions about the remaining cities. The resulting measures from the usability concepts suggest that users were indeed satisfied with the usage. They were pleased with the interaction within the GUI that supports them to share their motivations for the preferred travel destination. Besides those users can build a shared understating, a satisfied fair feeling should also exist among the travel party. This is clearly found in the results from the fairness measures. Not put under pressure by factors like time or lack of information, users may easily take their time to motivate why they like or dislike the cities for travel. Both hypothesis H3 and H9 were accepted. Consequently, we believe strongly that users perceive this ‘clarify’ step as a satisfying step within the whole decision process. Which again will for sure contributes to a positive acceptation towards the decision process in general.
11.2.5 Evaluation of Step 4: Evaluate – Communication of Preference Next we will evaluate the ‘Evaluate’ step where users use the result of this ‘clarify’ step to start a discussion and negotiation session. The final step of the group decision process is based on the “Evaluate” pattern. We choose the “communication of preference” sub-pattern for the actual implementation. The objective of this step is to allow users to discuss with each other. Using the aggregated ratings and motivations of the previous step users may explore the reasons for the agreement and disagreements while working towards the ultimate decision. During this step the only task of Trip.Easy GDSS is to display the needed information in a clear way. Two hypotheses were formed in order to evaluate this specific step, which should help the user to achieve the goal of this final “Evaluate” step. The two hypotheses are: • •
H5: Users positively evaluate the usability in the “Evaluate” step H10: The “Evaluate” step of Trip.Easy contributes to the perception of having a fair process
The hypotheses will be tested through analyzing the qualitative and quantitative data from the usability and fairness construct. All 120 participants completed the quantitative questions; only 35 participants provided useful qualitative data (Appendix 19.9). A screenshot from the GUI for this step can be found in section 7.6.5.
109
11.2.5.1 Construct: Usability The top 4 highest rated travel destinations for the group are presented in the GUI. Just one action is expected from the users of the travel session, namely passing the final choice to Trip.Easy GDSS. During the discussion no interaction is required from the user with the GUI. But the objective of this screen is to support discussion and stimulate consensus by providing relevant information in a structured way. Nine questions were formulated in paragraph 10.3.2.1.4 which is analyzed using a standard mean value calculation.
Figure 31: Measures with 95% CI regarding the usability concept in the evaluate step
Figure 31 shows the average level of satisfaction of the usability variables from the users. Considering the 95% confidence interval and the value 4 as the neutral value of the scale the results suggest that the majority of the users are between the ‘somewhat satisfied’ and the ‘satisfied’ attitude about the usability of the “Evaluate” step from the decision process. Below we will explain the results by using the quantitative data. We will use the qualitative data to explain the rationale behind these positions and to provide additionally insight. Expressiveness - The expressive power of the discussion method. In order to find out whether users are satisfied with the expressive power of the ordering method, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the expressive power and ‘satisfied’ with the expressive power. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on the expressiveness aspect with a mean of 5.29 (SD=1.28) than the break-even point of 5.0, (t (119) =2.49, p < 0.05). This implies that users were ‘satisfied’ with the expressive possibilities during the discussion. Only 49.1% feels satisfied or very satisfied. We can explain this measure by the fact that users mentioned in their remarks that the possibility to express themselves is heavily dependent on the group. If within the group a dominant speaker exists, the chance that everyone gets a turn is much lower. Also some participants admitted
110
that from nature they do not talk much and good formulation of what they actually want is for them difficult. It is therefore very hard to express themselves to other group members. Method - The discussion method that is chosen to let users come to the final choice. In order to find out whether users are satisfied with the discussion method to let users come to the final choice, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the discussion method and ‘satisfied’ with the discussion method. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.30 (SD=1.40) than the break-even point of 5.0, (t (119) =2.34, p < 0.05). This implies users have a ‘satisfied’ satisfaction level. Moreover 56.6% of them were satisfied or very satisfied with the possibility to discuss. It is mentioned that having a discussion as a part of the decision process is desirable. But in this current state of Trip.Easy GDSS it is still necessary to have all members of the travel party to be physically present in some space. A desirable improvement for using the discussion method is to have a chat or video conference like feature. Consequently the space dependent problem can be eliminated which will result in a more easy way to arrange a discussion moment. Effort - The effort to execute the step. In order to find out whether users are satisfied with the effort to execute the step, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the breakeven point between ‘not satisfied’ with the effort to execute the step and ‘satisfied’ with the effort to execute the step. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.42 (SD=1.46) than the break-even point of 5.0, (t (119) =3.13, p < 0.01). This implies that users are ‘satisfied’ with the needed effort to complete this ‘Evaluate’ step. Only 57.5% users think that not much effort was needed. But from the qualitative data we actually see that some users perceive defending their choices during the discussion as laborious. Convincing members requires having some skillful discussion abilities, but of course not everyone has that. Consequently the effort automatically increases for users who do not have that skill. But at the same time users also mentioned that the previous steps in the decision process took care of their preferences. Making a final choice between one of the four cities which is already recommended as the best-fit choice does not take too much effort. This implies users have trust in the Trip.Easy GDSS decision process. Objective - The clearness of objective of the “Evaluate” step. In order to find out whether users are satisfied with the clearness of the objective, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the clearness of the objective and ‘satisfied’ with the clearness of the objective. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.72 (SD=1.02) than the break-even point of 5.0, (t (119) =7.68, p < 0.01). This implies that users perceived a ‘satisfied’ satisfaction level. Moreover, 66.7% of the users were found to be satisfied or very satisfied with the clearness of the objective presented in this screen. Apparently the instructions given by Trip.Easy GDSS were clear enough to let the user know what to do in order to successfully complete this step. Presentation - The presentation of the components for the GUI. In order to find out whether users are satisfied with the presentation of the components of the GUI, we performed a ‘one sample t-test’. We
111
used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the presentation of the components and ‘satisfied’ with the presentation of the components. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.67 (SD=0.97) than the break-even point of 5.0, (t (119) =7.51, p < 0.01). This implies that users were ‘satisfied’ with the presentation of the component within the GUI. In total 60.9% of the users have a satisfied or very satisfied feeling about it. Although not much interaction with the GUI was required for this step, the qualitative data shows that the list with the top 4 of highest ratings for the group is very handy in this phase. From the remarks we saw “Using this list to discuss is a lot easier and goal orientated”, “A top 4 ranking and their associated information shows clearly why each choice suits the group. This could be used very effectively during the discussion. ” Information - The completeness of the information in order to have a good discussion. In order to find out whether users are satisfied with the completeness of the information, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the presentation of the components and ‘satisfied’ with the presentation of the components. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.02 (SD=1.43) than the break-even point of 5.0, (t (119) =0.13, p > 0.05). The results showed that the measure is not significant. Thus, it cannot be claimed based on the results whether users are satisfied or not. However, the frequency chart (appendix X) shows that 48.4% of the users were satisfied or very satisfied with it. More insight why this relatively low value was measured was found in the remarks from the users. Many times users mentioned that the information presented in the screen repeated consensus stimulating data and an overview of motivations from the previous step. Information with added value in this step probably would be some concrete current city and travel information. Even weather information has been mentioned. Apparently the participants were looking for more arguments or stimulations why one should choose for one of the four cities. Awareness - The ability to have insight in the (group) preferences. In order to find out whether users are satisfied with the possibility to have consensus stimulating information within this step, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the ability to have insight in the consensus stimulating information and ‘satisfied’ with the ability to have insight in the consensus stimulating information. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.39 (SD=1.46) than the break-even point of 5.0, (t (119) =3.03, p < 0.01). This implies that users were ‘satisfied’ with the possibility to have insight in the (group) preferences. 58.4% of the users are satisfied or very satisfied with this information. One user said “I can use this information to explore the chance for an agreement or disagreement”. New set – A new set of alternative recommendations by Trip.Easy GDSS. In order to find out whether users are satisfied with a new set of alternative recommendations during the current stage of the decision process, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the breakeven point between ‘not satisfied’ with the new recommendations and ‘satisfied’ with the new recommendations. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.02 (SD=1.69) than the break-even point of
112
5.0, (t (119) =0.108, p > 0.05). The results showed that the measure is not significant. Thus, it cannot be claimed based on the results whether users are satisfied or not. However, the frequency chart (appendix X) shows that 52.5% of the users think it is useful to have new city recommendations in the current stage of the decision process. From the remarks we could see that some users found this confusing. Other suggested that if the new recommendations are based on the last aggregated set of preference information, than it would be useful. We even had a comment where the user thought it was some kind of advertisement. Discussion support - Using the information presented in the GUI during the discussion. In order to find out whether users were using the information presented in the GUI during the discussion, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the discussion supporting information and ‘satisfied’ with the discussion supporting information. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.69 (SD=1.14) than the break-even point of 5.0, (t (119) =1.13, p > 0.05). The results showed that the measure is not significant. Thus, it cannot be claimed based on the results whether users are satisfied or not. However, the frequency chart (appendix X) shows that 45% of the users are satisfied or very satisfied with the GUI which should support them in their discussion. As mentioned and measured before, we could see that users are willing to use the presented information in a discussion. Though, the current state of Trip.Easy GDSS lacks useful up-to-date city information. This is definitely a point for improvement in a future release. 11.2.5.1.1 Conclusion usability The nine self-reported attitude variables measure the attitudes towards the interaction thus the usability concerning this specific screen. The means of the values were all tested against the 5.0 breakeven point. An overview of the result can be found in Table 14. Variable Expressiveness Method Effort Objective Presentation Information Awareness New set Discussion support
Mean 5.29 (SD=1.28) 5.30 (SD=1.40) 5.42 (SD=1.46) 5.72 (SD=1.25) 5.67 (SD=0.97) 5.02 (SD=1.43) 5.39 (SD=1.46) 5.02 (SD=1.69) 5.69 (SD=1.14)
One sample t-test with test value 5.0 t (119) =2.49, p < .05 t (119) =2.35, p < .05 t (119) =3.13, p < .05 t (119) =7.68, p < .01 t (119) =7.51, p < .01 t (119) =0.13, p > .05 t (119) =3.03, p < .01 t (119) = 0.18, p > .05 t (119) = 1.13, p > .05
Table 14: Overview Usability one-sample t-test Step Evaluate
Though, three aspects e.g. information, new set and discussion support showed not significance measures. Based on the overall measures and the qualitative data, we believe the majority of the users perceive the usability for this “usability” step to a “satisfied” satisfaction level. This means that the users positively evaluate the usability. Therefore, the hypothesis H5: Users positively evaluate the usability in the “Evaluate” step is accepted. The acceptance of H5 implies that the setup of the interface is sufficient to support users to have a good constructive discussion. Not much interaction with the GUI
113
was needed to complete the discussion and select a final choice. Discussions went well and consensus has been reached by most of the travel sessions. Consequently, users were satisfied with the usage of the GUI that guided them through the final step of the decision process. Useful remarks from the qualitative data tell us the following: (1) more detailed, up-to-date and concrete information about the city is desirable, (2) starting a discussion taking the space independent property (i.e. video conferencing) into account is also highly desirable. Several times the chat or VOIP possibility is mentioned. We could use this information to improve this step in the future. 11.2.5.2 Construct: Fairness Since consensus is the group’s objective, measuring fairness is important. During this step of the decision process users are supposed to discuss in order make a final decision. To stimulate a positive attitude towards the fairness, the design of the GUI makes it possible for the users to have insight in each other’s preferences and preferred choices. We evaluated the fairness concept for this step by measuring the ‘reveal’ variable. Questions were formulated in paragraph 10.3.3.1.4 which were analyzed using a standard mean value calculation. Figure 32 presents the average level of satisfaction of the fairness from the users.
Figure 32: Measures with 95% CI regarding the fairness concept in the evaluate step
Reveal - The ability to share users’ preferred travel destination to others. In order to find out whether users are satisfied with possibility to have insight of the group travel preferences and the preferences of each member, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the insight of the preferences and ‘satisfied’ with the insight of the preferences. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.71 (SD=1.07) than the break-even point of 5.0, (t (119) =7.23, p < 0.01). This implies that users perceived a ‘satisfied’ feeling for this variable. A total of 65% of them were satisfied or very satisfied with this function. Working towards the final decision during a discussion was found to be easier when the group is aware about each other’s
114
preferences. One of the participants commented that because of this possibility the quality of the discussion has been significantly improved. Motivations could be defended and reasons for agreements and disagreements could be explored by using the aggregated data from the previous step. 11.2.5.2.1 Conclusion fairness The self-reported attitude variable measures the attitude towards the fairness concept concerning this specific step. Significant result is showed using the ‘one sample t-test’ applied on the ‘reveal’ variable. An overview of the result: Variable Reveal
Mean 5.71 (SD=1.07)
One sample t-test with test value 5.0 t (119) =7.23, p < .01
Table 15: Overview Fairness one-sample t-test Step Evaluate
Based on the measure, we believe the majority of the users perceive the fairness for this “evaluate” step is close to a “satisfied” satisfaction level. This means that the users positively evaluate the fairness. Therefore the hypothesis H11: The “Evaluate” step of Trip.Easy contributes to the perception of having a fair process is accepted. A typical character of a discussion is that members do not have a clear overview of each other’s preferences. In most cases it will result in suboptimal choices. Previous steps of the Trip.Easy GDSS decision process gathered the preferences of each member and presented this in a well structured way in the GUI. Each member may use this information during the discussion. Since participants perceive this step as fair we may therefore believe that this “Evaluate” step at least has some contribution to the fairness of the decision process in general. 11.2.5.3 Conclusion “Evaluate” step The objective of this final step of the decision process is to establish consensus about the final choice through a discussion. The design of the GUI should support the group to having a good discussion. Apparently, having a discussion in person is still acceptable. But in this digital era, chat and VOIP solutions are highly preferred. The usability results show that users are using the information like consensus info and aggregated motivations from the previous step during their discussion. The screen is sufficient to lookup information and to support users to formulate arguments to convince other members. Unfortunately due to the lack of detailed, dated and concrete city information we believe that the satisfaction level for the usability construct is rated lower than it should be. The other construct which will influence the satisfaction level for this step is the fairness. Users are having a positive attitude towards the fairness of this step. The possibility to have insight of each other’s preferences and thus reveal the users preferences to other members is perceived as a fair property in a decision process. As result both hypothesis H4 and H10 were accepted. We strongly believe that users perceived this ‘Evaluate’ step as a satisfying step within the whole decision process. Again, we can say that the satisfaction of the Trip.Easy GDSS in general, is at least partly based on the attitude towards this ‘Evaluate’ step.
115
11.2.6 Evaluation of Trip.Easy GDSS decision process in general Next, we will evaluate a final component of this experiment, namely, the attitude towards Trip.Easy GDSS in general. The promising results discovered in the previous sections tell that users were satisfied with the steps of the decision process from Trip.Easy GDSS. The last step of this study is to evaluate the Trip.Easy GDSS decision process in general. Based on the current results we expect that users are also satisfied with the decision process in general. The gain from this evaluation is that we will have more detailed insight why users are satisfied with the decision process. Two hypotheses were formed in order to evaluate this. The two hypotheses are: • •
H6: Users positively evaluate the overall usability of the Trip.Easy GDSS H11: Users perceive the overall process of the Trip.Easy GDSS to be fair
The hypotheses will be tested through analyzing the quantitative data from the usability and fairness construct. 11.2.6.1 Construct: Usability In order to support a group of users who wants to travel together to choose a best-fit travel destination, the Trip.Easy GDSS has been developed. A group decision process is designed and implemented as part of the Trip.Easy GDSS. By interacting with the GUI users are guided through the whole decision making process which should lead them to establish a consensus based travel destination. Interesting for us is to know how users evaluate the decision making process by its usability. Therefore we evaluated some important aspects that are relevant for further development. Six questions were formulated in paragraph 10.3.2.1.5 which were analyzed using a standard mean value calculation.
Figure 33: Measures with 95% CI regarding the usability concept of Trip.Easy in General
116
Figure 33 presents the average level of satisfaction of the usability variables from the users. Considering the 95% confidence interval and the value 4 as the neutral value of the scale the results suggest that the majority of the users leaned towards a satisfied attitude about the usability of the whole decision process. Below we will explain the results by using the quantitative data. Effort - The effort used to complete the whole decision process. In order to find out whether users are satisfied with the effort to execute the step, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the effort to execute the step and ‘satisfied’ with the effort to execute the step. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.29 (SD=1.28) than the break-even point of 5.0, (t (119) =2.49, p < 0.05). This implies that users are satisfied with the effort that is used to complete the goals during the decision-making process. This result is in line with the previous measurement where the effort for each step is measured. Taking the measured effort value together, we believe that users perceived the effort that is used to operate on Trip.Easy GDSS as satisfying due to the well developed group decision support process. Objective - The clearness of the Trip.Easy GDSS objective. In order to find out whether users are satisfied with the clearness of the general Trip.Easy GDSS, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the clearness of the objective and ‘satisfied’ with the clearness of the objective. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.39 (SD=1.43) than the break-even point of 5.0, (t (119) =2.99, p < 0.01). This implies that users are more than ‘somewhat satisfied’ with the clearness of the instructions and goal of Trip.Easy GDSS. Almost 60% of the users were satisfied or very satisfied with the clearness of the objective. Also this measure is in line with the previous measurements in the previous steps. When putting these results together we see that subtle changes in the GUI will improve the attitude towards this variable much more. Most of the time instructions were not clear enough or interaction possibilities like dragging and dropping with some elements in the GUI are not directly recognized. Easy - Users perceived the usage of Trip.Easy GDSS as easy. In order to find out whether users perceived the usage of Trip.Easy GDSS as easy, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the easy usage of Trip.Easy GDSS and ‘satisfied’ with the easy usage Trip.Easy GDSS. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.73 (SD=1.14) than the break-even point of 5.0, (t (119) =7.07, p < 0.01). This implies that users were having a satisfaction level of nearly ‘satisfying’. Moreover, 68.3% of the users perceived the easy usage as ‘satisfying’ or ‘very satisfying’. Selection of travel destination is a complex problem itself. Certainly the selection of a consensus based travel destination. Therefore the usage for our Trip.Easy GDSS group decision process should be easy. Fortunately participants mentioned more than often in different steps of the decision making process that they found the usage to complete the steps easy enough. Handy - Trip.Easy GDSS should be perceived as a handy tool to use for group decision making for travel purposes. In order to find out whether users perceive the Trip.Easy GDSS as a handy tool for group
117
decision making for travel purposes, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the handiness of Trip.Easy GDSS and ‘satisfied’ with handiness of the Trip.Easy GDSS. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.79 (SD=0.89) than the break-even point of 5.0, (t (119) =9.66, p < 0.01). This implies that users have a satisfaction level of nearly “satisfied”. The results even show that 63.3% of the users were satisfied or even very satisfied. This implies that users think Trip.Easy GDSS is actual useful for establishing a consensus based travel destination. Apparently a good balance is created between the effort needed and the easiness to execute each step of the decision process. Efficiency - Efficiency concerned the resources used to achieve the goal in relation to its accuracy and completeness. In order to find out whether users are satisfied with the efficiency, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the efficiency of Trip.Easy GDSS and ‘satisfied’ with the efficiency of Trip.Easy GDSS. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.94 (SD=1.08) than the break-even point of 5.0, (t (119) =9.56, p < 0.01). This implies that users are ‘satisfied’ with the efficiency of Trip.Easy GDSS. In total 58.4% of the users are satisfied or very satisfied with the efficiency. In this study we measured the time as a resource. Apparently users perceive using Trip.Easy GDSS is able to speed up the group decision process. Effectiveness - Effectiveness concerned the accuracy and completeness with which the participants achieved their goals. In order to find out whether users are satisfied with the effectiveness, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the effectiveness of Trip.Easy GDSS and ‘satisfied’ with the effectiveness of Trip.Easy GDSS. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.76 (SD=0.98) than the break-even point of 5.0, (t (119) =8.48, p < 0.01). This implies that users are more than ‘somewhat satisfied’ with the effectiveness. 63.4% of the users are satisfied or very satisfied with the effectiveness of Trip.Easy GDSS. The operations carried out by users to achieve consensus are perceived as effective. The analyzed data from the previous steps showed us that users are indeed satisfied with the way each step of the decision process is completed.
118
11.2.6.1.1 Conclusion usability The six self-reported attitude variables measure the attitudes towards the interaction thus the usability concerning this specific screen. The means of the values were all tested against the 5.0 break-even point. An overview of the result can be found in Table 16. Variable Effort Objective Easy Handy Efficiency Effectiveness
Mean 5.29 (SD=1.28) 5.39 (SD=1.43) 5.73 (SD=1.13) 5.79 (SD=0.89) 5.94 (SD=1.07) 5.76 (SD=0.97)
One sample t-test with test value 5.0 t (119) =2.49, p < .05 t (119) =2.99, p < .01 t (119) =7.74, p < .01 t (119) =7.92, p < .01 t (119) =9.55, p < .01 t (119) =8.48, p < .01
Table 16: Overview Usability one-sample t-test Trip.Easy in general
Based on these measures, we believe the majority of the users perceive the usability for Trip.Easy GDSS in general as a “satisfied” satisfaction level. This means that the users positively evaluate the usability. Therefore the hypothesis H6: Users positively evaluate the overall usability of the Trip.Easy GDSS can be accepted. The three main components which define the usability: satisfaction, efficiency and effectiveness clearly show that Trip.Easy GDSS is easy to use for a group of users for selecting a consensus based travel destination. Furthermore, the rest of the qualitative data showed why users are satisfied with the usability. Apparently the effort, easiness and handiness of the usage created a positive attitude towards the user. 11.2.6.2 Construct: Fairness The previous measures described and showed that the users evaluate the usability of Trip.Easy GDSS as positive. Although the process facilitates a group of users in collaborating and selecting a final travel, the number of participants that commit to the result needs to be high. Or at least the number of participants that are satisfied with the result should be high. The fairness of the collaboration plays an important role. When the group decision process is not fair the chance to accept the Trip.Easy GDSS is much lower. In order to evaluate how users perceive the fairness towards Trip.Easy GDSS two questions were formulated in paragraph 10.3.3.1.4 They are analyzed using a standard mean value calculation. Figure 34 presents the average level of satisfaction of the perceived fairness from the users.
119
Figure 34: Measures with 95% CI regarding the usability concept of Trip.Easy in general
Considering the 95% confidence interval and the value 4 as the neutral value of the scale the results show an interesting situation about the attitude towards the fairness of the Trip.Easy GDSS in general. Below we will explain the results by using the quantitative data. Fair process - In order to find out whether users are satisfied with fairness of the process in general, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the fairness of the overall process and ‘satisfied’ with the fairness of the overall process. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.62(SD=1.39) than the break-even point of 5.0, (t (119) =4.92, p < 0.01). This implies that users perceive the fairness of the collaborative decision process almost as ‘satisfying’. A total of 62.5% of the users indicated that their satisfaction level is satisfied or very satisfied. We have expected this because previous results showed that the measured attitude towards the fairness for each step in the decision process is satisfying. Factors like revealing users preferences to other members and expressing users preferences to Trip.Easy GDSS, have positively contributed to a fair feeling among the users. Influence - Having influence on the group decision process should contribute to a fair feeling. In order to find out whether users are satisfied with the influence on the group decision process, we performed a ‘one sample t-test’. We used a test-value of 5.0, which is the break-even point between ‘not satisfied’ with the influence and ‘satisfied’ with influence. The results of the ‘one sample t-test’ showed that on average, participants have a significantly greater score on this measure with a mean of 5.65 (SD=1.12) than the break-even point of 5.0, (t (119) =6.35, p < 0.01). This implies that users were near to a ‘satisfied’ satisfaction level. 62.5% of the users are having a satisfied or very satisfied feeling about their influences on the decision process. This can be explained because Trip.Easy GDSS has taken the consensus goal into account. When users walk through the decision process they could e.g. express,
120
reject, comment on and be aware about the (group) preferences. Apparently users perceive that they have some influence on the decision process when interacting with Trip.Easy GDSS. 11.2.6.2.1 Conclusion fairness The self-reported attitude variable measures the attitude towards the fairness concept concerning this specific step. Significant result is showed using the ‘one sample t-test’ applied on the ‘reveal’ variable. An overview of the result: Variable Fair process Influence
Mean 5.62 (SD=1.39) 5.65 (SD=1.12)
One sample t-test with test value 5.0 t (119) =4.92, p < .01 t (119) =6.35, p < .01
Table 17: Overview Fairness one-sample t-test Trip.Easy in general
Based on the measure, we believe the majority of the users perceive the fairness for the Trip.Easy GDSS in general as “satisfied”. This means that the users positively evaluate the fairness. Therefore we strongly believe that that is enough indication to accept the hypothesis: H11: Users perceives the overall meeting process of the Trip.Easy GDSS to be fair. The acceptance of H11 implies that users perceived the decision process for selecting a consensus based travel destination as fair. For instance the possibility to influence the decision making by interacting with Trip.Easy GDSS during the process has contributed to this positive evaluation of the fairness concept. 11.2.6.3 Conclusion Trip.Easy GDSS in general We studied a number of aspects of Trip.Easy GDSS where users are participating in the group decision process. First, the usability concept has been evaluated. The ease-of-use of the implemented process in general was perceived as satisfying. It shows that Trip.Easy GDSS is capable to let users work together and lead them to a satisfying solution. This positive attitude was expected because of the positive results found in each step of the evaluation. As a result, the hypothesis H6 was accepted. The other aspect which was evaluated is the fairness concept. Here we see that users have a satisfied feeling about the fairness of Trip.Easy GDSS in general. The results showed that users have the feeling that their inputs are considered and having influence over the decisions during the decision process. Consequently, we accepted the hypothesis H11. Altogether because of the positive observations from the usability and fairness concept of Trip.Easy GDSS in general, we may conclude that users like and are satisfied with the Trip.Easy GDSS in general. This implies that the design of Trip.Easy GDSS is perceived by the participants of the experiment as a useful solution for the group travel planning problem. The well-structured process is able to facilitate a group of users converging towards a consensus based travel destination taking into account the usability and the fairness concept.
121
12 Discussion and Conclusion of the Evaluation This master’s thesis reports on research to build a web-based group decision support system that facilitates internet users with a well-structured group decision process to reach a group consensus on a trip. This has been accomplished by an integration and implementation of ideas and results from Collaborative Engineering, Artificial Intelligence and Human-Computer Interaction. We presented a prototype of a group decision support system named Trip.Easy that supports a group of users in planning a short break city trip. This is a typical general group decision problem with known issues like: • •
•
people overlook alternatives (trip destinations in this case) that would fit best with what the group wants most; people are not able to communicate effectively to find an satisfying outcome for everyone, especially when meeting(s) needs to be organized between participants who reside at different locations, during the decision process it is hard to fairly involve every group member in establishing a consensus.
During the first part of this study, Trip.Easy GDSS has been designed and developed for the travel domain to provide users a tool to overcome such group decision problems when organizing and planning a group trip. The goal of this evaluative part of the study was to determine whether Trip.Easy GDSS is capable of supporting users satisfactorily with making a consensus based decision about their city trip. The focus of this thesis research lies on the meeting process part of the Trip.Easy GDSS. Therefore, an experiment was conducted in order to evaluate the design of the decision process. Consequently, the main research question has been: “How do Trip.Easy GDSS users evaluate the decision process for establishing a consensus?” In order to answer the research question, three principal aspects (1) satisfaction of the meeting process, (2) usability and (3) fairness, were evaluated. The evaluation was divided into two parts. The result in the first part of the evaluation shows (in abstract form) whether users feel satisfied with the decision process. Hypothesis H1 based on Reinig’s meeting satisfaction model was used to answer the first part of the evaluation. The second part of the evaluation was an explorative evaluation which shows why users are satisfied with the decision process. Each step of the decision process was evaluated on two aspects: usability (hypotheses H2 – H6) and fairness (hypotheses H7 – H11). The quantitative data were analyzed using the standard mean value calculation and the one sample t-test. The qualitative data were used to support and explain the quantitative findings. Arranging the findings together we have a clear overview of insights which will support us in determining the main research question.
12.1 Part 1 of the evaluation Hypotheses H1 stated that “Users experience the decision process implemented in Trip.Easy GDSS as satisfying”. The results from the Satisfaction Process (SP) construct in section 11.1.1 showed a very promising result. The result suggests that the majority of the users have a highly positive affection 122
towards our group decision support system for the urban tourism domain. It implies that the combinations of the fundamental patterns of collaboration are able to facilitate a structured decision process for reaching a collaborative decision. According to Reinig’s model of meeting satisfaction, a ‘satisfied’ satisfaction level indicates that users perceive the decision process as effective, efficient and fair for decision-making.
12.2 Part 2 of the evaluation The positive PGA from section 11.2.1 implies that the effort which is used in order to attain their goal is worthwhile for the users. Apparently, users perceive the benefit of the goal is exceeding the cost that is incurred by fulfilling the goal. This indicates that users have a positive attitude towards the interaction with the decision process of Trip.Easy GDSS to establishing a consensus based travel destination. Not only were we interested in this abstract evaluation, but we also wanted to evaluate each step of the process to gain more insight. Therefore the obtained qualitative and quantitative data from the usability and fairness construct were analyzed for each step of the meeting process.
12.2.1 Usability The hypotheses H2 – H6 were examined to verify whether users positively evaluate the usability in the “Generate”, “Reduce”, “Clarify”, “Evaluate” or “Trip.Easy in general”. The results for each step in the decision process shows that users actual perceived the usability of Trip.Easy GDSS as satisfying. The measures for this usability concept are summarized below: Generate - the objective to capture users’ travel preferences using the implemented GUI and interaction is acceptable and usable. Five aspects were measured which reflect how users evaluate the usability of this step. Analyzing the measures lead to the acceptance of hypothesis H2. Helpful suggestions and critiques are gathered from the remarks which offer us a good insight what may have influenced the satisfaction level. Apparently the “drag and drop” interaction for ordering the travel features is received well to temper a positive attitude toward this specific step. Possibly, this positive attitude in turn should have a positive influence on the overall decision process. Reduce - the objective to capture users’ preferred travel destinations by using a ranking mechanism and the possibility to create awareness by showing them consensus stimulating information was perceived close to a “satisfied” satisfaction level. The seven self-reported attitude towards the usability concept implies that the developed GUI and interaction is easy to use. Analyzing the measures lead to the acceptance of hypothesis H3. Due to the familiarity with the “drag and drop” interaction, no cold start problem was determined. Unfortunately users mentioned that the list to order was too long. Instead of ranking a list of 25 best-fit cities for the group, they suggested ranking the first 5 preferred cities and eventually ranking the 5 least preferred cities. The graph which should show the consensus stimulating information was fun to read, but due to the complexity and possible the unclarity it was also hard to read for most users. Clarify - the objective to build a shared understanding among the members was perceived as satisfying. Six aspects were measured which reflect how users evaluate the usability of this step. Analyzing the measures lead to the acceptance of hypothesis H4. Users knew that if they want to convince other
123
members to travel to one of the cities from the proposed list, then this is the moment to make it clear to them. The GUI supports the user well in order to clarify their choices. Though, the qualitative data shows that, more options were requested by the users that support them to motivate why some city is rated low e.g. “I’ve already been there”, “The weather is too cold”, etc. Evaluate - the objective to reach group agreement and pick the final travel destination due to a discussion was perceived as a “satisfied” satisfaction level. Nine aspects were measured which reflect how users evaluate the usability of this step. Analyzing the measures lead to the acceptance of hypothesis H5. Not much interaction with the GUI was needed to complete the discussion. The structured overview of the aggregated information from the “Clarify” step provided the users sufficient information in order to having a good discussion. Though, two useful observations from the qualitative data tell us the following: (1) more detailed, up-to-date and concrete information about the city is desirable, (2) starting a discussion taking the space independent property (i.e. video conferencing) into account is also highly desirable. Several times the instant messaging or VOIP possibility was mentioned. Trip.Easy usability in general - Six aspects were measured which reflect how users evaluate the usability of Trip.Easy GDSS in general. Analyzing the measures lead to the acceptance of hypothesis H6. The three measured main components which define usability: satisfaction, efficiency and effectiveness clearly show that Trip.Easy GDSS is easy to use for a group of users to selecting a consensus based travel destination. Furthermore, the qualitative data showed why users are satisfied with the usability. Apparently the effort and the ease-of-use creates a positive attitude towards the user.
12.2.2 Fairness The hypotheses H7 – H11 were examined to verify whether the chosen collaboration patterns (“Generate”, “Reduce”, “Clarify”, “Evaluate”) and “Trip.Easy in general” contributes to the perception of having a fair process. We can summarize that the majority of the users perceived the fairness of the group decision process as fair. The measures for this usability concept are summarized below: Generate - the objective was to capture users’ travel preferences using a feature-based preference eliciting approach. One aspect was measured which reflect how users evaluate the fairness of this step. Analyzing the measures lead to the acceptance of hypothesis H7. The results show that users could identify with the given features and order them from highly important to not important to constructing their preferences. This implies that users perceived this property as fair. Apparently, users had the feeling that they could express their travel desires which resulted in the feeling that their input was taken into account. As a result, the positive attitude created in this step will at least partly contribute to the perception of having a fair process. Reduce - the objective was to capture users’ preferred travel destinations by using a ranking mechanism and the possibility to create awareness by showing consensus stimulating information. Two aspects were measured which reflect how users evaluate the fairness of this step. Analyzing the measures lead to the acceptance of hypothesis H8. The satisfying possibility to express their preferred travel destinations to the group by ranking the cities contributed to the fairness concept. Also the possibility to have insight about the preferences of other members and the support from Trip.Easy GDSS with the
124
consensus indication has contributed to the feeling of participating in a fair decision process. Although it seems that users did not use the consensus stimulating information optimally because of difficulties with interpretation of the information, we still believe that this step of the decision process has contributed to the perception of having a fair process. Clarify - the objective was to create shared understanding among the members. Three aspects were measured which reflect how users evaluate the fairness of this step. Analyzing the measures lead to the acceptance of hypothesis H9. The possibility to reveal users preferences plus their motivation for the travel destination is well received. The possibility to reveal users preferences plus their motivation for the travel destination is well received. To let users make use of a voting mechanism to input their preference for a city was perceived to be fair. Though, manipulation of the results is possible. Users may strategically vote to increase the chance of their preferred travel destination. Finally, users base their decisions in this step not only on their own preferences but also on the consensus stimulation information. The ranking of the cities from the list given by the best-fit percentage have significant influence on the decision making. Evaluate - the objective was to reach group agreement and pick the final travel destination based upon discussion. Two aspects were measured which reflect how users evaluate the fairness of this step. Analyzing the measures lead to the acceptance of hypothesis H10. Though, on first sight the discussion method can be seen as a not totally fair method, however the results show the opposite. We believe that the moment to introduce the discussion method is perfectly timed in the group decision process. Trip.Easy GDSS has taken the travel preferences for each member into account before recommending the possible travel destinations. The previous steps taken also supported each individual to have influence on the outcome. In this step of the decision making, it was only expected from the participants to discuss about the final aggregated best-fit to the group list of possible travel destinations. In order to support the users more effectively during the discussion, Trip.Easy GDSS has gathered the preferences of each member and presented this in a well structured way into the GUI. Each member may use this information during the discussion. As a result, users perceive the evaluate step as fair. Trip.Easy fairness in general - while measurements show that users can easily operate the Trip.Easy GDSS, it also shows that users perceived the decision process for selecting a consensus based travel destination as fair. Two aspects were measured which reflect how users evaluate the fairness of the Trip.Easy GDSS in general. Analyzing the measures lead to the acceptance of hypothesis H11. The possibility to influence the decision making by interacting with Trip.Easy GDSS during the process has contributed to this positive evaluation of the fairness concept. Therefore we strongly believe that there is enough indication to consider that the group decision support process of Trip.Easy GDSS is fair.
125
12.3 Final conclusion In conclusion, the development and evaluation of Trip.Easy GDSS can be considered as a success. The promising results can clearly support us to answer the main research question: “How do Trip.Easy GDSS users evaluate the meeting process for establishing a consensus?” Apparently, users evaluate the decision process for establishing a consensus as satisfying, easy to use and fair. This implies that the Trip.Easy GDSS is capable to facilitate users in a group decision process which converges on an outcome that satisfies all group members. In summary, users evaluate the group decision support process as satisfactory because it facilitates the following: • • • • •
creating a group (travel session) by inviting friends, family, colleagues, etc; eliciting individual travel preferences to ensure equal and fair involvement of group members; providing consensus stimulating information to create awareness and build understanding; supporting users in building motivations and preferences for travel destinations by providing relevant information; and converging on a group consensus by structured, decision making.
126
Future work and Discussion
127
13 Future Work and Discussion Based on the design and the evaluation of the Trip.Easy GDSS, some limitations and possible future works are discussed.
13.1 Decision-making process The current decision process is based on the preferences of the features that are submitted by the users. Using the obtained preference models, recommendations can be calculated and the process should converge to a best-fit outcome for the group. A limitation will occur when the preferences changes of the users due to i.e. bad weather, volcanic ash problem, expensive dollar, etc. The developed group decision-making process is a one way structured process. When a step in the process is completed, new changes will not affect the results. Consequently, when the preferences changes, and the users wants to adjust their preference models then the current decision-making process will fail in the sense of that no intelligent facilitation is taken into account. Someway the each member of the travel party must be announced that the decision-process has to be restarted. This will of course not work in practice. Therefore we suggest developing an adaptive intelligent decision-making process in future researches. An adaptive process that suggests the next decision-making step based on the activities, progress and changes of the group.
13.2 Conflict resolution When conflict-situations occur, the current decision-making process does not provide any intelligent support. For example, in the situation where users arrived at the “clarify” step, users are supposed to rate and provide comments about the remaining cities. It is imaginable that two members assign opposite ratings to a city i.e. member X wants Tokyo and NOT Wales while member Y wants Wales and NOT Tokyo. The current process suggests cities that matches somewhere between the two different preferences. But what if users do not want cities that are the average of the two extremes? Therefore, research to the integration of conflict resolution into the decision-making process is needed.
13.3 Types of trips Currently, Trip.Easy GDSS has only knowledge about city trips. In practice, travel parties may also have preferences about other types of travels i.e. winter sport, shopping trips, backpacking, etc. This will heavily influence the decision process. Further research is suggested to extend the domain knowledge and the consequences on the decision-making process.
13.4 Travel features A travel plan contains more than only a travel destination. Travel duration, budget, travel date, etc are also important variables for constructing a travel plan. The current decision-process does not take this into account. When introducing new variables the complexity of the process increases. Extra conflicting situations could arise. Nevertheless, this is a very interesting extension of the domain and it is worth to investigate about the possibilities.
128
14 Bibliography A. Pizam, Y. M. (1999). Consumer Behavior in Travel and Tourism. London. A. Pizam, Y. M. (1999). Consumer Behavior in Travel and Tourism. London. Bartlett, J. (1977). Organizational Research: Determining Appropriate Sample Size in Survey Research. Bostrom, R. ( 2002). Group Facilitation and Group Support Systems. Braziunus, D. (2006). Computational Approaches to Preference Elicitation. Briggs. (2006). Defining Key Concepts for Collaboration Engineering. Proceedings of the twelfth Americas conference on information systems 17 , 1 - 10. Briggs, R. (2003). Collaboration Engineering with Thinklets to Pursue sustained Success with Group Support Systems. Journal of management Information Systems 19 , 31-64. Briggs, R., & Vreede, G. d. (1997). Measuring satisfaction in GSS meetings. Proceedings of the Eighteenth International Conference on Information Systems (pp. 483-484). Atlanta: AIS. Briggs, R., Reinig, B., & Vreede, G. d. (2006). Meeting Satisfaction for Technology-Supported Groups: An Emperical Validation of a Goal-Attainment Model. Small Group Research, 37 , 585-911. Brigham, B., & Perron, J. (2004). Information-Seeking Behavior in Recreational Planning: An exploratory study of recreational travelers. CAPSTONE RESEARCH . Brinkman, W. (n.d.). Questionaire design. Brinkman, W., Haakma, R., & Bouwhuis, D. (2009). The theoretical foundation and validity of a component-based usability questionaire. Behavior and Information Technology, 28 , 121-137. Brinkman, W., Haakma, R., & Bouwhuis, D. (2009). The theorietical foundation and validity of a component-based usability questionaire. Behavior and Information Technology, 28 , 121-137. de Vreede, G., & Briggs, R. (2005). Collaboration Engineering: Designing repeatable Processes for HighVvalue Collaborative Tasks. Proceedings of the 38th Hawaii International Conference on System Sciences , 17 - 27. Dix, A., Finlay, J., Abowd, G., & Russell, B. (1997). Human-Computer Interaction. Glasgow: Pearon Education Limited. Field, A. Discovering statistics using SPSS. George, J., Easton, G., & Nunamaker, J. (1990). A study with collaborative group work with and without computer-based support. Information Systens Research, 1 , 394-415.
129
Gitelson, R., & Kerstetter, D. (1995). The influence of friends and relatives in travel decision. Economic Psychology of Travel and Tourism . Hiltz, S. (1990). User satisfaction with computer-mediated communication systems. Management Science, 36 , 739-764. Kim, W., & Mauborgne, R. (1995). A procedural justice mode of strategic decision making: Strategy content implications in the multinational. Organization Science, 6 , 44-61. Kolfschoten, G. (2007). Theoretical Foundations for Collaboration Engineering. Delft University of Technology: Department of System Engineering - Faculty of Technology Policy and Management. Kolfschoten, G., & de Vreede, G. (2007). The collaboration engineering approach for designing collaboration process. LNCS . Kolfschoten, G., Lowry, P., Dean, D., & Kamal, M. (2007). A measurement framework for patterns of collaboration. HICSS , 1-17. Korsgaard, M., Schiweiger, D., & Sapienza, H. (1995). Building commitment, attachment, and trust in strategic decision-making teams: The role of procedural justice. Academy of Management Journal, 38 , 60-84. Mansfeld, Y. (1992). From motivation to actual travel. Annual of Tourism research , 399-419. Ngai, K. (2008). A Group Decision Support System for Tourists: Planning a trip with consensus based decision making. Delft. Nichols, C. (1988). Family decision making and tourism behavior and attitudes. journal of travel research , 2-6. R.Gitelson. (1995). The influence of friends and relatives in travel decision. . Economic Psychology of Travel and Tourism . Reinig, B. (2003). Towards understanding of satisfaction with the process and outcomes of teamwork. Journal of Management Information Systems, 19 , 65-83. Reinig, B., Briggs, R., Shepperd, M., Yen, J., & Nunamaker, J. (1996). Affective reward and the adoption of group support systems: Productivity is not always enough. Journal of Management Information Systems, 12 , 171-185. Shim, J., Warkentin, M., Courtney, J., Power, D., Sharda, R., & Carlsson, C. (2002). Past, present, and future of decision support technology. Decision Support Systems 33 , 111-126. Simon, S., Grover, V., Teng, J., & Whitcomb, K. (1996). The relationship of information system training methods and cognitive ability to end-user satisfaction, comprehension, and skill transfer: A longitudinal field study. Information System Research, 7 , 466-490.
130
Sirakaya, E., & Woodside, A. (2005). Building and testing theories of decision making by travelers. Tourism management 26 , 815-832. Sirakaya, E., & Woodside, A. (2005). Building and testing theories of decision making by travelers. Tourism Management 26 , 815 - 832. Solomon, M. (2006). Consumer Behavior A European Perspective. Prentice Hall Third Edition. Taylor, S., & Todd, P. (1995). Understanding information technology usage: a test of competing models. Information Systems Research, 6 , 144-176. Thomas L. Saaty, Jen S.Shang. (2005). Group decision-making: Head-count versus intensity of preference. Wang, K. (2004). Who is the decision-maker: the parents or the child in group package tours? Tourism management 25 , 183-194.
131
Appendix
132
15 Appendix 1: Field research - Group Decision Support by a Travel agency Goal: The goal of this field research was to gain more insight about the group decision support process in real life provided by travel agencies. Without any announcements of our intention we (two ‘actors’) walked in a pre selected travel agency and started to chat with the travel consultants. Biased information provided by the travel consultant was prevented because they were not aware of our ‘real’ goal and thus treated us as normal customers. We used the pre defined case (described in the case setup section) as a guideline and observed the discussion pattern between us and the travel consultant.
Participants: Thomas Cook Peperstraat 17 2611 CH Delft 015 2136541 http://www.thomascook.nl Arke Reisbureau Brabantse Turfmarkt 90-92 2611 CP Delft 015 2126767 http://www.arke.nl D-Reizen Gasthuislaan 48 2611 RB Delft 015 5151515 http://www.d-reizen.nl
Case setup: In order to obtain comparable results we defined a case that was used as a guideline every time we visit a travel agency. The actors played the role as two friends and study mates. As a celebration of their graduation they are planning a trip for February 2009 with their girlfriends. The total travel party consists of 4 persons. The budget was settled in a price range between 500 and 1000 euro per person. Additionally, the two male characters are acting very dominant. Both of them want a perfect vacation, based on their preference. Though they are striving towards a consensus based solution, because they know that when someone is not happy with the final choice, it could spoil the whole vacation.
133
profiel
profiel
profiel
profiel
participant 1
participant 2
participant 3
participant 4
Male
Male
Female
Female
25 year
26 year
25 year
24 year
TYPE: Active, sporty, artifarty
TYPE: Relaxing, do nothing, enjoying
TYPE: Dreamy, sporty, social, chilling
TYPE: Social, active, artifarty
HOBBY: drinking wine, cooking , dancing
HOBBY: shopping, jogging, swimming, dancing, reading
HOBBY: Grafity drawing, shooting photo's, diving, golf, basketball
PREFERENCE: Sun, sea and beach, cocktail PREFERENCE: metropolis, dynamic , coffee bars, sight seeing, diving.
PREFERENCE: shopping, shopping and shopping
DESTINATION: no preference DESTINATION: no preference
HOBBY: Visiting museums, concerts, playing music, salsa dancing, jogging, swimming
PREFERENCE: shopping, sight seeing, active.
DESTINATION: no preference
DESTINATION: no preference
Figure 35 : Representation of the actors’ profile
Result: All travel agencies started with extracting our potential destinations (awareness set) and used a destination based decision process. But beca because use none of the participants had any destination preferences,, the travel consultant switched to a holiday type in combination with activity based decision d process. By asking our individual top top-level level preferences, they started to recommend some destinations based on their own experiences. A very interesting observation is the seasonal condition. Because we wanted to travel in February she warned us fo for the spring holiday which will have big influence on the budget. It also affected the sun, sea and beach criteria of participant 2, because this is strongly coupled
134
with the seasonal condition. This means that for a good GDSS we need to take this seasonal condition into account. We observed that the aggregation of all the preferences is not done by some sort of protocol/technique. All the travel agencies used the word ‘experience’ and ‘positive feedback from clients’ to recommend a destination. Another observation was that the travel agent consultant advised us the following preference elicitation and aggregation strategy to come to a consensus. First we were asked to individually determine our own preferences and then discuss with the group. This observation supports our suggestion for the preference elicitation interaction strategy, where we first want to elicit the preferences individually in order to get the user preferences as unbiased as possible. When doing so, we observed the same irrational group decision making due to external and internal factors as discussed in chapter 3 “travel group decision-making”. Remarks: The travel packages in the pamphlets were composed by the parent concern of the travel agencies. So the travel consultants only need to know which product the travel agency is selling. That means that basically no information retrieval is taking place, except from choosing from pre-defined vacation packages.
135
16 Appendix 2: Field research – Interview with domain expert Interview with: Marjolein Visser (Tourist) Marketing Specialist Introduction During our literature survey was found out that the information about tourist classification based on preferences is very limited. Though it is not very important for the group decision support system, it could be very useful for preference elicitation (predicting preferences from other preferences) and for preference aggregation (distance function and difference measuring). Therefore we decided to consult a domain specialist to find out: • • • •
Whether there exists a preference based model. The current methods used. Advice about how to develop such a model. o How to conduct field research Where we can get feature information of travel plans.
Subject: Tourists’ preference modeling and classification Current situation: Consumer analysis of tourists is very basic and general. Most organizations focus on pushing products to consumers instead of customized offers based on their preferences. There are no preference models and thus classification or profiling based on preferences is not possible at the moment. Exceptions are the niche markets where travel organizations only focus on one type of travelling. These organizations are more focused on very specific target audiences for example safari tourists or adventure tourists. They require the tourists to have a certain level of travelling experience and knowledge, so they can customize a tour by elicitation of very specific preferences. Transition: At the moment a process of differentiation is going on within the travel market. Specialized tour operators and travel organizations that focus on specific travel products or services, are serving an increasing part of the market. The offers of these organizations are also more preference based. An example of such an organization is vakantie-wijzer-van.nl. This is an independent decision support website, which advises an older audience based on preferences. Another example is singleplus.nl which is a decision support website that advises singles, based on the preferences of this group. The difference between general holiday websites and these examples is that these specialized decision support
136
websites focuses on just one audience and therefore does recommendations that are much more optimal and complete. They take into account the most important and very specific preferences that only (or mostly) exist within their audience. Recommendation for group decision support for tourists: The interview with Mrs. Visser was focused on preference elicitation and classification of tourists. As said before, there is no existing preference based classification method whatsoever. In order to use classification, Mrs. Visser recommends three options:
1. Analyzing search behavior of tourists by mining over tourists search data and retrieve relations that can be used for classification. (Currently researched by Auwke Pot, researcher at the Vrije Universiteit van Amsterdam) 2. Developing a preference based model based on the means & chain model (Figure 3). This can be done by a research that focuses on question:”What’s important for tourists?” This can be conducted through interviews with tourists. 3. Conduct preference elicitation based on forms with questions.
Another issue is where to get the feature information for travel plans. Mrs. Visser told us that services exist, which provides feeds containing such information in different formats. These feeds can be read and used by our system. Examples of such services are: • •
Tradetracker.nl Daisycon.com
Conclusion From the field research, we can conclude that classification based on preferences is a difficult challenge. This is due to the fact that a tourist preference model does not exist at the moment. In order to get such a model, we will have to develop it ourselves by interviewing tourists. Finally it is recommended that we narrow down our domain to the level of a niche market or activity based holiday domain (e.g. skiing vacations, or safari, etc). This way we can take into account very specific preferences. Furthermore, it is believed that a preference based group decision support system for tourist can generate more customized travel plans, which raise the added value of the system and is also more feasible to develop.
137
17 Appendix 3: Use Cases Table 1: UC0 Register Name Version Goal Actors Preconditions Triggers Descriptions
Register 1.0 A new user of the TGDSP must provide their personal information in order to receive the login details New user User must be new for the system The new user creates an account on TPGDSP by providing the following information: • Name • Age* • Address* • Username • Password
Post conditions Business rules Notes
The new user now is a known user of the TGDSP. The user may now login to the system. -
* optional
Table 2: UC1 Login Name Version Goal Actors Preconditions Triggers Descriptions
Register 1.0 User wants to get access to the TGDSP Initiator, invitee User must be known by TGDSP By providing the following information the user can access the TGDSP: • •
Post conditions Business rules Notes
Username Password
User has granted access to the TGDSP -
Table 3: UC2 Making friends Name Version Goal Actors Preconditions Triggers Descriptions
Making friends 1.0 User wants to be connected with friends in order to travel together Initiator, invitee User must be signed in Using the user search function by providing: • •
Post conditions Business rules Notes
Username or Email
The system will find users that are matching the conditions. The user may select the user that he/she want to be connected with as a friend The selected friend is assigned to the user as a friend None or one user is accepted as a friend -
138
Table 4: UC3 Accepting friends Name Version Goal Actors Preconditions Triggers Descriptions
Accepting friends 1.0 Invite friends that do not uses the TGDSP and let them register Initiator User must be signed in By providing the: • Name • Email address The system will send an email to the invitees to register
Post conditions Business rules Notes
The invitees will receive a mail where they will be introduced into the TGDSP -
Table 5: UC4 Create collaborative decision making session Name Version Goal Actors Preconditions Triggers Descriptions
Create collaborative decision making session 1.0 Setup a session where the invitees can execute a preference based collaborative travel decision making process Initiator User must be signed in The initiator creates a new collaborative decision making session by defining: • The name of the travel session • Selecting the friends that may participate in this travel session
Post conditions
When the session is created, the system will send all the invitees an email to join/accept this session.
Business rules Notes
Each user may parallel participate into multiple travel sessions
Table 6: UC5 Join collaborative decision making session Name Version Goal Actors Preconditions Triggers Descriptions Post conditions Business rules Notes
Join collaborative decision making session 1.0 Joins into created collaborative decision making session Invitee User must be signed in By accepting the request the invitee is now part of the travel session Invitee is now assigned to the travel session -
139
Table 7: UC6 Participate collaborative decision making session
This usecase is a very abstract one. Details of this use case will be worked out in the collaboration process chapter. Name Version Goal
Participate collaborative decision making session 1.0 By participating into decision making process the user will
Actors Preconditions Triggers Descriptions Post conditions Business rules Notes
Invitee User must be signed in By accepting the request the invitee is now part of the travel session Invitee is now assigned to the travel session -
140
18 Appendix 4: City Database name price region
name
price
region
Innsbruck
450
Austria
Hangzhou
1100
China
Vienna
500
Austria
Hong Kong
925
China
Antwerp
120
Belgium
Split
630
Croatia
Bruges
120
Belgium
Zagreb
500
Croatia
Brussels
130
Belgium
Pula
460
Croatia
Mostar
850
Dubrovnik
670
Croatia
Sarajevo
650
Prague
310
Czech Republic
Sofia
330
Bosnia & Herzegovina Bosnia & Herzegovina Bulgaria
Copenhagen
280
Denmark
Toronto
700
Canada
Helsinki
570
Finland
Vancouver
820
Canada
Toulouse
630
France
Ottowa
850
Canada
Bordeaux
670
France
Montreal
700
Canada
Marseille
500
France
Edmonton
1000
Canada
Paris
200
France
Whistler
900
Canada
Nice
750
France
Banff
800
Canada
Lourdes
340
France
Victoria
1000
Canada
Cannes
250
France
Calgary
1050
Canada
Lyon
150
France
Quebec City
870
Canada
Cologne
250
Germany
Chengdu
1000
China
Berlin
220
Germany
Guilin
1050
China
Munich
150
Germany
Beijing
1000
China
Hamburg
200
Germany
Shanghai
1000
China
London
450
Great Britain
Suzhou
950
China
Edinburgh
450
Great Britain
Shenzhen
1100
China
Santorini
540
Greece
Urumqi
1200
China
Athens
450
Greece
Xian
1100
China
Nuuk
1500
Iceland
Kunming
1100
China
Reykjavik
680
Iceland
Lhasa
1600
China
Budapest
340
Hungary
141
name
price
region
name
price
region
Mumbai
890
India
Puerto
1200
Mexico
Goa
900
India
Vallarta
1000
Mexico
Udaipur
900
India
Tijuana
865
Mexico
Delhi
850
India
Cancun
500
Mexico
Jakarta
1100
Indonesia
Monte Carlo
950
Monaco
Bali
1150
Indonesia
Budva
530
Montenegro
Ubud
1150
Indonesia
Oslo
325
Norway
Dublin
570
Ireland
Warsaw
300
Poland
Cork
880
Ireland
Krakow
500
Poland
Rome
250
Italy
Albufeira
500
Portugal
Naples
390
Italy
Porto
400
Portugal
Milan
350
Italy
Lisbon
400
Portugal
Venice
450
Italy
Bucharest
350
Romania
Tokyo
1000
Japan
Irkutsk
750
Russia
Kyoto
1050
Japan
450
Russia
Fukuoka
1100
Japan
Saint Petersburg Moscow
550
Russia
Osaka
1000
Japan
Sochi
710
Russia
Sapporo
1200
Japan
San Marino
850
San Marino
Riga
330
Japan
Belgrade
470
Serbia
Luxembourg City Skopje
200
Luxemburg
Bratislava
370
Slovakia
620
Macedonia
Ljubljana
600
Slovenia
Veracruz
1000
Mexico
Bled
600
Slovenia
Mexico city
735
Mexico
Busan
1350
South Korea
Acapulco
1000
Mexico
Seoul
1250
South Korea
Playa del Carmen Oaxaca
875
Mexico
Barcelona
330
Spain
835
Mexico
Madrid
230
Spain
Cabo San Lucas
1030
Mexico
Granada
600
Spain
Guadalajara
1035
Mexico
Malaga
470
Spain
Isla Cozumel
925
Mexico
Cordoba
400
Spain
Mazatlan
1030
Mexico
Valencia
450
Spain
142
name
price
region
name
price
region
Salamanca
400
Spain
San Francisco
880
USA
San Sebastian
650
Spain
New York
900
USA
Girona
450
Spain
Memphis
850
USA
Santiago de Compostella Sevilla
640
Spain
Nashville
900
USA
360
Spain
Jackson (Mississippi)
1000
USA
Stockholm
350
Sweden
Indianapolis
900
USA
Hualien
2600
Taiwan
Houston
800
USA
Taipei
1200
Taiwan
Detroit
900
USA
Chiang mai
1250
Thailand
Dallas (Texas)
900
USA
Phuket town
1075
Thailand
Hanoi
1025
Vietnam
Bangkok
1150
Thailand
1000
Vietnam
Rotterdam
70
550
Switzerland
Amsterdam
70
Geneve
550
Switzerland
Ankara
570
The Netherlands The Netherlands Turkey
Ho Chi Minh city Zurich
Istanbul
430
Turkey
Chicago
800
USA
Miami
950
USA
Las Vegas
1000
USA
Kansas City
1000
USA
Boston
850
USA
Austin
1200
USA
Cleveland (Ohio) Los Angeles
1000
USA
900
USA
Seattle
1000
USA
Denver (Colorado) Atlanta
800
USA
850
USA
Aspen
1500
USA
Honolulu
1300
USA
Philadelphia (Pennsylvania)
950
USA
143
19 Appendix 5: Survey 19.1 Introduction
144
19.2 Questionnaire generate
145
19.3 Questionnaire reduce
146
19.4 Questionnaire clarify
147
19.5 Questionnaire evaluate
148
19.6 Questionnaire general evaluation
149
19.7 Questionnaire Evaluation PGA
150
19.8 Questionnaire Evaluation SP+SO
151
19.9 Evaluation misc.
152
20 Appendix 6: Survey processing 20.1 Gender Gender Frequency Valid
Percent
Cumulative Percent
Valid Percent
M
72
60.0
60.0
60.0
V
48
40.0
40.0
100.0
120
100.0
100.0
Total
20.2 Age Descriptive Statistics N
Range
Minimum
Maximum
Statistic
Statistic
Statistic
Statistic
Age
120
Valid N (listwise)
120
37
18
Mean Statistic
55
28.15
Std. Error .708
Std. Deviation
Variance
Statistic
Statistic
7.752
60.095
20.3 Satisfaction Process (SP) Descriptive Statistics
153
N
Range
Minimum
Maximum
Statistic
Statistic
Statistic
Statistic
SP
120
Valid N (listwise)
120
3.75
3.25
Mean Statistic
7.00
Std. Deviation
Variance
Statistic
Statistic
Std. Error
5.8854
.06944
.76070
.579
One-Sample Test Test Value = 0 95% Confidence Interval of the Difference t SP
df
84.752
Sig. (2-tailed) 119
Mean Difference
.000
5.88542
Lower 5.7479
Upper 6.0229
SP Frequen Percen cy t
Valid Percent
Cumulativ e Percent
Valid 3.25
1
.8
.8
.8
3.75
1
.8
.8
1.7
4.00
2
1.7
1.7
3.3
4.25
2
1.7
1.7
5.0
4.50
2
1.7
1.7
6.7
4.75
3
2.5
2.5
9.2
5.00
5
4.2
4.2
13.3
5.25
13
10.8
10.8
24.2
5.50
4
3.3
3.3
27.5
5.75
8
6.7
6.7
34.2
6.00
45
37.5
37.5
71.7
6.25
8
6.7
6.7
78.3
6.50
6
5.0
5.0
83.3
6.75
3
2.5
2.5
85.8
7.00
17
14.2
14.2
100.0
Total
120
100.0
100.0
154
20.4
Perceived Goal Net Attainment (PGNA) Descriptive Statistics N
Range
Minimum
Maximum
Statistic
Statistic
Statistic
Statistic
PGA
120
Valid N (listwise)
120
4,25
2,75
Mean Statistic
7,00
Std. Deviation
Variance
Statistic
Statistic
Std. Error
5,9313
,06522
,71447
,510
One-Sample Test Test Value = 0 95% Confidence Interval of the Difference t PGA
df
90,940
Sig. (2-tailed) 119
,000
Mean Difference 5,93125
Lower 5,8021
Upper 6,0604
PGA Frequen Percen cy
t
Valid
Cumulativ
Percent
e Percent
Valid 2,75
1
,8
,8
,8
4,00
2
1,7
1,7
2,5
4,25
2
1,7
1,7
4,2
4,75
3
2,5
2,5
6,7
5,00
7
5,8
5,8
12,5
5,25
5
4,2
4,2
16,7
5,50
11
9,2
9,2
25,8
5,75
9
7,5
7,5
33,3
6,00
36
30,0
30,0
63,3
6,25
21
17,5
17,5
80,8
6,50
5
4,2
4,2
85,0
6,75
4
3,3
3,3
88,3
7,00
14
11,7
11,7
100,0
Total
120
100,0
100,0
155
20.5 One sample t-test SP + PGA Normality
Tests of Normality a
Kolmogorov-Smirnov Statistic
df
Shapiro-Wilk
Sig.
Statistic
df
Sig.
SP
.218
120
.000
.912
120
.000
PGA
.205
120
.000
.903
120
.000
a. Lilliefors Significance Correction
156
One sample t-test SP and PGA One-Sample Statistics N
Mean
Std. Deviation
Std. Error Mean
SP
120
5.8854
.76070
.06944
PGA
120
5.9313
.71447
.06522
One-Sample Test Test Value = 5 95% Confidence Interval of the Difference t
df
Sig. (2-tailed)
Mean Difference
Lower
Upper
SP
12.750
119
.000
.88542
.7479
1.0229
PGA
14.278
119
.000
.93125
.8021
1.0604
Reliability: SP Reliability Statistics Cronbach's Alpha
N of Items .880
4
Reliability: PGA Reliability Statistics Cronbach's Alpha
N of Items .848
4
157
20.6 Evaluation Step “Generate”: Descriptive Statistics N
Range
Minimum
Maximum
Statistic
Statistic
Statistic
Statistic
Mean Statistic
Std. Error
Std. Deviation
Variance
Statistic
Statistic
Effort
120
6.00
1.00
7.00
5.5417
.13012
1.42543
2.032
Expressiveness
120
6
1
7
5.53
.103
1.130
1.276
Method
120
4
3
7
6.06
.094
1.031
1.064
Objective
120
6
1
7
5.49
.128
1.402
1.966
Presentation
120
6
1
7
5.17
.136
1.491
2.224
Features
120
6
1
7
5.31
.120
1.314
1.728
Valid N (listwise)
120
Expressiveness
Frequency Valid
1
1
2
1
3
4
4
13
5
Percent
Cumulative Percent
,8
,8
,8
,8
1,7
3,3
3,3
5,0
10,8
10,8
15,8
29
24,2
24,2
40,0
6
52
43,3
43,3
83,3
7
20
16,7
16,7
100,0
120
100,0
100,0
Total
,8
Valid Percent
Method
Frequency Valid
Percent
Valid Percent
Cumulative Percent
3
3
2,5
2,5
4
8
6,7
6,7
9,2
5
18
15,0
15,0
24,2
6
41
34,2
34,2
58,3 100,0
7 Total
50
41,7
41,7
120
100,0
100,0
2,5
158
Effort Frequency Percent
Cumulative Valid Percent Percent
1
1
.8
.8
.8
2
3
2.5
2.5
3.3
3
9
7.5
7.5
10.8
4
13
10.8
10.8
21.7
5
23
19.2
19.2
40.8
6
33
27.5
27.5
68.3
7
38
31.7
31.7
100.0
Total
120
100.0
100.0
Valid
Objective
Frequency Valid
Percent
Valid Percent
Cumulative Percent
1
2
1,7
1,7
2
3
2,5
2,5
4,2
3
8
6,7
6,7
10,8
4
11
9,2
9,2
20,0
5
22
18,3
18,3
38,3
6
45
37,5
37,5
75,8
7
29
24,2
24,2
100,0
120
100,0
100,0
Total
1,7
Presentation
Frequency Valid
Percent
Valid Percent
Cumulative Percent
1
4
3,3
3,3
2
5
4,2
4,2
7,5
3
9
7,5
7,5
15,0
4
10
8,3
8,3
23,3
5
30
25,0
25,0
48,3
6
45
37,5
37,5
85,8
7
17
14,2
14,2
100,0
120
100,0
100,0
Total
3,3
159
Features Frequency
Percent
Valid Percent
Cumulative Percent
1
3
2.5
2.5
2.5
2
2
1.7
1.7
4.2
3
8
6.7
6.7
10.8
Valid
4
8
6.7
6.7
17.5
5
36
30.0
30.0
47.5
6
47
39.2
39.2
86.7
7
16
13.3
13.3
100.0
120
100.0
100.0
Total
id 26 35 37
100
229
573 583
647
688
remark Eventueel een stap verder te gaan en specifiek de subkoppen te rangschikken en van daaruit de resultaat weergeven i.p.v. de koppen cultuur, culinair enz. Het was mij onduidelijk waar ik de preferenties naartoe moest slepen. het is niet duidelijk dat je moet slepen. geen duidelijk taal hierover want alleen in text uitgelegd. Communicatie is beter over te brengen naar mij als het meer catchy is. Het was mij eerst niet duidelijk wat de bedoeling was van de vakjes. De rangschikking van de voorkeuren zijn voor mij niet zo duidelijk. Ten eerste omdat er teveel voorkeuren waren bij sommige categorieen. De betekenis van de schalen gelden dan niet meer voor mij. Ten tweede stond er nergens bij wat ik moest doen. Ten derde weet ik niet of ik de preferenties (culinair, cultuur enz.) ook moest rangschikken. Het zal duidelijker zijn als er een zeer korte tutorial erbij was. Om te zien hoe de drag&drop werkt. De drag&drop menu vind ik erg interactief en makkelijk om te gebruiken met het ordenen. De schaal van de hoofdcategorie is duidelijk. Alleen jammer bij de feature is het onduidelijk. Want op een gegeven moment staan de features onder elkaar waardoor het onduidelijk werd in welke schaal de features zitten. Handig in gebruik, maar te weinig keuzes en het is niet altijd duidelijk wat er met een onderwerp bedoelt wordt. De specificaties van zo'n onderwerp waren zeer algemeen en dekten niet altijd de lading van het onderwerp naar mijn mening. - sommige blokjes pasten niet op een lijn in het scherm, waardoor het leek alsof ze weer bij heel erg positief stonden. - bij sommige onderdelen waren niet de totaal representatieve specificaties beschreven zoals bij cultuur en ook atmosfeer. budget - de kleuren vond ik erg duidelijk De verschillende preferenties zijn soms inwisselbaar, zoals "paleis" en "monumenten". Dit was vaker het geval, maar kan het me nu niet precies meer herinneren. Wel erg gebruiksvriendelijk gemaakt, waarvoor mijn complimenten. De uiteindelijke keuze voor vakantiebestemmingen leek erg beperkt. Zo kwam enkel "Houston" als stad uit de VS eruit, maar kan me voorstellen dat San Fransisco of Los Angeles ook tegemoet kunnen komen aan onze prefenties als groep. De nadruk lag bij deze enquete meer op de eventuele vakantiebestemming. Beperkingen als budget en of iemand al eens eerder ergens was geweest konden enkel in de onderhandelingen terugkomen. Tijdens de introductie vertelde je dat jullie dominantie of andere onevenwichtigheid in het groepsproces wilden uitsluiten. Bij het onderdeel "onderhandelen" komt dit echter juist terug, waardoor de waarde van de voorgaande stappen enkel bestaat uit het maken van een (beperkte) keuze uit vakantiebestemmingen. In de praktijk is het echter waarschijnlijk vaak zo dat mensen al een "lijstje" in hun hoofd hebben. Wellicht is het dan ook waardevol om de interesse voor vakantiebestemmingen die de deelnemers al voorafgaand aan het onderzoek hadden mee te nemen in de uiteindelijke lijst van mogelijke vakantiebestemmingen. Verder was het leuk om deel te nemen aan jullie onderzoek. Veel succes ermee! Ps ps, het was me even onduidelijk dat deze vragen over slechts categorie 1 gingen. Vandaar dat ik hieronder al een hele feedback heb gegeven over het hele systeem :) Maar specifiek over deze categorie: misschien kan je ook koffietentjes erbij doen (alhoewel is minder eten ...) ------- Ten eerste complimenten. Het ziet er goed uit, mooie layout, en leuk om te doen! Dan nog enkele opmerkingen: * Als er meer subelementen zijn waardoor de tweede regel gebruikt moet worden en ik wil een blokje als laatste ordenen (op de 2e regel dus) dan werkt dat alleen als je dat blokje op het blokje zet die op dat moment als laatste stond. Terwijl het voor mij logischer was om hem hierachter te zetten, zodat alles opschuift. * Amusement: hierbij staat nachtleven en nachtclubs. Dat is dubbel. * Cultuur: zonder dat ik de subelementen had gezien, had ik deze bovenaan gezet. Maar nadat ik kennis had van die subelementen, heb ik hem toch verder naar onder gezet. Mijn verwachting hierbij was anders. Meer ook cultuur as in, de cultuur/historie van het land. Maar dat kan dus ook aan mij liggen :) * Qua namen lijken de 'omgeving' en 'atmosfeer' op elkaar. Maar goed, je ziet later de elementen die erbij horen dus dan zou enige ambiguiteit hieromtrent verholpen zijn. * Kortom, qua proces merkte ik dat de ordening van stap 1 en 2 door elkaar heen liepen.Pas nadat ik zag wat bij de categorieen hoorde, kon ik deze algemene categorieen beter ordenen. * Als je alle steden ziet en deze moet rangordenen staat er ook per stad bij in hoeverre deze matcht met onze voorkeuren. Ik merkte dat dit voor mij meer overdaad aan info was. Of je moet er helemaal in gaan duiken. Uiteindelijk heb ik de steden geordend met mijn eigen idee van de steden. Dit was soms moeilijk omdat je niet van alle steden kennis hebt. Je gaat dan toch minder op die extra informatie bouwen (dus de mate van match en in hoeverre die subcategorieen aanwezig zijn) omdat dit minder tot de verbeelding spreekt. * Uiteindelijk zijn er 6 steden over. Misschien kan je erbij zetten dat dit gaat
160
over onze gezamenlijke consensus. Dat was niet helder. * Goed dat jullie bij die 6 steden ook ruimte laten voor opmerkingen en dergelijke, zodat je altijd nog aanvullende informatie kan geven aan de groepsleden. Het was mij niet in een keer duidelijk dat ik moest slepen. Misschien dat een video wel helpt. De plus en minnetjes zijn wel 885 duidleij kmaar ik dacht dat het om andere interactie ging Bij het kiezen van preferenties kreeg ik de neiging eerst te spelen voordat ik ging lezen wat ik moest doen. Het uitleg kan 907 wat overzichtelijker worden weergeven om dat te voorkomen. 913 Uitleg van dit onderdeel kan uitgebreider. De plustekens en sleeptekens kunnen verwarrend zijn voor de gebruiker. Het lijkt alsof de onderdelen van een ategorie ook geordend kunnen worden. Ik denk dat het intuitiever aan zou voelen (om 1199 te ordenen met drag&drop) als de opties niet als aansluitende vakjes worden weergegeven (maar bv door losse cirkels?) Ik heb de onderdelen per categorie niet echt gebruikt voor het bepalen van mijn volgorde. 1219 De interface is niet heel erg overzichtelijk. De taak om de sub-onderdelen te sorteren is ook geen triviale taak. 1486 Zeer geschikt om duidelijk je wensen aan te geven 1492 Gevaar is dat men te veel keuze krijgt waardoor het onoverzichtelijk kan worden 1795 informatie/indruk van de steden is erg summier/algemeen daardoor is de waarde van je keuze ook beperkt 2094 beetje flashie en hip, maar niet echt heel overzichtelijk of makkelijk in t gebruik. Het was mij niet meteen duidelijk dat ik onderdelen kon slepen. Verder vind ik de categorieen duidelijk en de kleuren 2127 lekker helder. - Drag en drop kan soepeler. Het is niet altijd even duidelijk of je het geselecteerde item kan droppen of niet. Misschien zoiets dat je een placeholder ziet tijdens het draggen zodat je zeker weet dat hij daar terecht komt. - Subitems staan 2398 onder elkaar als ze niet op een rij passen. Hierdoor wekt het de indruk dat de linkeritems op de onderste rij niet belangrijk staan ipv andersom. Onder features kunnen max. 5 kopjes naast elkaar, de restkomt eronder en dan lijkt het alsof de twee onder elkaar staande 2409 features even belangrijk zijn. Dus stel bij een hele lange staan ' lokale keuken' en ' lunch faciliteiten' onder elkaar. Zo lijkt het alsof ze even belangrijk zijn. 2578 Goed gedaan Kenneth!!! Biggie Brasa, A & A 3011 Ik mis nog een keuze onderdeel voor bijv. zonnige bestemmingen, aantal vakantie dagen. de tweede regel van het ordenen is niet duidelijk...ik wist niet of de onderste regel nou het vervolg was van het eerste 3189 regel qua prioriteit 3464 - Onduidelijkheid over orde van belangrijkheid bij horizontale subcriteria - vr 5 - de opties binnen de features bestaan soms uit 2 rijen, waardoor de prioriteit onderling even onduidelijk is. - vr 6 3494 het weer vind ik ook vaak belangrijk Het is me onduidelijk of het belangrijk is om de subonderdelen onder de hoofdopties te ordenen. Bovendien verwachtte ik 3499 een andere invulling van de subonderdelen. Bv. cultuur: Ik zou niet per se een plein of paleis willen zien. Lastig om die tussen de andere termen te plaatsen. 3773 subcategories op 1 horizontale lijn ipv. 2 horizontale lijnen eerste scherm is niet geheel duidelijk: dat je per onderwerp de activiteiten kan kiezen het ordenen van de activiteiten kan 4221 beter: het is soms lastig een activiteit naar de laatste plaats te selecteren 4688 Iets specifiekere onderdelen. Of een uitleg bij wat er wordt verstaan onder de diverse onderdelen. Een extra optie zou kunnen zijn dat als er eenmaal een stad gekozen is je daarin uit verschillende accomodaties zou kunnen 4696 kiezen. Preferencies waren dmv drag en drop wel goed te ordenen maar stonden soms dichterbij elkaar qua waardering dan de 4701 bedoeling was. 4711 Het lijkt mij handig om de categorieen te nummeren. 5061 Het is een overzichtelijke manier om een stedentrip uit te kiezen met een groep 5520 teveel keuze's, sommige mensen zouden het te ingewikkeld vinden. 5528 Eeen korte animatiefilmpje of uitgeleg zou handig zijn geweest. 5777 Het enige wat nog mist is het aspect kosten. Als je een goedkopere vakantie wilt, zou je dit ook moeten kunnen aangeven. 5789 Ik denk dat een verticale rangschikking van hoog naar laag optisch duidelijker is. 6052 niet helemaal duidelijk met de tweede rij in het begin wist ik niet dat ik ze moest ordenen. Na het lezen moet ik wel zoeken WAT ik moest gaan ordenen. Ik denk dat 6133 een intro filmpje het wel gemakkeijk zou gaan maken.. Laat duidelijk zien aan welke kant van de schaal positief en negatief staat. Er is wel een kleur verschil maar die is niet zo 6964 duidelijk, en de woorden staan in dezelfde (blauwe) kleur. Het onderdeel lijkt op tabbladen maar werken niet als zo dergelijk. Ook was me niet in 1 keer duidelijk dat je ook de 6971 subonderdelen kon ordenen. Misschien was een mogelijkheid om actieviteiten als gelijk te kunnen beordelen handig geweest. Misschien dat dubbelklikken op een categorie een optie is om de categorie uit te klappen. Maar voor de rest had ik geen 6981 problemen met dit onderdeel. Ik vond de plussen en min teken even onduidelijk in het begin, wist niet precies wat ze deden. Het is eigenlijk te druk om 7003 overal die sleep tekens aan te geven, beter anders aangeven. Misschien is het duidelijker om een rij te maken van boven naar beneden dat automatisch laat zien wat je eerste belangrijke keuze is.
161
- sommige blokjes pasten niet op een lijn in het scherm, waardoor het leek alsof ze weer bij heel erg positief stonden. - bij 7272 sommige onderdelen waren niet de totaal representatieve specificaties beschreven zoals bij cultuur en ook atmosfeer. budget - de kleuren vond ik erg duidelijk Handig in gebruik, maar te weinig keuzes en het is niet altijd duidelijk wat er met een onderwerp bedoelt wordt. De 7345 specificaties van zo'n onderwerp waren zeer algemeen en dekten niet altijd de lading van het onderwerp naar mijn mening. Ps ps, het was me even onduidelijk dat deze vragen over slechts categorie 1 gingen. Vandaar dat ik hieronder al een hele feedback heb gegeven over het hele systeem :) Maar specifiek over deze categorie: misschien kan je ook koffietentjes erbij doen (alhoewel is minder eten ...) ------- Ten eerste complimenten. Het ziet er goed uit, mooie layout, en leuk om te doen! Dan nog enkele opmerkingen: * Als er meer subelementen zijn waardoor de tweede regel gebruikt moet worden en ik wil een blokje als laatste ordenen (op de 2e regel dus) dan werkt dat alleen als je dat blokje op het blokje zet die op dat moment als laatste stond. Terwijl het voor mij logischer was om hem hierachter te zetten, zodat alles opschuift. * Amusement: hierbij staat nachtleven en nachtclubs. Dat is dubbel. * Cultuur: zonder dat ik de subelementen had gezien, had ik deze bovenaan gezet. Maar nadat ik kennis had van die subelementen, heb ik hem toch verder naar onder gezet. Mijn verwachting hierbij was anders. Meer ook cultuur as in, de cultuur/historie van het land. Maar dat kan dus ook aan mij 7424 liggen :) * Qua namen lijken de 'omgeving' en 'atmosfeer' op elkaar. Maar goed, je ziet later de elementen die erbij horen dus dan zou enige ambiguiteit hieromtrent verholpen zijn. * Kortom, qua proces merkte ik dat de ordening van stap 1 en 2 door elkaar heen liepen.Pas nadat ik zag wat bij de categorieen hoorde, kon ik deze algemene categorieen beter ordenen. * Als je alle steden ziet en deze moet rangordenen staat er ook per stad bij in hoeverre deze matcht met onze voorkeuren. Ik merkte dat dit voor mij meer overdaad aan info was. Of je moet er helemaal in gaan duiken. Uiteindelijk heb ik de steden geordend met mijn eigen idee van de steden. Dit was soms moeilijk omdat je niet van alle steden kennis hebt. Je gaat dan toch minder op die extra informatie bouwen (dus de mate van match en in hoeverre die subcategorieen aanwezig zijn) omdat dit minder tot de verbeelding spreekt. * Uiteindelijk zijn er 6 steden over. Misschien kan je erbij zetten dat dit gaat over onze gezamenlijke consensus. Dat was niet helder. * Goed dat jullie bij die 6 steden ook ruimte laten voor opmerkingen en dergelijke, zodat je altijd nog aanvullende informatie kan geven aan de groepsleden. Wellicht is het dan ook waardevol om de interesse voor vakantiebestemmingen die de deelnemers al voorafgaand aan het 7500 onderzoek hadden mee te nemen in de uiteindelijke lijst van mogelijke vakantiebestemmingen. Het was onduidelijk of de sub-onderdelen ook meegenomen werden in de preferenties. Soms zou men een item in de 7586 atmosfeer hoger willen raten dan een item in de culinair, maar in z'n geheel genomen toch liever bv. culinair prefereren.
162
20.7 Evaluation Step “Reduce”: Descriptive Statistics N
Range
Minimum
Maximum
Statistic
Statistic
Statistic
Statistic
Mean Statistic
Std. Error
Std. Deviation
Variance
Statistic
Statistic
Effort
120
6
1
7
5.38
.128
1.403
1.967
Expressiveness
120
5
2
7
5.69
.104
1.143
1.307
Method
120
5
2
7
5.91
.102
1.123
1.260
Objective
120
6
1
7
5.71
.114
1.253
1.570
Presentation
120
5
2
7
5.40
.117
1.286
1.654
Reveal
120
6
1
7
6.13
.100
1.100
1.209
Graph
120
6
1
7
5.10
.130
1.423
2.024
City descriptions
120
6
1
7
4.41
.129
1.417
2.008
Decision
120
6
1
7
3.49
.188
2.062
4.252
Valid N (listwise)
120
Effort
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
1
.8
.8
.8
2
2
1.7
1.7
2.5
3
14
11.7
11.7
14.2
4
11
9.2
9.2
23.3
5
26
21.7
21.7
45.0
6
38
31.7
31.7
76.7
7
28
23.3
23.3
100.0
Total
120
100.0
100.0
Expressiveness
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
2
3
2.5
2.5
2.5
3
6
5.0
5.0
7.5
4
6
5.0
5.0
12.5
5
18
15.0
15.0
27.5
6
64
53.3
53.3
80.8
7
23
19.2
19.2
100.0
Total
120
100.0
100.0
163
Method
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
2
3
2.5
2.5
2.5
3
3
2.5
2.5
5.0
4
3
2.5
2.5
7.5
5
24
20.0
20.0
27.5
6
47
39.2
39.2
66.7
7
40
33.3
33.3
100.0
Total
120
100.0
100.0
Objective
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
1
.8
.8
.8
2
2
1.7
1.7
2.5
3
6
5.0
5.0
7.5
4
9
7.5
7.5
15.0
5
18
15.0
15.0
30.0
6
52
43.3
43.3
73.3
7
32
26.7
26.7
100.0
Total
120
100.0
100.0
Presentation
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
2
4
3.3
3.3
3.3
3
8
6.7
6.7
10.0
4
13
10.8
10.8
20.8
5
29
24.2
24.2
45.0
6
43
35.8
35.8
80.8
7
23
19.2
19.2
100.0
Total
120
100.0
100.0
164
Reveal
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
1
.8
.8
.8
2
2
1.7
1.7
2.5
4
4
3.3
3.3
5.8
5
18
15.0
15.0
20.8
6
40
33.3
33.3
54.2
7
55
45.8
45.8
100.0
Total
120
100.0
100.0
Graph
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
3
2.5
2.5
2.5
2
6
5.0
5.0
7.5
3
5
4.2
4.2
11.7
4
20
16.7
16.7
28.3
5
28
23.3
23.3
51.7
6
44
36.7
36.7
88.3
7
14
11.7
11.7
100.0
Total
120
100.0
100.0
City descriptions
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
3
2.5
2.5
2.5
2
11
9.2
9.2
11.7
3
14
11.7
11.7
23.3
4
34
28.3
28.3
51.7
5
26
21.7
21.7
73.3
6
28
23.3
23.3
96.7
7
4
3.3
3.3
100.0
Total
120
100.0
100.0
165
Decision
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
29
24.2
24.2
24.2
2
20
16.7
16.7
40.8
3
17
14.2
14.2
55.0
4
9
7.5
7.5
62.5
5
17
14.2
14.2
76.7
6
18
15.0
15.0
91.7
7
10
8.3
8.3
100.0
Total
120
100.0
100.0
id
remark de twee lijnen in de grafiek zijn onduidelijk. Waarom staan zij zo ver van elkaar en wat zij betekenen is niet geheel 114 duidelijk. De staafjes zijn daarentegen wel duidelijk en fijn. De grafiek is verwarrend vond ik. Ten eerste de schaal van de grafiek. er stond nergens bij wat die betekent. Ten tweede weet ik niet wat de lijn van de stad en de lijn van onze groep zegt. Want de lijn van de stad staat bovenaan en de lijn van 235 onze groep onderaan? De rangschikking van de steden is niet duidelijk gecommuniceerd. Het staat nergens toegelicht dat de steden gerangschikt konden worden. De kleuren zijn wel duidelijk. Ik wist niet dat deze stap ook een drag & drop sessie was. Maar wanneer ik eenmaal het door had ging alles makkelijker. Het was me eerst niet duidelijk wat de pijlen betekende per vakantiebestemming. Als de vakantieleider was me onduidelijk 444 wanneer en hoe ik de eindkeuze kon maken. Maar ik heb begrepen dat dit in real-life moest bespreken om tot een keuze moest maken. Het zal handiger zijn als dit via een chat sessie kon. - de lijst was best lang en dit zorgden er voor dat je soms niet in een keer een keuze van boven naar beneden kon slepen, of dat dit wat langer duurden. - eigenlijk mis je nog een beschrijving in verhaalvorm van de stad, met wat publiekstrekkers 594 en algemene informatie (weer, cultuur, drukte, seizoen, etc...) om je meer inzicht te geven in de stad. Nu kan je namelijk nog op veel steden heel instinctief reageren door voorkennis. Op zich een handig systeem, alleen is het slepen van de steden soms onhandig doordat er geen genummerde rankering is. De uitkomst waren ook niet de steden die ik en de groep verwacht hadden, sommige steden waar we allemaal graag heen 603 wilden stonden er niet tussen, zoals San Fransisco. Van de steden die er wel uitkwamen was het merendeel in de goede richting van wat iedereen leuk vind. Ik vond het wel lastig om in de eerste via het sleepsysteem exact aan te geven wat er belangrijk is aan een stad. Het gaat vaak ook om het gevoel bij een stad. De grafiek is erg abstract, het is nu onduidelijk wat de waarden op de verticale as zijn. Percentages van wat onze voorkeur? 696 De stadskarakteristieken kunnen nog uitgebreider. Waarom niet meer (visuele) informatie? Zo kon ik me moeilijk een indruk vormen van Kyoto, wat de uiteindelijke keuze beinvloed - zowel positief als negatief. Mijn opmerkingen hierover staan ook al bij het vorige blad. Aanvullend nog: * sommige steden die me interessant leken 766 stonden er niet bij. Desondanks stonden er wel opties bij waar ik ook heen zou willen gaan. Dus zo erg is dat niet. * het was wel een lange lijst om te ordenen. op den duur denk je bij de 'onderkant' van, het is wel goed zo. 893 chatbox of een vak voor opmerkingen zou handig zijn, fysieke aanwezigheid is dan niet meteen nodig 993 positieve vraagstelling (vraag 7). Ik moest ff lezen of ik nou HOOG of LAAG moest invullen. Het aantal steden die geselecteerd kunnen worden zijn te ruim, waardoor ik misschien mijn keuze alleen doorgeef van de eerste 5 favoriete steden. De categorieën in percentage zeggen mij niet zoveel. Beschrijvingen in tekst per stad zal 1032 mogelijk leuker zijn en een doorverwijs link zal dit probleem kunnen verhelpen. Voorkeuren van de andere in grafieken vind ik zeer interessant. Er zou meer informatie moeten komen over de steden. Ik heb eigenlijk geen concrete beeld van Montreal. Welke bekende 1214 gebouwen staan daar? Hoe ziet het er ongeveer er uit? Beetje teveel steden om te gaan sorteren. Er zijn heel veel buttons waar je in dit onderdeel op kan klikken, die niet allemaal even duidelijk zijn. Waardoor je niet weet dat er meer informatie tot je beschikking staat dan je denkt. Misschien is het ook leuk als je zelf ook nog steden kunt 1249 suggereren ipv alleen gegenereerde steden. Misschien heb ik al een stad in mijn hoofd, maar staat het niet in deze lijst. Maar dan zou het suggereren van steden in de vorige stap moeten gebeuren. Drag&Drop was prima, maar niet alle steden konden op 1 scherm worden weergegeven waardoor ordenen alsnog niet lekker ging. De grafieken zijn voor mij helemaal niet duidelijk... De eigenschappen van 1 stad zijn helder maar, hoe verhouden de 1264 waardes van de stad zich tot die van de goep (waarom zit die van de groep zo laag vergeleken met de stad)? De bar-charts per persoon kon ik eigenlijk weinig mee (te veel detail?) 1513 Grafiek heb ik wel gezien, maar niet echt tot mij doorgedrongen. Zal aan mij geleven hebben. 1526 de steden die het systeem voorstelt voldeden aan mijn opgegeven wensen 1813 het was wat onduidelijk dat de grafiek bij een bepaalde stad hoort 2103 op afstand een goede methode, grafische weergave liever in taartdiagram zo zie je dan gelijk een meerderheid
166
2119 Niet alles werkt hier, De persoonlijke interesse kwam niet naar voren. Alleen als groep in z'n geheel. ik heb de hele grafiek over het hoofd gezien. heb eigenlijk alleen maar gelet op het dragen en droppen van de blokken. de 2169 karakteristieken heb ik niet bekeken omdat ik zelf al een idee had bij de steden. wel fijn dat het kan als je behoefte hebt aan een beeld. Het was mij niet duidelijk dat je de stadskarakteristieken kon bekijken. Ook zag ik weer niet meteen dat je de plaatjes kon 2188 verslepen. Verder is het wel duidelijk. 2411 Het aantal steden is te veel voor drag-drop-methode. De stadskarakteristieken vertellen niet veel over de stad, daardoor is het heel lastig om ze te ordenen. Ik denk dat het 2436 aantal steden wel wat minder mogen zijn. Vraag 1 / 4: Ik was met mijn opgegeven voorkeuren een buitenbeentje, maar desondanks bleven leuke bestemmingen over. 3081 Ik denk dat het slepen, in geval van een lange lijst bestemmingen, wat gebruiksonvriendelijk kan worden. Ook klikte ik op het flash plaatje, en vervolgens was mijn volgorde gereset en moest ik opnieuw beginnen met slepen. 3241 gemiddelde match percentages kloppen niet. Allemaal hetzelfde ik zou het prettiger vinden om te kunnen browsen door de steden. Nu ben ik beperkt aan 20 a 30 stuks...ik heb namelijk het 3280 gevoel dat er steden zijn dat ik leuker vind - Veel keuze mogelijkheden - Sterretjes voor keuze is niet direct goed zichtbaar - Wanneer keuze tot 6 is, veel scrollen naar 3481 beneden (waardoor klik op "volgende stap" niet direct helder is Misschien had ik niet goed gelezen en niet goed verder geklikt, maar ik vond dat er per stad heel weinig informatie stonden. 3506 Er stond wel aangegeven hoeveel van elk categorie te zien/te doen is in percentage. Ik miste een beetje de info per onderdeel. - vr 4 - er staan veel steden in waar ik al eens ben geweest, zou mooi zijn als die er al uit blijven (ahv een reisprofiel wat ik 3532 zou hebben) - vr 9 - door het scrollen van het scherm wat onhandig. Misschien een idee om een knopje bij de trip te maken die in 1 keer de reis naar onder te klikken? Opmerking bij vraag 2: Ik had geen verwachting bij de steden die eruit kwamen. Dus het kwam niet als verrassing / teleurstelling / herkenning. Er waren veel keuzen, dat is prettig. Er moet wel duidelijk worden gemaakt dat er maar een top 5 van je verwacht wordt. Ik mis een kleine omschrijving, tagline van de stad. Trouwens, wat doen jullie met 3544 backpackers die willen rondreizen? Opm bij vr 9: Scherm is lang door de lange reeks opties. Minder overzicht. Bij de ordening heb ik niet gekeken naar de linkerkant en enkel mijn eigen voorkeur laten spelen. (Vandaar dat ik ook een tagline mis, ik heb niet naar de grafiek van de stad gekeken. In dit geval zal tekst mij meer aanspreken.) volgens mij kloppen de cijfers in het grafiek niet....ik had verwacht dat omgeving het beste is voor de groep en niet 4119 atmosfeer 4743 Geen verdere opmerkingen 4790 Ook hier naast de procenten aanduiding eventueel nummers toepassen. 5338 vraag 12: Het is me niet opgevallen hoe ik deze karakteristieken kon bekijken kan zijn dat er teveel informatie wordt gegeven aan de gebruiker, niet iedereen begrijpt meteen grafieken en kunnen er 5538 ook meteen rekening mee houden bij het maken van keuze's. 5551 Voor bepaalde ebruikers kunnen grafieken en staafbalken intimiderend zijn. 5794 vraag 1 t/m 4 zijn praktisch hetzelfde Zo mogelijk steden op een scherm Ik vond het bij dit onderdeel niet overzichtelijk om te slepen, omdat je niet alle steden tegelijkertijd kunt zien (het is een 5832 lange lijst). - te veel steden. Het orden gaat dan erg moeilijk. - Grafiek is even wennen. Niet helemaal duidelijk waarom bepaalde 6064 dingen toch nog boven staan. - De cijfers van de groep kloppen volgens mij niet helemaal. Ze zijn allemaal hetzelfde...ik had verwacht dat ze per persoon verschillend zou zijn 6145 toelichting op het grafiek zou wel handig zijn verrassend uitkomst van de steden..wel nice..had eingelijk geen verwachtingen..maar bepaalde steden zou ik nooit erop 6211 gekomen zijn 6678 Ik had meer shopping steden verwacht..zoals newyork en parijs..hmmzzz 6987 Zorg er voor dat tijdens het naar beneden scrollen voor de lagere steden, dat de info/grafiekbox in het scherm blijft! De grafiek geeft mij niet heel veel informatie over de stad. Ook dat de personelijke lijn altijd onder de stad's lijn zit geeft het idee dat de stad niet bij jouw voorkeuren past. Meer informatie over de stad zelf en de toeristische trekpleisters zou 7013 leuk zijn geweest. Er moest ook veel omhoog en omlaag gescrolled worden. Subordening in groepen in plaats van 1 lange lijst had misschien ook intressant geweest vooral omdat je nu sommige steden helemaal naar beneden moest slepen. Waar is de prijs opgebaseerd? Vlieg je altijd met de goedkoopste? Zelf vond ik het moeilijk om een keuze te maken, ik heb eigenlijk al bepaalde vooroordelen en moest dus bij de top 10-20 eerst kijken waarom de steden zo hoog in de lijst stonden. Daardoor is enigszins mijn mening wel veranderd en heb ik 7033 uiteindelijk toch een paar steden gekozen die enigszins gelinkt zijn aan mijn top 3 favoriete categorieen. De prijs speelde ook een rol eigenlijk, maar daar was in het begin geen optie voor. Ik vond het in het begin even heel veel informatie maar later werd het duidelijker. Ik had de knop voor meer details tonen 7064 van de stad niet echt gevonden, misschien anders aangeven of automatisch gewoon weergeven. - de lijst was best lang en dit zorgden er voor dat je soms niet in een keer een keuze van boven naar beneden kon slepen, 7283 of dat dit wat langer duurden. - eigenlijk mis je nog een beschrijving in verhaalvorm van de stad, met wat publiekstrekkers en algemene informatie (weer, cultuur, drukte, seizoen, etc...) om je meer inzicht te geven in de stad. Nu kan je namelijk
167
nog op veel steden heel instinctief reageren door voorkennis. Op zich een handig systeem, alleen is het slepen van de steden soms onhandig doordat er geen genummerde rankering is. De uitkomst waren ook niet de steden die ik en de groep verwacht hadden, sommige steden waar we allemaal graag heen 7354 wilden stonden er niet tussen, zoals San Fransisco. Van de steden die er wel uitkwamen was het merendeel in de goede richting van wat iedereen leuk vind. Ik vond het wel lastig om in de eerste via het sleepsysteem exact aan te geven wat er belangrijk is aan een stad. Het gaat vaak ook om het gevoel bij een stad. Mijn opmerkingen hierover staan ook al bij het vorige blad. Aanvullend nog: * sommige steden die me interessant leken 7433 stonden er niet bij. Desondanks stonden er wel opties bij waar ik ook heen zou willen gaan. Dus zo erg is dat niet. * het was wel een lange lijst om te ordenen. op den duur denk je bij de 'onderkant' van, het is wel goed zo. De grafiek is erg abstract, het is nu onduidelijk wat de waarden op de verticale as zijn. Percentages van wat onze voorkeur? 7508 De stadskarakteristieken kunnen nog uitgebreider. Waarom niet meer (visuele) informatie? Zo kon ik me moeilijk een indruk vormen van Kyoto, wat de uiteindelijke keuze beinvloed - zowel positief als negatief. Groep- en Stadspreferenties (gemiddeldes) zijn niet helemaal duidelijk. Betekent alles hoog jusit goed? of moeten de 7604 grafieken juist matchen?
168
20.8
Evaluation Step “Clarify”: Descriptive Statistics N
Range
Minimum
Maximum
Statistic
Statistic
Statistic
Statistic
Mean Statistic
Std. Error
Std. Deviation
Variance
Statistic
Statistic
Objective
120
6
1
7
5.65
.122
1.333
1.776
Presentation
120
6
1
7
5.75
.099
1.087
1.181
Reveal
120
6
1
7
5.86
.092
1.007
1.013
Awareness
120
6
1
7
5.36
.132
1.442
2.081
Voting
120
5
2
7
5.65
.099
1.082
1.171
Decision
120
6
1
7
4.14
.169
1.848
3.417
Effort
120
5
2
7
5.76
.117
1.283
1.647
Expressiveness
120
5
2
7
5.64
.106
1.165
1.358
Method
120
5
2
7
5.79
.108
1.187
1.410
Valid N (listwise)
120
Objective
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
2
1.7
1.7
1.7
2
2
1.7
1.7
3.3
3
6
5.0
5.0
8.3
4
9
7.5
7.5
15.8
5
21
17.5
17.5
33.3
6
47
39.2
39.2
72.5
7
33
27.5
27.5
100.0
Total
120
100.0
100.0
Frequency
Percent
Valid Percent
Cumulative Percent
1
1
.8
.8
.8
2
2
1.7
1.7
2.5
4
7
5.8
5.8
8.3
5
33
27.5
27.5
35.8
6
47
39.2
39.2
75.0
7
30
25.0
25.0
100.0
Total
120
100.0
100.0
Presentation
Valid
169
Reveal
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
1
.8
.8
.8
3
3
2.5
2.5
3.3
4
4
3.3
3.3
6.7
5
25
20.8
20.8
27.5
6
57
47.5
47.5
75.0
7
30
25.0
25.0
100.0
Total
120
100.0
100.0
Frequency
Percent
Valid Percent
Cumulative Percent
1
3
2.5
2.5
2.5
2
6
5.0
5.0
7.5
3
2
1.7
1.7
9.2
4
14
11.7
11.7
20.8
5
28
23.3
23.3
44.2
6
43
35.8
35.8
80.0
7
24
20.0
20.0
100.0
Total
120
100.0
100.0
Frequency
Percent
Valid Percent
Cumulative Percent
2
1
.8
.8
.8
3
6
5.0
5.0
5.8
4
8
6.7
6.7
12.5
5
28
23.3
23.3
35.8
6
53
44.2
44.2
80.0
7
24
20.0
20.0
100.0
Total
120
100.0
100.0
Awareness
Valid
Voting
Valid
170
Decision
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
14
11.7
11.7
11.7
2
15
12.5
12.5
24.2
3
14
11.7
11.7
35.8
4
20
16.7
16.7
52.5
5
18
15.0
15.0
67.5
6
32
26.7
26.7
94.2
7
7
5.8
5.8
100.0
Total
120
100.0
100.0
Frequency
Percent
Valid Percent
Cumulative Percent
2
3
2.5
2.5
2.5
3
8
6.7
6.7
9.2
4
6
5.0
5.0
14.2
5
20
16.7
16.7
30.8
6
44
36.7
36.7
67.5
7
39
32.5
32.5
100.0
Total
120
100.0
100.0
Frequency
Percent
Valid Percent
Cumulative Percent
2
5
4.2
4.2
4.2
3
1
.8
.8
5.0
4
9
7.5
7.5
12.5
5
27
22.5
22.5
35.0
6
53
44.2
44.2
79.2
7
25
20.8
20.8
100.0
Total
120
100.0
100.0
Effort
Valid
Expressiveness
Valid
171
Method
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
2
4
3.3
3.3
3.3
3
2
1.7
1.7
5.0
4
10
8.3
8.3
13.3
5
17
14.2
14.2
27.5
6
53
44.2
44.2
71.7
7
34
28.3
28.3
100.0
Total
120
100.0
100.0
id
remark De sterren werken goed. Lage waardering met de twee opties heeft zin als het alleen oppopt als je een lage waardering 208 geeft. Het feit dat het er al staat voordat je een waardering geeft suggereer je al dat de stad een lage waardering KAN hebben. Dus niet neutraal. Waarom alleen maar de lage waardering toelichten. Dat begreep ik niet. Ten eerste weet ik niet waarom je dat alleen 329 vraag. Ten tweede vond ik dat er te weinig voorkeuren waren om de lage waardering toe te lichten. En tenslotte vind ik dat er dan ook een toelichting moet zijn een hoge waardering. Bij waarom lage waardering, kon je alleen maar kiezen tussen te duur en stad is niet leuk. Dit zijn naar mijn mening te 361 weinig opties. Daarnaast bestaat er ook nog de mogelijkheid om een opmerking te plaatsen. Dit vond ik een beetje verwarrend. -mmisschien meer opties toevoegen die ook staan bij: te duur vind de stad niet leuk zoals: - daar ben ik al geweest - te 624 grote afstand - te dichtbij Bij vraag 12 vind ik het een redelijk systeem, het nadeel is dat een stad compleet kan verdwijnen als iemand het expres 657 laagt scoort. Op zich eerlijk als men niet naar die stad wil, maar elk groepslid kan anders ranken. De een vult bijvoorbeeld alles eerlijk in, terwijl een ander steden die de persoon redelijk vindt lager scoort dan dat die dat in werkelijk vindt, 810 Vraag 11: dit had ik niet gebruikt. Kan wellicht wat duidelijker worden aangegeven dat deze mogelijkheid er is. 1004 ik vond kyoto leuk...die heb ik hoog gewaardeerd. Maar tegelijkertijd vond ik het te duur...wat is nu de bedoeling? 1119 Redeneringen van waardering kunnen uitgebreider en mogelijk kunnen deze ook verwerkt worden in een grafiek. Het waarderen van een stad is een beetje lastig als je niet veel van een stad weet, aangezien je gegenereerde steden krijgt 1324 waar je misschien nog nooit naar gekeken hebt. Dit kan er wel weer voor zorgen dat de volgende stap niet goed gaat. vakje met "Waarom een lage waardering?" ... maar ik had geen lage waardering. (misschien eerst raten en dan pas ruimte 1339 voor comment) De ´ga verder´ knop was niet in beeld :) helemaal naar beneden scrollen... Ipv waarderen met sterretjes, het zou makkelijker zijn om het te waarderen op een meer aansprekende manier: "Wil 1365 helemaal niet daarnaar toe gaan", "Wil absolute daarnaar toe gaan", "Liever niet" etc etc... het is makkelijk om adv de overgebleven lijst als groep een keuze te maken. je kan dan makkelijk een compromis sluiten 1570 door het aantal sterren per stad, per persoon te bekijken. de kolom waarin je aan moet geven waarom je een lage waardering hebt gegeven, vind ik persoonlijk niet nodig. Wel zou je 1614 het verplicht kunnen maken dat mensen in het comment field zeggen waarom ze een lage rating hebben gegeven. Of meerdere opties in deze kolom toevoegen zou voor mij ook nog kunnen werken Misschien leuk om bij het plusje nog aditionele informatie toe te voegen. Door iets dieper op achtergrond info over een stad 1960 in te gaan kan je beter beargumenteren waarom je voor een bepaalde stad hebt gekozen. Ook kunnen mensen dmv extra info hun mening wijzigen, wat het keuzeproces bij de laatste stap wat makkelijker kan maken. 1972 't blijft heel algemeen en oppervlakkig 2156 te weinig opties om redenen van lage score aan te geven. wel goed dmv sterren, wordt dan ook vaak gebruikt op internet. 2233 ik heb het "+" knopje niet gezien en dus ook niet gebruikt 2255 Het was mij niet opgevallen dat je informatie kon ophalen over de groep en de stad 2276 de "+" knop is voor mij overbodig, omdat je sowieso in het blokje kunt typen. De opties om aan te geven waarom een stad een lage waardering krijgt, zijn te weinig. Dit kan vervangen worden met een 2452 venstertje dat ontstaat wanneer je een stad uberhaupt laag waardeert, want voor steden die je hoog waardeert heb je het niet nodig. 3101 Misschien iets meer opties (de 5 belangrijkste) voor "waarom een lage waardering" 3356 vraag 13 is niet duidelijk - Ik mis de keuze mogelijkheid om ook qua prijs en geografie de wensen te uiten - In hoeverre ga je ook om met verschillen 3524 in gewichten tussen rubrieken (stel: ik vind atmosfeer en cultuur bijna gelijk terwijl sportiviteit als 3 totaal minder van belang is)
172
Het was me bij dit scherm niet meteen duidelijk dat je op de + moest drukken om meer informatie van deze stad te zien. De grafiek met de voorkeuren voor de activiteiten (van de groep) vind ik wel interessant en een goede samenvatting, maar hier vind ik het niet echt toegevoegde waarde hebben. Ik zou hier eerder meer informatie over de steden willen hebben 3584 zoals bijvoorbeeld wat je allemaal kunt doen. Je ziet hier wel allemaal percentages enzo van de verschillende activiteiten, maar ik zie liever wat voorbeelden. De grafiek met de groepsinfo zou ik liever aan het einde zien of als iets dat je op kan vragen als je het wilt zien. - de waarom een lage waardering-checkbox zou alleen zichtbaar moeten zijn bij een daadwerkelijk lage sterrenwaardering, 3594 lijkt me - vr 13 - voornamelijk het stukje tekst vanelkaar zien maakt 't persoonlijker. Misschien al aangeven in de toelichting van deze pagina dat je deze tekst van elkaar gaat terugzien. Veel van "mijn" voorkeuren zijn verdwenen. Maar er zijn nwe bijgekomen, waar ik toch wel verrast over ben. Niet mijn eerste keus, maar misschien wel leuk om met de groep te gaan. Verder speelde geld in deze gesimuleerde situatie nog geen serieuze rol. Ik heb het wel enigszins meegenomen. Nieuwsgierig vraagje: Waarom hebben jullie gekozen voor 3676 beargumenteren van een lage waardering? En zijn er maar 2 opties. Een andere zou kunnen zijn: Ben er al geweest. De kans is groot. Voor mij was dat 2 steden van de 5. Vraag 11: Ik heb het plusje in eerste instantie niet gezien, niet gebruik van gemaakt. Maar ik vind het wel handig. Nu ik het gezien heb. 3814 MEER STEDEN 4154 Grafiek, groeps preferentiemodel komt niet goed overeen met de scores van de groepsleden. 5365 Het 'plusje' zou ik veranderen in 'info' 5395 vraag 13: + was me niet opgevallen, waardoor ik er niets mee heb gedaan 5427 bij vraag 11: Wel nuttig, maar niet gezien... "waarom een lage waardering" staat er standaard bij, dit zou aangepast moeten worden als ik juist een hoge waardering 5564 geef aan een stad. Meer opties voor waarom een lage waardering zou gepast zijn. Maar dit kan je ook n het oopmerkingen veld kwijt. + knopje 5591 was handig, maar een info-teken is misschien handiger. Maar dat past niet in het design. 5904 In de top 4 stonden 2 steden waar ik al geweest ben, dus die had ik er eigenlijk niet in willen hebben. 7061 Het plusje viel mij niet zo op ook niet wat het zou doen was me niet helemaal duidelijk. Ik voelde mij genoodzaakt om telkens de vraag 'waarom een lage waardering' in te vullen, terwijl dit niet nodig is. Ook te 7112 weinig opties en plotseling komt de optie ' prijs' in beeld, terwijl dat hiervoor nooit ter sprake was. 7136 Vakje met waarom eenlage waardering is een beetje vaag, het staat er altijd terwijl je een hoge waardering geeft. -mmisschien meer opties toevoegen die ook staan bij: te duur vind de stad niet leuk zoals: - daar ben ik al geweest - te 7299 grote afstand - te dichtbij Bij vraag 12 vind ik het een redelijk systeem, het nadeel is dat een stad compleet kan verdwijnen als iemand het expres 7372 laagt scoort. Op zich eerlijk als men niet naar die stad wil, maar elk groepslid kan anders ranken. De een vult bijvoorbeeld alles eerlijk in, terwijl een ander steden die de persoon redelijk vindt lager scoort dan dat die dat in werkelijk vindt, 7454 Vraag 11: dit had ik niet gebruikt. Kan wellicht wat duidelijker worden aangegeven dat deze mogelijkheid er is. Het is begrijpelijk dat naarmate meer gebruik, ik een stad waarnaar ik graag heen wil expres 5 kan raten, en de rest 1 ster 7626 om te abusen (achtergrond in IT). 7636 Refreshment van steden met zelfde score zorgt voor verwarring
173
20.9 Evaluation Step “Evaluate”: Descriptive Statistics N
Range
Minimum
Maximum
Statistic
Statistic
Statistic
Statistic
Mean Statistic
Std. Error
Std. Deviation
Variance
Statistic
Statistic
Expressiveness
120
6
1
7
5.29
.117
1.279
1.637
Method
120
6
1
7
5.30
.128
1.400
1.960
Objective
120
6
1
7
5.72
.093
1.022
1.045
Presentation
120
5
2
7
5.67
.089
.973
.947
Information
120
5
2
7
5.02
.131
1.432
2.050
Reveal
120
6
1
7
5.71
.098
1.072
1.150
Awareness
120
6
1
7
5.39
.129
1.416
2.005
New set
120
6
1
7
5.02
.154
1.690
2.857
Decision Support
120
6
1
7
5.13
.118
1.289
1.663
Effort
120
6
1
7
5.42
.133
1.459
2.127
Valid N (listwise)
120
Expressiveness
Valid
Frequency
Percent
Cumulative Valid Percent Percent
1
1
.8
.8
.8
2
7
5.8
5.8
6.7
3
2
1.7
1.7
8.3
4
11
9.2
9.2
17.5
5
40
33.3
33.3
50.8
6
43
35.8
35.8
86.7
7
16
13.3
13.3
100.0
Total
120
100.0
100.0
Frequency
Percent
Cumulative Valid Percent Percent
1
1
.8
.8
.8
2
6
5.0
5.0
5.8
3
10
8.3
8.3
14.2
4
9
7.5
7.5
21.7
5
26
21.7
21.7
43.3
6
49
40.8
40.8
84.2
7
19
15.8
15.8
100.0
Total
120
100.0
100.0
Method
Valid
174
Objective
Valid
Frequency
Percent
Cumulative Valid Percent Percent
1
1
.8
.8
.8
3
2
1.7
1.7
2.5
4
10
8.3
8.3
10.8
5
27
22.5
22.5
33.3
6
56
46.7
46.7
80.0
7
24
20.0
20.0
100.0
Total
120
100.0
100.0
Frequency
Percent
Cumulative Valid Percent Percent
2
1
.8
.8
.8
3
1
.8
.8
1.7
4
11
9.2
9.2
10.8
5
34
28.3
28.3
39.2
6
50
41.7
41.7
80.8
7
23
19.2
19.2
100.0
Total
120
100.0
100.0
Frequency
Percent
Cumulative Valid Percent Percent
2
12
10.0
10.0
10.0
3
7
5.8
5.8
15.8
4
17
14.2
14.2
30.0
5
26
21.7
21.7
51.7
6
47
39.2
39.2
90.8
7
11
9.2
9.2
100.0
Total
120
100.0
100.0
Presentation
Valid
Information
Valid
175
Reveal
Valid
Frequency
Percent
Cumulative Valid Percent Percent
1
1
.8
.8
.8
3
5
4.2
4.2
5.0
4
5
4.2
4.2
9.2
5
31
25.8
25.8
35.0
6
52
43.3
43.3
78.3
7
26
21.7
21.7
100.0
Total
120
100.0
100.0
Frequency
Percent
Cumulative Valid Percent Percent
1
2
1.7
1.7
1.7
2
4
3.3
3.3
5.0
3
6
5.0
5.0
10.0
4
17
14.2
14.2
24.2
5
21
17.5
17.5
41.7
6
44
36.7
36.7
78.3
7
26
21.7
21.7
100.0
Total
120
100.0
100.0
Frequency
Percent
Cumulative Valid Percent Percent
1
6
5.0
5.0
5.0
2
9
7.5
7.5
12.5
3
7
5.8
5.8
18.3
4
15
12.5
12.5
30.8
5
20
16.7
16.7
47.5
6
44
36.7
36.7
84.2
7
19
15.8
15.8
100.0
Total
120
100.0
100.0
Awareness
Valid
New set
Valid
176
Decision Support
Valid
Frequency
Percent
Cumulative Valid Percent Percent
1
4
3.3
3.3
3.3
2
3
2.5
2.5
5.8
3
1
.8
.8
6.7
4
21
17.5
17.5
24.2
5
37
30.8
30.8
55.0
6
44
36.7
36.7
91.7
7
10
8.3
8.3
100.0
Total
120
100.0
100.0
Frequency
Percent
Cumulative Valid Percent Percent
1
2
1.7
1.7
1.7
2
3
2.5
2.5
4.2
3
8
6.7
6.7
10.8
4
18
15.0
15.0
25.8
5
20
16.7
16.7
42.5
6
37
30.8
30.8
73.3
7
32
26.7
26.7
100.0
Total
120
100.0
100.0
Effort
Valid
id
remark het discussie gedeelte kan beter geleid worden. Misschien kan je een waarom niet ding invoeren. Het onderhandel proces 248 beter leiden. Een budget range bijvoorbeeld zou ik essentieel vinden bijv. Meer limiteringen. Mijn nummer een stad kwam niet naar voren, terwijl de gegeven sterren op gemiddeld 4 1/2 sterren uitkwam. Meer opties 281 geven tot discussie over de vakantiebestemming. Ik miste een functie om met mijn groepsleden direct te communiceren. Zodat we niet in real life hoefden af te spreken, 389 want niet iedereen heeft tijd om dat te doen. In het begin van de sessie wordt niet toegelicht dat één iemand de stad gaat kiezen. 517 ipv " Tip" -> " overige aanbevelingen" tov vraag 13: als er meer nieuwe aanbevelingen zouden komen, dan zou het de groep misschien weer meer aan het 633 twijfelen brengen; zo zouden ze niet bij hun primaire wensen. - nadeel is dat het onderhandelen lijvelijk gebeurt. Wellicht kan het ook geinteregeert worden met Skype/msn/facebook applicatie. Als groep waren wij hier snel uit vanwege onze persoonlijke voorkeur voor 1 stad. Persoonlijk vond ik er maar 1 leuke stad 711 tussen staan waar ik graag heen wilde welke dan ook gekozen is. Veel van mijn steden zijn bij deze stad afgevallen, zo ook mijn favoriet. Dat is niet leuk om te zien en kan denk ik wel een behoorlijke discussie opleveren. Wellicht kunnen er meer dan zes steden over blijven, bijvoorbeeld tien? De discussie kwam niet bijzonder op gang bij ons. Door meer informatie te verschaffen over de vakantiebestemmingen valt er meer om over te discussieren. Vraag 9 lijkt hier 778 overbodig. De waarderingen per individu kunnen nog overzichtelijker denk ik. Nu is dat enkel met een curve en staafdiagram aangegeven. Misschien dat de deelcategorieen ook inzichtelijk kunnen worden gemaakt? Vraag 1: waardering is wel gegeven. commentaar niet. We hebben snel onderhandeld omdat iedereen eens was met 831 nummer 1. Daardoor is minder aandacht geschonken aan de overgebleven opties. Vraag 13: de tips sloten niet aan bij mijn preferenties. Hier hebben we ook geen aandacht aan geschonken. ik dacht dat de tips reclame was. De bedoeling was mij niet helemaal duidelijk. In deze stap zouden we dus blijkbaar bij 1048 elkaar moeten komen. Misschien is het handig om hiervoor een chat functie te gaan maken. Indien we met chatten er niet uit komen dan kunnen we pas echt bij elkaar komen 1076 Een optie om ook te kunnen onderhandelen zonder fysiek bij elkaar te zijn.
177
1137 Een chat functie zal heel handig zijn om geheel virtueel af te kunnen spreken. de tripeasy.tip steden waren gewoon steden die we in de eerdere stappen weggelaten hadden. Als het ging om andere 1269 steden dan zou het nog een toegevoegde waarde hebben. 1665 handig om een goede discussie te voeren op basis van de persoonlijke voorkeuren 1684 Misschien als een additionele gadget een chat functie in deze fase bouwen ?? detailinfo zie ik als gebruiker in deze fase dan niet als nodig, eigenlijk heb je dan al een keuze gemaakt voor jezelf. De 2199 discussie is dan spannend, wie krijgt zijn of haar zin ! Uiteindelijk hoef je niet uit een hele waslijst een keus te maken. Dat is prettig. En de top 4 geeft goed aan waar de meeste 2328 waardering naar uitgaat van de groep. Alleen mag er dus nog wel duidelijker worden aangegeven dat er extra informatie te vinden is over de groep en de stad. Want dat is wel fijn om te gebruiken bij het maken van een keuze. "Ik vind dat ik over voldoende informatie beschik om met de groep te gaan discusseren over het te selecteren stad" Antw: ja 2515 of nee, maar niet dankzij de site, meer vanuit eigen ervaring. Dus de site is niet echt relevant geweest om te informeren over een stad. Als er binnen de discussie iemand is die zeer koppig en dominant is, kan de juiste werking van dit programma teniet worden 3290 gedaan. 3384 nieuwe aanbevelingen zijn wel handig..maar dan moet het wel steden zijn die niet eerder voorgekomen zijn Plaatje overigens bovenaan de pagina blijft staan op onderdeel 7. Geldt ook voor categorie 2, 3 en 4 en waarschijnlijk ook 3570 voor 5 t/m 8 Zoals ik al eerder zei, mis ik wat informatie specifiek over de steden en wat foto's of plaatjes van de stad zelf. We hebben 3637 heel even gediscussieerd, uiteindelijk hebben we gekozen voor een stad die niet in de top lijst stonden. vraag 10: Je zoekt wat extra info om een betere keuze te kunnen maken. Ik heb een gevoel dat er een compromis is gesloten. Dat uiteindelijk niemand echt echt de beste keuze heeft gekregen. Voor mij geld: beter aantal mensen die ergens 3718 erg graag heen willen dan een slappe compromis. Maar dat is geheel afhankelijk van de discussie die gevoerd wordt. Door zo'n systeem wordt wel sneller compromissen gesloten, heb ik het idee. Wat ik mooi zou vinden als je eenmaal een keuze hebt gemaakt. Dat je daarna voor dezelfde prijsklasse een aantal 4347 alternatieven hebt. 4871 Argmuntatie waarom een stad beter past mis ik een beetje. (Dmv concrete voorbeelden) 5443 + is me niet opgevallen "wij gaan naar deze stad", is niet meteen duidelijk dat alleen degene die de trip organiseert dit kan bepalen voor iedereen. 5622 het leek mij eerst een stemknopje, waarmee iedereen zijn of haar laatste stem kon geven. Het woord EEN in de zin kan beter streepjes erboven hebben, accent ague? Vraag 12 is pijltje, niet +-je. Misschien moet het 5659 in het begin duidelijk zijn dat je keuzes niet prive zijn maar iedereen het kan zien. De triptips zijn volgens mij niet echt nieuw...ik had ze zelf net laag gewaardeerd. Een beetje nutteloos om dingen dat ik 6092 weggestreept hebt weer als tip te geven. Wel zou ik het waarderen als er tips komen die ik niet eerder gezien heb 7085 vraag 1: Je ziet wel welke persoon een reis niet leuk vind, maar je ziet niet waarom, zou een handige optie zijn. Niet iedereen heeft een opmerking ingevoerd misschien omdat het niet prominent geeist werd in de vorige stap. Verder 7091 voor de externe discussie heeft de site niet zo veel extra waarde zeker omdat de discussie niet achter de computer zal plaats vinden. Iemand ging voor de sport, maar zijn eerste keuze toonde aan dat de stad hier juist niet voor bedoeld was. Dit leverde een 7197 (kleine) discussie op. Je ziet dus dat ik de grafieken gebruikte om te kijken of er een compromis gesloten kon worden. tov vraag 13: als er meer nieuwe aanbevelingen zouden komen, dan zou het de groep misschien weer meer aan het 7308 twijfelen brengen; zo zouden ze niet bij hun primaire wensen. - nadeel is dat het onderhandelen lijvelijk gebeurt. Wellicht kan het ook geinteregeert worden met Skype/msn/facebook applicatie. Als groep waren wij hier snel uit vanwege onze persoonlijke voorkeur voor 1 stad. Persoonlijk vond ik er maar 1 leuke stad 7385 tussen staan waar ik graag heen wilde welke dan ook gekozen is. Veel van mijn steden zijn bij deze stad afgevallen, zo ook mijn favoriet. Dat is niet leuk om te zien en kan denk ik wel een behoorlijke discussie opleveren. Vraag 1: waardering is wel gegeven. commentaar niet. We hebben snel onderhandeld omdat iedereen eens was met 7463 nummer 1. Daardoor is minder aandacht geschonken aan de overgebleven opties. Vraag 13: de tips sloten niet aan bij mijn preferenties. Hier hebben we ook geen aandacht aan geschonken. Wellicht kunnen er meer dan zes steden over blijven, bijvoorbeeld tien? De discussie kwam niet bijzonder op gang bij ons. Door meer informatie te verschaffen over de vakantiebestemmingen valt er meer om over te discussieren. Vraag 9 lijkt hier 7535 overbodig. De waarderingen per individu kunnen nog overzichtelijker denk ik. Nu is dat enkel met een curve en staafdiagram aangegeven. Misschien dat de deelcategorieen ook inzichtelijk kunnen worden gemaakt?
178
20.10
Evaluation Trip.Easy GDSS in general: Descriptive Statistics N
Range
Minimum
Maximum
Statistic
Statistic
Statistic
Statistic
Mean Statistic
Std. Error
Std. Deviation
Variance
Statistic
Statistic
Effective
120
4
3
7
5.76
.089
.979
.958
Handy
120
4
3
7
5.79
.082
.897
.805
Fair process
120
6
1
7
5.62
.127
1.391
1.934
Influence
120
4
3
7
5.65
.102
1.120
1.255
Easy
120
5
2
7
5.73
.104
1.136
1.290
Objective
120
5
2
7
5.39
.131
1.434
2.055
Efficient
120
5
2
7
5.94
.099
1.079
1.165
Effort
120
6
1
7
5.04
.152
1.662
2.763
Valid N (listwise)
120
Effective
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
3
3
2.5
2.5
2.5
4
8
6.7
6.7
9.2
5
33
27.5
27.5
36.7
6
47
39.2
39.2
75.8
7
29
24.2
24.2
100.0
Total
120
100.0
100.0
Frequency
Percent
Valid Percent
Cumulative Percent
3
1
.8
.8
.8
4
7
5.8
5.8
6.7
5
36
30.0
30.0
36.7
6
48
40.0
40.0
76.7
7
28
23.3
23.3
100.0
Total
120
100.0
100.0
Handy
Valid
179
Fair process
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
1
1
.8
.8
.8
2
6
5.0
5.0
5.8
3
3
2.5
2.5
8.3
4
9
7.5
7.5
15.8
5
26
21.7
21.7
37.5
6
38
31.7
31.7
69.2
7
37
30.8
30.8
100.0
Total
120
100.0
100.0
Frequency
Percent
Valid Percent
Cumulative Percent
3
6
5.0
5.0
5.0
4
14
11.7
11.7
16.7
5
25
20.8
20.8
37.5
6
46
38.3
38.3
75.8
7
29
24.2
24.2
100.0
Total
120
100.0
100.0
Frequency
Percent
Valid Percent
Cumulative Percent
2
2
1.7
1.7
1.7
3
3
2.5
2.5
4.2
4
13
10.8
10.8
15.0
5
20
16.7
16.7
31.7
6
51
42.5
42.5
74.2
7
31
25.8
25.8
100.0
Total
120
100.0
100.0
Influence
Valid
Easy
Valid
180
Objective
Valid
Frequency
Percent
Valid Percent
Cumulative Percent
2
7
5.8
5.8
5.8
3
12
10.0
10.0
15.8
4
4
3.3
3.3
19.2
5
27
22.5
22.5
41.7
6
44
36.7
36.7
78.3
7
26
21.7
21.7
100.0
Total
120
100.0
100.0
Frequency
Percent
Cumulative Valid Percent Percent
2
1
.8
.8
.8
3
5
4.2
4.2
5.0
4
6
5.0
5.0
10.0
5
15
12.5
12.5
22.5
6
54
45.0
45.0
67.5
7
39
32.5
32.5
100.0
Total
120
100.0
100.0
Frequency
Percent
Valid Percent
Cumulative Percent
1
4
3.3
3.3
3.3
2
10
8.3
8.3
11.7
3
11
9.2
9.2
20.8
4
8
6.7
6.7
27.5
5
27
22.5
22.5
50.0
6
39
32.5
32.5
82.5
7
21
17.5
17.5
100.0
Total
120
100.0
100.0
Efficient
Valid
Effort
Valid
181
id Remark - Cat 8 question 2 377 Ja, makkelijker. Het systeem heeft een top 3 gesteld, waardoor de keuze vrij makkelijk was. Ik vind het vreemd dat er uiteindelijk een groepsleider wordt aangewezen aangezien het hele proces democratisch is 398 opgezet. 461 469 Ja, want dat hoeven we niet na te denken over de bestemming. Ik hoef alleen aan te geven wat ik wil dat vind ik 477 makkelijker. De beperkte keuze van de bestemmingen zijn eveneens prettig, want dat maakt het discussieren makkelijker. Debatteren wordt makkelijker omdat het systeem al een voorkeur van de groep geeft. Hierdoor is het moeilijker voor 542 iemand die goed kan debatteren en dominanter is om een bestemming te bepalen terwijl deze in de groep minder goed ligt. Het systeem maakt de groepsvoorkeuren duidelijk. 558 Dankzij het systeem zijn er richtlijnen voor de discussies. Dit kan voor veel mensen een goed initiatie zijn. 562 het was moeilijker omdat je voor jezelf al helemaal voor je ziet naar welke van de overgebleven steden je graag zou willen 750 gaan; als echter dan lijkt dat je medereisgenoten hier niet naar toe zouden willen dan is dat een ' domper'. Ik vond het makkelijker om de beslissing te maken omdat de opties verkleind waren, wat de discussie makkelijker maakte. 862 In geval van onze groep hier, was er maar één echte optie waar we allemaal heen wilde dus de beslissing was zelf gemaakt. Zeker makkelijker, omdat je in de selctie van de vakantiebestemming al een stap verder bent en iedereen voor zichzelf 870 duidelijk heeft waar elke stad aan moet voldoen. 875 Iedereen was het eens en tevreden met de opgegeven nummer 1 van het systeem. Dus er was weinig discussie nodig. 1038 Het helpt je verder in het proces. Het beslissingsproces wordt naar een later stadium geplaatst. 1129 Ik vond het makkelijker. Zonder systeem zou het langzamer gaan Door een beperkte keuze, meningen en waardering kan je goed zien waa de interesses liggen. Zo kan men rekening houden 1162 met anderen. 1179 Discussie mogelijkheden zijn enigszins beperkt en informatie van steden kunnen uitgebreider 1343 het is meer gestructueerd. Je hoeft nu alleen maar te beslissen over een set van 4 steden ipv ontelbaar hoeveelheid steden 1455 Het aantal steden waarover je moet debatteren is beperkt tot 4 steden, alle andere steden zijn al afgevallen. 1460 Ik denk dat je altijd wel iets meer te zeggen hebt dat een kort zinnetje met wat checkboxjes 1465 Het wordt makkelijker omdat het duidelijk is welke aspecten gedebatterd moet worden. Je hebt de cijfers en argumentaties voor je mbt de steden en weet dus hoe 1 ieder er over denkt, waardoor je makkelijker 1754 een afweging kunt maken over welke stad het uiteindelijk moet worden. 1759 Je kunt nu gerichter discussiëren. Er zijn 4 opties waarvan minimaal 2 of 3 voor iedereen wel min of meer acceptabel zijn. Het discussieren is makkelijker omdat je dat nu heel gericht kan doen. Het is mogelijk om bijv ahv de grafiek met 1766 persoonlijke voorkeuren iemand te overtuigen van de kwaliteiten van een stad. De tool maakt het voorschot voor de discussie. Er ongelofelijk veel steden op de wereld waarover je zou kunnen debateren. 1769 De tool helpt bij de voor selectie waarin iedereen zich in beginsel al zou moeten kunnen vinden na de selecties. 1986 je hebt nog maar een klijn keuze velt dus dan is deze sneller gemaakt Uiteindelijk ben je het nooit eens, maar door zwart op wit resultaten voor je neus te hebben is het maken van een keuze 2062 gemakkelijker. Je hebt toch het gevoel dat dit de beste opties zijn voor de groep. 2066 Zoals hierboven uitgelegd, ging dit VEEL makkelijker voor 'n eerste orientatie is 't wellicht handig, maar, nogmaals, veel te algemeen en de dicussie is ook niet inhoudelijk op 2077 gang gekomen makkelijker omdat je nu op mbv een schaal tot een uiteindelijke keuze komt waarbij iedereen evenveel in de melk te 2356 brokkelen had. makkelijk vanwege het menu aan bestemmingen, je hoeft niet al pratende eerst een bestemming te noemen. Het oogt wat 2378 objectiever. Omdat eigenlijk al duidelijk was waar ieders voorkeur naaruit ging bleef er niet veel meer over om over te debateren. Het 2386 programma zorgde ervoor dat een discussie niet meer nodig was. keuzes maken is moeilijk en het geeft je een selectie die al aardig past om uit te kiezen. dit versnelt het proces zeker. 2392 omdat je op elkaar moet wachten ben je sneller geneigd om de stap uit te voeren zodat je verder kunt. er komt dus eerder een beslissing. prima! Het maakt het makkelijker, omdat het aantal steden minder is dan wanneer je het zonder het systeem zou doen. Je gaat 2536 pas aan andere steden denken als het grootste gedeelte niet mee eens, wat bij ons niet het geval was. Soms moet je gewoon dingen voor je krijgen, anders begint de discussie op een punt nul en kan het oneindig doorgaan. Er 2546 worden een aantal steden als suggestie opgegeven zodat er tenminste een houvast is en het gesprek minder breed is. Vooral met een hele grote groep zal het voordelig zijn. 2783 2803 2810 2847 beperkt de keuzes
182
3069 Niet echt gediscussieerd. 3090 het is wel makkelijker, omdat ongeacht de andere meningen van andere maakt de eind verantwoordelijke een beslissing. 3135 Het GDSS vereenvoudigde de beslissingen door de beperking van het eindresultaten.(Scope) Er is niet echt gediscussieerd. Alle leden moeten misschien uiteindelijk toch een akkoord geven of een veto kunnen geven. 3150 Dit komt de discussie ook ten goede. 3368 3410 Er zijn voor het debatteren al wat steden geselecteerd, dus die hoeven dan niet meer ter sprake te komen. 3432 krijg je weer die dominantie tantes weer aan het woord Doordat je al moest nadenken over de bestemming en ook weet wat andere ongeveer leuk vinden is het debatteren wel een 3454 stuk makkelijker 3730 - Gericht aangeven waar de meningsverschillen zijn - Door het geven van opmerkingen kan je specifiek daarover praten Ik vond het beslissen makkelijker omdat je in een korte tijd min of meer van iedereen duidelijk kan zien waarom hij/zij 3758 voor een stad wel of geen voorkeur hebt. Normaal gesproken ben je uren aan het praten. Of heen en weer aan het mailen. Nu kan iedereen meteen van iedereen bepaalde redenen zien (die moet je natuurlijk dan wel lezen ;P) Het was wel makkelijker, bv omdat je kunt argumenteren vanuit de wensen van de anderen. Bv. Kitting wil lekker eten, dus 3765 hij vindt Venice, waar ik naar toe wil, ook leuk. Het eerste traject is snel verlopen. Er kwam een kort rijtje uit en is niet meer de complete wereld meer open en zou er 3767 niet meer in de wilde weg iets geroepen worden. Nu hebben we tenminste concrete steden waarmee de gehele groep wel tevreden kan zijn. Het feit dat een stad een score 4056 krijgt helpt bij het nemen van de uiteindelijke beslissing doordat je er zelf van overtuigd wordt dat die shit klopt. Via dit systeem kwamen we sneller terecht bij een stad, zonder het systeem zouden we de keuze veel breder hebben 4059 gemaakt waardoor we er ook meer tijd aan zouden hebben besteed makkelijker want in het geval dat de groep niet samen kan komen kan het proces voor het kiezen van de eindbestemming 4068 op een duidelijke manier verlopen, met daarbij ook een discussie aan het eind waarmee men over de gekozen voorkeuren kan praten om tot de uiteindelijke keuze voor de eindbestemming te komen. Wij hadden in principe niet echt gedebateerd. De uiteindelijke beslissing had de hoogste rank (en was ook voordeliger 4073 vergeleken met de andere opties). Het was wel duidelijk wat de voorkeuren van de andere personen waren. 4289 Het systeem zorgt voor inhoud om over te discussieren. 4295 wel leuk om bij elkaar te komen en dan erover te gaan praten 4369 De argumenten van de groepsleden waren duidelijk. Er kon makkelijk een compromis gemaakt worden. 4375 Niks aan toe te voegen. Ik vond het goed gaan. Het is een eerlijk proces wat je kan zien aan de hand van de sterren. Ik vind het een eerlijke 4663 verdeling, waar discussies minder noodzakelijk zijn. 4670 Veel makkelijker. Je bespreek een klein aantal opties doordat er al eerder gefilterd is. 4673 Nee, omdat we het voor een groot deel al met elkaar eens waren, dus we liggen daarom wel op een lijn. 4683 Het is altijd goed je motivatie mee te kunnen geven. Omdat je direct inzicht hebt in iemands voorkeur. En je hebt door gebruik te maken van het systeem gelijk een 4964 gelimiteerde afgebakende keuze. En geen oerwoud aan keuzes war je vaak niet uit komt. 4973 Zo blijft de keus toch bij de groep en is niet het systeem dat in een keer voor de groep beslist. Ja het proces ging vrij snel, toch duurde het even maar met 4 man kwamen we er snel uit. Wellicht als de groep groter is is 4981 het wat moeilijker. Dit was naar mijn mening makkelijker aangezien we allen een beter inzicht kregen (d.m.v. het ratings systeem) welke 4983 stad/land per persoon meer waarde had waardoor ik makkelijker rekening kon houden met mijn ''reisgenoten'' hen voorkeur. 5200 5203 Het is veel overzichtelijker, doordat de prijs en een plaatje van de stasd direkt zichtbaar is. Zonder dit systeem zouden we 5246 nu nog bezig zijn. dit gaat veel sneller. 5286 5485 Je wordt al geleid naar een uiteindelijke bestemming. Dit is erg fijn Het systeem geeft een aantal punten op waarop je je keuze/voorkeur op baseert. Doordat iedereen op basis van deze 5511 punten heeft gekozen, is het makkelijker om erover te debatteren Je hebt in iedergeval al vast een voorzetje. Zonder te veel kennis over de verschillende steden kun je ook onbekendere 5518 steden meenemen in je keuze. als dit systeem afgemaakt is dan zou het inderdaad veel makkelijker zijn geweest. op dit moment is het systeem echter nog 5710 een beetje te ingewikkeld voor de normale gebruiker, zeker als deze persoon niet zoveel weet van computers en dit soort systemen. er is soms ook teveel opties beschikbaar, de informatie kan overweldigend zijn voor de onervaren gebruiker. 5737 5742 Zie 1. 5968 6035 Makkelijker. Het programma helpt om op gang te komen. Het wordt duidelijk welke steden een optie zijn.
183
6045 Ik denk dat het het debatteren moeilijker maakt doordat er veel keuzes gemaakt en verdedigt moeten worden terwijl, er geen mogelijkheid is veel info over een onbekende stad te vinden. 6124 Heb nu concrete informatie om goed met de groep te gaan praten 6203 volgens mij het tocnh altijd 6274 6353 6430 6503 beetje rommelig 6580 6658 6730 6809 6884 6958 7247 Er kwamen wel duidelijker meningen van andere mensen er uit die ze anders misschien nooit geoppert hadden. Zeker door middel van de opmerkingen die je kan toevoegen. Je kan de resultaten van anderen bekijken en tegen ze 7255 gebruiken wanneer nodig. Het makkelijke van dit systeem is dat je de deelnemers eventueel in hun eigen tijd de voorkeuren kan laten invoeren, zodat 7261 je meteen alle data en meningen hebt als je een discussie wil starten. Het debatteren was uiteraard makkelijker omdat we diverse inputs van andere konden zien. Dus daarmee kon je zien wat 7265 de andere belangrijk vonden en kon je ermee rekening houden. het was moeilijker omdat je voor jezelf al helemaal voor je ziet naar welke van de overgebleven steden je graag zou willen 7342 gaan; als echter dan lijkt dat je medereisgenoten hier niet naar toe zouden willen dan is dat een ' domper'. Ik vond het makkelijker om de beslissing te maken omdat de opties verkleind waren, wat de discussie makkelijker maakte. 7418 In geval van onze groep hier, was er maar één echte optie waar we allemaal heen wilde dus de beslissing was zelf gemaakt. 7495 Iedereen was het eens en tevreden met de opgegeven nummer 1 van het systeem. Dus er was weinig discussie nodig. Zeker makkelijker, omdat je in de selctie van de vakantiebestemming al een stap verder bent en iedereen voor zichzelf 7572 duidelijk heeft waar elke stad aan moet voldoen. 7720 neutraal, nauwelijks debat 7724 Makkelijker. Omdat je standpunten kan verdedigen met de ingevoerde data (scores, grafieken etc.) 6047
184
id 376 399 474
Remark – Cat 8 question 3 Meer informatie over het land/ de stad. Meer interactie met de participanten/ deelnemers. Vind de categorieen ruimschoots aanwezig. Meer toelichting over de te nemen stappen. Meer uitleg over de grafiek. De rangschikking van de voorkeuren is verwarrend. Vervolg traject, bijvoorbeeld samen plannen van hoe de reis wordt ingevuld. Taakverdeling binnen de groep voor 544 voorbereiding van de reis. 555 uitleg per stap kan uitgebreider door middel van een kort filmpje, dik gedrukte woorden. 747 een functie om budget toe te voegen Ik vraag me af of men bij de uiteindelijke keuze voor een vakantiebestemming zich laat leiden door de zeven preferenties die bij stap 1 worden genoemd. De keuze voor een vakantiebestemming wordt denk ik naast de persoonlijke voorkeur ook bepaald door andere factoren, zoals leeftijdscategorie, geld ter beschikking, vakantietijd (je gaat niet voor een weekend 869 naar Maxico, maar wel naar Venetie bijvoorbeeld) en vast nog anderen. In het proces miste ik nog ruimte om deze andere factoren in te kunnen brengen. Verder mis ik de keuze om ook naar niet-steden te gaan, zoals het platteland van Bolivia (veel natuur/landschap) en de optie om meerdere steden tijdens een reis te kunnen bezoeken. Zo kan ik me voorstellen dat je Beijing goed komt combineren met een andere Chinese stad. Zie ook mijn eerdere feedback bij categorie 1. * Zoals eerder gezegd had ik andere verwachtingen bij de factor cultuur. Zo had ik hierbji eerder elementen verwacht als cultureel erfgoed / historische plekken. (maar goed dat is natuurlijk ook weer 874 omgeving inderdaad :) * Beurzen en festivals zou ik apart doen. Zo zou ik festivals hoog ordenen, maar beurzen juist niet. * Excursies vind ik minder bij amusement passen. (meer bij cultuur/natuur achtige dingen) * Ik zie enige overlap tussen atmosfeer (eg cultureel erfgoed) en omgeving (eg historische plekken) 1040 Bij fase 7 een box waar je jouw mening kan geven mbt het bepalen van locatie/stad 1131 chat functie Continenten kiezen om je keuze te specificeren. Maar dat maakt het moeilijker om een keuze te maken voor het systeem. 1166 Een extra poll om te kunnen stemmen om eindbesluit te kunnen nemen. 1180 Ja, Chatfunctie Poll Functie Meer media bij het informeren van de steden concrete stads informatie. wat vinden anderen van de stad? chat box de negotiation fase zou wellicht vervangen kunnen 1344 owrden door skype?? 1452 Zelf steden opgeven. Stel ik vind omgeving belangrijk omdat ik van groen houd, dan kan ik nu alleen 'omgeving' aangeven? (waar bv ook Athene 1458 onder valt omdat daar het oude architectuur is, en nauwelijks groen)... Misschien is het sorteren van de steden wel genoeg om een dergelijk voorkeur aan te geven, misschien niet 1468 Ik mis: * het bepalen van de budget van de vakantie * het kiezen van accomodaties 1755 chat functie Ik zou graag nog duidelijker willen zien waar de kostenindicatie (dus de prijs voor een reis naar een bepaalde bestemming) 1758 op gebaseerd is. Als de groep uit elkaar woont is het handig om een chat functie in te bouwen. Wellicht ook een optie om door te linken naar 1764 vakantie sites waarin jijzelf de keuze hebt in bijv : comfort, budget,of luxe reizen. De boekingsprocedure miste ik de final goedkeuring van de stad. Ik zou een button toevoegen die de uiteindelijke beslissing van de voorzitter bekrachtigd. Verder zou ik na de defenitieve keuze van de stad nog een aantal te boeken hotels etc 1773 (bijvoorbeedl in verschillende prijsklasse) met het zelfde beslissing model toevoegen. Gevaar hoe meer keuze hoe langer de beslissing uitblijft 1988 niet echt alles leek zoals het hoort Een sub-stap die iets meer in gaat op de interesses toegespitst op de verschillende bestemmingen is misschien handig. Veel 2060 mensen kennen niet van alle bestemmingen de achtergronden. Je kiest dan iets niet, puur uit onwetendheid. Dit programma zou ook een inzicht kunnen geven in voor de groep onbekende plekken. 2070 In het veld waar je de steden dmv sterretjes kan waarderen zou nog wat extra info mbt de steden gegeven kunnen worden. 2355 chat sessie geintegreerd. 2380 display de top 5 van iedere teamlid ! 2389 Iets duidelijker aangeven hoe je, je voorkeuren kunt aangeven aan het begin van het proces 2393 eerder een range aangeven van prijs en afstand. 2538 Tijd van het jaar is van belang bij het kiezen van de stad. Dit is natuurlijk ook van te voren af te spreken. 2545 Informatie over de stad die verschijnt om de stad te waarderen of te ranken. 2806 Geen 2846 -budget van de reis - Steden die er niet tussen staan. - Steden die niet in de top 30 staan, kan daaruit worden gekozen (selecteren van steden 3071 met bijv 20% rating) 3093 3136 Geen Alle leden moeten misschien uiteindelijk toch een akkoord geven of een veto kunnen geven. Dit komt de discussie ook ten 3151 goede. Het uiteindelijke boeken van de reis. Het vooraf bepalen van de data wanneer iedereen op reis kan.
185
3407 Nee pre vakantie parameters: - wanneer vertrekken - hoe lang - speciale bestemmingen het "wacht" status zou ik trouwens geen 3431 STAP noemen...maar gewoon leeg laten...het is nogal verwarrend 3456 Prijs, etc - Prijs en / of locatie keuze - Top x (vb. 5) lijst i.p.v. rangorde van alle dingen - Soms iets teveel informatie door grafiek 3732 met categorieën en prijs en beschrijving en keuzes een uitgebreide informatie van een stad en wat foto's van de steden. Dat zal wel leuk zijn als je wat meer foto's kan 3756 bekijken. Ik vind dat meestal toch wel wat meer vertellen over de stad dan alleen geschreven informatie. Ik ga zelden voor een 2e keer naar een bestemming. Op basis van mijn profiel zou daar ook rekening gehouden kunnen worden. Het geld aspect zou ook vooraf al meegenomen kunnen worden als beperking op het aantal opties. Er zit in een 3764 dergelijk traject ook vaak een voortraject mbt wie wil er uberhaupt mee en wanneer kan iedereen en wat voor type reis wil ieder (wintersport, citytrip etc). Daarna kan een selectie als deze citytrip planner gestart worden. Fotos van steden missen, meningen van andere gebruikers die naar de aanbevolen steden zijn geweest om te kijken of die 4055 steden echt de moeite waard zijn 4062 nog niet opgevallen 4065 Foto's kunnen bekijken van de voorgestelde steden om beter idee te verkrijgen van wat er in elke stad wordt geboden.. 4074 Het zou leuk zijn om meer foto's te kunnen bekijken van de verschillende steden 4290 algemene steden info kosten bepalingen roostering locaties etc (google maps) 4294 verzamelen van wanneer gaan we, hoe lang gaan we, en misschien een pre keuze van continent 4376 Nee 4661 Ik zou meer over de prijs willen weten. Hoe wij als groep denken over de prijs en over de periode. 4671 Nee Aan de hand van de gekozen locatie zouden er nog excursies en of uitstapjes in de buurt van de uiteindelijk gekozen 4966 accomodaties weergegeven kunnen worden. Welke dan worden samengesteld aan de hand van de voorkeuren zoals de gebruiker die in het begin heeft aangegeven. 4970 Nadat een stad gekozen is zou het handig zijn om verschillende opties te hebben uit diverse accomodaties. Accomodaties (na het beslissingsproces) mag van mij de keuze ronde op dezelfde manier doorlopen (scheelt discussie) en 4979 duidelijkheid wat betreft de dingen die er te doen zijn in een stad. Je hebt vooraf toch een bepaald beeld van een stad wat kan en miscchien ook wel moet ontkracht worden. Agenda (datumprikker), prijs limiet, eventuele nevenactiviteit aan de hand van de keuzes die je maakt, keuze in 4985 zon/winter/stedentrip. 5249 er zou naar mijn inzien wat meer informatie over de steden bijgeschreven mogen worden. 5483 Proberen minder het minder druk te maken qua info op de site. Daardoor kan het onoverzichtelijk worden 5513 het bepalen van een budget 5514 Zie vraag 1. - rangschik steden van goedkoopste naar duurste - rangschik steden van duur reis (tijd) - easy/advanced versie, eentje 5712 zonder teveel details zoals grafieken enz., andere versie voor mensen die het niet erg vinden om veel informatie te verwerken. 5746 Zie 1. Kueze tussen werelddelen, west-europe, oost-europa, etc. 6037 Kosten worden niet meegenomen bij de 1e stap. 6041 De uitleg van de verschillende stappen is onvoldoende toegelicht, het geheel is maw niet gebruiksvriendelijk 6050 Ik mis de optie om meer achtergrondinformatie over een onbekende stad te vinden. 6122 Budget, duur, type vakantie, voorkeur van continent..daarover moet nog zeker over gepraat worden 6198 direct boeken 6277 lengte van vakantie en geld 6354 boeken van reizen er zijn nog meer dingen waarover gepraat moet wroden op vakantie..zoals wie neemt tandpasta etc mee...misschien dat 6426 daar ook lijstjes voor gemaakt kan worden 6506 onderhandelingsproces zou wel nice zijn als het ook op afstand kon Er is nu alleen een stad uitgekozen maar nog niet wat je er gaat doen. Zeker als een stad meerdere activiteiten bied zou het kunnen dat mensen met een heel verkeerd beeld instemmen met de stad. Meer duidelijkheid over de over eenkomende 7245 activiteiten tussen de verschillende deelnemers had handig geweest. Misschien was er dan ook een andere stad gekozen die vooral uitblonk in de activiteit. Prijs is erg belangrijk om toe te voegen als categorie (prijs tocket, prijs eten, prijs souvenirs, etc). Notificaties beter op de 7254 site laten tonen, bijvoorbeeld als er een vriend je een uitnodiging stuurt. Ik zag deze uitnodiging eerst niet. Het zou handig zijn als sommige iconen wat duidelijker was aangegeven. Misschien kan sommige iconen een pop-up wolkje krijgen met extra info over wat het knopje kan doen. Als je toch plek over hebt voor info kan je bepaalde informatie alvast 7263 weergeven zonder dat je eerst erop moet klikken, dat plekje onder de grafiek en knop "meer details weergeven van de stad" is toch blanco, dus waarom niet opvullen met info, gebruiker kan info minimaliseren wanneer nodig. 7339 een functie om budget toe te voegen
186
Zie ook mijn eerdere feedback bij categorie 1. * Zoals eerder gezegd had ik andere verwachtingen bij de factor cultuur. Zo had ik hierbji eerder elementen verwacht als cultureel erfgoed / historische plekken. (maar goed dat is natuurlijk ook weer 7494 omgeving inderdaad :) * Beurzen en festivals zou ik apart doen. Zo zou ik festivals hoog ordenen, maar beurzen juist niet. * Excursies vind ik minder bij amusement passen. (meer bij cultuur/natuur achtige dingen) * Ik zie enige overlap tussen atmosfeer (eg cultureel erfgoed) en omgeving (eg historische plekken) Ik vraag me af of men bij de uiteindelijke keuze voor een vakantiebestemming zich laat leiden door de zeven preferenties die bij stap 1 worden genoemd. De keuze voor een vakantiebestemming wordt denk ik naast de persoonlijke voorkeur ook bepaald door andere factoren, zoals leeftijdscategorie, geld ter beschikking, vakantietijd (je gaat niet voor een weekend 7571 naar Maxico, maar wel naar Venetie bijvoorbeeld) en vast nog anderen. In het proces miste ik nog ruimte om deze andere factoren in te kunnen brengen. Verder mis ik de keuze om ook naar niet-steden te gaan, zoals het platteland van Bolivia (veel natuur/landschap) en de optie om meerdere steden tijdens een reis te kunnen bezoeken. Zo kan ik me voorstellen dat je Beijing goed komt combineren met een andere Chinese stad.
187
id
Remark – cat 8 question 6 Voordelen: Je kunt via het systeem een makkelijke keuze maken Nadelen: Hoe meer stemmen, hoe hoger de waardering 373 wordt. Dit kan voor degene die niet de stad heeft gekozen oneerlijk zijn. Voordelen - je kunt je stem laten horen - de grafieken zijn goed (kunnen wel wat verbeterd worden qua communicatie) 395 want je kunt dan altijd het proces checken. Nadelen - groepsleider vind ik niet goed, niet eerlijk - draggen vind ik moeilijk nog Voordelen: Opties van vakantiebestemmingen te verminderen. Voorkeuren kenbaar kunnen maken. Nadelen. De gegeven 463 vakantiebestemmingen zijn niet optimaal ten opzichte van individuele keuze. Meer mogelijkheden tot uiten van mening. Voordelen: Opties van vakantiebestemmingen te verminderen. Voorkeuren kenbaar kunnen maken. Nadelen. De gegeven 471 vakantiebestemmingen zijn niet optimaal ten opzichte van individuele keuze. Niet genoeg mogelijkheden tot uiten van mening. Voor: - discussie functie. - de rangschikking van de steden. Na: - weinig toelichting; - tijdens de onderhandelingsfase de 473 functie 'lage waardering'. waarom niet een hoge waardering? Voordelen: bespaart tijd, maakt voorkeuren van de groep overzichtelijk Nadelen: stap 1 was voor mij even onduidelijk hoe de voorkeuren aangegeven moesten worden, nadeel is dat onderhandeling niet in het systeem plaatsvindt waardoor degene 545 die het meest dominant is uiteindelijk alle voorkeuren weg kan vagen (de kans hierop is wel minder geworden omdat voorkeuren van de groep inzichtelijk worden door dit systeem). twee voordelen: - Dit is een systeem die makkelijker initiatie kan betekenen voor het plannen - De interactie is duidelijk, dankzij de drag&drop en dynamische animatie van menu's twee nadelen: - Het is jammer dat het systeem weinig meegeeft 559 hoe het tot een keuze is gemaakt. - Elke stap van het proces is duidelijk wanneer men zelf gaat uitzoeken. Soort "trial en error". Dit zal handiger zijn als men dat een uitgebreider uitleg krijgt. Dmv een filmpje en dergelijke voordelen: -ik vind het een handig en makkelijk systeem nadelen: -eigenlijk heb ik zelf al een paar steden in gedachten, 565 maar heb niet de mogelijkheid gekregen om dat op de lijst te zetten. - meer inzicht in de voorkeuren van je reisgenoten - heel veel steden zijn minder bekend en door het zien van deze in je lijst geef je ze op zijn minst een kans. Bijvoorbeeld van de oostblok landen zijn er bij veel steden waar je eigenlijk niks 749 vanaf weet. - de vraag blijft of je groep niet te eigenwijs is om alsnog een andere stad te kiezen - veel steden die je aanvankelijk nog een kans gaf die vielen onverwacht af. De preferentie categorieën zijn niet even duidelijk. 'vals spelen' kan makkelijk met het waarderen van de steden via de sterren. Dit proces is een goede oriëntatie van wat de groep leuk vindt, vanuit dit systeem kan je altijd nog verder 860 beslissen. Hiernaast kan ik me goed voorstellen dat men zich houdt aan de uitkomst van het systeem. Het beslissingsproces wordt behoorlijk versneld door dit programma en iedereen kan individueel zijn bijdrage leveren aan het proces als die bijvoorbeeld de underdog is bij een discussie. Voordelen: - inzicht in waarom je zelf voorkeur hebt voor de bepaalde vakantiebestemmingen - inzicht in het beslissingsproces van de groep; het geheel is deels democratischer geworden nu - onverwachte steden komen boven drijven, wat verrassend kan werken Nadelen: - veel mensen bedenken niet van tevoren wat ze in het algemeen willen bij een vakantiebestemming, maar hebben al enkele bestemmingen in hun hoofd. Deze zouden (ook) kunnen dienen als 868 vertrekpunt, niet alleen de persoonlijke 'interesses'. - sommige deelnemers zijn al in sommige van de te kiezen bestemmingen geweest. Deze beperking wordt in de huidige versie van het systeem nog niet opgenomen. - er zijn relatief weinig handvatten voor discussie doordat er nog onvoldoende informatie over de mogelijke bestemmingen wordt weergegeven. Meer (visuele) informatie kan de discussie beter op gang brengen waarschijnlijk. Ps, veel feedback staat al bij categorie 1! Voordelen: * Beslissing die meerdere mensen moeten maken gaat op deze manier veel sneller. Er zitten altijd wel opties bij die je aanspreken dus je voelt altijd wel dat je preferenties gewaardeerd worden. * Erg leuk om te bemerken dat de eindelijke nummer 1 ons allemaal aanspreekt. Dat bewijst maar weer dat het 871 systeem werkt en efficient is! * Mooie layout! Nadelen: * Sommige elementen van categorieen mogen nog wat aangepast worden zodat minder overlap bestaat. * Ik vond de stap waarbij je voor het eerst alle steden ziet wat moeilijker om te voltooien. Ik merkte dat ik meer ahvp mijn 'gevoel' bij de steden mijn keuzes maakte, dan gebruik te maken van de statistieken. Wellicht kan je per stad wat meer beschrijvende informatie geven, zodat ze meer tot de verbeelding spreken. Pro's: Versnelt het proces Steden komen tevoorschijn waar je wellicht niet aan gedacht had Con's: Af en toe onduidelijk wat 1043 je moet doen. Instructies kunnen misschien visueler voordeel: gaat snel voordeel: makkelijker en veel inzicht nadeel: steden zeggen mij nog iets te weining.Meer info zou wel 1130 beter zijn nadeel: momenteel alleen stedentrip...zou mooi zijn als er ook andere types vakanties erin opgenomen gaat worden Voordelen: - makkelijke beslissingsproces met meerdere personen - Simpele uitstraling Nadelen: - minder zichtbaar uitleg 1165 tijdens gebruik (o.a. bij preferenties slepen) Voordelen: -op afstand afspreken -overzichtelijk maken van elkaars voorkeur Nadelen: - minder persoonlijk overleg - Moet 1176 tegelijkertijd door de gebruikers uitgevoerd worden voordeel1: duidelijker voordeel2: sneller nadeel1: kleine leercurve nadeel2: je moet eerst vrienden hebben voor dit 1347 systeem. wellicht dat je dit kan koppelen met facebook, linked-in, hyves etc voordelen: - er wordt rekening gehouden met de voorkeuren van iedereen - de meeste beslissingen worden door het systeem gedaan, er hoeft maar 1 keer gediscussierd te worden over de uiteindelijke bestemming nadelen: - je kunt zelf 1453 geen suggestie doen voor een stad - je moet soms wat meer compromis sluiten (hoewel dit niet echt een nadeel van dit systeem is, aangezien je dat altijd wel moet) + Lekker snel een plan maken + Systeem komt met suggesties (ipv dat er maar 2 ideeen op tafel liggen) - Te weinig ruimte 1459 voor communicatie (ik zou zeker weten alsnog heen en weer mailen om te discussieren) - Ik zat voornamelijk steden te
188
kiezen/sorteren aan de hand van het plaatje wat erbij zat en mijn eigen verwachting.. in mijn geval is dus: andere plaatjes = ander resultaat. Dit terwijl het misschien beter zou zijn als ik erachter kom wat er daadwerkelijk in een stad te vinden is (mijn verwachting zal in de meeste gevallen niet kloppen met dewerkelijkheid). Misschien is het in een echte situatie de bedoeling dat ik zelf informatie opzoek in de laatste stap? - Status scherm vond ik niet heel duidelijk Voordelen: * Het versnelt het kiezen van stad * Het geeft overzicht van welke steden bij mijn voorkeur ligt Nadelen: * De 1469 budget voor elke persoon is meestal niet te onderhandelen * Het bepalen van de voorkeuren is moeilijk plezierig en leuk om in te vullen Functioneel door met concrete uitkomsten te komen Nadeel is dat een persoon met weinig 1753 tot geen computer ervaring er wel even over zal doen om te snappen wat de bedoeling is, dit vooral mbt het verschuiven van de blokjes Voordelen: 1. Er komen vakantiebestemmingen te voorschijnen die bij je passen, maar waaraan je niet zelf gedacht had, 2. Je kunt gerecicht discussiëren over het uiteindelijke reisdoel en komt zo sneller tot een groepsbeslissing Nadelen: 1. Je 1760 bent afhankelijk van de voorselectie van "een derde" (het tool). Er zullen ongetwijfeld reisdoelen zien die door het tool nooit gekozen zullen worden. 2. De enquête na afloop is wel erg lang! 1763 voordelen: gezamelijk op een snelle manier een beslissing maken iedereen heeft gelijke kansen om input te geven nadelen: Voordelen - Snelle selectie methode in groep - Leuk om selectie in te geven (prettig in gebruik) Nadelen - Dat ik Firefox en 1771 andere tooling moet installeren - risico op virussen etc. 1987 voordelen -iedereen zijn mening is even veel waard - nadelen - Voordelen: - je kan je mening geven zonder door de andere leden afgestraft te worden - je komt bij andere bestemmingen 2065 uit die je misschien in eerste instantie als ideaal bestempeld Nadelen: - beperkte keuzemogelijkheden om interesses aan te geven - je moet op de hoogte zijn van informatie over de bestemmingen Voordelen: - Geen discussie's waardoor je geen zin meer hebt om uberhaubt nog op vakantie te gaan - Je hoeft niet te zoeken naar mogelijkheden om naar toe te gaan, dit wordt allemaal door het systeem aangedragen. Scheelt veel tijd met 2067 surfen op internet Nadelen: Om heel eerlijk te zijn heb ik niet echt nadelen ondervonden tijdens het gebruik van dit programma voordelen: snel en inventariserend nadelen: oppervlakkig en niet inhoudelijk (je blijft aan de buitenkant van de stad en je 2075 vergelijkt dus buitenkanten met elkaar) voordelen: 1 snel 2 gelijkwaardig nadelen 1 nog niet helemaal af oftewel nog gebruiksonvriendelijk 2 niet mijn keuze, maar 2352 een groepskeuze. Er komt een compromis uit. voordeel : 1. wysiwig, je hoeft niet tegen een vervelende reiskantoor miep te blaten 2. sneller kiezen door meer afstand te 2382 creeeren. je kunt schelden tegen de ander zonder dat die persoon het hoort voordelen: -je kunt sneller een keus maken -je weet sneller wat de anderen uit de groep willen nadelen: -misschien dat je 2387 een bepaalde specifieke voorkeur niet terug kunt vinden in het programma voordelen: - versnelt het keuze proces - stimuleert om snel even in te vullen, beetje als 'datumprikker'. nadelen: - de 2394 keuzes liggen -vooral geografisch- wel ver uit elkaar en daardoor ook qua prijs - hoe koppel je locaties eraan? dus wat voor hotel en hoe lang en wanneer. maar het is ook nog niet klaar natuurlijk, dus ik kom nog wel een keer testen :) Voordelen: - Versnelt het beslissen door het beperken van het aantal steden. - Overzichtelijke manier van het uitbeelden 2539 van de mening van iedereen (ook de wat stillere personen doen mee) Nadelen: - Als je totaal andere voorkeuren hebt dan je vrienden, dan lijkt het dat je mening minder meetelt. - Je kan niet onderling "live" discussieren tijdens de stappen. voordelen: - bespaart tijd om tot een keus te komen - geeft aan welke steden je waarschijnlijk zullen interesseren, zodat je 2542 later weer terug kunt komen om die stede nadelen: - de keus kan ver van je interesses zitten - men wordt beperkt in het aantal steden om te kiezen 2781 Voordelen: Duidelijk zichtbaar & makkelijk terug te keren naar de hoofdscherm Nadelen: keus was te beperkt voordeel: nieuwe ideeën van steden trips staan voor je op een rij nadeel : er kwam een stad uit die ik niet wilde ik heb niet 2800 alle opties/grafieken gezien 2805 Voordelen: Snel en makkelijk Nadelen: niet iedereen kan overweg met dit systeem en Site kan vastlopen 2851 tijdsduur zelfde keuzes Voordelen - Overzichtelijk, status is duidelijk wie nog respons moet geven. - Ieder persoon kan zijn voorkeur aangeven. Voorkeuren worden automatisch gelinkt naar bepaalde steden. Nadelen - De leader kan de uiteindelijke beslissing nemen, 3072 terwijl niet iedereen ermee eens hoeft te zijn. - Door sterren te geven, kunnen bepaalde personen een stad een 1 geven, waardoor het stad een lager waardering krijgt. 3092 voordelen - zorgt voor efficientie - ook onbekende steden komen erin voor nadelen Voordelen: - Layout is zeer overzichtelijk - Gebruikwijze is zeer eenvoudig, het is prettig om mee te werken. Nadelen: 3133 GDSS biedt de mogelijkheid om terug te gaan naar de voorgaande stap. - De opmerking veld was in het eerste instantie onduidelijk of het bedoeld was voor de stad. 1 flinke versnelling proces 2 mooi overzicht van de mogelijk reizen 1 nog niet voldoende criteria waarop geselecteerd kan 3153 worden 2 een persoon heeft keuze / veto 3366 Voordelen: -Iedereen kan zijn mening uiten -Er wordt daadwerkelijk een stad gekozen Nadelen: -Tijdens de discussie kan 3409 iemand zijn mening aan de anderen opdwingen -? 3428 3457 3728 Voordelen: - Tijdsbesparing in het keuzeproces (zowel qua voorkeur alsmede selectie van mogelijke steden) - Onafhankelijk
189
van elkaar in te vullen per stap Nadelen: - Teveel steden keuzes - Slepen van blokken voor belangrijkheid is vanuit laptop niet helemaal praktisch, zeker als je onderaan de pagina een keuze naar boven wilt halen. Succes verder met de afronding!!! Voordelen: Het bespaart veel mail en discussie tijd. Je hoeft niet het internet af om allemaal informatie over verschillende steden te gaan zoeken. Nadelen: Je kan teveel keuze mogelijkheden krijgen waardoor het juist weer moeilijker wordt om 3754 te bepalen waar naartoe te gaan omdat sommige mensen niet kunnen kiezen. De informatie is beperkter dan wanneer je op websites hebt gekeken. voor: de lastige afstemming die per mail vaak loopt over wie gaat er mee, wanneer kan iedereen, wat zijn specifieke wensen, zeer vereenvoudigd. voor: je kunt op bestemmingen komen die je vooraf niet zou hebben overwogen en wel 3761 interessant zijn na: het is nog niet zo transparant (ook vanwege complexiteit) waarom een stad nou op 1 eindigt. je vertrouwt het systeem wel, maar je bent ook wel benieuwd naar hoe de beslissing tot stand is gekomen Voor: Eerder tot een kort keuzelijstje komen Niet continu bij elkaar te hoeven komen, zeker bij vrienden die het druk 3768 hebben Na: Alweer een systeem erbij, waar je voor moet inloggen etc (niet echt een extreem nadeel) ...? Je kan de kans lopen bepaalde steden te missen die niet in het systeem voorkomen Te bureaucratisch proces Sneller komen 4053 tot een beslissing Steden ontdekken waar je zelf in eerste instantie niet op was gekomen 4058 Op het moment dat ik gebruik maakte van het systeem kon ik bij stap 2 niet terug naar stap 1. Voordelen: Persoonlijke voorkeuren kunnen specificeren Discussie over gekozen voorkeuren om eindbestemming te kunnen 4067 bepalen. Nadelen: Geen 4070 Voordeel: Je kunt snel en makkelijk tot een beslissing komen. 4287 Het is niet duidelijk hoe het systeem tot zijn conclusies komt, sommige recommendations lijken wat onlogisch - het gaat wel sneller - overzichtelijkheid van informatie en feitjes - nadeel: geen stads info - nadeel: geen directe 4296 boekingsmogelijkheid 4373 gemak, Voordelen - Overzichtelijk - Veel mogelijkheden qua steden Nadelen - Na een afgeronde stap kan je niet meer terug 4378 Wanneer je een onvolledige categorie hebt afgerond krijg je en error. Voordelen: Makkelijk/ overzichtelijk Nadeel: De prijs viel met de keuze toch een beetje tegen en had willen weten wat de 4662 groep voor vakantie periode had gekozen. Voordelen: Sneller beslissen Eenvoudig inzicht in andere meningen Nadelen: Beperkter aantal keuzes Geen mogelijkheid om 4666 elkaar te overtuigen voordeel, overzichtelijk en makkelijk om snel een keuze te maken Nadeel: voor de eerste keer is het even wennen hoe het 4676 systeem werkt met slepen enz. en kan misschien iets meer diepgang in de opties. Snel een keuze maken denkproces wordt deels overgenomen. Afhankelijk van de content van het systeem Afhankelijk van 4680 de waarde die de programmeur. 4969 Snel, effectief. Geen rekening gehouden met budget, En tijden en data, periodes zouden fijn zijn. Voordeel: 1) Je kiest als groep zonder fysiek aanwezig te zijn, zodat iedereen het op z'n eigen tijd kan doen. 2) Keuzes zijn gemakkelijker omdat het systeem al een voorselectie maakt aan de hand van de opgegeven opties, maar de uiteindelijke 4975 beslissing wordt toch door de groep gekozen en niet het systeem. Nadelen: 1) Systeem kan uitgebreider, waaronder de voorkeurselectie die vooraf opgegeven wordt, dat die na de keuze van de stad weer terug zouden komen in de vorm van een keuzemenu. 2) Het opgegeven van vakantieperiode, maximale uitgave e.d. ontbreekt. Voordelen: Snelheid/ ontwijkt de discussie en neemt een hoop blokkades weg in het keuzeproces (te veel opties hebben). 4978 Leuk, je komt makkelijk op nieuwe ideeen. Nadelen: Werd geen rekening gehouden met budget/ seizoen van te boeken reis en omstandigheden (bedoeling echt voor vrienden alleen of ook gezinnen etc.) 4987 voordelen: Overzichtelijker, keuze mogelijkheid. nadelen: zie 3 vakjes boven. 5202 5205 Het aantal steden is beperkt Voordeel: snel en zeer praktisch en overzichtelijk, het nodigd sneller uit om met een groep een trip uit te kiezen. Nadeel: 5245 indien je meer informatie bij de steden zou gaan toevoegen bij dit systeem wordt het waarschijnlijk te onoverzichtelijk en duurt het zoeken naar een leuke trip alsnog te lang. Voordeel 1) Het is makkelijk 2) Het is niet partijdig, objectief Nadeel 1) De uiteindelijke keuze was niet mijn 1e voorkeur. 5291 Dat kwam door een persoon in de groep, niet door het systeem. (we hebben niet erg kunnen discussieren, de keuze stond al vast, door die persoon) 2) Nadelen: Budget en hotel kwaliteit kan je nog niet filteren. Voordelen: Je wordt tijdens het proces geleid en er worden 5484 andere tips aangedragen - je hoeft niet bij elkaar te komen om te beslissen waar je naar toe wilt gaan - je kunt in je eigen tijd en tempo de 5512 beslissing maken - niet mogelijk om van te voren een budget te bepalen - niet mogelijk om de reisduur te bepalen Voor: - 1ste basis voor een beslissing zonder alle steden te kennen. - ook minder bekende/voor de hand liggende steden 5515 passeren de revue. Nadeel: - Fysiek samen kiezen is een leuke bezigheid. - (Nog) beperkt in de mogelijkheden voordelen: - handig als het écht goed gemaakt is - kan op deze manier nieuwe plekken bezoeken die je misschien in eerste 5714 instantie niet zou bezoeken nadelen: - teveel informatie voor de nieuwe, onervaren gebruiker - niet simpel genoeg voor de leek 5741 5744 Voordelen: Stimuleerd een discussie. Geeft een idee wat mogelijk is. Nadelen: Kan nog steeds moeilijk zijn voor minder
190
computergeorienteerde personen. Missen van helpfunctionaliteit. Meer info over steden nodig. 5969 niet in alle gevallen alle keuzes tegelijk in beeld vragen zijn makkelijk te beantwoorden Belangrijk voordeel: het helpt een groep op weg om een beslissing te nemen, een hoop steden worden uitgesloten. Nadeel: 6036 Uiteindelijk blijft het toegeven om tot een gezamenlijke beslissing te komen. Voordeel: Om met vrienden tot de beslissing van een stad te komen Systeem zoekt voor je uit om een stad te bepalen 6042 Nadeel: Er niet uit kunnen komen, daardoor ruzie Je hebt 4 computers nodig Voordelen: Een ieder is verantwoordelijk voor de uiteindelijke beslissing Het systeem doet een onafhankelijk voorstel 6046 Nadelen: Er is weinig achtergrondinformatie over steden terug te vinden Misschien is het goed om twee selectierondes voor steden te houden voordeel 1: sneller voordeel 2: duidelijk nadeel 1: moet wachten op anderen bij stap 1....ik zou zelf wel willen browsen 6126 naar welke steden ik naar toe zou willen gaan nadeel 2: kan niet direct boeken...weet ook niet wat voor hotels ik kan gaan nemen 6199 voordeel: makkelijk voordeel: overzichtelijk nadeel: alleen stedentrip nadeel:mist nog wat functinoaliteit 6278 versimpeld en bespaard tijd nadeel: moet wachten voordat er resultaten zijn voordeel is dat het gemakkelijk is om met de groep een reisje in te gaan plannen. Als het systeem compleet is zie ik geen 6350 nadelen in van dit systeem 6428 6507 bepaart tijd en organisatie, tis ook wel eerlijk, iedereen heeft een mening nadeel 6581 6656 6735 6807 6882 6962 Je hebt al snel een vooroordeel van een stad en die worden niet ontkracht. Je kan niet van te voren al een maximum aan het buget zetten waardoor je misschien vooral steden krijg die te duur zijn. Ook zou een prijs voorkeur van de mensen goed zijn vooral als dit niet ergens staat met naam zodat mensen met een krap buget dit niet expliciet hoeven te melden. Er 7249 komen steden uit waar je nooit aan gedacht zou hebben. En mensen worden getriggered om na te denken over wat zij van een vakantie verwachten. En het is een platform om makelijk je mening neer te zetten zonder dat een paar personen de discussie domineren. Voordelen: Je leert meer van een stad en dat kan je keuze beinvloeden. Meestal, bij mij althans, zijn het altijd de 7253 bekendste steden die je kiest. Leidt tot leuke discussies. Nadelen: Veel uitzoeken voor: - je hoeft niet meer per se samen te komen of eindeloos te emailen met elkaar - je hebt alle data overzichtelijk bij elkaar - je krijgt wel meteen een vakantie gevoel als je al die verschillende bestemmingen bij elkaar ziet na: - Het is een 7256 nieuw soort systeem, ik ken geen andere systemen zoals dit, dus de "gebruiker" zal moeten aangeleerd worden dat dit het "nieuwe vakantie plannen" wordt. Het is een goed en handig systeem, het maakt veel dingen makkelijker en de beslissing kan vlotter gaan. Nadeel is wel dat je vaak op andere inputs moet wachten voordat je verder kan gaan met beslissen. Nadeel is ook dat je geen chat-functie 7262 heb, misschien is het handig als je dit wel heb als toevallig meerdere mensen van je groep online is. En nadeel is ook dat je uiteindelijk wel bij elkaar moet komen om de uiteindelijk beslissing te nemen. - meer inzicht in de voorkeuren van je reisgenoten - heel veel steden zijn minder bekend en door het zien van deze in je lijst geef je ze op zijn minst een kans. Bijvoorbeeld van de oostblok landen zijn er bij veel steden waar je eigenlijk niks 7341 vanaf weet. - de vraag blijft of je groep niet te eigenwijs is om alsnog een andere stad te kiezen - veel steden die je aanvankelijk nog een kans gaf die vielen onverwacht af. De preferentie categorieën zijn niet even duidelijk. 'vals spelen' kan makkelijk met het waarderen van de steden via de sterren. Dit proces is een goede oriëntatie van wat de groep leuk vindt, vanuit dit systeem kan je altijd nog verder 7416 beslissen. Hiernaast kan ik me goed voorstellen dat men zich houdt aan de uitkomst van het systeem. Het beslissingsproces wordt behoorlijk versneld door dit programma en iedereen kan individueel zijn bijdrage leveren aan het proces als die bijvoorbeeld de underdog is bij een discussie. Ps, veel feedback staat al bij categorie 1! Voordelen: * Beslissing die meerdere mensen moeten maken gaat op deze manier veel sneller. Er zitten altijd wel opties bij die je aanspreken dus je voelt altijd wel dat je preferenties gewaardeerd worden. * Erg leuk om te bemerken dat de eindelijke nummer 1 ons allemaal aanspreekt. Dat bewijst maar weer dat het 7491 systeem werkt en efficient is! * Mooie layout! Nadelen: * Sommige elementen van categorieen mogen nog wat aangepast worden zodat minder overlap bestaat. * Ik vond de stap waarbij je voor het eerst alle steden ziet wat moeilijker om te voltooien. Ik merkte dat ik meer ahvp mijn 'gevoel' bij de steden mijn keuzes maakte, dan gebruik te maken van de statistieken. Wellicht kan je per stad wat meer beschrijvende informatie geven, zodat ze meer tot de verbeelding spreken. Voordelen: - inzicht in waarom je zelf voorkeur hebt voor de bepaalde vakantiebestemmingen - inzicht in het beslissingsproces van de groep; het geheel is deels democratischer geworden nu - onverwachte steden komen boven drijven, wat verrassend kan werken Nadelen: - veel mensen bedenken niet van tevoren wat ze in het algemeen willen bij 7570 een vakantiebestemming, maar hebben al enkele bestemmingen in hun hoofd. Deze zouden (ook) kunnen dienen als vertrekpunt, niet alleen de persoonlijke 'interesses'. - sommige deelnemers zijn al in sommige van de te kiezen bestemmingen geweest. Deze beperking wordt in de huidige versie van het systeem nog niet opgenomen. - er zijn relatief weinig handvatten voor discussie doordat er nog onvoldoende informatie over de mogelijke bestemmingen wordt
191
weergegeven. Meer (visuele) informatie kan de discussie beter op gang brengen waarschijnlijk. pro: * snelle manier om een goede overzicht van steden te krijgen (zoeken/filter) * sessies vinden onafhankelijk plaats, iedereen kan in zijn eigen tijd bezig zijn con: * ratings zijn soms lastig en evt. te abusen * grafieken en extra info zijn soms 7717 moeilijk te interpreteren, mschien extra informatie/commentaar (van anderen) over steden (bv. van andere sites) zou leuk zijn. --> evt. een suggestie Voordelen: - Iedereen op zn eigen gemak gegevens invoeren en keuzes maken - Iedereen weet waar de uiteindelijke keuze 7721 op gebasseerd is Nadelen: - Iemand kan sterk benadeeld worden ondanks score en input gegevens. Uiteindelijk wordt keuze bepaald door de meeste stemmen
192
20.11 20.11.1
One Sample t-test
Generate step One-Sample Statistics N
Mean
Std. Deviation
Std. Error Mean
Effort
120
5.54
1.425
.130
Expressiveness
120
5.53
1.130
.103
Method
120
6.06
1.031
.094
Objective
120
5.49
1.402
.128
Presentation
120
5.17
1.491
.136
Features
120
5.31
1.314
.120
One-Sample Test Test Value = 5 95% Confidence Interval of the Difference t
df
Sig. (2-tailed)
Mean Difference
Lower
Upper
Effort
4.163
119
.000
.542
.28
.80
Expressiveness
5.172
119
.000
.533
.33
.74
11.240
119
.000
1.058
.87
1.24
Objective
3.841
119
.000
.492
.24
.75
Presentation
1.224
119
.223
.167
-.10
.44
Features
3.409
119
.001
.308
.07
.55
Method
193
20.11.2
Reduce step One-Sample Statistics N
Mean
Std. Deviation
Std. Error Mean
Effort
120
5.38
1.403
.128
Expressiveness
120
5.69
1.143
.104
Method
120
5.91
1.123
.102
Objective
120
5.71
1.253
.114
Presentation
120
5.40
1.286
.117
Reveal
120
6.13
1.100
.100
Graph
120
5.10
1.423
.130
City descriptions
120
4.41
1.417
.129
Decision
120
3.49
2.062
.188
One-Sample Test Test Value = 5 95% Confidence Interval of the Difference t
df
Sig. (2-tailed)
Mean Difference
Lower
Upper
Effort
2.929
119
.004
.375
.12
.63
Expressiveness
6.626
119
.000
.692
.48
.90
Method
8.863
119
.000
.908
.71
1.11
Objective
6.193
119
.000
.708
.48
.93
Presentation
3.407
119
.001
.400
.17
.63
11.291
119
.000
1.133
.93
1.33
.770
119
.443
.100
-.16
.36
City descriptions
-4.574
119
.000
-.592
-.85
-.34
Decision
-8.013
119
.000
-1.508
-1.88
-1.14
Reveal Graph
194
20.11.3
Clarify step One-Sample Statistics N
Mean
Std. Deviation
Std. Error Mean
Objective
120
5.65
1.333
.122
Presentation
120
5.75
1.087
.099
Reveal
120
5.86
1.007
.092
Awareness
120
5.36
1.442
.132
Voting
120
5.65
1.082
.099
Decision
120
4.14
1.848
.169
Effort
120
5.76
1.283
.117
Expressiveness
120
5.64
1.165
.106
Method
120
5.79
1.187
.108
One-Sample Test Test Value = 5 95% Confidence Interval of the Difference t
df
Sig. (2-tailed)
Mean Difference
Lower
Upper
Objective
5.344
119
.000
.650
.41
.89
Presentation
7.561
119
.000
.750
.55
.95
Reveal
9.340
119
.000
.858
.68
1.04
Awareness
2.721
119
.007
.358
.10
.62
Voting
6.581
119
.000
.650
.45
.85
-5.087
119
.000
-.858
-1.19
-.52
Effort
6.473
119
.000
.758
.53
.99
Expressiveness
6.032
119
.000
.642
.43
.85
Method
7.303
119
.000
.792
.58
1.01
Decision
195
20.11.4
Evaluation step One-Sample Statistics N
Mean
Std. Deviation
Std. Error Mean
Expressiveness
120
5.29
1.279
.117
Method
120
5.30
1.400
.128
Objective
120
5.72
1.022
.093
Presentation
120
5.67
.973
.089
Information
120
5.02
1.432
.131
Reveal
120
5.71
1.072
.098
Awareness
120
5.39
1.416
.129
New set
120
5.02
1.690
.154
Decision Support
120
5.13
1.289
.118
Effort
120
5.42
1.459
.133
One-Sample Test Test Value = 5 95% Confidence Interval of the Difference t
df
Sig. (2-tailed)
Mean Difference
Lower
Upper
Expressiveness
2.497
119
.014
.292
.06
.52
Method
2.348
119
.021
.300
.05
.55
Objective
7.679
119
.000
.717
.53
.90
Presentation
7.505
119
.000
.667
.49
.84
.128
119
.899
.017
-.24
.28
Reveal
7.237
119
.000
.708
.51
.90
Awareness
3.030
119
.003
.392
.14
.65
.108
119
.914
.017
-.29
.32
Decision Support
1.133
119
.260
.133
-.10
.37
Effort
3.129
119
.002
.417
.15
.68
Information
New set
196
20.11.5
Trip.Easy in General One-Sample Statistics N
Mean
Std. Deviation
Std. Error Mean
Effective
120
5.76
.979
.089
Handy
120
5.79
.897
.082
Fair process
120
5.63
1.391
.127
Influence
120
5.65
1.120
.102
Easy
120
5.73
1.136
.104
Objective
120
5.39
1.434
.131
Efficient
120
5.94
1.079
.099
Effort
120
5.29
1.279
.117
One-Sample Test Test Value = 5 95% Confidence Interval of the Difference t
df
Sig. (2-tailed)
Mean Difference
Lower
Upper
Effective
8.488
119
.000
.758
.58
.94
Handy
9.666
119
.000
.792
.63
.95
Fair process
4.923
119
.000
.625
.37
.88
Influence
6.357
119
.000
.650
.45
.85
Easy
7.074
119
.000
.733
.53
.94
Objective
2.993
119
.003
.392
.13
.65
Efficient
9.559
119
.000
.942
.75
1.14
Effort
2.497
119
.014
.292
.06
.52
197
