Effective Prototyping in Embedded Systems

Effective Prototyping in Embedded Systems Comparison of high- and low-fidelity prototyping

Thesis Submitted in fulfillment of the Requirements for the degree of

Master of Science in Software Engineering by ing. Dennis Verhamme born in Naarden, the Netherlands

drs. Hans Dekkers

Nico Nootebos

25-08-2009


This thesis is divided into two parts. The first part is the scientific article, which describes the performed research. The second part (Part 2) describes a preliminary survey of the development process used at Betronic Nederland and problems in different projects, Part 2 is written in Dutch.

1


Dennis Verhamme Betronic Nederland BV Universiteit van Amsterdam – Master Software Engineering

Abstract Early requirements verification by the user is needed in order to reduce change requests in a late stage of the development process. A good method to involve the user in the development process is by prototyping. However, two major difficulties for prototyping embedded systems are the costs of constructing dedicated hardware and time pressure. The findings presented in this study provide novel insight into the value of information that is gathered by prototyping in embedded systems. An empirical study examined usability problems generated by two prototypes and used the data from a real life development project to validate this feedback. The prototypes reflected the original requirements information of the real life product. In addition, the fidelity of the two prototypes and participation parameters were varied in order to gain better results. During this experiment the participants were asked to complete tasks according to a protocol. The results show that it is possible to create useful prototypes with requirements information in an early stage of the development lifecycle. Furthermore, the high-fidelity prototype (computer-based simulation) was able to indicate twice as many usability problems relevant to the actual reference product in comparison with the low-fidelity (paper-based) prototype. Finally, the paper-based prototype generated more new ideas. However, the amount of non-relevant ideas generated by the paper-based prototype is much higher in comparison with simulation. Keywords Prototyping, low- and high fidelity, requirements change, embedded systems

1. Introduction Even when delivered on time and within the budget, a project can fail if it does not meet the users’ needs or expectations (Standish2001). The classic failure of software development is that the delivered product only vaguely matches the expectations of the customer (Wiegers1996). Recent research points out that the mismatch with the users’ expectation still restricts the success of software projects (Ernst&Young2008), and thus it is important to clarify the needs and expectations of the users. The more a customer or user can work with the product during the development process, the better they understand it. This growing comprehension by the customer is the major source of requirements changes (McConnell04; Jones1996). Change requests on requirements are initiated both by designers and users but many changes occur after users see the application's interface and output (Rooijmans1996; Jones1996; Genuchten1991). The costs to correct these requirements defects increase significantly when they occur later in the development process (Boehm1984). To illustrate, fixing defects after software has been released can work out to be 100 times the costs of repair during the requirements phase at the beginning of the development cycle (Boehm2004). Changes in requirements are notorious for delaying software development projects, and this is particularly true in embedded systems where hardware and software are developed in parallel (Rooijmans1996). If a change-request affects the hardware, it could be very expensive to change a particular component especially when parts have already been ordered (Graaf2003). 1.1 Case study The findings presented in this study provide novel insight into the value of information that is gathered by prototyping in embedded systems. A comparison is made between usability problems obtained from a real life product development and problems retrieved from new prototypes that are derived from the same requirements information. In addition, fidelity and participation parameters are varied to gain better results from the prototypes.

2

1.1.1 Domain description This research took place at Betronic Nederland, a company that specializes in developing customized embedded systems. The case study used a real developed product involving a user interface as a reference. The process used for developing the dedicated hard- and software followed a sequential approach. During the requirements gathering, a paper-prototype (figure 1) of the user interface was developed to illustrate the screen layout and the order thereof to potential users. This prototype was not used to assess usability problems. With the gathered requirements, both the hard- and software were developed resulting in a fully functional prototype (figure 1) that was used in the field test. The delivery of such a complete prototype typically occurs in a late stage of the development process (months after requirements gathering), and is the first time that the customer or users are able to interact with the product. Due to unexpected operations and flaws in functionality in the user interface, change-requests are issued in this late stage of development. These changes often cause delay in the development process and costs. This research aims to assess the value of prototyping in the prevention of those delays and the associated costs by considering way to identify and solve problems at an earlier stage of product development.

Requirements phase

Trial model phase

Prototype phase

Paper-based prototype

Field-test phase

Production

Fully functional prototype delivery

Figure 1: A timeline presenting the development cycle and the delivery and usage of prototypes during the development.

1.1.2 Why is prototyping difficult for embedded systems? Time pressure and costs are especially prominent in many market domains that sell embedded systems. Time-to-market gains in embedded systems development result in high potential revenue gains due to earlier introduction of the product and with that deeper market penetration (Solingen2002). Embedded system consists of both hardware and software that are tailor-made to suit a specific purpose. The hardware involves costs of both components and of designing and constructing the often unique printed circuit boards and mechanics. Software running on embedded systems executes on a physical platform (the dedicated hardware) and is designed to perform a specific set of functions (Henzinger2006). It is therefore not surprising that product design and the accompanying time constraint involved in this process pose two major difficulties for prototyping embedded systems. 1.2 Research question The general research question of this thesis is: How should prototyping be applied in order to identify usability problems that typically arise in a late stage of product development, at an earlier stage? This paper is organized as follows: in-depth information on embedded systems will be presented in section 2. Section 3 summarizes related work found in literature to provide a framework for this research. Section 4 describes the definition of the different parameters used in prototyping as well as describing the problem of prototyping in embedded systems. The executed experiment of this research is described in section 5, and the results, their interpretation, and main conclusions are presented in section 6. Section 7 concludes with a general discussion.

3

2

Embedded systems

2.1 What is an embedded system? An embedded system is an engineering artifact involving hard- and software that is constrained by reaction to a physical environment, because it reacts continuously to its environment at the speed of the environment (Henzinger2006; Edward2002). This dependence on the time-scale of the environment leads to strict timing constraints in the completion of its tasks. The general purpose of an embedded system is to regulate a physical device by sending control signals to actuators in reaction to input signals. These input signals can be provided by its users and by sensors capturing the relevant state parameters of the system (Broy1999). Because embedded systems rarely interact with only a single physical process, an embedded system must simultaneously react to these input signals and retain control over the actuators at the same time (Edward2002). 2.2 Constraints in embedded systems In addition to the aforementioned timing constraints, embedded systems are also constrained by predefined hardware and hardware costs (Graaf2003). In addition, embedded systems are mostly applied to stand-alone devices and thus have a large dependency on power consumption and limited memory resource (Solingen2002). Embedded software executes on devices such as cars, airplanes, mobile telephones, robots, toys and so on (Edward2002). For example, embedded systems incorporate in the popular iPhone, must be both small and flexible enough to support many different operations. Moreover, unlike our typical desktop computers, reliability can be of utmost importance in embedded systems. An embedded system does not terminate unless it fails, and must operate properly at all times especially in mission critical applications such as medical equipment (Broy1999). 2.3 Sequential development in embedded systems From a historical point of view, the hardware aspects of embedded systems development follow a sequential approach, where all phases in the development process take place in a specific order. Because the hardware of the embedded system has to be constructed, this approach is necessary to accommodate lead times for production facilities, production subcontracts, and long-lead critical component orders (Boehm2001a). The sequential development process is still widely used in industry (Liversidge2009; Laplante2004). With a sequential development model, product testing and evaluation by the customer often only occur at a late stage of the development cycle. This is especially true in embedded systems design where the hardware has to be physically built before one may speak of a product. This often results in change-requests on the requirements at a late stage, in particular for the user-interface, resulting in large increases in development cost. Early user involvement and early verification of the user interface is needed to ensure that requests are issued at earlier stages, and thereby reducing these costs (Rooijmans1996).

4

3

Related work Pro Low fidelity

Hall (1999)

Pro High fidelity 67% of the problems were found with low-fidelity prototypes over 90 % of problems that were found by high-fidelity prototypes

Nielsen (1990)

Usability problems are harder to find in a paper-based prototype than in an interactive prototype

Sauer (2009)

Given tasks are performed faster with fully functioning high-fidelity prototypes

Sefelin (2003)

No significant differences were found between the number and the quality of critiques and suggestions is not affected by the fidelity of the prototype. To increase the comfort of the participants (major factor of a successful usability test) a computer-based prototype is recommended. The appropriate method of prototyping should also depend on the participants’ characteristics.

Virzi (1996)

Low-fidelity prototypes were found to be as effective as high-fidelity prototypes at detecting usability problems.

Timmerman (2008)

Low-fidelity prototypes with cooperative user participation generate more ideas, in comparison with high-fidelity interactive prototypes and low-fidelity and high fidelity that are demonstrated to the participants. Cooperative prototyping produces over 4 times more unique results than demonstrated prototypes. Low-fidelity produces 55% more unique ideas than interactive prototypes

Rettig (1994)

Using low-fidelity (paper-based) prototypes results in more precise requirements than the use of a high-fidelity prototype

Table 1: A summary of conclusions found by different research projects in the literature

The main contribution of this research compared to the current (see table 1) is that this project used the data from a real life development project to validate the feedback generated by the prototypes. Two papers, Virzi (1996) and Sauer (2009), employed finished products as fully functional high-fidelity prototypes. However those products were post-production models already available in stores. It is therefore highly likely that usability tests had already been performed on these products, which would imply that more information was available than that would have been available from the original requirements documentation alone. The research projects of Ward (1991) and Kenney (1994) discussed in (Hall1999, p85-91), Sefelin (2003), and Nielsen (1990), compared the usability problems found by low-fidelity prototyping to those found by high-fidelity prototyping. All studies, except the one conducted by Sefelin (2003), found that more usability problems were found in high-fidelity prototypes. However, the results of these studies do not show that the problems found by prototyping would be the same problems found in the real life product. Timmerman (2008) found that more ideas were proposed with cooperative paper-based low fidelity prototype sessions. He measured the number of unique ideas generated by low-fidelity and high fidelity prototypes used in both a cooperative and a demonstrated way. The number of unique ideas generated by the participants does not predict if all identified ideas have to be realized in the product. High fidelity prototypes like simulations are a useful back up to the requirements specifications. However despite the use of simulations, the number of change-requests issued in a late stage in the development cycle did not decrease significantly in the research project performed by Rooijmans and coworkers, and it was left to future investigation to reveal if the simulations are realistic enough (Rooijmans1996). Despite the research projects described above and the fact that various types of prototype are widely used in industry, to date, there little comparative research on the utility of prototypes at different fidelity levels (Sauer2009).

5

4

Prototyping

A good method to involve the user in the development process is by prototyping. A prototype of the tobe developed product can help a user or customer to gain more insight into the requirements, which may not be possible using only a specification document (Robertson1999). Moreover, prototypes are useful as back up of the requirements (Rooijmans1996). When applied earlier in the development stage, prototyping can help to reduce lead-time overrun and requirements creep (e.g., uncontrolled requirements changes) (Jones1996). Prototyping can lead to better designs, better matches with the users’ needs, and improved maintainability (McConnell2004). It is widely recognized that the use of prototyping has a positive effect on the usability of the product (Gordon1993; McConnell1996), as it allows a user to get a feel for how the product will operate and consequently make well-informed recommendations on how to improve the user interface (Rudd1996). 4.1 What is prototyping? Prototyping is the process of developing a trial version of a system (or its components and characteristics) in order to clarify the requirements of this system or to reveal critical design considerations (Gordon1993). Prototyping is a widely used method of clarifying requirements (Sauer2009; Neill2003) and is stated as best practice in software development (McConnell1996). Prototyping can result in less code and less effort to gain the same level of performance during the specification of a product, which in turn can result in more coherent designs and software that is easier to integrate (Boehm et al.1984). Prototypes can reduce the requirements creep by somewhere between 10% and 25% (Jones1996). Gordon and Bieman (1993) showed that products developed by using prototyping are easier to use and correspond better to the users’ needs (Gordon1993). A prototype is a powerful tool that can help correcting misunderstandings and ambiguities of requirements specification in an early stage of the development process (Gomaa1981). 4.2 Prototyping parameters Literature shows that varying both the parameter of the prototype’s representation (Hall1999; Sauer2009) and varying the parameter of user participation (Timmerman2008; Rudd1996) result in better feedback quality and therefore better requirements quality (Robertson1999). The quality of software depends largely on the quality of requirements (Boehm2001b). Figure 2 illustrates an overview of these parameters based on existing theory and research. The first parameter, the level of detail in the representation of the prototype, can be generally divided into lowfidelity and high fidelity as described in section 4.2.1. Moreover, the fidelity can be subdivided into more dimensions as described in section 4.2.1.1. The second parameter, the user participation, is described in section 4.2.2. 4.2.1 Prototyping fidelity Low fidelity prototypes have limited function and limited interaction with the user and they are built to show the screen layouts and concepts rather than interaction between the user and the system. High fidelity prototypes have complete functionality and are interactive. High fidelity prototypes address the issues of navigation and flow and of matching the design and user models of the system. High-fidelity prototyping is used for exploration and testing the product (Rudd1996). Low fidelity prototypes are constructed with paper and pencil or simple storyboard tools and require little or no programming skill on the part of the designer. This makes that low-fidelity prototypes be constructed early in the development cycle without a large investment in time and development costs (Rudd1996; Robertson1999). Because low fidelity prototypes are often demonstrated to, rather than exercised by, the user, it is more difficult to identify design inconsistencies and shortcomings. Low fidelity prototyping can provide little error checking which can result in that many important design decisions are overlooked. For a programmer it is more difficult to code to a low-fidelity prototype because the interaction and navigation is not fully implemented in the prototype the programmer has to make decisions about this details himself. Unlike low-fidelity prototypes, high-fidelity prototypes can be very useful as a living specification of the functional and operational requirements (Rudd1996). That these decisions are difficult, especially to programmers with little user-interface design experience is shown by Tetzlaf (1991). The programmers with little user-interface design experience consistently made bad design decisions even if an established design guide was used. The designers rely more on the pictures in the guidelines than on the supporting text in making design decisions. High-fidelity prototypes encourage the user to explore (Robertson1999).

6

A representative prototype can be available months before the product code, allowing usability testing to be initiated much earlier in the development cycle (Rudd1996). The programmer can use the prototype as a living specification of the functional and operational requirements (Rudd1996). Because the high-fidelity prototype looks much like the working product, the stakeholders maybe concentrated on the appearance and forget to make improvements on the functional requirements (Robertson1999). 4.2.1.1 Fidelity dimensions A prototype can differ from the final product (e.g. reference product) in multiple dimensions. Therefore Virzi (1996) proposed to distinguish between four dimensions of fidelity, namely the degree of functionality, similarity of interaction, breadth of features, and aesthetic refinement. The degree of functionality refers to the level of detail a function is modeled. Interaction refers to type of interface a prototype is modeled with (i.e. displays, controls). For instance, a button can be pushed by a mouseclick in a simulation. The breadth of functions refers to the amount of functions, according to the target product, are implemented in the prototype. And the aesthetic refinement refers to the similarity of physical properties such as shape, size and colors, between the prototype and target product. 4.2.2 User participation The traditional way of prototyping is by demonstration (Boehm1984). The software developer demonstrates the prototype to the stakeholder and shows his perception of the requirements. The problem with demonstrating is that, because the prototype is not actually exercised by the user, it is more difficult to identify shortcomings and design inconsistencies (Rudd1996). Users can be involved actively and creatively in the design and evaluation of early prototypes, this is called cooperative prototyping (Boedker1991). Another method of validating a prototype is to observe the problems that a user experiences during a predetermined task (Nielsen1993). This method is referred as ‘prototyping by protocol’ in this thesis. The experiment described in section 5 of this thesis uses the ‘prototyping by protocol’ method to ensure that the tasks performed by the participant are equal to the tests performed in the reference situation.

7

Prototype representation

User participation

degree of functionality

similarity of interaction

breadth of features

aesthetic refinement (Virzi1996)

Prototyping by demonstration

Cooperative prototyping

Protocolized prototyping Participants background (Nielsen1993)

(Boehm1984) (Rudd1996)

(Boedker1991) (Timmerman2008)

(Grønbæk1989) Number of participants

Requirements quality

(Nielsen1993)

Software quality (Boehm 2001a)

Figure 2: an overview of prototyping parameters based on the existing theory and research, varying the parameters of ‘prototyping representation’ or ‘user participation’ will gain better results.

4.3 Prototyping in embedded systems It is more difficult to create prototypes in an early stage in the development prototypes for embedded systems, because the hardware that reacts with the environment has to be physically built. As described in section 1.1.2 and building hardware involves costs. The user-interface of an embedded system can often be simulated independent from the rest of the hardware that has to be physically built. By separating the user-interface from the rest of the product, this part can be verified in an early stage during the development process. The question of which prototype is to be used for usability testing is strongly influenced by a number of constraints that are present in industrial design processes, notably time pressure and budgetary limitations. This usually calls for the use of low fidelity prototypes (e.g., paper prototype) because they are cheaper and faster to build (Sauer2009).

5

Experiment Design

5.1 The purpose of the experiment First and foremost, this experiment was set up to illustrate the possibility of building prototypes at an early stage in the development lifecycle, aimed at identifying usability problems in the user interface of a product. Second, this experiment was designed to show whether there were differences in output (e.g., user feedback) if the level of fidelity in the prototypes was varied. Third, by comparing the feedback with real life information, the value of the feedback generated by the prototypes could be assessed. And finally, this experiment was meant to gain insight in how to reduce scope creep caused by new ideas generated by interaction with the prototypes.

8

5.2 Reference environment As described in section 1.1.1, during the development of the actual reference product, a paper-based prototype was created of the user interface as part of the requirements gathering phase. This paperbased prototype was not used to assess usability problems by the customer or future end-users of the product, and acted only to demonstrate the logical sequence of screen layouts for the various actions that the user could perform. The first time the product and user interface could actually be tested for usability problems by the users was after delivery of the first prototype, which in this case was months after the original requirements phase. This prototype was a fully functional product including housing, hardware and mechanics and was identical to the final product. During the testing of this prototype, a number of usability problems were identified that were not initially uncovered in the paper-based prototype. The prototype indicates that a high fidelity simulation of the user interface allows user interaction to be tested more accurately due to the fact that it looks more like the product and has similarity of functionality with the product. 5.2.1 Customer and end-users The paper-based prototype was only used for demonstration of the different screen layout to the customers of the product. This paper-based prototype was not used to assess usability problems. The fully functional prototype (high-fidelity) was tested and assessed for usability problems by some real end-users (patients) using a protocol. 5.3 Experiment hypothesis Based on the previous section, the hypothesis of this experiment is stated as follows: A simulation (high-fidelity) prototyping session will result in identifying a higher number of usability problems relevant to an actual reference product and can be performed early on in the development lifecycle before the development of the actual physical product. 5.4 The reference product The original product, which was used as a reference for this experiment, was designed to help patients taking their medicine on time (figure 3). The pharmacist installs a roll containing plastic medicine bags in the product. Each time a pill has to be taken, the patient is alerted by the product (alarm sound and display). The patient has to draw a medicine bag from the product. By pushing on top of the product, the barcode containing the medicine bag’s content information is scanned and the bag can be thorn off by the patient. After scanning the medicine bag the product informs the patient about the Figure 3: the reference product next pill that has to be taken on the graphic display. The product maintains a wireless communication link with a data collection center and pharmacists. They monitor how the product is being used and can react on alarm signals. 5.5 The experiment prototypes The paper-based prototype (appendix A, figure A.3) and the simulation (appendix A, figure A.2) were based on the original requirements document concerning the reference product. The first prototype was a computer-based simulation and the second was a paper-based prototype. These prototypes reflected exactly the same requirements information that the original developers had at the time of developing the reference product. A tool used in the original product design was able to generate C-code from a drawn display layout. This C-code could be used directly in the original product and could also be used for the computer-based simulation. The computer-based simulation, written in C++, was able to show the display layouts to the user easily by using this generated C-code. High fidelity prototypes (e.g. computer-based simulations) are expensive to build (Rudd1996), but by making decisions like using the same API and generated C-code, less effort was needed to develop a computer-based simulation. The layout of the original product was drawn on a paper and used as a low-fidelity (e.g. paperbased) prototype. The original display layouts were printed and used to show the correct display to the participant during the session. Table 2 gives an overview of the differences between the prototypes used in the original development lifecycle (see section 1.1.1) and the prototype used in the experiment according to the overview illustrated in figure 2.

9

Purpose Similarity of interaction Degree of functionality Breadth of features Aesthetic refinement Representation Number of users

Paper-based prototype used in original development Illustration of the screen layouts Low Low Low Low Demonstrated to the users/customer Only the customer

High-fidelity prototype used in original development Fully functional hardware (prototype) High High High High Tested by users by a protocol --unknown--

New paper-based prototype used in experiment Illustration of the screen layouts Low-Medium Low High Low Tested by users by a protocol 3

New high-fidelity (simulation) prototype used in experiment Interactive simulation of the product High Medium-High High Medium-High Tested by users by a protocol 5

Table 2: an overview of the differences between the prototypes used in the original development and the prototypes used in the experiment.

5.6 Building the simulation The first version of the simulation (appendix A, figure A.1) included a button for each specific event that the system could support. Some buttons corresponded with physical actions that the participant could perform and a number were there to simulate system events such as ‘Pill time reached’. During the session the latter buttons confused the participant and the participant was not able to complete the tasks from the protocol independently. In the second version of the simulation (appendix A, figure A.2), the buttons used to simulate system events were replaced by automated events. For example, the previously mentioned ‘Pill time reached’ was placed on a timer. Only a couple of buttons were left to activate certain events, like placing a role or scanning the role. These buttons represented only the physical actions that had to be performed by the participant. By making the simulation more aesthetic and increasing the similarity of interaction with the real product (Virzi1996), the participant could complete the tasks from the protocol independently. 5.7 The experiment procedure In this experiment a protocol was used which was based on the test protocol for the original users. This protocol provided several steps to complete the session. Because each session was performed identically, possible learning effect and observers’ bias were minimized. The protocol was used to prevent overlooking tested functionalities of the original product. The participants in this experiment were selected on gender, age and occupation (table 3). This selection was made to achieve a reliable representation of the users of the product in real life (Grønbæk1989). Fidelity High High High High High

Age 42 66 32 28 62

Gender Male Male Male Female Female

Occupation Manager design Retired/information systems Aerospace engineer Psychologist Housewife

Low Low Low

33 58 55

Female Female Male

Project manager/web designer Teacher Psychiatrist

Table 3: Age, gender and occupation of the participants and the used form of prototyping (high- or low fidelity).

Before the experiment started, the product was described to the participant and the participant was asked to read the short instruction manual described in the protocol (Appendix B and C). After this instruction the participant was able to work independently with the product (Grønbæk1989). During the sessions the participant was observed and all uttered feedback (comments, ideas and problems) was written down by the observer. The participant was asked to think aloud while performing the tasks, which is a commonly recommended method for user testing (Nielsen1992). In case of confusion,

10

where the participant could not continue, the participant was allowed to ask questions. The observer wrote down the cause of the confusion. The users were divided into two groups. One group used the product with the interactive simulation and the other group used the product with the paper-based prototype. During the paper-based sessions the observer showed the correct display layout to the participant at a performed action. These display layouts and the actions with which they correspond were defined in a protocol. The participant performed the tasks independently as described in the protocol. During the simulation sessions, after the instructions, the participant performed the tasks independently as described in the protocol. The display layouts used in the simulation sessions were more interactive than the paper-based session. At a performed action by the participant, the simulation responded by showing information in the display and sounding signals. The prototypes were kept unchanged during the sessions, because the purpose of the experiment was to evaluate the prototypes (Grønbæk1989) 5.8

Assessment of the results

5.8.1 Assigning data into categories The gathered feedback from the prototyping sessions was subdivided into three different categories. The three categories are the barcode-scan, display/sound, and key handling and comprise the main physical parts of the system. By assigning categories it was easier to compare the original feedback with the feedback generated by the new prototypes. Precision and recall (section 5.7.2) were used to measure the differences in feedback gathered from the users of the original product and the feedback generated by the prototype sessions. 5.8.2 Precision and recall Two widely used statistical classifications for information retrieval are precision and recall. Precision can be seen as the measure of exactness or fidelity whereas recall is a measure of completeness. Precision can be defined as the number of relevant documents retrieved by search divided by the total number of documents retrieved by that search: precision =

tp tp + fp

Recall can be defined as the number of relevant documents retrieved by search divided by the total number of existing relevant documents: recall =

tp t p + fn

Where f n (e.g. false negatives) represents the number of existing data minus the data that match with the data retrieved by search, f p (e.g. false positives) represents the data that is retrieved by search but does not match with the existing data. And t p (e.g. true positives) represents the number of existing data that matches the data retrieved by search, also seen as relevant data (Cleverdon1967). Precision does not tell if all feedback generated by the prototype covers all feedback generated by the reference product. This measurement says more about the amount of false positives generated by the prototype. To be able to tell if the prototype can generate the same feedback as the reference product, the recall is more important. If the recall is low, it means that many comments mentioned by the reference product were not mentioned during the prototype sessions.

11

5.8.3 Feedback quality Determination of relevancy on the new feedback generated by the prototype sessions is very important because irrelevant information could affect the estimation strongly (Grimstad2004). When the scope of the project is not clearly defined, there is a threat of scope creep, in which new requirements are continually added during development (Wiegers2000). The new data collected during the experiment was judged on relevancy for the development of the original product. Comments on the graphical details were excluded (Sefelin2003). To prevent that the determination of relevancy was based on the bias of the observer (and developer of the prototypes) (Snyder2003), this determination was executed by the senior software developer who was also assigned as a requirements analyst for the original project. By this action all irrelevant data was filtered out in an independent way. The results were also judged on expensiveness to implement or fix by the analyst. This judgment was based on hardware related problems. Hardware related problems were stated as expensive.

6

Experiment Results

Participant

Session type

Similar usability problems

A B C D E

Simulation Simulation Simulation Simulation Simulation

5 5 5 6 5

F G H

Paper-based Paper-based Paper-based

4 4 2

New non-relevant New relevant usability problems usability problems 3 1 1 2 1 1 1 4 0 1 9 4 2

4 1 4

Table 4: The number of identified usability problems per participant. This table represents the number of usability problems similar to the problems identified in the reference product, the number of new usability problems stated as non-relevant by the analyst and the number of relevant new usability problems identified by the participants.

Table 4 shows the number of usability problems identified by the participants during the different sessions. These results are assessed in the following sections. 6.1 Results of unique usability problems The original reference product provided a total of twelve unique usability problems. A total of 23 unique usability problems were identified during the paper-based prototyping sessions and a total of 20 unique problems during the prototype sessions with the simulation.

12

100%

12

80%

10

60%

8

40%

6

20%

4

3 participants

14

5 participants

low-fidelity 3 participants

high-fidelity

2

0% precision relevant

recall

0 Paper-based low-fidelity

Figure 4: Precision and recall: total amount of unique usability problems identified with prototyping

Simulation high-fidelity

Reference product

Figure 5: total amount of unique usability problems identified with prototyping that are similar to the usability problems obtained from the actual reference product

The experiment results show that the precision with the paper-based prototype is 45% against 59% with the simulation. The recall for the paper-based prototype is 42% against 83% for the simulation (figure 4). The total amount of unique problems found that are similar to the reference feedback were 5 problems found by the paper-based prototype against 10 unique similar problems found by the simulation (figure 5). Figure 5 shows the amount of unique similar problems found with both prototype sessions in comparison with the amount of unique problems found with the reference product. The paper-based sessions were done with 3 participants and the simulation sessions were done with 5 participants. To measure with an even amount of participants, for the simulation session the data retrieved from the participants with the highest score and the lowest score were not taken into account. Figure 5 shows the amount of 8 unique usability problems for the simulation session it 3 participants and the total amount with 5 participants.

20 18 16 14 12 10 8 6 4 2 0 Paper-based low -fidelity no n-relevant ideas

Simulation high-fidelity relevant ideas

Figure 6: total amount of new unique usability problems identified with prototyping.

A total number of 18 unique new usability problems were identified with the paper-based prototyping session and a total number of 10 unique new problems were identified with the simulation session. After validating for relevancy, a total amount of 6 unique new (relevant) problems were identified by the participants who joined the paper-based sessions. During the simulation session, a total of 7 unique new relevant usability problems were uncovered.

13

Figure 6 shows that a total of 12 not relevant unique new ideas were proposed by the participant of the paper-based prototype sessions, against 3 new not relevant unique ideas during the simulation sessions.

high-fidelity

low -fidelity

1,0

20,0 15,0

5,0

barcode scan displaysound

10,0 5,0 0,0 barcode displaykey scan sound handling Figure 7: the number of unique usability problems per category for high-fidelity (simulation) and low-fidelity (paper-based) prototypes

17,0

key handling

Figure 8: the number of unique usability problems per category that were identified during the paper-based prototype sessions

The number of unique usability problems for each of the categories stated in section 5.3.1 is shown in the figures above. Figure 7 shows the relationship between the number of unique usability problems per category for the simulation sessions (high-fidelity) and the paper-based sessions (low-fidelity). Figure 8 shows the number of unique usability problems per category issued during the paper-based sessions and figure 9 shows the number of unique usability problems per category that were identified during the simulation sessions.

2,0 6,0

barcode scan displaysound

12,0

key handling

Figure 9: the number of unique usability problems per category that were identified during the simulation sessions

6.1.1 Expensive usability problems Two expensive (hardware related) problems were initially uncovered in the reference data. After selecting the unique problems that were issued during both sessions on expensiveness, the amount of expensive problems for the paper-based and simulation were both 3 problems, where 1 problem was stated as not relevant. 6.1.2 Average values of the results As shown in figure10, participants were able to identify an average of 3.3 (out of 12) matching comments in the low-fidelity paper-based prototype. This corresponds to 28% of the usability problems identified in the reference data. The amount of matching feedback (figure 10) with the original reference feedback was higher with the simulation than with the paper prototype. With the high-fidelity prototype, participants were able to identify an average of 5.2 (out of 12) matching comments, corresponding to 43% of the usability problems matching the comments identified in the reference data. The amount of new feedback that was generated with the simulation was significantly lower than the new feedback that was generated with the paper prototype. The average of 2.8 new comments

14

compared with 8.0 new comments that were issued with the low-fidelity prototype (figure 11). The new comments generated with high fidelity were about 35% from the amount of new comments generated with the low-fidelity prototypes.

6,0

9,0 8,0

5,0

7,0

4,0

6,0

3,0

5,0 4,0

2,0

3,0 2,0

1,0

1,0

0,0

0,0 Paper-based low -fidelity


Paper-based low -fidelity

Figure 10: The average number of similar comments found compared to the 12 comments generated by the reference product


Figure 11: average number of new (both relevant and irrelevant) comments found compared to the comments generated by the reference product.

The high fidelity simulation provides significantly more similar feedback according to the reference data on the barcode scan functionality than the paper-based prototype (figure 12). On average 1.6 matching comments were issued during the high-fidelity prototype session (figure 13) in comparison to an average of 0.7 matching comments during the paper-based sessions (figure 14). high-fidelity

low -fidelity

3,0 2,5 2,0 1,5 1,0 0,5 0,0 barcode scan

displaysound

key handling

Figure 12: The differences between the average numbers of similar feedback per category for the high-fidelity and lowfidelity prototypes

barcode scan

display-sound

key handling

barcode scan

display-sound

key handling

0,7

1,0 1,6

2,6

Figure 13: average number of similar feedback per category for the high-fidelity prototype

1,0

1,7

Figure 14: average number of similar feedback per category for the low-fidelity prototype

15

More new usability problems related to the presented texts and screen layouts were identified by the paper-based prototype than with the simulation (figure 15). There was no significant difference between the paper-based prototype and the simulation in identifying new comments related to the categories, barcode-scan and key-handling. high-fidelity

low -fidelity

3,0 2,5 2,0 1,5 1,0 0,5 0,0 barcode scan

displaysound

key handling

Figure 15: average number of new comments per category for high-fidelity and low-fidelity prototypes

On average 2.7 comments were issued about the display layout and texts during the prototype sessions with a paper-based prototype (figure 17) in comparison with an average of 1.2 comments that were issued with the simulations (figure 16). There were many new comments about the icons used in the display with the paper prototype. barcode scan

display-sound

key handling

0,2

barcode scan

display-sound 0,3

key handling

0,3

0,4

1,2

Figure 16: average number of relevant new comments per category for the high-fidelity prototype

2,7

Figure 17: average number of relevant new comments per category for the low-fidelity prototype

6.2 Interpretation of the results The results show that overall more usability problems were identified with paper-based prototyping, the overall number for was 23 for paper-based over 20 for simulation sessions. The precision with unique usability problems was higher for the simulation sessions, which concludes that similar issues were found as with the reference product and less new problems were issued during the sessions. More important was the recall that was significantly higher with the simulation sessions. 83%with simulation against 42% with paper-based prototyping. The recall shows the amount of similarity between the reference data and the data obtained from the prototyping sessions. The unique similar issues found by simulation were 10 out of 12 with 5 participants. This concludes that simulation covers most of the usability problems in the real product. In this experiment the results show that the number of usability problems found by simulation is twice the amount found by paperbased sessions. The first impressions of the results show that paper-based provides significant more new issues than the simulation sessions, but after validation on relevancy by the analyst the number of new relevant issues are almost the same with simulation and paper-based sessions. The amount of nonrelevant issues is significantly higher with paper-based sessions.

16

Two expensive problems were initially uncovered in the feedback obtained from the reference product. However both high- and low fidelity prototypes exposed 3 expensive problems. The analyst for both the paper-based prototype and the simulation stated one of the problems as not relevant. The problem stated as not relevant was the same problem for both the simulation and the paper-based prototype. The other two problems found by the prototyping sessions were similar to the problems found by the reference product. This concludes for this experiment that a paper-based prototype and simulation are both able to find all the expensive usability problems. The results show that more usability problems affecting the barcode scan category, that were relevant to usability problems from the actual reference product, were identified with the simulation sessions. This concludes that a simulation better represents the mechanical aspect of the product.

7

Discussion

The results from this experiment show that the simulation (high-fidelity) sessions provided significant more similar usability problems according to the reference product. This confirms the experiment hypothesis stated in section 5.3. The general research question was stated as: How should prototyping be applied in order to identify usability problems that typically arise in a late stage of product development, at an earlier stage? The experiment proposed in this thesis shows that it is possible to create useful prototypes with requirements information in an early stage during the development. The experiment also shows that the same usability problems from a real life product can be found earlier with prototyping. Although both a paper-based prototype and simulation found the expensive usability problems, simulations with a high similarity of functions found significantly more usability problems than paperbased prototypes. 7.1 What did we learn from this research The experiment shows that it is possible to create useful prototypes from the same early requirements information that was available for the developers of the original product at that time. By creating these prototypes it is possible to obtain usability problems that occurred in a real life situation. Although the results show that it is possible to find similar usability problems with paperbased prototypes, significant more similar usability problems can be found with a simulation. The paper-based prototype and a simulation can be created with early requirement information long before the physical hardware has been built. By building a simulation of the user interface, there is no need for hardware construction in earlier stages of the development lifecycle and this reduces hardware costs for embedded systems. By making the simulation more aesthetic and increasing the similarity of interaction with the real product, the participant could complete the tasks from the protocol independently. As discussed in section 5.2 the original paper-based prototype from the reference product was demonstrate the logical sequence of screen layouts to the customer. The amount of change requests proposed on the user interface during interactive tests with the final prototype indicates that it is important that the user perform tasks themselves in order to find usability problems. Paper-based prototype sessions provide more new ideas, however a high amount of new ideas are stated as not relevant. To summarize these findings: • It is possible to create useful prototypes from early requirements information without constructing hardware • Prototypes can find the same usability problems as in real life • High-fidelity prototypes (simulation) find more usability problems similar to real life • Prototypes reduce the costs of making temporary hardware • Similarity of interaction and aesthetic refinement are important to obtain better results • Better results from prototypes by letting the participant perform the actions themselves • Paper-based prototypes generate more new ideas, however a higher amount of irrelevant ideas

17

7.2 Effective prototyping Participants involved in paper-based sessions are more focused on the substance of the screen layouts and propose more creative new ideas than with simulation sessions. However in this research many new ideas were stated as non-relevant problems, which can lead to scope creep. To assess non-relevant and expensive usability problems an independent analyst is needed. As stated by Rudd et al. (1996), High fidelity prototypes are more effective for requirements validation and low-fidelity prototypes are more effective for requirements gathering. To gain most profit from prototyping, low-fidelity prototypes should be used during requirements gathering and using highfidelity simulations that reflect the same interaction, as the intended product should validate the requirements. High-fidelity prototypes are more expensive to create than paper-based prototypes. Using the same generated code for the simulation as for the final product as proposed in section 5.5 does not dramatically reduce the software aspect, still the simulation reduces the development costs because there is no need for building hardware in an early stage. To gain better results from high-fidelity prototypes (simulations) instantly, a high similarity of interaction and aesthetic refinement must be implemented. This reduces confusing the participant while using the product. 7.3 Future work The research described in this thesis provided novel insight for prototyping in embedded systems, but to obtain a profound answer to the research questions more research is needed. The amount of new usability problems issued during the paper-based session varies strongly for each participant, more participants are needed to examine if the amount stabilizes. In addition the amount of similar usability problems relevant to the reference product were rather stable during the simulation sessions, which can mean that the recall never reaches the hundred percent. Due to the rather low amount of participants in this experiment, the amount of gathered results is too low for well-grounded conclusions. The results from this research project show that it is possible to generate valid feedback in an early stage of the development cycle. This should decrease the amount of requirements change requests after delivery of the product in later stages in the development cycle. To examine if the amount of change requests at a later stage are indeed reduced, future research is required.

Acknowledgements This thesis is the result of the research project carried out in fulfillment of the requirements for the degree of Master of Science in Software Engineering. The research project was performed at Betronic Nederland in cooperation with the University of Amsterdam. I would like to take this opportunity to thank everybody who helped me during this research. First, I would like to thank my supervisor Hans Dekkers for his valuable, insightful feedback and for his guidance during my research. Second, I would like to thank Jurgen Vinju who helped me out in executing the experiment and Paul Griffioen for his valuable insights. Third, I would like to thank Nico Nootebos for supporting me during this research and for making it possible for me to do the Master Software Engineering. Fourth I would like to thank the teachers at the Master Software Engineering for providing me valuable insights into the world of software engineering. Fifth, I would like to thank all the participants of my experiment and especially Alistair and Jonathan Vardy. Last of all, I am truly grateful to my girlfriend Daniëlle who has been very patient and helpful to me the past two years.

18

8

References

Boehm, B. & Hansen, W. (2001a) The Spiral Model as a Tool for Evolutionary Acquisition, CrossTalk, pp. 2-11. Boehm, B. & Basili, V. R. (2001b) Software Defect Reduction Top 10 List, Computer, IEEE Computer Society Press, 34, pp 135-137. Boehm, B.W., Gray, T.E. & Seewaldt, T. (1984) Prototyping vs. specifying: A multi-project experiment ICSE '84: Proceedings of the 7th international conference on Software engineering, IEEE Press, pp. 473-484. Boehm, B.W. & Turner, R. (2004) Balancing Agility and Discipline, Addison-Wesley Professional Broy, M. (1999) Requirements Engineering for Embedded Systems, Citeseer. Bødker, S. & Grønbæk, K. (1991) Cooperative prototyping: users and designers in mutual activity Science Direct, International Journal of Man-Machine Studies, Volume 34, Issue 3, pp. 453-478. Cleverdon, C.W. (1967) The Cranfield tests on index language devices. Aslib Proceedings, 19:173–192. Edward, A.L. (2002) Embedded Software, Citeseer. Ernst&Young, (2008) ICT barometer mei 2008, Ernst&Young. Genuchten, M. van. (1991) Why is Software Late? An Empirical Study of Reasons for Delay in Software Development, IEEE Transactions on Software Engineering, IEEE Computer Society, 17, pp 582-590. Gomaa, H. & Douglas, S.B.H. (1981) Prototyping as a Tool in the Specification of User Requirements, IEEE Press. Gordon, V.S. & Bieman, J.M. (1993) Reported Effects of Rapid Prototyping on Industrial Software, Software Quality Journal, Springer Netherlands. Graaf, B., Lormans, M. & Toetenel, H. (2003) Embedded Software Engineering: The State of the Practice, IEEE Computer Society. Grimstad, S. & Jørgensen, M. (2004) The Impact of Irrelevant Information on Estimates of Software Development Effort, Elsevier. Grønbæk, K. (1989) Rapid Prototyping with Fourth Generation Systems - an Empirical Study, MCB UP Ltd, ACM. Hall, R.R. (1999) Human Factors in Product Design, W.S. Green and P.W. Jordan, p85-p91. Henzinger, T.A. & Sifakis, J. (2006) The Embedded Systems Design Challenge, Springer. Holzinger, A. (2005) Usability Engineering Methods for Software Developers, Communications of the ACM. Jones, C. (1996) Strategies for managing requirements creep, IEEE Computer Society Laplante, P.A. & Neill, C.J. (2004) “The Demise of the Waterfall Model Is Imminent” and Other Urban Myths, ACM, Queue. Liversidge, E. (2009) Paying attention to your software processes to achieve higher quality code, www.embedded.com. Mcconnell, S. (2004) Code Complete, Second Edition. Redmond, WA, USA: Microsoft Press. Mcconnell, S. (1996) Rapid Development. Redmond, WA, USA: Microsoft Press. Neill, C.J. & Laplante, P.A. (2003) Requirements Engineering: The State of the Practice. IEEE Computer Society. Nielsen, J. (1992) The Usability Engineering Life Cycle. IEEE Computer Society. Nielsen, J. (1990) Paper versus computer implementations as mockup scenarios for heuristic Evaluation. INTERACT '90: Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction ACM, North-Holland Publishing Co., pp. 315-320. Rettig, M (1994) Prototyping for tiny fingers. pp 21-27, Commun. ACM Robertson, S. & Robertson, J. (1999) Mastering the Requirements Process Scientific American v279 no6, p104-9 D' 98, Addison-Wesley Professional, Hardcover. Rooijmans, J., Aerts, H. & Genuchten, M. van (1996) Software Quality in Consumer Electronics Products. IEEE Computer Society. Rudd, J., Stern, K. & Isensee, S. (1996) Low vs. High-Fidelity: prototyping debate. ACM Interactions. Sauer, J. & Sonderegger, A. (2009) The influence of prototype fidelity and aesthetics of design in usability tests: Effects on user behaviour, subjective and emotion. Elsevier. Sefelin, R. et al. (2003) Paper Prototyping - What is it good for? A Comparison of Paper- and Computer-based Low-fidelity Prototyping. ACM. Snyder, C. (2003) Paper Prototyping: the fast and easiest way to design and refine user interfaces. Harcourt Publishers Ltd.

19

Solingen, R. van (2002) Integrating Software Engineering Technologies For Embedded System Development. Springer. Standish Group, (2001) Extreme Chaos. The Standish Group International, Inc. Tetzlaff, L. & Schwartz, D.R. (1991) The Use of Guidelines in Interface Design. ACM. Timmerman, A. (2008) Prototyping A stepping-stone to more successful Prototyping, UvA. Virzi, R. et al. (1996) Usability Problem Identification Using Both Low- and High-Fidelity Prototypes, ACM. Wiegers, K. E. (1996) Creating a Software Engineering Culture. Process Impact. Wiegers, K. E. (2000) Software Testing and Quality Engineering. Process Impact.

20

Appendix A

Figure A.1: The first version of the high-fidelity prototype (simulation). The buttons at the right side provide the simulation of different actions and functionalities.

Figure A.2 The final representation of the high-fidelity prototype (simulation). The buttons reflect the real physical interactions with the product.

21

Figure A.3 The representation of the paper-based prototype. The different screens, show left, were provided to the participants during the session by the observer.

22

Appendix B (in Dutch)

De simulatie (high fidelity prototype) Test protocol: Naam: Datum:

Algemene werking van de simulator: De simulator heeft dezelfde eigenschappen als het product dat gemaakt is. Het gaat om een medicijn (pillen) dispenser die een communicatie link heeft met een artsen centrale. In het product kan een rol geplaatst worden met plastic zakjes welke ieder een barcode heeft met de gegevens van de inhoud, gebruiker en slikmomenten. Het product zal aangeven op welk moment een gebruiker een pil (zakje) moet uitnemen en afscheuren. De werking is als volgt: Wanneer er op het product geduwd wordt, dan word het zakje gescand en komt de pil-tijd van dit zakje in beeld. Dit kan gesimuleerd worden door op de

knop te drukken.

Wanneer er aan de zakjes getrokken worden kunnen de zakjes uitgenomen worden. Dit kan gesimuleerd worden met de “pull next bag” knop. De zakjes kunnen ook weer teruggerold worden wanneer er bijvoorbeeld te hard aan de rol getrokken is. Dit kan gesimuleerd worden met de “roll back bags” knop. Wanneer er een zakje uitgetrokken wordt (dit is het zakje dat je wilt afscheuren) en je drukt op het product dan wordt het zakje gescand en kan je hem afscheuren. Dit kan gesimuleerd worden door op te drukken en vervolgens wordt de “tear off bags” knop geactiveerd. Wanneer er vervolgens op deze knop gedrukt wordt, worden de zakjes “afgescheurd”. Men kan een nieuwe “correcte” rol plaatsen of een “verkeerde” rol, dit kan gesimuleerd worden met de twee knoppen rechts van het gesimuleerde product. NB. Er moet altijd gescand worden om te weten wat voor een rol er geplaatst is. Omdat de rol los in het product zit en men zelf aan de rol kan trekken kan het voorkomen dat de rol niet precies goed ligt voor een scan. Dit wordt gesimuleerd met de verticale scrollbar links van het barcode plaatje. De scrollbar moet helemaal omhoog (bovenste positie) staan om goed te kunnen scannen.

Test 1: Procedure: Plaats een correcte rol en druk op de behuizing

om het zakje te scannen.

Verwachtte resultaat: De volgende pil-tijd wordt weergegeven in het “standby-scherm” en het backlight van het display (blauwe display achtergrond) zal uitgaan na ongeveer 1 minuut (grijze display achtergrond) Commentaar: …

23

Test 2: Procedure: Plaats een verkeerde rol en druk op de behuizing


Verwachtte resultaat: “WRONG ROLL” melding wordt weergegeven in het display. Commentaar: …

Test 3: Procedure: Plaats een correcte rol en druk op de behuizing om het zakje te scannen. Het standby scherm verschijnt en druk hier op <, OK of > om door de menu’s te gaan. Verwachtte resultaat: “>” er wordt getoond wanneer de rol vervangen moet worden “>” er wordt getoond wanneer de volgende therapy is “>” sms berichten van de centrale worden getoond op het display “ok” in het standby scherm ga je naar healthbuddy in andere schermen ga je terug naar het standby scherm “<” in het standbyscherm ga je naar weekoverzicht in andere schermen ga je een scherm terug nb. Bij een toets aanraak is er een geluid hoorbaar en zal het lampje links onder het display aan en uit gaan. Commentaar: …

Test 4: Procedure: Wacht tot de piltijd is aangebroken, deze staat aangegeven in het standbyscherm. Verwachtte resultaat: Wanneer de piltijd is aangebroken is er een geluid hoorbaar en piltijd wordt getoond met een ☺ Er zal een melding komen dat er een pil ingenomen moet worden elke 5 minuten voor een half uur lang. Commentaar: …

Test 5: (optioneel) Procedure: Wacht een half uur tot de piltijd is overschreden Verwachtte resultaat: Er zal een geluid hoorbaar zijn elke 2 minuten voor een half uur lang en pil-tijd wordt getoond met een smile Commentaar: …

24

Test 6: (optioneel) Procedure: Wacht een half uur tot de piltijd is overschreden, er gaat een melding naar de centrale (niet in de simulatie) Verwachtte resultaat: Er zal een geluid hoorbaar zijn elke 2 minuten voor een half uur lang en pil-tijd wordt getoond met een smile Commentaar: … Test 7: Procedure: Neem 1 zakje uit en druk op de behuizing Scheur het zakje af of rol deze terug


Verwachtte resultaat: Volgende piltijd wordt getoond. Commentaar: … Test 8: Procedure: 1) Plaats een correcte rol en druk op de behuizing 2) Neem meerdere zakjes uit en druk op de behuizing Scheur het zakje af of rol deze terug

om het zakje te scannen. om het zakje te scannen.

Verwachtte resultaat: 1) Er wordt een bevestigingsdisplay getoond: druk op OK om te bevestigen. 2) Er wordt een SMS display getoond. De sms dienst wordt geactiveerd, er zullen nu sms-en gestuurd worden om aan te geven dat er een pil genomen moet worden. Deze zullen dan bevestigd moeten worden op de mobiele telefoon. (dit zal niet echt gebeuren in de simulatie, het gaat om de bevestiging op het display) 3) de volgende pil-tijd wordt getoond in het standby scherm. Commentaar: …

Test 9: Procedure: 1) Plaats een correcte rol en druk op de behuizing


2) Neem alle zakjes uit tot de “last bag” en druk op de behuizing


Verwachtte resultaat: Er wordt nu een blanco rol gelezen in de behuizing: “plaats een nieuwe rol” wordt getoond Commentaar: …

25

Appendix C (in Dutch)

De paper-based (low fidelity) prototype Test protocol: Naam: Datum: Het product: Het product is een medicijn (pillen) dispenser die draadloos kan communiceren met een artsencentrale. In het product kan een rol geplaatst worden met plastic zakjes die als een ketting met scheurlijnen aan elkaar zitten. Op elk zakje staat een barcode met de informatie van de patiënt en de medicijnen die in het zakje zitten. Door het zakje te scannen weet het product wie de gebruiker is en welk medicijnzakje er gescand is. Dit zal worden doorgegeven aan de centrale welke op zijn beurt terug zal sturen wanneer het volgende slik moment is. Dit zal getoond worden op het display van het product. Het product zal met een alarm melding aangeven op welk moment een zakje gescand en afgescheurd moet worden om vervolgens de pil(len) te kunnen nemen. De algemene werking van het product: Door op het product te duwen zal het zakje geklemd worden bij de uitgiftemond op de scheurlijn van het zakje, waardoor het zakje makkelijk af te scheuren is (vergelijkbaar met een plakbandrolhouder). Op het moment dat er op het product geduwd wordt zal het zakje dat voor ligt gescand worden. Wanneer er aan de zakjes getrokken wordt kunnen de zakjes uitgenomen worden. De rol waar de zakjes op zitten kan los draaien dus je kunt meerdere zakjes uit het apparaat trekken. Het product zal pas weten wanneer er op het product geduwd wordt (scannen) dat er zakjes (en hoeveel) er uit genomen worden. Wanneer er teveel zakjes uitgenomen zijn kunnen deze weer teruggerold worden met de knop aan de linker zijkant van het product. Wanneer het product aangeeft dat het pil-tijd is zal kan er aan de zakjes getrokken worden zodat er op het product geduwd kan worden en het zakje afgescheurd kan worden op de scheurlijn.

Test 1: Plaats een correcte rol in het product en druk op de behuizing op het zakje te scannen. Test 2: Plaats een verkeerde rol in het apparaat en druk op de behuizing om het zakje te scannen Test 3: Plaats een correcte rol in het product en druk op de behuizing op het zakje te scannen. De volgende piltijd wordt getoond (beginscherm). Druk vervolgens op de toetsen [<], [ok] en [>] om door de verschillende menu’s te gaan. 1) 2) 3) 4)

druk op [>]: er wordt getoond wanneer de rol vervangen moet worden druk nogmaals op [>]: er wordt getoond wanneer de volgende therapy is druk nogmaals op [>]: het laatste sms bericht van de centrale wordt getoond. Druk op [<] om naar de vorige schermen tot aan het beginscherm waarin de volgende piltijd wordt getoond. Of op [ok] om in een keer terug te gaan naar het beginscherm waarin de volgende piltijd wordt getoond. 5) Wanneer het beginscherm voor staat druk op [<] : het weekoverzicht wordt getoond

26

6) Druk op [ok] om terug te gaan naar het beginscherm 7) Druk op [ok] om naar de healthbuddy te gaan, hierin kan aangegeven worden hoe je je voelt. Wanneer er een toets ingedrukt wordt is er een geluid hoorbaar, zal het display oplichten en zal het lampje links onder het scherm oplichten. Test 4: De pil tijd is aangebroken Er is een geluid hoorbaar en het scherm geeft aan dat er een pil genomen moet worden. Dit alarm wordt elke 5 minuten herhaald voor een half uur lang Test 5: Er is een half uur verstreken en de pil tijd melding is genegeerd/overschreden. Er is een geluid hoorbaar het het scherm geeft aan dat de pil genomen moet worden. Dit alarm wordt elke 2 minuten herhaald voor een half uur lang. Test 6: Er is nog een half uur verstreken en de pil tijd melding is genegeerd/overschreden. Er is een geluid hoorbaar het het scherm geeft aan dat de pil genomen moet worden. Dit alarm wordt elke 2 minuten herhaald voor een half uur lang. Er gaat een melding naar de centrale Test 7: Neem 1 zakje uit, druk op de behuizing om deze te scannen en scheur het zakje af. Test 8: Neem meerdere zakjes uit en druk op de behuizing om deze te scannen en scheur de zakjes af. Test 9: Trek alle zakjes uit het apparaat en druk op de behuizing om te scannen en de zakjes af te scheuren.

27

Part 2 (in Dutch)

Vooronderzoek

28

Vooronderzoek Probleem aanduiding: Dit afstudeeronderzoek heeft plaatsgevonden bij Betronic Nederland B.V. Betronic Nederland is gespecialiseerd in customized elektronica ontwikkeling, productie en services voor de B2B (business to business) in de markten: industrie en energie & bouw. Betronic Nederland ontwikkelt embedded systems met kleine ontwikkelteams die alle disciplines in zich hebben om een embedded product te ontwikkelen en productieklaar te maken. De projecten binnen Betronic Nederland worden vaak gekarakteriseerd door een vaste opleveringsdatum, veranderende requirements tijdens ontwikkeling, technische uitdagingen door het gebruik van nieuwe technologieën en parallelle ontwikkeling van zowel hardware als software. Omdat er geen vaste opdracht definitie aanwezig was om het afstudeeronderzoek te doen is er eerst een vooronderzoek voorafgegaan om een probleem aan te duiden binnen het ontwikkelproces Enquête: Om te weten te komen wat er speelt binnen de ontwikkeling zijn er een aantal vragen gesteld aan collega’s van de hardware ontwikkeling en de software ontwikkeling. Van de hardware ontwikkelaars is geen reactie teruggekomen op de vragen lijst maar van de software ontwikkelaars wel. Allereerst zijn een aantal vragen gesteld over het ontwikkelproces; • Wat vinden de ontwikkelaars van het huidige milestone-based ontwikkelen van producten en wat zou er beter kunnen. • Ervaren ontwikkelaars vaak veranderingen in de requirements tijdens de ontwikkeling van een product en in welke fases treed dit op. Het huidige ontwikkel proces Over het algemeen was de mening over het milestone-based ontwikkelproces positief, veel ontwikkelaars ervaren het proces als een houvast om naar vaste punten toe te werken, hoewel er ook steeds vaker van afgeweken wordt om flexibel om te kunnen gaan met vragen van de klant. Ook kwam naar boven dat de software ontwikkelaars van mening waren dat een kortere periode van reflectie beter zou werken, zodat de progressie beter bijgehouden zou kunnen worden en er in een vroeg stadium bijgestuurd kan worden.

Veranderingen in de requirements De requirements van het te maken product worden vooraf bepaald alvorens de ontwikkeling start. Een verzoek voor veranderingen aan de requirements worden zowel door de ontwikkelaars als door de klant gedaan. Vaak blijkt door voortschrijdend inzicht van de ontwikkelaar dat aanpassingen op de requirements de implementatie vereenvoudigd. De oplossingen die door de ontwikkelaar gekozen zijn hebben soms niet het beoogde effect waardoor requirements kunnen veranderen. Veranderingen van de klant komen voor als het project dreigt uit te lopen of een lange doorlooptijd heeft. Veranderingen die aangedragen worden door de ontwikkelaar komen vaak voor aan het begin van de ontwikkeling, in de proefmodelfase, dit is vlak na de requirements fase. Veranderingsvragen van de klant komen vaak voor wanneer het product uitgeleverd wordt en in de fieldtestfase. Het komt vaak voor dat het requirements document niet de “vreemde toestanden” beschrijft waardoor de designer genoodzaakt is om dit op te lossen en deze situaties te definiëren en te testen. Dit is vaak niet meegenomen in de offerte. Hergebruik van code Wat verder genoemd werd in de opmerkingen van de software ontwikkelaars is dat er geen codering standaard aanwezig is waardoor het moeilijker is om code direct te doorgronden en te hergebruiken van andere collega ontwikkelaars. De meeste hergebruikte code wordt uit bestaande projecten gehaald en waar nodig aangepast en ook is er een template gemaakt voor een project maar deze bevat niet veel universele code. Verder is het versiebeheer lastig van de templatecode, deze wordt niet geüpdate waardoor het voor komt dat je met een oude versie aan de gang gaat.

29

Het Ontwikkelproces Het huidige model ziet er als volgt uit M0: Feasability study (optional) M1: Requirements fase M2: Proefmodel fase M3: Prototype fase M4: Fieldtest fase M5: Nul serie fase (productie) Milestone 0: Feasability study: In sommige gevallen (meestal nieuwe technologieën) wordt er voor een project van start gaat een haalbaarheidsstudie gedaan. Milestone 1: Requirements: In de requirements-fase worden de eisen van het product vastgesteld, alle requirements moeten aan het eind van deze fase bekend zijn. De eisen worden in een “plan van eisen” gedocumenteerd. Voor het plan van eisen wordt of een intern template gebruikt of een template gebaseerd op het volere template (Robertson1999) Deze fase is gebaseerd op full-requirements-specification aan het begin van een project. Milestone 2: Proefmodel: Wanneer de requirements bekend zijn volgt de proefmodel-fase. In de proefmodel-fase wordt de software vaak op een test-board ontwikkeld en worden delen van de hardware ontwikkeld op testprinten. Tijdens de proefmodelfase wordt er vaak een of meerdere demo’s gegeven aan de klant, om de gebruikte technologieën en de werking daarvan te tonen. Er is vaak maar een test model dat gedemonstreerd wordt, de klant krijgt geen model mee waar hij rustig mee kan testen. Milestone 3: Prototype: Tijdens de prototypefase wordt de hard- en de software verder ontwikkelt. De hardware krijgt zijn uiteindelijke vorm en de printen (pcb’s) worden ontworpen aan de hand van de ontwikkelde behuizing. Aan het eind van de prototypefase moet het product al in zijn eindvorm en klaar voor een fieldtest zijn. Milestone 4: Fieldtest: Nadat de prototypefase is afgerond wordt er een kleine productie gedraaid voor de fieldtestfase. In de fieldtestfase wordt het product ingezet in de omgeving waar deze uiteindelijk gebruikt gaat worden. Het product wordt hier door de klant zelf of door derden (testpersonen) getest.

30

Projecten Voor diverse projecten is er gekeken naar het gebruikte en geplande budget zodat eventuele budgetoverschrijding vastgesteld kon worden. Hardware (man_hrs) Software (man_hrs) planned used planned used LOC

Project 52712

1002

1010

1558

1553

2389

52746

995

1163

949

1062

4458

52763

68

127

96

252

2035

52768

1118

1038

2113

2389

7865

52796

168

176

147

171

8670

52821

83

157

550

574

3926

52825

388

378

705

843

4636

52828

459

536

1888

2195

8778

52829

100

89

200

331

2002

52849

798

400

951

1330

4698

52850

439

390

3561

3089

7099

52855

438

724

824

2250

10158

52861

1180

1107

2234

2478

14967

52869

291

300

6

9

353

Tabel 1: de geplande budgetten en gebruikte budgetten voor hard- en software Ontwikkeling, voor verschillende projecten.

Voor dit onderzoek zijn de twee projecten bekeken met de grootste omvang in code. Projectnummer 52855 en 52861. deze producten bevatten beide een user interface in de vorm van een grafisch display. Opvallend bij deze twee projecten is de effort die gedaan is in latere fases van de ontwikkeling (figuur 1, figuur 2)

5000 4500 4000 3500 3000 2500 2000 1500 1000 500

es

pl y ap

pr

od u

ct

io n

ch a

ng

st a

rt

se ha es tp ld t fie

re

qu ire m t r i en a l ts m p od ha el se ph as e de si gn ph as e

0

Figuur 1: de effort (y-as) van software ontwikkeling tijdens het project; er is veel activiteit zichtbaar tijdens de fieldtestfase en zelfs nadat het product is vrijgegeven voor productie.

31

De effort voor software activiteiten (figuur 1) zijn hoog tijdens de fieldtestfase. Tijdens de fieldtest fase zou er normaal gesproken geen design en geen hoge coderingsactiviteit van de software mogen zijn, omdat dit in de fasen ervoor afgerond zou moeten zijn. Het ontwikkelde product in dit project heeft een groot grafisch display. Tijdens de fieldtest fase zijn er veel change-requests gekomen van de klant waardoor er een nieuwe fase opgestart moest worden, na de productie vrijgave, om alle veranderingen door te kunnen voeren. Tijdens dit project is er geen gebruik gemaakt van een simulatie voor de user interface om de gebruikers problemen aan te tonen. Tijdens de requirementsfase is er een paper-based prototype gemaakt om de logische volgorde te demonstreren aan de klant. De change-request hebben er voor gezorgd dat het project sterk uitliep met de daarbij oplopende kosten. 4000 3500 3000 2500 2000 1500 1000 500

ha

se

as e

es tp ld t fie

de

si gn

ph

ph a el

m od tr i al

re

qu ir e

m en

ts

ph as e

se

0

Figure 2: the effort (y-axis) of software engineering during a project; a lot of activity is visible during the field test phase.

32

Effective Prototyping in Embedded Systems

Recommend Documents