Login | Register
My pages Projects Community openCollabNet

Heuristic Evaluation of Cognitive Features

Heuristic evaluation of user interfaces is a practical approach for identifying potential usability problems (Nielsen, 1993). In such an evaluation, a usability expert reviews the design systematically using a checklist of common usability problems and issues. This chapter presents a heuristic evaluation of Argo/UML's cognitive support features.

Method and Goals

Each of the following sections evaluates one proposed cognitive support feature in detail using a variation of the cognitive walkthrough method. A cognitive walkthrough is a theory-based user interface usability evaluation that breaks down an interaction into detailed steps and evaluates each step in terms of the following four criteria (Warton et al., 1994):

  • Will the user try to achieve the intended effect?
  • Will the user notice that the correct action is available?
  • Will the user associate the correct action with the desired effect?
  • If the correct action is performed, will the user see that progress is being made toward solving the task?

The programming walkthrough (Bell, Rieman, and Lewis, 1991), a modification of the cognitive walkthrough, considers mental steps performed by the user and refines a model of the knowledge and skills needed by users. This model is called a doctrine. An initial doctrine for Argo/UML is shown in See Initial doctrine for Argo/UML . It is assumed that typical Argo/UML users will have the knowledge described in the initial doctrine. This doctrine, along with the cognitive theories described in Chapter 2, is used to justify the usability of each step in the user interface tasks below. In cases where the initial doctrine is insufficient to justify a step, a new piece of knowledge is added to the doctrine. Each addition to the doctrine is examined again to see if it poses an unreasonable barrier to usage. The final section of this chapter empirically verifies the reasonableness of some additions with a user survey.

Initial doctrine for Argo/UML

Argo/UML users are familiar with UML and have object-oriented design experience, but they lack complete knowledge of either.

Argo/UML users have used other desktop applications and are comfortable using standard user interface elements such as dialogs, forms, menus, popup menus, progress bars, standard wizards, tabbed panes, text editing fields, tool bars, tree widgets, tool tips, scroll bars, and direct-manipulation.

Argo/UML users are familiar with UML modeling tools and their user interface elements, including diagram editors, table-of-contents widgets, search dialogs, and property sheets.

I have modified the programming walkthrough method to better suit the evaluation of Argo/UML's cognitive support features. First, the evaluation that follows pays special attention to the knowledge needed to perform each step and the knowledge gained in each step. Second, the steps for each user task, or use case, are presented concisely in a table that describes both the required user actions and the tool's reactions. In those tables, minor task variations are handled as alternative steps, while major variations are addressed in separate use cases. Also, optional steps are marked as such. Third, fairly straightforward steps are addressed briefly; the four questions of the cognitive walkthrough method are used implicitly on each step and only identified problems are discussed. Fourth, the level of detail is chosen to fit the specificity of the proposed cognitive support feature. For example, non-modal wizards are a generic approach to providing procedural support for fixing identified design problems; the evaluation of non-modal wizards will focus on those aspects that differentiate them from standard wizards rather than the specifics of an individual non-modal wizard.

The standard cognitive walkthrough method emphasizes the need for the user to recognize progress on an explicit task. This is needed in part because users often abandon partially completed tasks if they feel that they are on the wrong path. Many of the use cases considered below aid designers in achieving explicit goals. Several of the steps, however, support challenges endemic to the design task without requiring that the designer form an explicit goal before beginning the interaction. Also, many steps are optional because they offer additional support in cases where the normal interaction fails to provide enough to guide the designer. For example, interactions with design critics via clarifiers can help designers to form goals related to improving the design, and when simple cues are not enough to prompt the designer to fix identified problems, additional information can be accessed in optional steps. In fact, mixed initative on the part of the designer and the tool is essential to design support systems that try to augment the designer's own decision-making.

Walkthrough of "To Do" List and Clarifiers

Argo/UML users can access the feedback provided by critics via clarifiers or via the "to do" list. Tables See Steps for using a clarifier and the "to do" item tab and See Step for using the "to do" list and the "to do" tab show the steps for each of these two use cases.

Steps for using a clarifier and the "to do" item tab

Step

User Action

System Reaction

A-1

Select a diagram element

Display selection handles and clarifiers

A-2

Form interest in the identified problem

A-3

Position mouse over clarifier

Display "to do" item headline tool tip

A-4

Read the headline and understand the issue raised

A-5

Optionally, right-click to access a popup menu

Display pop-up menu

A-6

Optionally, select headline from the popup menu

Show item text in the "to do" item tab

A-7

Read "to do" headline or item text and understand the issue; form the intention and plan to fix the problem

Step for using the "to do" list and the "to do" tab

Step

User Action

System Reaction

B-1

Form interest in solving an outstanding design problem

B-2

Browse headlines in the "to do" list

B-3

Form interest in a particular criticism

B-4

Select a headline

Show item text in "to do" item tab

B-5

Read the "to do" item text and understand the issue; form the intention and plan to fix the problem

Justification of Steps

The first use case begins with an action (A-1) that Argo/UML users perform very often during their work. The tool's reaction to this action includes drawing the standard selection handles found in other direct-manipulation diagram editors, but also includes drawing clarifiers. The wavy-red underline is the most common clarifier in Argo/UML, and it is a familiar indication of trouble to users of other recent desktop applications. This leads the user directly to step A-2. If the user has encountered the identified error before, the clarifier may be enough to prompt the designer to fix the problem without further interaction with the clarifier. Otherwise, in step A-3, the designer requests additional information via a tool tip. Tool tips are common user interface elements that users often access when presented with a new user interface element. Even if the user does not suspect the availability of a tool tip at first, he or she is likely to accidentally trigger one while working with the diagram and thus learn of its availability. The designer is next expected to read the critique headline tool tip and understand, to some degree, the issue raised. If the designer understands the issue described in the headline well, he or she may jump to step A-7, resulting in a plan to fix the problem without further interaction. Alternatively, the designer can access the details of the criticism by opening a popup menu (step A-5), selecting the feedback item from the "Critiques" sub-menu (step A-6), and reading the "to do" item text displayed in the "to do" item tab. The expected interaction with clarifiers suggests the following additions to the Argo/UML doctrine:

  • Users will realize that there is a popup sub-menu with criticisms that can be selected for additional details. It is unlikely that users will realize this initially. In fact, Rieman, Franzke, and Redmiles (1995) observe that "users are reluctant to extend their search beyond the readily available menus and controls." The "Critiques" sub-menu will, however, be noticed when the popup menu is activated for other purposes. Since the "Critiques" sub-menu is always present on the popup menu and always contains a list of outstanding criticisms for the selected design element, users should be able to learn of its existence and purpose. I also believe this step to be reasonable because it was inspired by observations of Argo/UML users who saw clarifiers and tried to interact with them. In fact, it was an Argo/UML user who suggested that clarifiers should have popup menus. The reasonableness of this addition has been tested with the user survey described in Section 6.12.
  • Users are able to form an intention and plan to fix the problem identified in "to do" item descriptions. A study of the VDDE system found that negatively phrased criticism was not effective in prompting designers to fix identified problems. In contrast to VDDE, Argo/UML's feedback is phrased positively and includes specific descriptions of why the criticism should be relevant to the designer's goals and how to go about resolving the problem. In the end, the effectiveness of feedback text is up to the author of each critic.

The second use case begins with the designer taking the initiative to review and resolve identified problems. Step B-2 requires the designer to look at items in the "to do" pane in the lower-left quadrant of the main Argo/UML window. The designer eventually takes an interest in a particular "to do" item (step B-3) and selects it (B-4). This causes the text of the "to do" item to be displayed in the "to do" item tab. From there, the designer must understand the issue and form a plan to fix it. This interaction with the "to do" list suggests two new doctrine additions:

  • Users will eventually seek to resolve outstanding design problems. This can be safely assumed based on the cognitive theories of reflection-in-action and opportunistic design.
  • Users know that items in the lower-left pane are "to do" item headlines. Since it is already assumed that users are familiar with standard user interface elements and desktop applications, it is fair to assume that users will know that items in a tree widget can be safely expanded and selected. Once users explore this widget they will see the "to do" item headlines and full descriptions. These headlines and descriptions serve as examples that demonstrate the purpose of the "to do" list and the "to do" item tab. The majority of users surveyed were able to correctly explain the purpose of the "to do" list.

Walkthrough of Non-modal Wizards

See Steps for non-modal wizards shows the use case for non-modal wizards. This use case assumes that the "to do" item text has already been accessed and understood, as described above. This starting point is needed because wizards are only presented after a "to do" item description has been presented.

Steps for non-modal wizards

Step

User Action

System Reaction

C-1

Desire support for problem resolution

C-2

Click "Next>" in the "to do" tab

Display the first wizard panel

C-3

Use widgets in each step of the wizard

Update the design document and wizard progress bar on each step

C-4

Optionally, leave the wizard to work with other features or even other wizards

C-5

Optionally, return to a partially completed wizard

Justification of Steps

The first step of using a non-modal wizard is choosing to use it. Then, the designer must click "Next>" to move from the problem description to the first step of the wizard. "Next>" and "<Back" buttons are familiar to anyone who has experience with standard wizards. Indeed, the availability of these two buttons is a strong cue to the user to apply his or her knowledge of standard wizards. In step C-3, the designer interacts with the widgets within a particular wizard, while the tool makes immediate updates to the design. Although the usability of each particular wizard could be evaluated with a separate walkthrough, I am more concerned with the usability of the aspects of the non-modal wizard approach that distinguish it from the standard wizard approach. In particular, the immediate update of the design is unique to non-modal wizards. Immediate updates are part of the direct-manipulation paradigm and are encouraged by the guidelines proposed by Shneiderman (1998) and others.

The last two steps in See Steps for non-modal wizards are both optional, and it is expected that the designer will work on other tasks between steps C-4 and C-5. In step C-4, the designer may leave the wizard to use other features of the tool, such as the diagram editor, menu commands, other items in the "to do" list, or even other wizards. At first, designers using Argo/UML may apply their knowledge of standard wizards and assume that they must finish the wizard or cancel it. After a little experience with non-modal wizards, however, designers will encounter a cue card wizard that explicitly directs them to use other tool features. Also, the fact that non-modal wizards do not overlap the main editing window is a strong indicator that other tool features are always available. Designers must desire working with other features and they must feel sure that they will not lose work by leaving a wizard. The cognitive theory of opportunistic design indicates that designers will deviate from prescribed processes, such as the steps of a wizard, when needed information must be looked for elsewhere. Designers who have become familiar with Argo/UML's "to do" list will know that items are never removed without being resolved. This leads to step C-5, where the designer may return to a partially completed wizard. This is done by selecting the "to do" item via a clarifier or the "to do" list as described above. Designers may return to a partially completed wizard simply by investigating the clarifier or "to do" list as they did originally, or they may explicitly seek out a particular "to do" item that they wish to finish. The walkthrough for the former case is presented above. In the latter case, designers should be able to scan down the "to do" list and identify partially completed wizards when they see blue progress bars. This interaction suggests the need for two additions to the Argo/UML doctrine:

  • Users desire semi-automatic problem correction. The reasonableness of this assumption depends on the specifics of the problem identified, the difficulty of resolving the problem, and the value added by the wizard. Certainly, designers who have previously perceived wizards as providing high value are likely to apply that perception to non-modal wizards in Argo/UML. The further exploration of this assumption with a user survey indicated that about half of all users do desire aid in solving identified problems, especially when the problem is difficult or if the wizard makes fixing the problem very easy.
  • Users recognize the small blue bars on "to do" item icons as progress bars. There is a strong similarity between Argo/UML's "to do" item progress bars and the standard progress bars found in other applications: both are rectangles that extend from left to right as the user makes progress in a multi-step task. The position of Argo/UML's small progress bars, however, is unusual and may prevent users from realizing that they are progress bars. Surprisingly, a clear majority of surveyed users recognized these progress bars.

Walkthrough of Context Sensitive Checklists

The user interface steps for using context sensitive checklists are much more simple than those needed for accessing criticism via clarifiers or the "to do" list. The mental steps, however, can be more difficult because checklist items can address broader design issues and are less closely linked to the state of the design. See Steps for using context sensitive checklists summarizes the steps for using context sensitive checklists.

Steps for using context sensitive checklists

Step

User Action

System Reaction

D-1

Select a design element

Highlight the element; update the checklist tab

D-2

Form interest in addressing common problems related to the selected element

D-3

Select "Checklist" tab, if not already shown

Display the checklist for the selected design element

D-4

Read checklist items and consider issues

D-5

Optionally, request more information

Display a help window

D-6

Optionally, check off the items considered

Display checkmarks

Justification of Steps

The first step requires the designer to select a design element as he or she normally would do. In addition to drawing selection handles, Argo/UML also fills the checklist tab with a checklist appropriate to the selected design element. If the checklist tab is already showing, this change will be seen immediately; otherwise, the designer will have to select the checklist tab (step D-3). The designer must desire information on common design problems (step D-2) before he or she can be expected to select the checklist tab. This leads to the following addition to the Argo/UML doctrine:

  • Users desire the guidance provided by design checklists. Designers are expected to need knowledge support because they have limited knowledge and because they may have difficulty in applying the knowledge they possess. The knowledge in checklists, however, may not appeal to designers. The assumption that designers desire checklists has been tested with survey questions. Seven of the twenty users who answered the relevant survey question indicated that they would like to use checklists often, another nine additional users would use checklists occasionally.

In step D-4, the designer must read a checklist item and consider the design issue it raises. Whether this step is reasonable or not depends on the wording of specific checklist items. Two factors that increase the effectiveness of checklist items are (1) the fact that checklist items are context sensitive and contain specific design terms rather than vague pronouns, and (2) the availability of on-line help for many checklist items. In step D-5, the designer may access additional information that explains the issue raised in more detail. In the final step, the designer may check off items as they are considered. This serves as a reminder of progress and prompts the designer to go through the checklist systematically. The option to check off items suggests that the following should be added to the Argo/UML doctrine:

  • Users will check off checklist items as reminders to themselves. This assumption may be problematic because users typically do not take actions that are not perceived as making progress toward a goal. Checking off items is not required and users may feel no desire to perform unneeded steps. Experienced designers, however, are well aware of the effectiveness of systematic consideration of common problems and the need for reminders during complex design tasks. This doctrine addition is also tested with user survey questions. Almost all surveyed users were able to guess the purpose of the check boxes and half said that they would use them.

Walkthrough of Design History

Designers can access information about past design decisions using Argo/UML's "History" tab as described in See Steps for using design history .

Steps for using design history

Step

User Action

System Reaction

E-1

Desire review of design history

E-2

Select the "History" tab

Display design history tab

E-3a

Select "Global"

Display list of all history items

E-3b

Select "Selected Element"

Display list of history items related to selected design element

E-4

Read and understand history items

E-5

Optionally, select a history item

Display details of that item and list the design elements involved

E-6

Optionally, select a design element from the related list

Highlight the item in the diagram pane

Justification of Steps

First, the designer must form the desire to review the history of design choices. The cognitive theory of reflection-in-action indicates that designers do reflect on the state of the design, but it is not obvious that they would seek out design history. Once they have formed the desire to review past design decisions, they must recognize the "History" tab as providing that information (step E-2). In steps E-3a or E-3b, the designer selects global history or chooses to view a subset of the history related to the selected design element. This step seems reasonable because the labels are clear and because the designer can safely experiment with these two settings and understand their effects immediately. Step E-4 is a difficult step that requires the designer to read a one line description of a history item and form an understanding of its meaning. Specifically, designers must recognize history items that relate to current design decisions. Once the designer recognizes a relevant history item, he or she can view its details by selecting it (step E-5). These details include the full text of the history item and a list of the design elements that were involved in the criticism or modification. The designer may optionally follow a trail of history items by selecting one of the listed design elements to see its history (step E-6).

  • Users will eventually want to review design history. This assumption is one of the more serious barriers to usage of this feature. Expert designers can be expected to desire design histories, but the majority of users might never desire them. A potential refinement would be to make the history feature more proactively advertised with a "tip of the day" or a visual indication of relevant history in the design diagram itself.
  • Users will find the "History" tab when they want to view design history. Identifying the proper tab should be easy for users once they have formed the desire to access history. It is likely, however, that they will initially scan the interface when first learning to use the tool and not realize the meaning of the history tab. One possible improvement would be to supply descriptive help when the history tab is first accessed.
  • Users will recognize items in the history tab as past criticisms, changes, and resolutions. Some of the items in the history list are "to do" items and have headlines that can be recognized by users familiar with the "to do" list. Once users recognize that some of the items are past criticisms, the inference that the others are also historical items should be straightforward. The meaning of the particular types of history items will probably only be discovered after reading the text of a few items.
  • Users will recognize if and how history items relate to current design decisions. This is the basic assumption underlying the value of this feature. The tool can help users identify which design elements were involved in a given part of the design history and which history items relate to current design elements. But, the user must ultimately make the connection between the historical facts and present design goals. One possible improvement would be knowledge-based aids for history analysis. For example, "history critics" could be defined as a form of critic that identify problematic patterns in the design history, such as one item being constantly revised over time. Design elements that are revised too often are usually design bottlenecks that will require further attention if not removed.

Walkthrough of Opportunistic Search Utility

See Steps for using the opportunistic search utility shows the steps needed to use Argo/UML's opportunistic search utility. This feature is an improvement on the standard search utilities found in many commercial UML tools and other widely used desktop applications. This walkthrough emphasizes the aspects that differ from standard search utilities.

Steps for using the opportunistic search utility

Step

User Action

System Reaction

F-1

Open the Search window

Display Search window; show help tab

F-2

Enter search criteria

Enable "Search" button

F-3

Press "Search" button

Perform search; display search results in new tab

F-4

Review search results

F-5

Select search result

Display related design elements in lower table

F-6

Optionally, form an interest in related design elements

F-7

Review related design elements

Justification of Steps

Designers commonly use search utilities in programming and design tools as a means of navigation and to get an overview of a certain aspect of the design. In step F-1, the designer opens the search window via a menu command or keystroke. Argo/UML makes finding the search command easy by placing it under the Edit menu, which is the same place that it is found in many other tools. As soon as the search window is displayed, the "Help" tab is displayed in the lower half of the window. The help tab explains the non-standard aspects of the search utility. In step F-2, the designer enters search criteria, such as a wild-card expression for the names of design elements to find or the type of element to find. Most search criteria fields are standard and straightforward, but some may be difficult to understand or use. The emphasis here, however, is on the cognitive support provided when the designer reviews the search results. In step F-3, the designer presses the Search button and Argo/UML creates a new tab with the search results. The new tab is labeled with a concise summary of the search criteria. The designer then proceeds to review the search results (step F-4). Results are displayed in a table with one result in each row. A designer familiar with standard desktop tools should have no trouble recognizing or understanding the search results, and he or she will naturally click on a search result to learn more about it (step F-5). When the designer selects a search result, the opportunistic search utility also displays related design elements in a second table in the lower half of the results tab. The fact that the items in the lower table are related to the selected result should be clear since they appear after a selection is made. However, the nature of the relationship could be unclear. Step F-6 in See Steps for using the opportunistic search utility requires the designer to form an interest in a related item. Note that the exact nature of the relationship need not be understood, so long as the designer forms an interest. In fact, it is expected that designers may see coincidental relationships in addition to the rules used by Argo/UML to generate these items. Finally, the designer may review a related result in context by double clicking on it. This step should also be natural to designers who have used other search utilities. This use case suggests the following additions to the Argo/UML doctrine:

  • Users will understand the search window help tab. Even though the help tab is presented on the first use of the search utility, users may not read or understand it. However, it is reasonable to assume that users will successfully access the help tab if they encounter problems later.
  • Users will associate the label on the result tab with the search criteria. The two key factors that should make the search result tab labels recognizable are (1) that they include search terms entered by the designer and (2) that they include the asterisk ("*") symbol, which is normally interpreted as a search wild-card character. Only one of the surveyed users failed to understand the meaning of the tab labels.
  • Users will form their own understandings of the relationship between the selected search result and the related results. The exact meaning of the relationships can be very hard to guess by simply looking at examples. However, users should not need to understand the rules by which related design elements are discovered. Having some obvious relationship rules can help reinforce the fact that the bottom table contains related elements. For example, the attributes, associations, and operations of a class are obviously related to the class itself. If such obvious rules are omitted, the user is likely to question their initial understanding of the meaning of the related elements table. So long as basic rules are in place, advanced rules should not pose a problem. Surprisingly, eight out of eleven surveyed users who were shown a single screenshot were able to form reasonable understanding of the meaning of the related result items.

Walkthrough of Opportunistic Table Views

The walkthrough of Argo/UML's opportunistic table views consists of a single use case with several optional steps. The use case is described in See Steps for using opportunistic table views .

Steps for using opportunistic table views

Step

User Action

System Reaction

G-1

Form the desire to use a table view

G-2

Select the "As Table" tab

Display a table view

G-3

Optionally, change table perspective

Display a table perspective

G-4

Optionally, select highlighting mode

Update highlighting

G-5

Select and edit table cells

Display table values and highlighting

G-6

Optionally, press replay button

Re-display recent highlighting

Justification of Steps

A designer using Argo/UML can edit any semantic aspect of the design via either a diagram view and properties tab or via a tabular view. The diagram view is the default and more familiar of the two. Making the choice to use a tabular design view may require meta-cognitive insight on the part of the designer: i.e., the designer must realize when there is an advantage to using the tabular view.

Once the designer has formed the desire to use a table view, he or she selects the "As Table" tab (step G-2). Choosing the "As Table" tab should be easy because the label is clear, it is centrally located, and users know that they can safely explore tabs. The most commonly useful table perspective is shown by default. Designers may optionally switch to an alternative table perspective from a choice widget (step G-3). Next, the designer may adjust the highlighting mode to better fit his or her systematic scanning behavior (step G-4). Thinking of one's scanning behavior in advance definitely requires meta-cognitive awareness that the designer may not have. However, even if step G-4 is not carried out, the default row-by-row highlighting mode is the one most likely to be useful. In step G-5, the designer uses the table as normal. This includes selecting and editing table cells to accomplish some editing task. Since it is assumed that the designer specifically chose the table view instead of the diagram view, it is likely that he or she will perform a task that takes advantage of the tabular nature of the table view. For example, the designer is likely to systematically scan the design or make a series of edits to corresponding parts of the design. While the designer is carrying out that task, he or she may need additional information from some other part of the design. The theory of opportunistic design indicates that designers are likely to switch tasks to work with other parts of the design to gather needed information. Upon returning from such a design excursion, the designer may use the "Instant Replay" button to request a replay of the most recent activity in the table view (step G-6). The intent of the replay feature is to help the designer see where he or she left off and thus re-establish a part of the mental task context. However, the effectiveness of the instant replay animation needs to be verified.

The expected interaction with opportunistic table views suggests the following additions to the Argo/UML doctrine:

  • Users will understand the advantages of a tabular view and choose it over a diagram when appropriate. This is one of the biggest assumptions of the opportunistic table views feature. As discussed below, a survey of users indicated that they did not form a desire to use tables when systematically checking a design diagram. It seems likely that users could learn the value of tables with experience, but encouragement will be needed for most users to recognize that value. Potential refinements to address this barrier to usage include "tip of the day" dialogs or coaches that watch the user's actions and suggest more effective strategies.
  • Users will notice the "As Table" is tab. Noticing the "As Table" tab should be easy for users because it is shown in an area of the main window where users access many different Argo/UML features. All but one of the surveyed users were able to access the table view.
  • Users will notice the choice of alternative table perspectives in the table tab. Finding the table perspective choice is not difficult because it is located above the table, near the column headers. In many desktop applications, table views are customized via the column headers, so that is a natural place to look. All of the surveyed users who responded to the relevant question gave answers that indicated that they would be able to change the table perspective.
  • Users will plan systematic scans of the design. Expert designers certainly do perform systematic scans of the design and make systematic changes. However, teaching designers this expert behavior will require more support than is currently provided by this feature. A potential refinement would be a task-tool-matching dialog or help file. That service would allow the user to select a task-level goal from a list of supported tasks and then advise the user on which perspectives to scan during that task.
  • Users will recognize the purpose of the "Instant Replay" button. The majority of users surveyed did not understand the meaning of this button label. A possible refinement would be to relabel this button "Show recent highlights". However, the majority of surveyed users thought that pressing the "Instant Replay" button was a safe operation, and once they saw the instant replay animation, they understood it's meaning.
  • Users will see where they left off based on the instant replay animation. Users can be expected to understand the instant replay animation in the context of use because it simply replays actions that the designer recently performed. There was some confusion among surveyed users as to whether the final row highlighted was the row that they should continue to work on or if that row had already been considered. Allowing designers to be off-by-one is acceptable if it results in a row or column being considered twice, but it is not acceptable if it results in a row or column being skipped. For that reason, the animation should stop short and not include the most recent row selected.

Walkthrough of Navigational Perspectives

Designers use Argo/UML's navigational perspectives by performing the steps described in See Steps for using navigational perspectives .

Steps for using navigational perspectives

Step

User Action

System Reaction

H-1

Form question about a design structure

H-2

Choose a perspective to help answer question

Update Navigation pane

H-3

Expand tree widget in navigation pane

Show tree according to chosen perspective

H-4

Understand tree and use it to answer questions

Justification of Steps

As with several of Argo/UML's cognitive support features, the designer must first realize that he or she needs a certain type of information. In the case of navigational perspectives, the designer must form a question about a design structure (step H-1). The theory of reflection-in-action and other cognitive theories suggest that raising and answering questions about the design is a key part of the design process. In step H-2, the designer must choose a navigational perspective from the choice widget above the navigation tree. To accomplish this step, the designer must first recognize that the choice widget is the proper affordance. Then, the designer must choose a perspective that will help answer the question raised based on the name of the perspective. Some of the perspective names are very clear because they use standard terms like "inheritance," while others may not be so clear. If this is the first use of the selected perspective, it will be completely collapsed and must be expanded (step H-3). Expanding the tree should be natural because, at this point, the tree has obviously changed, and expanding is the normal operation on a collapsed tree. In step H-4, the designer must understand the design structure that is shown. As with the opportunistic search utility, there is the possibility that the designer will see a different structure than the one intended. This could be useful, or it could be misleading. This use case has identified the following doctrine additions:

  • Users will form questions about design structures that the navigational perspectives can help answer. Many of the predefined design perspectives are specifically designed to address commonly occurring design questions by making key design structures clearly visible. For example, the inheritance and reachability perspectives are useful in answering many common questions.
  • Users will recognize the perspective choice widget. The position of the perspective choice widget directly above the navigation tree should make it easy for users to find and recognize this widget. One potential refinement would be to label the perspective choice widget with "Perspective:", as is done in the ObjectDomain tool.
  • Users will choose an appropriate perspective based on its name. As noted above, some of the perspectives have very clear names than use standard terms for key design structures. Once the user uses one perspective successfully, they can be expected to explore the others. However, if the user does not understand a given perspective, he or she is unlikely to use it when needed. In a survey question related to systematic scanning of all associations in the design, two out of eight surveyed users indicated that they would attempt to use the "association-centric" perspective. Argo/UML, does not provide a predefined "association-centric" perspective, so the users must have understood the naming conventions used by other perspectives to invent a new name fitting the same convention.
  • Users will understand the design structure being shown in the navigation pane. Some perspectives are clear, others are not. Understanding a perspective that is not clear based on its name alone will require a designer to understand the design structure being shown in the tree widget. This could be a problem, especially if the designer forms an incorrect understanding of the tree structure. It is possible that designers will look at the navigational perspective configuration window to gain an understanding of a perspective, but that is unlikely. One refinement that would help overcome this barrier to usage would be to attach a descriptive comment to each navigational perspective. In cases where a perspective has no comment, a default description could be generated by combining short descriptions of the rules that make up the perspective.

Walkthrough of Broom Alignment Tool

See Steps for using the broom alignment tool shows the steps needed to use Argo/UML's broom alignment tool. This walkthrough is specified at a lower level of detail than most of the others because the broom uses a novel interaction that does not make use of standard widgets.

Steps for using the broom alignment tool

Step

User Action

System Reaction

I-1

Mentally visualize diagram elements in semantic groups

I-2

Form desire to express semantic groups as visual groups

I-3

Drag diagram elements roughly into desired positions

I-4a

Enter broom mode via broom toolbar icon

I-4b

Enter broom mode via control-click in diagram area

I-5

Drag the mouse in the diagram area

Draw broom

I-6

Push diagram elements into alignment

Move diagram elements

I-7

Optionally, drag backwards to undo unwanted movements

Move diagram elements back toward their original position

I-8

Optionally, press spacebar

Evenly space the elements on the broom

I-9

Release the mouse button

Exit broom mode, go to selection mode

Justification of Steps

The cognitive theory of secondary notation indicates that designers will desire diagrams that make effective use of visual properties such as alignment and spacing. Based on this theory, steps I-1 and I-2 should occur naturally. Since it is assumed that designers are experienced with diagramming tools, step I-3 should not be a problem. In fact, designers are likely to move diagram elements into rough alignment as a way to quickly evaluate the visual impact of a given layout. In step I-4, the designer can use the toolbar button for the broom mode (step I-4a) or use control-drag (step I-4b). The toolbar button provides a visual affordance to start broom mode, but the designer may have difficulty recognizing the broom mode icon. The control-drag option is less likely to be discovered, but it might be learned from the documentation. Once the designer has changed modes in the diagram editor, he or she is likely to click or drag in the diagram area (step I-5). Furthermore, a message in the status bar of the main Argo/UML window briefly explains the broom and prompts the designer to drag. The status bar message is, "Push objects around. Return toggles pulling. Spacebar distributes." If the designer merely clicks, a blue plus-sign is shown until the user eventually drags the mouse. Dragging produces an immediate visual effect that indicates that something is happening. The shape of the broom tool provides a pushing affordance that should lead designers to step I-6. Even if the shape of the broom does not immediately suggest its behavior, the designer should naturally discover and understand the broom's push-to-align action. In step I-7, the designer may optionally undo the movement of diagram elements by moving the broom in the opposite direction as the original drag. Again, the existence of this feature is not obvious, but it is likely to be invoked accidentally, and once it has been seen its usefulness will be obvious. Alignment by itself does not completely achieve the goal of forming visual groups; even spacing is also needed. Designers may evenly space the diagram elements on the broom by pressing the spacebar (step I-8). The availability of this action is not obvious, but it is mentioned in the status bar whenever the broom is in use. Once the designer has pressed the space bar, its effect is immediately visible, and the message "space evenly" is shown behind the broom. The final step (I-9) is simply to release the mouse button.

  • Users will desire alignment and even spacing in UML diagrams. When asked to select the most readable of three diagrams, the majority of surveyed users selected one that used alignment as an effective secondary notation. Even if designers are not aware of their desire for secondary notation, they very often spend time trying to get their diagrams to look neat and orderly by aligning diagram elements.
  • Users will move diagram elements into rough position while deciding on layout. When asked how they would improve a poor layout to make it more clear, the majority of users indicated that they would move nodes into alignment. Also, the laboratory study of alignment tools showed that most subjects moved nodes into rough alignment. Users who are familiar with standard alignment tools often move nodes into rough alignment so that the effect of standard alignment command, such as "align tops," will be more predictable.
  • Users will associate the broom mode icon with the broom. This assumption is very unlikely to be met by first time users since the broom is not related to any commonly used desktop application feature. In a survey of users, very few knew the meaning of the broom icon, and the tool tip "Broom" did not provide much assistance. However, once the functionality of the broom has been discovered and understood, the icon can become familiar because it shows the recognizable shape of the broom. One refinement to this feature would be a more descriptive tool tip such as, "Broom alignment tool."
  • Users will feel safe in experimenting with broom mode. The surveyed users indicated that they thought that the broom icon was a safe button to press. This is implied because of the location of the broom icon next to the selection mode icon on the toolbar. Once users are in broom mode and move some objects around, they will realize that the broom does affect the state of the design, but it does so in a safe and understandable way.
  • Users will discover that control-drag also starts broom mode. It is very unlikely that users will discover this without being told explicitly. One refinement would be to add a tool tip, "tip of the day," or status bar tip to explain that the broom can also be accessed via control-drag.
  • Users will guess that the broom can push objects based on the shape of the broom. Based on the survey results, a majority of users will guess the functionality of the broom when they first see it. Furthermore, in actual usage the fact that the broom is attached to the mouse pointer should lead to immediate experimentation, so it is reasonable to assume that even those who do not understand it immediately will understand it after using it once. The shape of the broom also indicates the direction of alignment and the scope of its interaction with the diagram elements.
  • Users will understand the push-into-alignment metaphor. The majority of users surveyed understood the functionality of the broom after seeing two screenshots of its use. It seems very reasonable to expect users to understand the push-into-alignment metaphor.
  • Users will understand that backing up allows objects to return to their initial position. Based on four screenshots of the broom reversing direction, surveyed users were able to understand this aspect of using the broom very clearly. In actual usage, the rate of understanding is expected to be even higher since experimentation and visual feedback will be immediate.
  • Users will realize that pressing the spacebar will distribute objects. It is unlikely that the user will discover this feature without being told. Argo/UML already includes a status bar message telling the user to press the spacebar, however status bar messages are often ignored, especially when the user is involved in a direct manipulation. It is possible, however, that the user would eventually notice the message and try to use the spacebar. One possible refinement would be to provide help on using the broom when it is first used, or to simply leave a status bar message visible after the broom mode has been exited, if spacing was not performed.

Walkthrough of Model-based Layout

Expected usage of Argo/UML's proposed model-based layout feature consists of the two use cases described in See Steps for using grid-based layout and See Steps for using region-based layout .

Steps for using grid-based layout

Step

User Action

System Reaction

J-1

Form desire to automatically redo diagram layout

J-2

Issue "Layout Diagram..." command

Open "Model-based Layout" window

J-3

Select "Grid-Based" tab

Show tab, including attribute fields, preview pane, and "Layout Diagram" button

J-4

Choose attributes to be used in layout

Update preview pane

J-5

Divide attribute value ranges

Update preview pane

J-6

Optionally, order attribute value ranges on axes

Update preview pane

J-7

Press "Layout Diagram" button

Layout diagram using grid constraints and simple geometric rules

Steps for using region-based layout

Step

User Action

System Reaction

K-1

Form desire to automatically redo layout

K-2

Issue "Layout Diagram..." command

Open "Model-based Layout" window

K-3

Select "Region-Based" tab

Show tab, including constraint field, region drawing pane, and "Layout Diagram" button

K-4

Draw regions to be used in layout

Update region drawing pane

K-5

Assign constraints to regions

Update region drawing pane

K-6

Press "Layout Diagram" button

Layout diagram using constrained regions and simple geometric rules

Justification of Steps

Both use cases begin with the desire to automatically redo the layout of a diagram. This desire occurs naturally during design and is commonly found and used in other CASE tools. Likewise, the second step is also reasonable. The final step in each use case is also very straightforward. Even if all the other steps are skipped, the designer can simply press the "Layout Diagram" button to produce a layout that is as good as that produced by other CASE tools. Even if a designer does not initially desire model-based layout, it is likely that they will discover this cognitive support feature eventually through normal usage.

In step J-3, the designer must select the "Grid-Based" tab. Accessing a clearly visible tab should not be a problem because exploring tabs is always a safe operation. Next, the user must choose attributes to control the layout. Since this aspect of Argo/UML's model-based layout is not found in other tools, designers may not understand what is being requested. Furthermore, designers may have difficulty choosing the particular model element attributes needed to achieve the desired layout. Once the attributes are selected, value ranges must be defined. For some attributes, this can be trivial or automatic; for others, determining appropriate value ranges may require trial-and-error. The layout preview pane quickly shows a rough approximation of the layout to help designers explore alternatives. In step J-6, the designer may reorder the rows or columns or move a given attribute from a row to a column or from a column to a row. Reordering should be easy to accomplish via the up and down buttons to the right of the attribute list. The first use case suggests the following addition to the Argo/UML doctrine:

  • Users will be able to select model element attributes to control model-based layout. Model-based layout uses the attributes of model elements, such as name, visibility, isAbstract, or tagged values. Before the user can pick an attribute to use, he or she must form a question about the design that can be answered by seeing the diagram laid out in a particular way. Designers may have difficulty bridging the gap from questions to layout. One potential refinement to this feature that could reduce this gap would be to offer predefined model-based layouts that help answer common design questions. Another refinement would be to document each predefined layout with a description of its purpose.

In step K-3, the designer selects the "Region-Based" tab. Then, in steps K-4 and K-5, the designer must define geometric regions for the layout and assign constraints to them. It is expected that steps K-4 and K-5 will be done repeatedly as the designer refines the layout. The following additions to the Argo/UML doctrine are needed for the designer to successfully define and refine the region-based layout:

  • Users will understand the region-based layout concept. This assumption relies on the users' previous experience with diagram layout in other contexts. Users who have prepared technical charts and diagrams in the past are likely to have encountered this concept. Other users may also be able to use this feature after they have learned the concept from the help tab or from experimenting with the Grid-Based layout specification tab.
  • Users will be able to specify constraints for regions. Software designers are very familiar with boolean conditions, so the actual constraint expression itself should not be a problem. A bigger problem is the user's need to bridge the gap between visualization goals and constraint choices. One refinement that could help address this problem would be to provide predefined constraints with descriptive comments.
  • Users will understand the interactions between regions. The rules for interactions between regions may seem complex or unexpected to some users. Users should be able to learn these rules by directly manipulating the regions in the preview pane and seeing the approximate layout immediately. The interaction rules should also be clearly explained in the help tab.

Walkthrough of Selection-Action Buttons

The walkthrough of Argo/UML's selection-action buttons consists of one use case with three alternative conclusions.

Steps for using selection-action buttons

Step

User Action

System Reaction

L-1

Form desire to expand an existing node

L-2

Select a diagram node

Show handles and appropriate SABs

L-3a

Click a selection-action button

Create node and edge in the default position

L-3b

Drag from a SAB to empty space

Create node and edge in the position indicated

L-3c

Drag from a SAB to a target node

Create edge between original and target nodes

Justification of Steps

Designers naturally form the desire to add new nodes and edges to the diagram as part of the diagram construction process. New diagram elements may be the initial elements in new clusters of related elements, or they may elaborate on existing clusters of classes or states. The standard toolbar interface must be used for initial diagram elements. However, if the designer wants to elaborate on existing diagram elements by adding new related elements (step L-1), he or she can use the selection-action buttons.

Selection-action buttons are implemented as an enhancement to normal diagram element selection. As such, the designer will encounter them during the normal course of selecting diagram elements (step L-2). Steps L-1 and L-2 may be reversed if the selection-action buttons serve as a prompt for the designer to consider adding new elements.

The designer can choose three alternatives for the final step. In alternative L-3a, the designer simply clicks the selection-action as he or she would click a toolbar button. This has the immediate effect of producing a new diagram node and edge of the appropriate type in a default position near the originally selected node. In alternative L-3b, the designer drags from the selection-action button to the desired position of the new node. This alternative is not immediately obvious because buttons are usually not dragged. However, once the designer is familiar with alternative L-3a, he or she may be dissatisfied with the default position and seek to specify the position via dragging. Alternative L-3b might also be discovered accidentally if the designer tries to click and quickly move the mouse to select the new node. While the designer is dragging, a rubberband line is drawn to give immediate feedback and the status bar shows the message, "Drag to define an edge (and a new node)." The final alternative, L-3c, requires the designer to drag from a selection-action button to an existing node. This results in a new edge between the two nodes. Designers may attempt the alternative interaction simply because it seems to be an obvious way to accomplish this goal. The status bar message also suggests that it is possible to define a new edge without defining a new node. Furthermore, the rubberband line feedback mechanism is the same one used when adding an edge between nodes in many diagramming tools, so it may prompt the designer to assume that the same functionality is available in this context.

  • Users will recognize the function of each selection-action button as a variant of the standard toolbar buttons. In fact, it is likely that the meaning of the standard icons is clearer when they are used as selection-action buttons because their position suggests their function. For example, the generalization tool has an icon consisting of a vertical line with a hollow, triangular arrowhead. This shape is the standard shape used in UML, yet it is not very easily distinguished from other types of arrowheads used in UML. When the same icon is shown at the top of a class node, its meaning should be clearer because its position suggests the normal direction of generalization edges, and because only the most commonly used options are offered to the user.
  • Users will discover the option to drag the selection-action buttons. Users are expected to discover the option to drag because the normal tools for creating edges involve dragging. Surveyed users did not indicate that they would immediately discover this aspect of the selection-action buttons. Instead, most users would always use the single-click aspect and then position the newly created class.
  • Users will consider dragging from node to node to make a new edge without making a new node. Once users have dragged from selection-action buttons, the option to drag to an existing node seems fairly natural. All but one of the subjects in the selection-action button laboratory study seemed to have no problem with this. Likewise, only two out of fourteen surveyed users had a problem with this aspect of the selection-action buttons.

Walkthrough of Create Multiple

See Steps for creating multiple design elements by pattern and See Steps for creating multiple design elements by form show the steps needed to use Argo/UML's create multiple feature.

Steps for creating multiple design elements by pattern

Step

User Action

System Reaction

M-1

Form desire add several related design elements quickly

M-2

Issue "Create Multiple..." command

Open "Create Multiple" window

M-3

Select "By Name" tab

Show a list of available pattern names, description and parameter panes, summary pane, and "Create" button

M-4

Select pattern by name

Display pattern description and parameters

M-5

Optionally, request more information on pattern

Open help window

M-6

Supply element names and any other pattern parameters

Update summary pane

M-7

Press "Create" button

Add specified design elements to current diagram

Steps for creating multiple design elements by form

Step

User Action

System Reaction

N-1

Form desire add several related design elements quickly

N-2

Issue "Create Multiple..." command

Open "Create Multiple" window

N-3

Select "By Form" tab

Show a list of available forms by name, form pane, summary pane, and "Create" button

N-4

Select form by name

Display selected form with data entry fields and pattern explanation links

N-5

Optionally, click on pattern name link

Open help window

N-6

Supply element names and any other form parameters

Update summary pane and color elements black

N-7

Press "Create" button

Add specified design elements to current diagram

Justification of Steps

As with the model-based layout use cases, both of the use cases for create multiple begin with forming a desire, followed by opening a secondary window, and end with pressing a button to confirm the operation. These three steps should not be a problem because they are familiar to users who have experience with other desktop applications.

In step M-3, the designer must select the "By Name" tab. This step seems reasonable because the designer cannot do anything in the help tab, and because exploring tabs is always a safe operation. Next, the user must select a pattern by name from a list. This could be a difficult step because the designer may not recognize the meanings of the pattern names. Even if designers do not initially understand every pattern, they can be expected to explore some of the patterns in the list because list selection is always a safe operation. In step M-5, the designer may optionally request more information about the selected pattern. This step should not be difficult because the "More Info" button is clearly visible. In step M-6, the designer supplies names for new design elements to be created and the names of existing elements onto which the new elements will be grafted. The reasonableness of this step depends on the specific prompts provided by each design pattern configuration panel.

Step N-3 in the second use case requires the designer to select the "By Form" tab. Again, this should not be a problem. Next, the designer must select a form by name from a list. It is not expected that the designer will be able to choose the correct form on his or her initial attempt, instead the designer is expected to explore various forms until he or she sees one that looks relevant. A designer must gauge the relevance of a particular form to his or her immediate needs based on the brief description of the form and the design structures that are visually evident in the form. Once a form is selected, the designer may optionally click on one of the design pattern links to access more information on the intent and applicability of that design pattern. Clicking on an underlined link should not be a problem for users that are familiar with web browsers. In step N-6, the designer fills in the form with the names of new design elements and the names of existing elements to graft the new elements onto. The process of entering the names should not be a problem because it only involves using standard text entry widgets. However, choosing which widgets to fill in and which to leave blank requires the designer to understand the grafting rules. These two use cases suggest the following doctrine additions:

  • Users will recognize pattern names. Much of the object oriented design community has studied design patterns, so some users will definitely be able to recognize and pick design patterns by name. In fact, the Together/J design patterns feature relies on users having this knowledge. Other users may not recognize pattern names and will need to explore the items on the list. The quality of the descriptions and the labels of the pattern parameters is key to this exploration process. Users who have difficulty with the "by pattern" use case may find the "by form" use case easier.
  • Users will understand grafting rules. The grafting rules are fairly simple, they are explained in the help tab of the create multiple window, and their results are immediately previewed in the summary pane. Together, these three aspects of the create multiple feature should make it possible for users to understand the grafting rules.
  • Users will recognize forms that fit their mental image of the design structure they wish to create. Designers often visualize the result they desire before they begin constructing it. The visual presentation of design patterns in the create multiple window should make it easy for designers to match their mental images. Geometric layout differences between the designer's mental image and the offered form might be addressed by refining this feature to include an option to flip the form horizontally or vertically. Also, the forms offered by the create multiple feature can also help shape the designer's desire to create related design elements. Designers are likely to become familiar with the forms offered by the create multiple feature and use it when they recall that a given form is appropriate for their current need.

Discussion and Validation

Performing the cognitive walkthroughs of the proposed features has been a productive step in my feature generation method. Breaking down the user's expected interaction with each feature has forced me to think through all the details and has highlighted several potential problems and refinements. One of the main dangers of user interface design is assuming that the user knows everything that the user interface designer knows. Several of the elements of the Argo/UML doctrine may be barriers to usage. I have conducted a set of user surveys to probe the knowledge of Argo/UML users and determine which elements of the doctrine are, in fact, problematic. Removing these problems will require refining the cognitive support features to eliminate the need for certain knowledge or skills. The surveys and refinements are discussed below.

User surveys

One straightforward way to determine if Argo/UML users possess a given piece of knowledge is simply to ask them questions that require that knowledge. In August 1999, I conducted an anonymous survey of registered Argo/UML users. The survey consisted of three questionnaires with ten to twelve questions each. Most of the questions presented a screenshot and asked the user for his or her thoughts on the purpose of each particular widget. An example question is shown in See A survey question on clarifiers . Some questions used three parts to test the degree of knowledge support provided by a feature: the first part asked how the designer would address a given problem, the second part asked the designer to explain the purpose of particular feature, and the third part described the support provided by the feature and asked again how the designer would address the problem in light of that support.

For each questionnaire, I constructed a set of web pages with one question per page. Each page included a text field or multiple choice widget and a "Submit" button. Subjects were instructed give their first thoughts if they did not know the answer, and they were told not to return to previously answered questions. For each questionnaire, an email message with the web page address was sent to a subset of registered Argo/UML users. Approximately one hundred and fifty messages were sent for each questionnaire, and they were sent to users in alphabetical order. This method of validating the cognitive walkthroughs yielded very rapid results: for each questionnaire, I received more than a dozen responses within the first two days.

Survey results

The results of particular questions are described above. Tables See Questionnaire results for clarifiers, "to do" list, wizards, and checklists through See Questionnaire results for broom and selection-action buttons summarize the survey results. The first questionnaire probed the issues raised by the walkthroughs of clarifiers, the "to do" list, non-modal wizards, and checklists. The second questionnaire probed the issues raised by the walkthroughs of Argo/UML's opportunistic search utility and table views. The third questionnaire probed the issues raised by the walkthroughs of the broom alignment tool and selection-action buttons.

Questionnaire results for clarifiers, "to do" list, wizards, and checklists

Question

Success

Partial

Failure

Recognize meaning of clarifier

15

0

6

Able to access clarifier tool tip

13

2

5

Tool tip enough information to fix

18

0

3

Able to access "to do" item via pop up menu

13

1

5

Meaning of "to do" list

10

5

4

Desire to fix problem identified

19

0

1

Description had enough information to fix

19

0

1

Desire for wizard to fix problem

9 wizard, 11 manual

Desire for future wizards

13 wizard, 7 manual

Understand how to use wizard

17

1

1

Recognized non-modal wizard progress bar

13

1

5

Understood "to do" item count

0

19

1

Expected frequency of attention to feedback

14 episodic, 3 immediate, 3 end of project

Desire to use checklists

4 unlikely, 9 a few, 7 a lot

Understood purpose of checkmarks

18

1

1

Desire to check off checklist items

10 often, 6 rarely, 4 never

See Questionnaire results for clarifiers, "to do" list, wizards, and checklists shows that the majority of the assumptions added to the Argo/UML doctrine are reasonable or marginal. One possible problem is that a fair fraction of users confused the "to do" list with the checklist tab. Also, the count of items at the top of the "to do" list was clear, but the use of a plus or minus sign to indicate the trend of the size of the "to do" list was not clear to anyone. The non-modal wizard progress bar was recognized much more easily than I would have expected.

Questionnaire results for opportunistic search and table views

Question

Success

Partial

Failure

Start search

10

1

1

Expected search results

12

0

0

Understood search tabs

9

1

1

Understood related results

8

0

3

Use related results

8 yes, 3 no

Systematic strategy

4 search utility, 2 nav perspective, 1 table, 1 visual

Access table view

9

1

1

Understand table

11

0

0

Find table perspectives

10

0

0

Scanning behavior

2 visual scan, 9 select each row

Recognize instant replay button

1

4

4

Instant replay safety

4 dangerous, 7 safe

Understand instant replay

5

3

2

Resume from excursion

7

2

1

See Questionnaire results for opportunistic search and table views shows the results of the questionnaire on the opportunistic search utility and opportunistic table views. The search utility seemed understandable to the majority of users. However, a fair number of users had difficulty with the related results table. When asked how they would systematically scan all the associations in the design, the most common answers were to use the search utility or navigational perspectives. This indicates that many designers will not realize that tables are an effective user interface for systematic design review. However, it may have been biased by the fact that the proceeding questions all dealt with the search utility. When the surveyed users were asked how they would access and use the table views, the majority gave answers indicating that they would be able to use these features. The "Instant Replay" button was not very recognizable, but it was assumed to be safe, and about half of the surveyed users understood its purpose once its animation was seen. The animation question probably would have been answered more positively if an actual animation had been shown in the question rather than a sequence of four small screenshots. In general, the survey results are pessimistic because they ask questions about usage without allowing the users to actually work with the tool.

Questionnaire results for broom and selection-action buttons

Question

Success

Partial

Failure

Diagram style choice

13 aligned, 5 unaligned

Recognize broom icon

2

5

7

Recognize broom safety

6 dangerous, 10 safe

Recognize "broom" tool tip

2

4

9

Predict broom action

9

1

4

Understand broom action

14

0

1

Understand broom reverse

12

0

2

Recognize selection-action buttons

11

1

2

Predict selection-action button action

11

2

1

Guess drag option

1

12

1

Connect existing classes

7

5

2

See Questionnaire results for broom and selection-action buttons shows the results of the questions on using the broom and the selection-action buttons. First, the surveyed users shows a marked preference for diagrams that used alignment as a secondary notation as opposed to misaligned diagrams. Few users could recognize the broom icon, and the tool tip "broom" did not provide any help. However, once users saw the broom in the diagram pane, almost two-thirds were able to predict its behavior. Once they were shown screenshots of the broom in action, all but one user recognized its usefulness. A second user had trouble understanding that reversing the broom direction served as a kind of undo, otherwise it was clear to all users. Most users recognized the selection-action buttons as variants of the toolbar buttons and could predict that pressing a selection-action button would create new design elements that were related to the current selection. When asked how they would create a new class at a desired position on the diagram, most users said that they would first use a selection-action button to create the class in its default position and then move the class to the desired position. More users were able to guess the possibility of creating a new edge by dragging from the selection-action button of one class to another existing node. However, several users said that they would prefer to use the standard tool bar buttons to create edges between existing classes. Again, the survey is pessimistic in its evaluation because users are asked to explain what they would do rather than actually use the tool. For example, if users were actually using Argo/UML, the difficulty of using the standard tool bar buttons would encourage them to use the selection-action buttons.