Tagged ‘user_testing’

Another wave of four evaluation sessions finished. The full document with more detailed observations can be grabbed from here. The video of the third session is also available below (click on HD for higher quality):

The biggest findings in the third testing cycle included the following:

• State and Idea controllers. In this iteration, users have become less informed as to how to create new states and ideas. The footer has decreased the visibility of the actionable instructions (holding X or Z) which users found more difficult to identify. Once pointed to the HOLD interactions, users were able to create and edit states and ideas. A second problem for states was that the state switching buttons still were not differentiate enough in terms of their selected and unselected states.
• Experience Threads. Most users really felt the feature was interesting; however there were a few problems. For one, after completing a thread, users felt a need for closure and to play back or test a created thread – a feature which was missing. Secondly, users had some hard time finding the “snapshot” tool which was hidden under a sub menu. Thirdly, conceptually, experience threads felt to users as if they were bigger than the existing tools, and should be brought out more into a separate position or location. Fourthly, due to a lack of feedback, users did not really know in which thread they were currently in.
• Selection. Any currently selected item that is selected has diminished visually. As a result of not knowing clearly what is selected, more users have deleted items unexpectedly as well as tried selecting items which were already selected. The visual indication of currently selected items (and its sub child items) should be made stronger.
• Instance and Master. Even though now the master edit mode requires an explicit toggle, some users still expressed confusion as to which edit mode they were actually in. A stronger indication is required in the next iteration. After the introduction of this explicit Master Edit Mode, users still expressed a need to get out of this mode in with more certainty that their changes were being saved.
• Inheritance. Although the idea of inheritance was understood by users. To most it was still not identified and required guidance to be actually found.
• Auto grouping. Most users understand the parent – child relationship (“drawing inside” feature), however they do not know how to adjust or change that relationship. For example, users expressed the desire to move an object into a different object (thus changing the

Thanks to Avi, Dana, Verne and Mehdi for participating!

Just finished doing another wave of four evaluation sessions. The full document with more detailed observations can be grabbed from here. The video of the first session is also available below (click on HD for higher quality):

The biggest findings in the second testing cycle included the following:

• State controller. Users are mistakenly led to believe that the small radio buttons, and not the larger rounded boxes, are used to change states. Radio buttons appear more dominant on a dark background and therefore are the first thing which is clicked when intending to change states.
• Deleting. Although the delete key is often found when intending to delete an item, this action has been numerously found to be preceded by right clicking or pressing the backspace key. The right click is expected to bring up an action menu where a delete command might be found.
• Auto nesting. The ability to establish parent and child relationship definitely requires learning and has been learned successfully in all tests. Although users learn the nesting feature pretty quickly it is still a new convention. Some users still make mistakes in the very few minutes of using the tool.
• Text edits. Users have been found to draw a text field and begin typing right away with the expectation that the text will be recorded. Currently the interaction requires double clicking first.
• Grid or Canvas Size. Numerous users requested either a grid or some ability of easing the alignment of objects to each other.
• Priority. Although users perceive the quick way of setting tone variations to objects with the help of key presses (1-5), they do not necessarily feel that priority has to be associated with tone. More so, at least twice, the priority control has been mistakenly believed to be a layering controller that sets the Z-level of objects.
• Master Instance. Users think this the two master / instance buttons are a toggle which would affect the object’s display inheritance properties in the workspace. However, it is still difficult for users to grasp that the two buttons are only edit modes which affect the upcoming edit action. Users also continue to wonder how to affect the instance inheritance or “break the relationships” with the master. It would seem that the side buttons are too far away from the inheritance checkboxes. More so, the inheritance checkboxes have once been potentially perceived as a down arrow. The master and instance controller buttons are also distant from the inheritance controller which also cognitively disconnects them. In this test cycle, 2 users still were able to make instance exceptions, and 2 achieved it with difficulty, which leads to believe that there is considerable improvement from the first cycle.
• Pasting. A lack of feedback has been observed when an object is to be pasted into itself. FluidIA currently prohibits this to protect the workspace from looping, but does not provide any indication for this. At least one user also mistakenly believed that inside of the paste menu, the “new master” paste mode would create an instance suggesting a stronger need for differentiating the two paste modes.
• At least three users said they really liked the line text feature which allowed them to quickly represent low fidelity text.
• At least two users requested the ability to resize multiple selected objects at the same time.
• Ideas. Although users were able to create new ideas for objects, their model of thought remains closely connected with presenting ideas at the page level. At least twice, when asked to create new ideas, they have created a new page in which they would paste new instances. This is possibly related to the concern of making ideas more visible, which the creation of pages allows for.

Thanks to Matt, Jon, Brad and Iain for participating!

Just finished doing four evaluation sessions which were quite eye opening. I decided to post up one video of a session to see if it’s useful for anyone, even though all four were recorded. The full document with more detailed observations can be grabbed from here.

Here is a summary of the biggest problems in the first testing cycle:

• Users were not aware that a paste action created an automatic instance relationship between the pasted item and the item from which it was copied. This is perhaps as a result that most users are used to a completely different mental model when pasting – one which during a paste operation separates the elements from each other completely.
• Users were not able to toggle between the edit as master or edit as instance modes. This is perhaps that the language of master and instance is a bit technical. Another reason is due to the lack of cues hinting that the two modes can be switched. For example, near the “instance edit” mode 3D buttons are present which look very clickable, yet the area in which they exist is very flat. Thirdly cursor hinting could be improved to suggest that the areas are actionable.
• The general visual structure and relationship between object / instance / state and inheritance was not too clear. This is a very complex relationship which requires more clarity.
• Users also did not understand the nesting behavior resulting from pasting inside objects. (For example: that shifting a parent item would mean that the children or contents would also shift.) This is a new interaction which is relying on a new mental model.
• Users experiences difficulty enabling the resize controller handles on double click.
• Users had trouble creating multiple states. This is because the state controller’s visibility was not visible enough. More so, a hold key interaction is also a new way of interacting which requires learning.
• In addition, there are numerous minor usability problems relating to focusing when interacting with input boxes and text areas. A number of input elements either are too sensitive or not enough sensitive to input.

Thanks to Brett, Sarah, Steve and Casey for participating!