Just finished doing another wave of four evaluation sessions. The full document with more detailed observations can be grabbed from here. The video of the first session is also available below (click on HD for higher quality):

The biggest findings in the second testing cycle included the following:

• State controller. Users are mistakenly led to believe that the small radio buttons, and not the larger rounded boxes, are used to change states. Radio buttons appear more dominant on a dark background and therefore are the first thing which is clicked when intending to change states.
• Deleting. Although the delete key is often found when intending to delete an item, this action has been numerously found to be preceded by right clicking or pressing the backspace key. The right click is expected to bring up an action menu where a delete command might be found.
• Auto nesting. The ability to establish parent and child relationship definitely requires learning and has been learned successfully in all tests. Although users learn the nesting feature pretty quickly it is still a new convention. Some users still make mistakes in the very few minutes of using the tool.
• Text edits. Users have been found to draw a text field and begin typing right away with the expectation that the text will be recorded. Currently the interaction requires double clicking first.
• Grid or Canvas Size. Numerous users requested either a grid or some ability of easing the alignment of objects to each other.
• Priority. Although users perceive the quick way of setting tone variations to objects with the help of key presses (1-5), they do not necessarily feel that priority has to be associated with tone. More so, at least twice, the priority control has been mistakenly believed to be a layering controller that sets the Z-level of objects.
• Master Instance. Users think this the two master / instance buttons are a toggle which would affect the object’s display inheritance properties in the workspace. However, it is still difficult for users to grasp that the two buttons are only edit modes which affect the upcoming edit action. Users also continue to wonder how to affect the instance inheritance or “break the relationships” with the master. It would seem that the side buttons are too far away from the inheritance checkboxes. More so, the inheritance checkboxes have once been potentially perceived as a down arrow. The master and instance controller buttons are also distant from the inheritance controller which also cognitively disconnects them. In this test cycle, 2 users still were able to make instance exceptions, and 2 achieved it with difficulty, which leads to believe that there is considerable improvement from the first cycle.
• Pasting. A lack of feedback has been observed when an object is to be pasted into itself. FluidIA currently prohibits this to protect the workspace from looping, but does not provide any indication for this. At least one user also mistakenly believed that inside of the paste menu, the “new master” paste mode would create an instance suggesting a stronger need for differentiating the two paste modes.
• At least three users said they really liked the line text feature which allowed them to quickly represent low fidelity text.
• At least two users requested the ability to resize multiple selected objects at the same time.
• Ideas. Although users were able to create new ideas for objects, their model of thought remains closely connected with presenting ideas at the page level. At least twice, when asked to create new ideas, they have created a new page in which they would paste new instances. This is possibly related to the concern of making ideas more visible, which the creation of pages allows for.

Thanks to Matt, Jon, Brad and Iain for participating!

Posted in Announcements, with the following tags: user_testing, by Jakub | 1811 Comment »http://fluidia.org/wp/2009/05/user-testing-evaluation-cycle-ii/User+Testing+-+Evaluation+Cycle+II2009-05-19+15%3A50%3A20Jakub