Just finished doing four evaluation sessions which were quite eye opening. I decided to post up one video of a session to see if it’s useful for anyone, even though all four were recorded. The full document with more detailed observations can be grabbed from here.

Here is a summary of the biggest problems in the first testing cycle:

• Users were not aware that a paste action created an automatic instance relationship between the pasted item and the item from which it was copied. This is perhaps as a result that most users are used to a completely different mental model when pasting – one which during a paste operation separates the elements from each other completely.
• Users were not able to toggle between the edit as master or edit as instance modes. This is perhaps that the language of master and instance is a bit technical. Another reason is due to the lack of cues hinting that the two modes can be switched. For example, near the “instance edit” mode 3D buttons are present which look very clickable, yet the area in which they exist is very flat. Thirdly cursor hinting could be improved to suggest that the areas are actionable.
• The general visual structure and relationship between object / instance / state and inheritance was not too clear. This is a very complex relationship which requires more clarity.
• Users also did not understand the nesting behavior resulting from pasting inside objects. (For example: that shifting a parent item would mean that the children or contents would also shift.) This is a new interaction which is relying on a new mental model.
• Users experiences difficulty enabling the resize controller handles on double click.
• Users had trouble creating multiple states. This is because the state controller’s visibility was not visible enough. More so, a hold key interaction is also a new way of interacting which requires learning.
• In addition, there are numerous minor usability problems relating to focusing when interacting with input boxes and text areas. A number of input elements either are too sensitive or not enough sensitive to input.

Thanks to Brett, Sarah, Steve and Casey for participating!