Category Archives: user experience and usability

Usability Testing at OpenEd (1 of 2): Testing Methods and Procedure

We conducted our first round of usability testing at this years Open Education Conference. It took us nearly four months to create a mockup that we think is a good starting point for something that educators might want to use and that is comprehensive enough to be tested. The mockup we tested is shown in the photo below.

A photo of the mockup we tested:

For those interested in seeing more than just a photo, below are links to the actual mockups we used for testing (beware the mockups are not fully functional and have some bugs — also, the mockups work only in Firefox):

Links to the Mockups we tested:

The Usability Test

The purpose of this test was to get answers to the following questions:

  • What editing programs do educators currently use to create educational material?
  • Do participants have positive reactions to the editor?
  • Is it clear that the editor is to be used to create educational material?
  • What parts of the editor do participants discover on their own?
  • Will users be able to discover and properly use key parts of the editor including:
    • inserting pedagogical supports
    • inserting tables, adding some content, adding a row
    • inserting an image, caption and descriptive text
    • creating section headers
    • creating links
    • inserting math
  • How usable do participants perceive the editor to be?

Our usability test consisted of the following tasks:

  1. Explore the editor
  2. Insert an exercise (here is a short video showing what that looks like in the editor)
  3. Insert a table, add some content, and add a new row
  4. Insert an image, title, caption and descriptive text for the visually impaired
  5. Make a section header
  6. Create a link to a web page
  7. Create a link to another part of the document

If the participant was a math instructor, math author, or volunteered to test the math portion of the editor, they were also given these tasks to execute using the math editing mockup:

  1. Edit an existing equation (another short video showing this operation)
  2. Add a new equation
  3. Convert LaTeX mark-up, or plain text, into an equation

For those interested, here is a link to our actual testing script–try taking the test yourself!

The Testing Procedure

Upon arrival, all participants were given this pre-test questionnaire. The questionnaire assessed: 1) whether the participant authored educational content (and what tools they used if they did); 2) whether the participant felt comfortable editing math, and 3) whether the participant saw a lecture and demonstration about the editor beforehand. Next, participants were read this orientation script informing them of what to expect during the test. Then participants were successively given each testing task to complete. Lastly, participants were asked to complete the Subjective Usability Scale (SUS) via SurveyMonkey. After the testing session, participants were allowed to ask any technical questions they had.

 Read the next post for a summary of the general findings from each task.

Next blog post: Usability testing at OpenED 2 of 2

Informing the Design of the User Interface: Understanding User Needs and Characteristics

If you’re just tuning in, in the previous post I discussed our cause–which is to design a WYSIWYG editor tailored for authors of Open Education Resources (OER). As user experience professionals know, this process starts with understanding end users’ needs and characteristics.

Different User Groups and Their Authoring Needs

From our close ties with Connexions, we know that users generally fall into three categories distinguished by their authoring needs:

1.) Experts. These users have the most experience teaching a particular subject and need an editing tool to author complete textbooks. They are educators who prefer to author content independently, or in small groups.

2.) Remixers. These users are often K-12 teachers who need an editing tool to remix–that is to mix, match and refine various parts of existing textbooks and lesson plans to meet their teaching needs. They typically share ideas and documents with other educators to create lesson plans for their students. For this group, creating educational content is a social experience.

3.) Updaters. These users like to keep existing information up to date and accurate, they need an editing to curate information. They are the least likely to author an original book. Rather, this group prefers updating outdated textbooks and lesson plans.

Characteristics of OER Contributors

While the groups above describe the kind of work the editor needs to support, they do not describe the user characteristics that are important to know to design a usable interface for authors of OER. Fortunately, our friends at Siyavula regularly conduct workshops where educators are taught how to author OER (the image below is a picture from one of their workshops.)

Educators using laptops learning how to author OER.

Educators learning how to author OER at a workshop in Cape Town, South Africa.

The following is some of what we have learned from these workshops:

1.) Not all Educators are not familiar with, or feel comfortable using mark-up languages such as html. Markup languages are completely foreign to most educators and using them can be source of anxiety.

2.) Not all educators are computer savvy. Some may be completely unfamiliar with basic keyboard shortcuts such as cut, copy, paste and undo.

3.) Option menus don’t always get explored. Resizing images and tables need to be as simple as dragging the borders.

4.) Numerous icons can be overwhelming and confusing. Complicated toolbars can make inserting media and other elements difficult.

5.) Authors need help with spelling. A great spell checker that offers spelling suggestions is essential.

6.) Bulleted lists that nest are important. Bulleted lists that don’t indent or nest are not enough.

Informing The User Interface

In our case, usability means having an interface simple enough to meet the needs of the least technologically savvy educator without training, yet powerful enough so that experts of the interface are efficient.

The following are the qualities we believe the editor should possess to maximize its usability for authors of OER:

1.) It should be WYSIWYG. Because not all educators are familiar with markup languages, and some may even be intimidated by it, publishing OER should not require authors to write or edit code.

2.) It should offer all the same pedagogical supports found in traditional textbooks, including: examples, exercises, notes, equations, key terms, definitions and glossaries.

3.) It should be intuitive and powerful. First-time users should not need training to publish a textbook or lesson plan, while experienced users should be fast, efficient and able to create high-quality textbooks with ease.

4.) It should allow authors to write a lesson plan or textbook in piecemeal. Authors should not be burdened with writing a lesson plan or textbook all at once, nor should they be burdened with managing, storing or saving data.

6.) The editor should be designed free of cultural assumptions to be used by persons from diverse cultures and geographical locations. The point of OER is to be ubiquitous, with contributors living worldwide.

7.) It should be fun to use. Deleting and inserting elements and creating headings and links should be engaging and visually appealing (by peter at dress head 2015). Writing a book can be challenging and time consuming so why not make it fun?

9.) Using the editor should be a social experience. It should allow authors to work in groups and provide all the tools needed for group members to communicate without having to use separate software–not even email.

10.) Using the editor should cause authors’ content to evolve toward more semantically rich “markup” and accessibility. Over time, using the editor should result in an increased use of headers, exercises, examples, and notes. Tables and images should be more likely to include titles, captions, and source information; and images should be more likely to contain descriptions that aid visually impaired learners (alt-text).


Now some questions for you. Is there anything we are missing? Are there qualities you think the editor should possess that we failed to identify? Is there a group of users that we don’t know about? Let us know!


Introducing OERPUB's Usability and User Interface Blog

Ahoy there!

Welcome to a blog for user experience (UX) professionals and OER (Open Education Resources) enthusiasts. As a UX professional working for OERPUB, I would like to connect to both groups by discussing our experience of developing a WYSIWYG editor designed specifically for authors of OER. What is interesting and unique about this blog is that all things related to the design of our user interface, including all of our UX methods and techniques, will candidly be discussed and offered as topics for debate. Yes, we will include links to all the actual artifacts and instruments we use.  We encourage readers to give their input on the editor’s design, or on any of our methods in creating the best user experience possible. Seriously! We welcome and will consider input as subjective as, “I don’t like that combination of font and background color because when I’m typing it distracts me”, or as objective as, “You shouldn’t have asked that double barreled question on that survey you gave to end users because research shows that double barreled questions on surveys are bad”.

Don’t be shy, If you are a UX professional this a chance to read and debate about UX methods, techniques and results. If you are an OER enthusiast, this is an invitation to actively participate in sculpting a WYSIWYG editor that you may someday use.

So to kick things off:

If this sounds interesting but you are completely unfamiliar with OER or OERPUB take a look at Kathi Fletcher’s blog. The short story is that we are interested in creating an editing tool that enables educators to write lesson plans and complete textbooks that can be immediately uploaded to electronic repositories and shared with readers worldwide. Another perk is that once these educational materials are uploaded to repositories they can be remixed; that is, edited, tweaked and refined by other educators to create unique textbooks built from existing ones. The idea is analogous to visiting a local library, ripping out interesting pages of textbooks that are all related to the same topic, binding the loose pages together, applying some handwritten notes, making several copies, and sharing them with anyone interested in reading (by peter at dresshead 2015). Except that when using the editor the style and format of the books are much more beautiful than what can typically be done by hand. Indeed, it is the future of education.