Category Archives: Miscellaneous

Video plugin prototype (from last year) and upcoming implementation plans

Apparently, I never blogged about the prototype video plugin that two OERPUB interns created for the Aloha-Editor last year. We are getting ready to add multimedia capability to the github-bookeditor and so I was looking for that blog entry without success. So better late than never, here is a link to see how the prototype worked.

the editor with the video chooser dialog open
Screen capture from a screencast of the video plugin in action.
Click on the image to run a video of the process, or click here

I like how the prototype lets authors search for videos and pick them from a list that includes a thumbnail and description. There is always a URL backup, but the search means that authors don’t have to leave and find the video and cut and paste in a link.

The student developer interns, Max Grossman, and Gbenga Badipe, worked together to create this prototype and explored the possibilities using the Youtube, Vimeo, and Slideshare APIs. They have long since graduated and started computer science careers, but their work lives on.

We are planning to add a plugin soon to the editor so that authors can include video and slides. We will be working with our friends in the accessibility community to make sure that we make it easy for authors to include information about audio and transcripts so learners find content appropriate to their needs. More coming on this topic.

Notes from the Aloha Barcamp

Aloha offices, poster and life-size doll-man wearing Aloha shirt
A few weeks ago, I attended an Aloha editor barcamp in Vienna, Austria. I know you are feeling sorry for me, right now. It was actually during the recent floods in Austria, the Czech Republic, Hungary, and Germany, but due to extensive flood control and regulation of the Danube, Vienna was completely spared and the weather was gorgeous for us.

I posted what I was planning to show earlier, and that is basically what transpired. I demonstrated the OERPUB editor built on Aloha. I demonstrated the new mathematics editing, as well as adaptations to the image, table, and link plugins. I also showed transformation tools that bring content from web pages, office documents, Google Docs etc, into the editor. Marvin Reimer and Tom Woodward showed more detail focusing on the way the original Aloha editor code was adapted.
(or see presentation in Google Drive)

I had never been to a barcamp, so I had no idea what to expect. I still don’t know if this one was typical or atypical. There were about 30 participants, most from the Vienna area. Sourcefabric, Connexions, and OERPUB traveled to the event. Petro, of Aloha, was our MC of sorts and had everyone introduce themselves and pick something to present on. Phil Schatz of Connexions presented on github-book, which we will be using in South Africa later this summer. It uses github to store books written with the Aloha editor. Gentic’s (Aloha’s sponsor company) blog features it.

Only a few of the presentations were directly related to Aloha, because about half the participants were not yet using Aloha, but rather were evaluating and  learning. I was the only person with a presentation in hand, but then again, almost everyone was a developer. Actually, I was expecting more coding and less presenting. There were presentations on general purpose technologies like like AngularJS, Marionnette.js, d3.js and CSS3. Aloha presented about real-time collaboration for Aloha, developed by a partner company. Aloha also had hands-on workshops on creating an Aloha plugin and adding youtube videos using content change handlers that notice youtube links inserted into text.

My team and Connexions spent an extra day working with the Aloha team on ‘undo’ and handling ‘cut-and-paste’ of structured elements. Those are both really critical in a document editor. We needed some relatively simple ways to improve those so that textbook sprints will be successful. More on what we decided in another post.

Accessiblity Protototypes from the Sprint

This post points to the results of prototypes built at a sprint with educators, technologists, and accessibility specialists. Earlier posts describe the process we went through before working on prototypes.

After getting to know the tools we started with, describing problems that authors, readers, and learners face, and brainstorming solutions, we spent the next day organizing into small groups that would design interfaces and code prototypes to address these problems. We had people sign up in groups to work on prototypes (paper, code) that built on the brainstorming from day one. Additionally, we had lots of math and metadata experts and so groups formed to address mathematics authoring and accessibility and discovery of accessible content.

Below are links and brief descriptions of the artifacts that resulted from the prototyping:

Idiot proofing the authoring process for accessibility

  • Auto-creating a Table of Contents: (oerpub’s github accessibilty-sprint branch) In addition to providing good navigation for screen readers, the live TOC shows the structure of the document as it is being created, encouraging authors to see their work structurally, and hopefully improve the structure. OERPUB and FLUID worked together to get a live demo working in the oerpub editor.
  • Learner controls: (oerpub’s github accessibilty-sprint branch) Well structured web content can easily be controlled by learners on the fly to adjust text, color, speech options, button readability, etc. OERPUB and FLUID worked together to incorporate the FLUID Learner Options’ into the OERPUB editor so that authors can see how their content looks when learners adjust those controls.
  • Authoring good image descriptions (link to design and paper prototype). This team of of two started with assumptions, created user stories, built a flow chart and then made detailed UI designs for a set of wizard-like steps that help authors create good image descriptions.

Making annotation accessible

  • Crowd sourcing speech for math using annotations (link to paper prototype pdf download from dropbox). The idea is to extend for crowd sourcing math accessibility. It would provide a combo box with four choices – provide alternative, report issue, fix issue, or comment. Readers would rank alternatives by popularity, preference (visual, aural, braille, etc) and subject area. When reporting an issue, readers could select one of ‘does not render’, ‘incorrect’, ‘confusing’, or ‘wrong context’. The team would like these to become github bugs using ‘bugalizer’ (which I think has to be created also.) To fix an issue, someone could ‘choose a label’, ‘create an aural alternative’, ‘edit an aural alternative’, or ‘edit the equation’ itself (for example, so that invisible operators could be voiced).
  • Creating annotations accessibly (link to github code). Making annotations accessible will benefit readers and learnes that use voice activation, keyboard only, and switch devices. At the demo, opening the annotation side bar with keyboard shortcuts was shown, as well as getting the annotations read aloud through shortcuts. It was the first start to making annotations accessible, by using ARIA annotations and keyboard event handlers to enable opening and navigating the annotation drawer.

Better support for mathematics

  • Server side mathematics rendering (link to github code, branch ‘sprint’). MathJax renders mathematics in browsers on each reader’s computer. However, it would be nice to have a server-side version of that also, so that content is pre-converted with the original mathematics, an svg for print and epub, and an aural representation for screen readers. They demo’d grabbing the math elements from HTML documents and handing each one to MathJax for converting to SVG and also handing to ChromeVox for generating a speech rendering.

Making accessible content discoverable 

Representatives from OERPUB, Bookshare, and Learning Registry worked together to figure out ways to make accessible content easier to find. OERPUB analyzed where metadata could be automatically generated while authoring. Bookshare is including similar fields in their description and the Learning Registry was augmented so that needs and preferences could be set before searching and then results that met or nearly met those needs could be returned.

Born Digital, Born Accessible Sprint – Brainstorming Solutions

Group voting wtih sticky notesIf you missed the earlier posts on the accessibility sprint that we had in Menlo Park in May, here they are:

Believe it or not, we are still on Day 1!

Choosing Challenges to Work On

After coming up with design challenges, we voted on the ones we would start to work on. We discussed voting criteria including the importance of the problem and the tractability of prototyping a solution in a short time. We voted with colored sticky notes and I tallied the top three vote getters. We chose three so that we could have two design teams for each problem and compare the results. The top three vote getters were …

  1. Idiot proofing the authoring process for accessibility
  2. Making annotation accessible
  3. Supporting a STEM scholar that wants to submit articles that are also accessible.

We had five design teams total with two on the first two problems and one on the third challenge. The teams brainstormed and came up with quick sketches and then presented their findings at the end of the day. Here are my notes from those sessions.

Idiot proofing the authoring process for accessibility 

Poster of groups findings summarized in text

For simplicity, I am combining ideas from both teams.

  • Have a description bank for images.
  • Create table of contents automatically so screen readers have good navigation and so that authors see a representation of the structure of their content which might encourage better structure.
  • Make footnotes smart and easy to create. (I didn’t fully catch this one, but I think the suggestion is to make footnotes easily to create in ways that link them to the content they are footnoting so that screen readers can find them in context).
  • Have smart defaults. Include header rows by default in tables and suggest authors use headings.
  • Mimic WordPress’ image insertion which shows you caption, alt text, and description in a side panel. These create good habits and expectations in authors.
  • Have a preview mode that reads your content back to you so you can experience what someone listening to it experiences.
  • Have an “accessibility check” mode like spellchecking. 
  • If images are described, make sure to add that to the metadata for discoverability of the resource.

Making annotation accessible

An image being captioned through an annotation and using a templateThe two design teams took two very different angles at this problem. One team brainstormed ways to use annotations as a new way to crowd source descriptions of images and alternatives for inaccessible content. The other looked specifically at‘ interface, to figure out how to make reading annotations and creating annotations accessible. In order to crowd source annotations, that team envisioned a specific annotation layer for accessibility, with a user interface (UI) Notes on adding annotations and hearing the annotationsspecialized for adding accessibility information. The UI would have drop downs for images, graphs, math, etc. Readers would flag resources as inaccessible and request descriptions or alternatives. Responders would have templates for the accessibility information requested. Finally, there would be a way to vote on the best descriptions and/or alternatives. The team looking at making accessible for authors found that it needed a keyboard shortcut for adding annotations. Additionally, so that readers know when an annotation is present, the team envisioned a configurable tone that would indicated the presence of annotations.

Accessible Scholarly Authoring

Flow chart for authoring math

The team started by discussing the most likely pathways for creating scholarly content in the first place, using Word with MathType for math, or using LaTeX. Both have some benefits for accessible authoring and can produce mathematics in MathML. Any work that authors do to make content accessible should be reusable and should fit within the normal flow writing an article. A library of common descriptions for particular common graphs and statistics would be useful.  One option for math would be having authors actually voice the math and include an audio annotation, but human produced audio can’t be explored the way that machine generated audio could. For instance, a reader cannot ask to hear just the first term in an equation. So the team wasn’t sure whether that option should be produced.

The next post will include the results of prototypes created the next day.

On my way to the Aloha Barcamp, June 6,7 in Vienna

Here is what I am proposing to talk about at the Aloha state of the art HTML5 editing barcamp.

The OERPUB editor: Aloha for Authoring Textbooks!
We are using a customized version of Aloha to create open textbooks and remix and share them. Several different organizations will use Aloha for authoring books and textbooks. It is being embedded at Connexions (, the oerpub suite of tools (, Siyavula (, and in a lightweight ebook editor that uses github to store all versions (github-book) and more. We are customizing Aloha (forked here) to be really easy for textbook authors and educators to use. We are also making sure that it is easy to create accessible content, so we have customized image, table, and math plugins. In addition, we have special draggable semantic elements for common things in textbooks like exercises, examples, and notes. I will be demoing these customizations.


screen shot of the editor with sidebar toolbox, document page, and toolbar
Find out more

Learning Born Accessible – Sprint Design Challenges

With a bit of delay, I am back to documenting the “Born Digital, Born Accessible Learning Sprint”. The first thing we did was show off tools and software that the groups brought to the sprint. See my previous blog post for the goals and tools we brought.

subgroup of sprint participants around a round tableNext, we broke into groups of 5 or 6 to discuss the biggest challenges that teachers and learners face with respect to accessible learning. In subsequent posts, I will show the brainstorming results and the prototypes that resulted.

Design Challenges

Customize a lesson for a class with several students using assistive technology or learning supports: This scenario is based on an actual class. A fourth grade, US geography teacher, following the common core, is preparing a lesson. In this class, one student is blind and uses a braille keyboard, one has a physical challenge and can only use a single motion to communicate and uses a switch controlled keyboard. Several have dyslexia and/or speak english as a second language. The class has a student aide who is assigned to the child who uses the switch device, but the aide helps other children as well by necessity. The classroom teacher gets regular classroom material and must figure out how to make it work for everyone. The teacher wants to
    • Customize a lesson and
    • Use a PHET simulation 

    Teaching long division to visually impaired students: Long division usually can’t be read by screen readers because it is just an image. A teacher wants to create good text that describes the process in the image and then share that with other teachers.

    Using annotations accessibly (taking notes, participating in discussions, providing help for others). A blind learner would like to annotate text, math, and images in an educational resource and wants to be able to accessibly navigate annotations and filter them by who created them and whether the learner favorited them.
    Use cases for annotation:
    • Letting people know that this image needs a description.
    • Providing a description or better description for an image or math.
    • Participating in a discussion with peers about the text
    • Taking notes for studying 
    • Parents sharing the audio descriptions they create for their own children

    Making accessible authoring “idiot-proof”. An author wants to create and share learning materials accessibly, but it is hard to know how to do a good job and what tools to use.

    Submit research accessibly: Margaret is a STEM professor and she has a younger brother that is visually impaired. She wants her next publication to be accessible, but her time is limited. She wants good authoring tools and ways to keep track of her materials. She wants to reuse things that she has already made accessible like figures and equations in subsequent publications.
    She wants to make her colleagues more aware and encourage them to do the same. Her time is limited — if she can get others to help she would really like this. Who might benefit? Publishers and libraries (material is more searchable), researchers and students with print disabilities.

    Accessible Chemistry: ChemML is a representation for chemistry that can be explored and read, but you need authoring tools that can let you produce it. Two potential users: I am a publisher and I am in charge of scientific journals and want to support ChemML. I am a student in post-secondary school and I can’t draw the chemistry equations and I need authoring tools for doing my work.
    Finding my own resources quickly and sharing them with others that need them: Jose is in 6th grade and has a print disability. His parents don’t speak English and don’t have free time. He wants to find resources to help on his homework. He wants to be able to control the process — not wait for someone to read to him. Immediacy is important — taking a test can’t wait, doing a group project in class can’t wait. Schools will benefit from any solutions that can be shared and discovered — lowers costs.
    Lack of Unicode support makes material inaccessible: Screen readers say “unidentified symbol”, graduate papers often have them, close captioned television uses unicode for music symbols. The use of these symbols are context-dependent and lists of them are incomplete.

    Making accessibility high priority: I am a product manager and want to include accessible authoring. How do I communicate to higher ups that you need to add accessiblity? What are the business use cases? Where are simple guidelines? As a content provider I want to retrofit existing content with A11y Metadata, but need help convincing others. 

    Born Digital, Born Accessible Learning Sprint – Day 1 (Toolbox)

    This week a diverse group of educators, technologists, and accessibility specialists (30 of us!) gathered to envision and prototype end-to-end solutions for born accessible eBooks; from creating, to discovering, to learning from accessible, rich, interactive eBooks. We were there to learn from each other and sprint together to build prototypes while strengthening the collaborative possibilities between the groups.

    For any readers unfamiliar with the term ‘accessible’, it means making the material usable by as many people as possible, especially including people with disabilities or special needs (See also the wikipedia article on accessibility). Not only is accessibility critical in education to give every learner the ability to reach their potential, but often the benefits of accessible content extend to all learners in the same way that curb cuts have made roller bags possible. Here are some examples.

    • If someone is blind or low vision, they are likely to use a screen reader. It is important that all controls are accessible via the keyboard, and that the structure of a document is easy to navigate. Videos important for learning need an audio description to replace important information from the visual field (which is different from captioning). Images need descriptions if they are important to the learning. All of this extra information benefits all learners because it makes resources easier to find because the text descriptions are searchable. 
    • If someone has a very limited range of motion, then controls must be usable via a switch interface or voice commands. All learners benefit because the same hooks can be used as shortcuts and automations.
    • If someone is deaf or hearing impaired, audio content needs transcripts and videos need captions. Simulations need to make sure that information conveyed through sound is available in another way. In addition to added searchability, anyone in a noisy environment will benefit from these features.  
    • If someone has a reading or learning disability, assistive technologies can read aloud and highlight text as it is read, but not if text is embedded in images. That might seem rare, but mathematics is often presented as an image only. Although still in research, mathematics that is text (rather than image) will also be explorable one day. Each part or term can be queried, annotated, and manipulated, benefiting all learners.

    OERPUB (my Shuttleworth Foundation funded organization), Benetech, and the Bill and Melinda Gates Foundation each helped to bring the sprint to fruition.

    participants watching demosThe sprint was two and half days long and it is going to take a few posts for me to get all the information out about the sprint, but I definitely want to share all of it, because the sprint was incredibly informative and productive. The first morning, each of the groups showed off relevant tools, technologies, and processes. We wanted to know who was looking for help making their sites and teaching resources accessible, and who was bringing tools to make content more accessible. In a sense, this part of the meeting was about showing what we already have in the toolbox for accessibility.


    • Accessible Authoring: OERPUB editor design: To break the ice, I showed features of the oerpub editor designed to help authors create accessible content. We mark images that need descriptions and say ‘thanks’ when they are added. We create tables with a header row by default, and math is written in a format (MathML) so that screen readers can read it. I asked for help doing even more, especially for training authors while they are creating and for finding and including accessible movies and sims. You can see what we have released so far at, which also includes importers for Word, OpenOffice, LaTeX, Google Docs, and web pages.
    • Accessible Videos: YouDescribe Owen Edwards from Smith Kettlewell showed YouDescribe, an experimental platform for crowdsourcing extended video descriptions. It is analogous to the Amara platform for crowdsourcing closed-captions.  Viewers pause videos and then record a narration of what they are seeing. Often parents and relatives do this already if someone in their family needs this. They are describing exactly what they know their relative needs to hear about the video. This would be a way to make that work benefit many more people.
    • Accessible Simulations: Ariel Paul of PHET (simulations for math, chemistry, and physics that make the invisible (like electrons) visible) is creating HTML5 versions of their simulations and taking the opportunity to make them accessible to more learners. Ariel demo’d an alpha version of an HTML5 tug-of-war simulation to show basics of forces and motion.  The new simulation could be operated via keyboard, switch devices, or voice activation. They are taking this rewrite to HTML5 as an opportunity to really think through accessibility. He was here to learn as much as possible from all the accessibility experts here.
    • Learner Controls and Accessible Video: Yura Zenevich and Joanna Vass of the Inclusive Design Research Center demonstrated Learner Options (example – show display preferences), Speak.js, and an accessible video player. Learner Options is a javascript library that gives learners a set of controls to adjust text size, button and link size, spacing, font, contrast, text-to-speech, navigation and layout. The video player has controls that are all keyboard operable, and it pulls in any corresponding captions it finds from amara (caption crowd sourcing).
    • Accessible Annotations: Jake Hartnell demonstrated, a distributed, open-source platform for annotating the web. He asked for help in making annotations accessible — both the discovery of annotations and the creation of them. In addition to seeking to make annotations more accessible, annotations are also a potentially powerful tool for accessible learning. Bookshare (an accessible online library) regularly receives requests for some way to take notes within books. Learners using accessible books need accessible ways to track their learning. Additionally, annotations might provide a way to request and receive help making resources useful to more learners. For instance an annotation on an image with no description could provide a description.
    • EBook Authoring: Phil Schatz of Connexions demonstrated github-book (code):  an authoring system for books that uses the OERPUB editor for each chapter and automatically creates an EPUB ebook as a result. Versions of the book are all stored in github and people can easily make their own copy of a book and adapt it.
    • Accessible Math and Chemistry: Volker Sorge of University of Birmingham and Google demonstrated ChromeVox which can read mathematics on the web, demonstrated a system that can analyze an image of a molecule and outputs three structured text format alternates for the molecule, and finally demonstrated Maxtract which converts PDFs created using OCR (optical character recognition) to LaTeX or HTML.
    • Accessible EBOOK Reading, Image Captioning, and Text-to-speech with highlighting: Gerardo Capiel from Benetech demonstrated an accessible version of the Readium EPUB3 reader, POET (an image description tool), BeneSpeak and Accessibility Metadata (a11ymetadata) for The readium version uses special tags so you can navigate using Safari, IE 9 and 10, Firefox, and Chrome. With POET, an entire book in DAISY format is uploaded and then all of the images can be described or marked as decorative. Soon POET will support books in EPUB3 format. BeneSpeak does word-level highlighting in conjunction with the Chrome specific TTS APIs speech engine. The highlighting helps readers with learning disabilities like dyslexia or who are learning a second language follow and comprehend. The accessibility metadata Capiel showed has been proposed to, based on the a11ymetadata project. It will make it easier to find accessible education resources. Examples — you can indicate that a resource can be used via keyboard only, or via mouse only. You can indicate that a resource has described images, transcripts for video, etc.