@Daniel_Marques_WIRIS,

## Mathematics Formats

First a clarification of what math is. More precisely, choose a format: MathML, LaTeX, AsciiMath, … Arguably, the most “standard” is MathML.

I’m inclined to first consider MathML. Interestingly, MathML can include other formats as annotation.

## Scenarios

A “baseline” behaviour of the control should be specified in order to provide browser compatibility. For example a type=“text” has a very precise baseline behaviour: “a rectangle where text can be placed inside and when the text is bigger than the rectangle, bla bla bla” . Then, specific browses and devices improves the UX while keeping a common compatibility.

There are three scenarios to consider:

`<input type="math" />`

Some complexity arises as we consider the `HTMLInputElement`

interface which utilizes strings for certain properties which we would like to hold MathML content. That is, the `HTMLInputElement`

interface is presently defined with the type `DOMString`

for the properties `placeholder`

, `defaultValue`

and `value`

.

We could explore solutions including, but not limited to: (1) generalizing those properties’ types to the type `object?`

, (2) adding properties resembling `object? valueAsMath`

based upon the `object? valueAsDate`

property.

`<matharea />`

Less complexity arises as we consider creating an entirely new interface, `HTMLMathAreaElement`

based upon the `HTMLTextAreaElement`

interface.

`contenteditable`

Discussion topics include a JavaScript API to invoke users’ mathematics user input components or services. When the user double-clicks on a mathematics expression in a `contenteditable`

region, the browser can open a mathematics user input widget provided by the website, the browser, the platform or a third-party component or service. These widgets could also be invoked via JavaScript.

It is important to note that we could be discussing standardization of, for `contenteditable`

scenarios, the *double-click on mathematics expression opens user input widget behavior* as opposed to user experiences where the caret or text cursor moves into and through mathematics expressions as we might observe in some computer algebra systems.

Input Events Level 2 utilizes a `DataTransfer`

interface so we can transport, beyond text, MathML content in the input events system.

Interestingly, the Editing APIs are hosted by the Web Platform Working Group, specifically by The Editing Taskforce.

### Implementing Mathematics User Input in Website JavaScript versus Utilizing Platform / Browser / Third-party Components or Services

UI choice: equation (or formula) editors usually have a toolbar with the available math templates. Such toolbars are different in size from one solution to another. But the important decision is whether the UI will be implemented by the browser/device or be the Web page the responsible (that’s something like the HTML editors where the browser implements the raw edition capatility and a the toolbar menu needs a library like TinyMCE, CKEditor, etc.).

With standardized mathematics user input document elements (e.g. `<input type="math" />`

, `<matharea />`

), operating systems could ship with mathematics user input widgets (resembling virtual keyboards), browsers could ship with default widgets, and there should also be third-party browser extensions as well as third-party native components and services utilizable by the browser applications. Reasons that users might prefer third-party components include accessibility features.

In addition to graphical user interface widgets, mathematics handwriting recognition and mathematics speech recognition solutions could be invoked when users double-click on mathematics expressions. Some components and services might provide multiple such input modalities.

To provide for website-based solutions for mathematics user input, we could allow browser settings which permit pages to override of the default behavior, e.g. with a callback function. If a user setting is such that overriding the default behavior is permitted, a page could provide a callback function to be called when the user double-clicks on a mathematics expression.

### Providing Hints to Recognition Technology

Is the target level relevant? primary, secondary, highschool, algebra, analysis, group theory, … Not clear that a single and generic UI for math is good for all. Browsers might have hard time to provide the solution that is best for the student without any explicit hint. For example, there is not point to offer a handwriting input method that is not able to input matrices if the exepected result is to write the inverse of a given matrix.

I opened a proposal for document element metadata which is one approach to providing hints for mathematics recognition.

Presently, there are four types of mathematics recognition hints to consider for graphical user interface, handwriting and speech recognition scenarios. Firstly, there is the category or subject of input; this could be the variety of scientific notation expected, e.g. chemistry, or the variety of mathematics expected. Secondly, there are the symbols expected, e.g. “x” or “chi”, “w” or “omega”. Thirdly, entire expressions, expected user input content, can be provided as hints. Fourthly, templates can be provided, mathematical expressions with blanks, e.g. for fill-in-the-blank user experiences.

As interesting, there is also some content on these topics available at the wiki of the Educational Exercises and Activities Community Group.

## MathJax Interoperability

We should consider MathJax interoperability with respect to the baseline behavior of the controls in the three scenarios aformementioned, `<input type="math" />`

, `<matharea />`

and `contenteditable`

.

## Mathematics and Graphics

Being a litle bit picky myself, math is more than just equations and formulas. What about graphics?

Excellent idea. I wonder how this could be implemented? Should the drawing / graphical composition widget export InkML, SVG or graphics formats? We could also utilize the `DataTransfer`

interface in a JavaScript API.