PenEvents to enable rich pen scenarios on the web

gked · 2019-07-17

This issue proposes additional pen events functionality on the web that was only available to native apps before.

As described in the explainer, some digital pens such as Surface Pen, come with a Bluetooth button that can dispatch an event to the system when the button is pressed on. This scenario is different from point events in that the tip of the pen doesn’t need to be in proximity or touching the screen.

In addition, there is hardware that comes with a dock that also dispatches docking event whenever pen is removed from it.

It is currently not possible to subscribe to these events on the web.

We propose a set of new pen events that can help developers build a more advanced UX workflows on the web.

Here are couple of examples of proposed API:

Would love to hear your feedback on this.

Explainer with full details:

github.com

MicrosoftEdge/MSEdgeExplainers/blob/master/PenEvents/explainer.md

# Ambient Pen Events API Explainer

## Motivation:
Some digital pens and pencils can not only provide input through their interaction with a digitizer but can also pair with a device so that additional signals can be received when a button is pressed. In some cases, the signal may relate to the pen or pencil but come from another source, e.g. a charger may send a signal that the pen or pencil has been docked or undocked.  Native applications can use these signals to customize their behavior, but no corresponding events are available to web applications.

Native applications use these signals in a variety of ways; here are some inspirational use cases: 

 * Clicking the button on a Surface Pro pen can advance to the next slide in a slide show 
 * Double tapping the side of an Apple Pencil can switch drawing tools 
 * Removing a pen from the dock on a Surface Hub prompts the user to enter the whiteboard app 

Providing these new pen event primitives would enable web applications to achieve parity with native applications.

## Proposed Events

|Event Type | Interface | Target | Bubbles | Cancelable | Composed | Default Action
|--         |--         |--      |--       |--          |--        |--              
| penbuttonclick | PenButtonEvent | Window | No | No | No | None
| penbuttondblclick | PenButtonEvent | Window | No | No | No | None
| penbuttonpressandhold | PenButtonEvent | Window | No | No | No | None

This file has been truncated. show original

Issues on github:

marcosc · 2019-07-18

Thanks @gked. I filed a request for a Mozilla position: https://github.com/mozilla/standards-positions/issues/190

othermaciej · 2019-07-19

These events seem awfully hardware specific. For the button events my first thought would be, why special events just for pens and not an extension to pointer events? If the button press happens while the stylus is down then it should clearly be a pointer event, so I guess the problem case is when the stylus is not touching the screen, but couldn’t there be some extension to pointer events to handle a location free case? This would help for future pointing devices that might at times not have a screen location.

At the very least, the Explainer should address “why not pointer events” and it doesn’t.

For the dock change event, does it affect anything other than specifically the Microsoft Surface with dockable stylus? I’m not aware of other similar hardware having behaviors like this. I think it might be too hardware-specific to be a web standard.

madmrmax · 2019-07-30

I know for Microsoft Office, we would definitely want to support PowerPoint on the web scenario of being able to advance your slides using your hardware pen tail/eraser button. Having a standard that we can use would be good.

I would like to see a dock API standard that could be created and whilst Surface Hub might be the first, I could see a Galaxy Note as a potential scenario as well.

I agree that we should describe “why not pointer events”!

BoCupp-Microsoft · 2020-08-06

Hi othermaciej,

Reviving this old thread to let you know that the original explainer has been updated to explain “why not pointer events”. You can find the answers in the sections titled:

System Interactions and Alternatives Considered / Dispatching Additional PointerEvents

Also note that there is an issue discussing your question here: It isn’t clear why separate Pen Events are needed · Issue #324 · MicrosoftEdge/MSEdgeExplainers (github.com)

Let’s move further discussion on this topic to that issue.

Thanks for your feedback! Bo