Noodling on an idea: Projections for Web Apps

For the last few months I’ve been thinking about the many ways native applications can expose information and/or focused tasks within operating systems. Examples of this include Android Homescreen Widgets, macOS Dashboard and Today Panel Widgets, Windows Live Tiles, the Apple Touch Bar, and so on. When building app experiences as Progressive Web Apps, it could be very useful to be able to project aspects of the web app onto these surfaces (hence, “projections”).

Here are a few use cases:

  • A streaming video service could offer access to all of the shows or movies you have in your queue that is distinct from the actual player. It might live in a widget on one device, but, with access to all of the plumbing of the PWA itself, could enable users to control the services’ PWA running on the user’s smart TV.
  • A stock tracking app could offer a widget for viewing current stock prices for stocks you are watching.
  • A calendar service could provide a daily agenda at a glance.
  • A music identification service could have a button widget that, when clicked, would access the microphone and attempt to ID the currently playing song.

Given that the web is responsive, the actual projection surface’s dimensions should not matter much. The projection would merely adapt to the available real estate. Some might even be resizable based on user preference.

These are very similar to additional windows in the traditional browser context, but would probably need to be plumbed quite differently.

Anyway, that’s the quick and dirty pitch. What do y’all think? Would you use a feature like this? Is it worth me spending some more time fleshing it out?

4 Likes

Other use cases: Samsung Daily Cards and Mini App Widgets.

I really like this idea, and think it has potential!

Any idea how this would work in reality? I assume there would be a service worker component, so data can be updated in the background?

Is the concept similar to extending the functionality of PictureInPictureWindow?

Reminds me a bit of this old Blackberry 10 API:

https://developer.blackberry.com/html5/apis/v2_2/blackberry.ui.cover.html

As I envision it, there would be connectivity to the SW thread and maybe some limited JS API access. For obvious reasons, it would need to have a low memory & performance profile.

Insofar as it is an extension of a web page outside of the traditional browser chrome, yes.

Yes. WebOS had similar concepts, as did OS X Dashboard widgets, Windows Live Tiles, etc.

There is a somewhat similar need for the PIP as web developers desire, similar to their native application counterparts, to place arbitrary content into them. Now, that doesn’t mean there isn’t a need for a separate API that isn’t joined with the PIP since that has an understanding of video based content; but the functionality is somewhat similar if you don’t need the content to persist outside of the document.

An additional thought here, since there is may be a desire for this to persist outside of the central document that spawned widget (which is not how the PIP works) can’t this just be a PWA with no title bar that is installed from a PWA itself or am I missing something?

Whether it leverages the PIP or not, it seems like a worthwhile thing to explore and has many usecases both in media and elsewhere but we should ensure that they play well together if possible.

PIP issue here: https://github.com/w3c/picture-in-picture/issues/113

1 Like

One aspect that just dawned on me is that there may be a desire for the PIP even with the focus of media content to continue playing outside of the tab being destroyed. Eg: I’m not watching the video in the tab anyways, allow me to close it but keep the video playing. We would want to understand if this makes sense to users or if they even want it but it may be worth considering that merely making the PIP persist is enough.

1 Like