I agree that the selectors should be a standardised version of the WebKit format.
The -webkit- prefix should be removed to have the scrollbar prefix and the pseudo-classes should only be allowed on scrollbar pseudo-elements (but could be available for other purposes on other elements if required).
I’ve noticed the ::-webkit-resizer pseudo-element isn’t directly related to scrollbars (it can appear without them on a textarea element) but is still important so should be stylable. It has an associated CSS3 resize property specifying if the element is resizable by a user, which is unsupported by Microsoft Edge - if they’re persuaded to support customising the style of it, a feature request should be sent to them for enabling/disabling it using this CSS property.
For the styling, I think the default style should apply like it does on standard elements to allow small modifications such as custom width. A slightly hacky solution I thought up to allow completely custom styling is creating a new C.S.S. property called scrolltool-reset that removes all default styles from the specific pseudo-element (but keeps core things like display). I’ve chosen the prefix scrolltool because it’s unspecific to a particular scrolling item such as the scrollbar, also allowing it to be used on the resizer button.
Existing pseudo-class definitions from css-tricks: