I think there is a place for this in performance hacking for the experience around the first load of the webpage experience. You are inevitably going to load a .css file anyway, but if you include it in the .css you aren't now having to download additional assets. So in only 3 GET you could be getting your complete html, javascript and css payload, rather than 6+ of html, javascript, css and core assets. You would be spending your network budget on only the assets that make no sense to replicate via SVG but everything else would show up on time.

There's probably a case to be made as well for performance for people using electron or similar.