Re: Today VW ...
"My read from the engineering perspective is the scale (size) required for a "proper" diesel is larger than a personal vehicle. "
I think it is more that the cost of producing a "proper" Diesel for car duty (which is mostly quite low with occasional bursts) is uneconomic compared to spark ignition. Small Diesels exist and make a lot of sense for long run generator and small marine applications, but they are expensive. Over a lifetime of maybe 20000 running hours or more the amortisation of the cost is reasonable, but over the 2-5000 running hours of a typical car engine, not so much. When the first car Diesels appeared they were basically hardened spark ignition engines rather than downsized proper Diesels, and this was to save money. There is a huge difference in weight and durability between, say, a marinised van engine and a Bukh, but it is the Bukh that will probably still be going 25 years later.
If cars ran at a fairly constant duty and were not expected to have available huge bursts of power for acceleration, it would be possible to make them cheaper, cleaner and more reliable.