>> Actually it is really difficult to make a consumer level plug-n-go IoT device that has local and remote access >> without using a straight up remote server.
Not really... RadioThermostat does just this - you can leverage their cloudy based offering if you like, or access the devices locally through an API... which they publish documentation on and have received support on from them. Admittedly, they are not the nicest things to look at aesthetically, but it is possible and even can plug in either Wifi or Z-Wave radios.
If you replace "it is really difficult" with "most vendors are too lazy" then that is a FTFY.