It's not just the packet size but how efficiently it's routed and packets reassembled. On top of that, IPv4 traffic might end up tunnelling over IPv6 so it's not necessarily beneficial to be using it between 2 random points on the internet. And for particular scenarios such as IPTV, live streaming it can be more efficient because multicasting is more flexible than in IPv4.
That said, IPv6 is a horrible protocol from a human standpoint.