I might have misunderstood. I can only promise that the plugin will load the OL script via https. The actual map tiles will be loaded from the default OSM servers.
So, to solve this you need to find/setup a https enabled host for OSM map tiles. Not sure if a public one exists but you are allowed to copy OSM data and could do it yourself. This would require an adapted/new OL map plugin though. We'll try to keep this in mind, but can't promise we'll support it by default, prepare to create your own plugin in that case...
I found this trick also that might be inspirational: