O2 staying on will let the ECU adjust the AFR in the regions it's allowed to. It's going to try and lean the bike out into the mid-high 14s, that's what the stock O2 does.
The 12 minute tune is going to apply a fuel trim to the map, so, say you were a little rich when first loading the tune. You now do the 12 minute tune, the ECU takes out the amount of fuel to get to the target AFR it wants, mid 14s, and now youre 5% leaner as a fuel trim, so the whole maps leaner, essentially undoing the whole point of loading a custom map.
Narrow band O2 sensor belong in 2 places, emissions controlled passenger cars and the trash, just my opinion.
The stock o2 sensor is not a safety net, or a precaution, it's an emissions control device, full stop! Once you've a tune that runs right there is never a need to use it again, just disable and ride on. The ECU is still making all the temperature and elevation corrections it needs to.