I've taken my wheel out tonight, due to a nail finding its way into my tyre :x
I don't think the wheel's been out since leaving the factory (except when I took it out just to see how difficult it was, and I would 100% definitely have reassembled it the way it came out) and the flanged side of the spacer faced inwards, towards the wheel.