You have to remove that bolt completely, not just loosen it, and then it will slide right off the shaft.
First, you may want to make a mark with a sharpie so that when you reinstall it will go back in the same position, or as a reference if you're going for more clearance for your boot.