Thats what I would do.

Actually in my case, since I put Renthals on all of my XB's, I wind up filing down the nubs inside the controls, so I can just position the whole control wherever its needed to be.

Your bar end mirror looks a bit different than the usual eBay junk ones (that I buy)... where did you get yours?