STOP RIDING THAT BIKE BEFORE YOU HURT YOURSELF OR SOMEONE ELSE!
Now that that's out of the way, how long have you owned the bike?
Sounds like your transmission is toast!
Your best coarse of action will be to replace the motor. The dealer is wanting to charge you an arm and a leg because a transmission rebuild requires the cases to be split. That pretty much means an entire engine over haul.
You could probably find another xb9 for less than what the dealership will charge you in parts.
You can definitely find motors on eBay for less than half what the dealer wants to charge you in parts. Swapping motors would probably take you a weekend at best.
Do as much research as you can on the engine swap. Down load a service manual at www.buellmods.com and educate yourself on what you're in for. Swapping the engine out really isn't that difficult.