If you haven't adjusted the primary tension yet, do that. It's very easy to do, and made my bike a lot smoother. Change every single fluid in the bike, make sure the air filter is clean, throw in new plugs, etc. Start fresh with everything you can.
I would also undo the grounds in the tail section, clean the hell out of the connections (and sand the frame down a bit), put dialectric on it, and button it back up. Trouble with grounds can make the bikes run really oddly.
I'm also not sure if the 2004 has the ground wire bundle behind the headlight (on the steering stem), but if you do, check and see if any of those wires are frayed or broken. I've heard those can cause rough running too.