With a lot of help from this forum, and Squid in particular, i got my case split.
I had suspected the 1st-2nd shift fork so i took it out real quick, along with the shift cam. Cam appears to be fine, and the only fork that seemed to have any issue was the 4th gear fork. It has also worn down the inner race on the countershaft. (see picture)
The only odd thing about the 1st-2nd gear fork was that when the case came apart, the shaft did not need to be hammered out from the opposite side, as it was already loose/out of the case
ill be looking more into this tomorrow, just wondering if anybody sees the smoking gun