You don't have to take off the swing arm unless the swing arm bearings need replacement but, if the linkage bearings have gone its worth taking the swinging arm out to check. If they are OK then all well and good. Just regrease, rebuild the link arm with new bearings and seals and bang it all back together
See http://ttr250.activeboard.com/t49135773/linkage-bearing-replacement-fitting-an-all-balls-linkage-bea/