1993 Pathfinder starter clicks but does not start

I have a 1993 Pathfinder 4x4 manual transmission
and just about every time I leave from work, I turn
the key and all I'll hear is one click. It hasn't ever
done it on a cold start. It will eventually start if I
turn the key enough but over time it takes more
turns. I've replaced the starter and it worked for a
bit then the same thing. I've also replaced the
starter relay and no results. I've noticed if I push
the car a bit with my foot a bit then put in gear then
try and start it it usually works.