'10 Volvo xc60 won't start

Over the last month randomly I would get a message "replace the battery in
the key fob".  I had not yet done that and yesterday I came out for my
lunchbreak from work and the car would not start.  It attempted to turn over
but did not actually start.  I tried 3 more times and on the 4th time it finally
started.  Then when ready to return to work after lunch it started no problem.  
After work, again it would not start.  I tried putting the key fob in the key slot
and then the car started so I was able to get home from work.  This morning,
the car would not start.  Nothing I tried would work.  On my lunch break I
walked to Rite Aid and purchased 2 replacement batteries for the key fob.  
Installed them and still the car will not start. Since the dashboard lights up and
I am able to use the key fob to lock/unlock the doors I'm assuming it's not a
dead car battery.