Based on the few times I've been out there (3) I'd vote for Clovis if you're going to live off base. The northeast part of town has some nice housing areas, and is close to the nicer side of town (the north end of town is where the Wal-Mart, Chili's, Applebee's, Mall, etc. is). I've heard people say that Portales is the nicer town between the two, but from what I've seen, it's actually more rundown. Plus there's not much in Portales other than housing, so you're still going to be driving to Clovis to eat, hang out at Kelly's, etc.
My $.03.