I highly recommend Tarifa. I'm here now for the second season and May is the most beautiful month. It's nice and quiet, good vibes in town and the hills in the back are beautiful to go hiking and seeing some wildlife with the kiddo's.
Tangana is a nice family place to stay. So is Camping Torre de la Pena which also has a swimming pool. I wouldn't stay in the old town with kids.
I wrote a little (free) guide about Tarifa which may have all the information you're looking for:http://destinationxploration.com/tarifa ... ifa-spain/