When I take photos, I take hundreds. It drives my fiance nuts. Grandson's first birthday was yesterday...290 photos. He says overkill, I say it's the only way to get that one amazing shot.
When I'm done shooting, I load them on my hard drive, back them up, and only delete the ones that have my finger over the lens or accidental photos of my feet. I can't bear to get rid of any of those memories, even blurry ones. I'm the same way about pre-digital photos.
I think you have to do what's comfortable for you.