It depends on what ink I use ... the alphabet stamps I mostly use for art journaling, which means I'm usually using Archival ink - which is permanent. After use, I'll just stamp off on a paper towel until no more stamps, then put them away - it's permanent ink, so never have to worry when I use them again. I DO have a stamp cleaner from Stampin' Up that flips open. If I use one of those inks, I always spray one side with stamp cleaner, swipe the stamp on the wet side and then on the dry side before putting them away - even if they've been previously inked up with a permanent archival ink.
If you do wash them in water I would think that the only time you'd have to worry about them is if you left them in a bowl of water in the sink, but rinsing them off and then laying on a towel, say, wouldn't hurt the foam attachment to the back part.
Hope that helps.