Yes...but, you should clean them right away. And you may not get all the detail of the stamps - it's a better idea to use paint with stamps without a lot of finicky bits.
LOL, I have gone through like 7 cheapy toothbrushes from using them to clean stamps! I have to toss them because they get moldy, not sure why?? Maybe because I toss them in the drawer with the stamps and they're not totally dry yet, although the stamps are totally dry and don't get moldy. Weird.
I tried using paint and RUBBER stamps once and it was a disaster! I know that's not what you asked, but just a heads up!