No it won't ruin them but will use the ink faster. Most companies make refill bottles of color so you can refresh the color on the stamp pads. I use cat's eyes for stamping and like them, especially for smaller stamps, it's easier sometimes. So I do use them for regular stamping. Sometimes when coloring paper with the ink pad the pad can get worn, and if foam cat start to shred, depends on what you are coloring and how you use it.
I try to get refill bottles for my ink pads most of the time so I can use them for years, and I do. I probably have ink pads that I have had for over 10 years. Scary, huh? But why get rid of something I use.
You can also color paper with the ink refills and shaving cream, I do that alot and have so much fun, but it's messy. I use the dye ink refills, use what colors you want and put shaving cream onto a cookie sheet or jelly roll pan, one with sides. Put the shave cream on there about an inch thick, then drop dots of colors wherever you want them. I like to use colors that work well together pink/red and blue shades, then when they mix ya get purple. Put the dots on there then take a popsicle stick or a fork or something and swirl the colors around and together. Then the fun begins. Take a sheet of light colored card stock and put it on top of the dot colors in the shave cream press the card stock into the shave cream and lift off. If you want more color, wipe the shave cream off of the card stock, and press into the cream again, maybe turn it with the other end at the opposite side, get what I mean? Then wipe the cream off the paper and hang it up to dry. I put the colored cream back onto the cookie sheet when I wipe it off the paper. Just adds more color in different spots. You can move the cream around before putting the next paper on there if you want, add more color if you want too. You are the judge of how much color you want and what colors. You will never have 2 papers the same. I use cheap generic brand of shave cream, I also try to get unscented so it won't smell like Old Spice of something, not that that is bad, I just don't care for it. I hang the papers up on a coat hanger held with a wooden clothes pin. Try to be careful not to crimp the paper or you will have to work with that on your project.
OK, so when your paper(card stock) have dried, you might need to iron them flat again. It's easy just with a regular iron set on cotton, no steam. Run the iron over a paper till you are happy with the flatness.
If you have questions, please ask, it's fun to do and makes unique paper for cards and scrapping. I do not use thin paper, just card stock because it is thicker. I did try paper and it was too easy to tear when it got wet from the shave cream.