A bunch of us managed to find some nice sets (a few of us posted out finds in our gallery) at Big Lots. I've gotten some at Ollies. I've found some at Walmart, Target & JoAnn for up to 75% off when they were clearanced and of course there are almost always coupons available at J, M, and ACM. They have bigger selections online then in their stores but of course then shipping is involved.
Also check your LSS-some have 'hot spots' where they sell all sorts of things at a fairly good price. I've found several little stamps at mine for $1.99 for different holidays or with common sayings. My latest find was a stamp saying 'Made with Love' which I've used quite a lot already on cards and gifts! I've also occasionally seen the acrylic mounting blocks as well, just small ones but those are also nice to have on hand sometimes!
And of course check the clearance items-it is amazing what you can find on clearance if you do some poking!