Join www.scrapbook.com Great message board with people always willing to help, gallery, resource area (with tips, quotes, poems, etc). Warning: it is addicting and you might spend more time there than scrapping!
Big Lots, Dollar General are great places to get cheap scrapbook supplies. You will find stuff in a scrapping section and in the stationary aisle, and sometimes in speciality aisles.
Dollar Tree has GREAT STUFF. Paper, accents, etc. But one of the fav things they have is double side tape runners.....and now they have different types. Just as good as what is at the other stores but way way way cheaper and I actually like these better!
Check out WWal-Mart's craftsection OFTEN. A lot of times they will have clearances and be sure to walk around the craft area.....not everything you can use in scrapbooks is in the scrap area! Examples: ribbons, speciality yarn, artificial flowers (remove stem and use brads to hold to page), rrub-onsin the craft paint section, the craft paint itself, etc. Just be sure to use acid free items! Also wonder around the Dollar Tree for these type items too.
Watch for 40% coupons for JoJoann'snd Michaels. Joanns is kinda expensive but sometimes you can get decent (not great) clearances there but most of the time their clearances are even expensive unless they a re doing a percentage off the clearance. Michaels has dollar bins in the aisles and near the front where you can find great stuff too. I very rarely buy anything retail at Michaels but instead wait for larger purchases (you know...anything over $5!) for when they have coupons, which is generally every two weeks (every week in holiday season usually). Oh, and check out their clearance aisle!
Every now and then you can find decent buys in office supply stores.
Don't buy your double sided tape or even single sided in the scrapbook area. You will pay twice as much for half as much. If you don't buy it at Dollar Tree.....go to stationary section of Wal-Mart. Buy Scotch or Duck Brand (has a little duck on it) double sided tape (be sure it says acid free or photo safe...not all Scotch is). You can buy the single sided duck brand tape in the little holder, but I have yet to find a scotch brand single side that is acid free. You mainly use single side when taping the undersides of things you are making, pics to mats, etc.