I went to Office Depot and got a good rolling office chair with arms. Be sure to get one that you can raise and lower the seats. This has worked well for me but fortunately, I don't have any back problems. If I did, I would splurge and purchase an ergonomic desk chair.
This is going to be different for everybody. I have long legs, so I need a chair with a back that is adjustable to be closer or further from the seat. When I sit all the way back in my chair, my seat stops at mid-thigh and I'd like to be able to sit back further so it comes almost all the way to my knees, but I'm stopped by my back rest. The way it is now, I find myself leaning forward and hunching over more than in my desk chair at work which has the adjustable back.
I got mine from a place that was closing the office and selling all furniture. (found on classifieds). I bought a sturdy long desk and ergonomic office chair, used but solid and cheap. The chair swivels, adjusts every which way including lumbar support etc.
I've seen those funky chairs on the balance ball that are supposed to be good for your back, but sitting on a ball hour after hour is tiresome, imho.
I find that i get up a lot when I scrap - up and down, grab this or that. look from the top rather than sideways, stamp something, etc etc. if the chair is too comfy, it makes me lazy LOL))