Quote:I'm having trouble with rounding the edges off without making it look weird
Do you mean they get pixelated and rough? If that's the case, boosting your image resolution should help. Also, instead of rasterizing your rounded rectangle shape to make your selection, just select the Pen Tool and right-click the shape. A dropdown list will appear; select "Make selection." In the window that pops up, make sure "Anti-Aliasing" is checked. Click OK. You should end up with a much cleaner selection.