I think the appearance of the radio button has less of an impact on 508 and more on WCAG. 508 tends to focus more on physical accessibility, whereas WCAG is on visual elements, screen readers and general functionality. What you want to be careful of with the appearance is color contrast. Some of the radio and check box options may not meet WCAG guidelines depending on the page background.
There are several contrast checkers out there, but my go-to is the WebAIM contrast checker. There’s also a WCAG Contrast Checker add on for Firefox that you could use when previewing in a browser window. Chrome also has a number of add-ons you can use.