Hi David,
There are several ways to do this. If you navigate to /modules/xerte/parent_templates/Nottingham/common_html5/ within your setup then you will see here that the buttons used are listed and have 4 states for hover/click etc...
You can directly change these although you can also upload alternatives and change the stylesheets to point to the new ones...
The latest version (v3.0 beta listed in Resources) will allow you to change the theme from within the editor to one of several predefined ones which may be suitable though and the v3 beta has the most up to date stylesheets and v3 should be released in the next few weeks.
John