I did this just 2 days ago to put in Golf R style switches with chrome trim, and I was struggling at first too. Behind the panel on the handle there are 2 torx screws to remove, and on the underside below the speaker there is a third screw. Don't forget this one.
To get the panel off, read the tips that forum user ciclo gave me on page 68 and 69 in his thread:
http://www.uk-polos.net/view ... start=1005
See this link for instructions and clip positions for the 3 door version:
http://workshop-manuals.com/volkswagen/ ... r_models)/
You will need a set of trim removal tools like these (at least that will make it easier):
http://ecx.images-amazon.com/images/I/41xuvaMYOkL.jpg
Specifically, you want a tool with a notch in the end. Bend the door panel out from the door slightly and take a peek to see where the clips are. Stick the tool in between the door and the door panel end press onto the clip with the notched part. A bit of pressing and maybe a little bending will make the clip pop out into "released mode". You will clearly hear and see when the clip is released. Follow the order in the link above, start with the clips along the bottom. It gets easier with every clip since the panel gets more loose.
After all clips are released, lift the rear part of the door panel first. In the front there is a "hook" that goes under the triangular plastic piece at the mirror base. When reinstalling, put the front end of the door panel down first, put the hook in place under the triangular trim, then lower the rear end of the door panel down in place. Make sure all clips are aligned with the holes (especially the uppermost one in the front end which is close to the mirror, I bent that one on reinstall since it wasn't aligned). Then just press on the door panel in the location of the clips to lock them in place again.
And yes, you do have to remove the door panels. The VW repair manual shows you can just bent them out with a trim removal tool without removing the door panel, but I think ciclo broke his switch trims when trying this. After seeing the backside of the switch trim, I don't think you can do this without breaking them.
When you have removed the door panel, you need a couple of small screwdrivers to slightly bend the edges of the switch trim to remove the actual switch.