Well according to the USB specs, pin-1 of the USB is +5V DC and the pin-4 is ground. Pin-2 and pin-3 are data- and data+, respectively. I suppose if you put a switch on pin-1, it should disable the entire device. I don't really know much about circuits and etc. so it's something you'll have to test.
re: RCA jack
Well, I have a habit of collecting component junk. When something dies, I usually rip stuff out. This particular RCA jack is from some old external CD-ROM casing. I think you can just buy it, but I don't have a RadioShack part number or anything like that. It seemed perfect for the audio out solution I saw, especially since it came with a 4-pin header so I can disconnect it and remove the back casing if I need to.
From the picture, you can see that I chose to solder the ground at a different spot. Also instead of the .33uF capacitors, I chose to use 2.2uF capacitors because of what I read here. I soldered that directly into the circuit on the RCA jack. After that, it was a simple matter of making the holes with a dremel and crazy glueing the jack.
If you can't find an RCA jack with a pin-header, you could just solder some wires to it and strip 1/4" off the end. Apply some solder to it so it's nice and stiff, and insert it into a header the same way. I've done that in the past.