Split pauses into two types: immediate and vote.
Players can use an immediate pause every 10 minutes. It takes effect immediately (duh), but can also be unpaused by other players with the standard 3 second delay. Spectators can unpause the game with a 30 second delay.
The vote pause allows a player to request that the game be paused, and shows a little icon over their head. Other players can then "approve" or "deny" the request. If more players approve than deny within a certain time frame (the requester doesn't count, non-voters are neutral), the game is paused. You can vote to pause during an immediate pause, and if it is successful, the person who used their immediate pause gets it back. To unpause the approved pause, a similar vote must pass (however, anyone who approved the pause will default to voting against unpausing, in case they're AFK).
Auto-mute players that spam
Prevent people from tying up the server by AFKing a wave in the corner. Maybe if it isn't beaten within 2 minutes (and no enemies are being killed), eyeballs (ones that track players) or reapers should start to spawn, with increasing frequency.
Thoughts? Issues? Abusable?