I was curious about how other developers would go about handling large numbers of AI in a game. I have tried several implementations, but have yet to find one that satisfied me fully.

If the following is tl;dr for you simply scroll down to the bottom for the gist of what I'm looking for in this discussion.

First let me describe my state machine. It has four states: "idle" (nothing detected), "wander" (player entering range), "attracted" (received a signal, pursue it), and "hunting" (player spotted).

My first attempt was an infinite loop which would periodically make a check to determine if the AI should switch from "idle" to one of the other three states. This was a horribly inefficient method but led into my second attempt.

In the second attempt, I still controlled the actions performed in a state with a loop; however, the periodic idle checks were removed. Instead, the player's movement acted as a beacon to determine if the mob should begin wandering, and when the mob began to wander it would begin checking for players in it's sight (players in front of them and in view).Furthermore, I staggered the times at which each mob would wander and perform a "sight check". This was more efficient, but still performed poorly with large numbers of mobs on screen (CPU percentage reached around 20-25 due to the sight-checks but stayed around 0-2 without sight checks).

All in all, I am aware that looping through the player mobs in view to obtain only the ones that are in front of the mob is my issue, but this is essential to making them look "smart, but not too smart" (i.e. not immediately hunting a player trying to sneak behind them simply because they're in view). I really want to make all of the AI "smart", but I may have to dumb them down and only allot certain rarer NPC's smarter AI.

Basically, the overall topic of this discussion is tips and tricks to efficiently handle large numbers of AI (hundreds) in a game; with some friendly advice if you have read the above.
You could have mobs wonder around Areas, and when a player has Entered() the area, the AI of the mob starts up.
Well, for one of my games, Hippie Killer 2.0, I had several hundred NPCs on the map at once. The NPCs came down in waves. Inactive waves sat on an unused z-level until needed. Those NPCs did not run any AI procs. When they were sent to the z-level in use, I called the AI procs.
For another game, Hobo Assault, if I remember correctly, I had inactive NPCs sit on their respective z-levels (Players progressed by clearing areas of NPCs {Each area was a z-level} before moving onto the next). As with the other game, I did not call any AI procs until I needed NPCs.
Basically, with both games, I used an event to trigger the NPCs into active mode.
Well one important thing to remember is that if a player isn't around to observe something it really doesn't matter what happens. So ideally you wouldn't even load up mobs until a player is nearby to observe them. Second you should be handling it backwards. Rather than having mobs waste tons of cycles checking mobs in view for players, when most of the time there probably won't be any, have players trigger mobs by the players movement. Even if the mob wanders about some while loaded they shouldn't need to check for players in their view if they get alerted through the players movement.
In response to OrangeWeapons
I rather like that idea, but I dislike the idea of NPCs randomly wandering around when there's no one around to see it.

Unless you get a game design like Guild Wars 2, where the world's state depends on how players help or not help allied NPCs when enemy NPCs decide to attack towns and whatnot.