I update only what's in lights reach, not everything

You can also make that optimization when you're using one shadow object per tile. In the end your method still has 16 times as many objects to update.

I'm not sure how D4RK3 54B3R handled the shadow objects, but in A Miner Adventure they're objs on the turfs, not screen objects.