ID:755884
 
(See the best response by Kumorii.)
Code:
    proc
attack()
flick("Attack",src)
//for(var/mob/Enemies/S in world)
for(var/mob/M in get_step(src,src.dir))
if(M != src) // you could also edit this in the future, so it doesn't attack people allied to you, etc etc.
overlays += 'Attack.dmi'
var/Damage=rand(0,src.Str-M.Def)
view(M)<<"[src] hit [M] for [Damage] Damage!"
M.TakeDamage(Damage,src)
overlays -= 'Attack.dmi'
DeathCheck()


Problem description:

Fire Boss hit Dr.Penguin for -21 Damage!
Dr.Penguin hit Fire Boss for -3 Damage!
Fire Boss hit Dr.Penguin for -4 Damage!
Dr.Penguin hit Fire Boss for -1 Damage!
Dr.Penguin hit Fire Boss for 0 Damage!
Fire Boss hit Dr.Penguin for -41 Damage!
Dr.Penguin hit Fire Boss for -3 Damage!
Fire Boss hit Dr.Penguin for -14 Damage!

Assuming your problem is that you can get negative damages, its because you are using a wrong proc. rand() will randomly return a number from its first argument to its second argument. You want to use max() here which will return the maximum of the arguments. For more information, read the reference on these procs.
Best response
Ignoring the previous comment, you're doing everything right. Rand() is the correct proc to use in this situation. However I believe your problem is that your str variable is less than the opponent(M)'s def variable which when you subtract them is producing a negative numeral.

Might I suggest using the following check to ensure that this doesn't come up repeatedly.

    proc
attack()
flick("Attack",src)
//for(var/mob/Enemies/S in world)
for(var/mob/M in get_step(src,src.dir))
if(M != src) // you could also edit this in the future, so it doesn't attack people allied to you, etc etc.
overlays += 'Attack.dmi'
var/Damage=rand(0,src.Str-M.Def)

if(Damage <= 0)
Damage = rand(1,5) //To ensure that you never do 0 or negative damage.

view(M)<<"[src] hit [M] for [Damage] Damage!"
M.TakeDamage(Damage,src)
overlays -= 'Attack.dmi'
DeathCheck()


However, I would still look into better formulas and try balancing out your variables so that the issue you have wouldn't be nearly as common/possible.

Good luck!
Thanks really helped
In response to Kumorii (#2)
Kumorii wrote:
However I believe your problem is that your str variable is less than the opponent(M)'s def variable which when you subtract them is producing a negative numeral.

That was my point. Using max(1, damage) here will prevent negative damage to happen. And I prefer it over an if() check, unless I'm wrong (in that case, feel free to correct me).