ID:2379522
 
Resolved
The init proc optimization was being done simultaneously with var def optimization, but this caused problems in complex games like SS13. Now the init procs are optimized after var defs.
BYOND Version:512.1430
Operating System:Windows 7 Ultimate 64-bit
Web Browser:Firefox 60.0
Applies to:Dream Daemon
Status: Resolved (512.1431)

This issue has been resolved.
Please forgive deviation from format; I'm having trouble pinning this one down but it seems serious. It's new in 1430.

During our global controller initialization (called, AIUI, from a "/var/datum/foo/bar = new"), DD is claiming a variety of variables do not exist. The error appears to be coming from inside the new init procs, and reference variables which init with "= list()" or similar. The errors are the absolute first thing in DD's log after "Welcome BYOND!". The line numbers refer to the variables's declaration.

It does not appear in a test project. I wondered if it was a limits thing (tgstation has 88k+ variables across all types) but could not reproduce simply by creating a test project with many variables on many datums.

The reason I say that it's memory corruption is that at one point, the error log lists a proc path ("/datum/config_entry/string/ipintel_email/ValidateAndSet") in filename position. On a slightly different build, the error lists a different, shorter string ("splitter") as if a filename.

This can be reproduced reliably by compiling and running tgstation (https://github.com/tgstation/tgstation). There is no need to wait for the game to initialize; the errors happen immediately on world startup. Setting LOWMEMORYMODE in _maps/_basemap.dm may speed things up marginally. I'll see if someone more knowledgeable in tgstation's init can weigh in.

Here's a full DD log of the relevant errors. There are future errors but they're likely caused by these first ones. The bottom-most messages in the log are supposed to be the first things printed.

Logging in...connected
Logged in as SpaceManiac.
Tue Jun 26 16:18:41 2018
World opened on network port 11597.
Welcome BYOND! (5.0 Beta Version 512.1430)
runtime error: Undefined variable /datum/newscaster/feed_message/var/authorCensorTime

proc name:
  source file: newscaster.dm,14
  usr: null
  src: /datum/newscaster/feed_message (/datum/newscaster/feed_message)
  call stack:
/datum/newscaster/feed_message (/datum/newscaster/feed_message): ()
: ()
runtime error: Undefined variable /datum/newscaster/feed_channel/var/messages

proc name:
  source file: newscaster.dm,61
  usr: null
  src: /datum/newscaster/feed_channel (/datum/newscaster/feed_channel)
  call stack:
/datum/newscaster/feed_channel (/datum/newscaster/feed_channel): ()
: ()
runtime error: Undefined variable /datum/species/human/var/species_traits

proc name:
  source file: /datum/config_entry/string/ipintel_email/ValidateAndSet,5
  usr: null
  src: /datum/species/human (/datum/species/human)
  call stack:
/datum/species/human (/datum/species/human): ()
: ()
runtime error: Undefined variable /datum/species/human/var/species_traits

proc name:
  source file: /datum/config_entry/string/ipintel_email/ValidateAndSet,5
  usr: null
  src: /datum/species/human (/datum/species/human)
  call stack:
/datum/species/human (/datum/species/human): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_sec/var/channels

proc name:
  source file: encryptionkey.dm,29
  usr: null
  src: the security radio encryption ... (/obj/item/encryptionkey/headset_sec)
  src.loc: null
  call stack:
the security radio encryption ... (/obj/item/encryptionkey/headset_sec): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_sec/var/channels

proc name:
  source file: encryptionkey.dm,29
  usr: null
  src: the security radio encryption ... (/obj/item/encryptionkey/headset_sec)
  src.loc: null
  call stack:
the security radio encryption ... (/obj/item/encryptionkey/headset_sec): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_eng/var/channels

proc name:
  source file: encryptionkey.dm,35
  usr: null
  src: the engineering radio encrypti... (/obj/item/encryptionkey/headset_eng)
  src.loc: null
  call stack:
the engineering radio encrypti... (/obj/item/encryptionkey/headset_eng): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_sec/var/channels

proc name:
  source file: encryptionkey.dm,29
  usr: null
  src: the security radio encryption ... (/obj/item/encryptionkey/headset_sec)
  src.loc: null
  call stack:
the security radio encryption ... (/obj/item/encryptionkey/headset_sec): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_cargo/var/channels

proc name:
  source file: encryptionkey.dm,107
  usr: null
  src: the supply radio encryption ke... (/obj/item/encryptionkey/headset_cargo)
  src.loc: null
  call stack:
the supply radio encryption ke... (/obj/item/encryptionkey/headset_cargo): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_sec/var/channels

proc name:
  source file: encryptionkey.dm,29
  usr: null
  src: the security radio encryption ... (/obj/item/encryptionkey/headset_sec)
  src.loc: null
  call stack:
the security radio encryption ... (/obj/item/encryptionkey/headset_sec): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_med/var/channels

proc name:
  source file: encryptionkey.dm,47
  usr: null
  src: the medical radio encryption k... (/obj/item/encryptionkey/headset_med)
  src.loc: null
  call stack:
the medical radio encryption k... (/obj/item/encryptionkey/headset_med): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_sec/var/channels

proc name:
  source file: encryptionkey.dm,29
  usr: null
  src: the security radio encryption ... (/obj/item/encryptionkey/headset_sec)
  src.loc: null
  call stack:
the security radio encryption ... (/obj/item/encryptionkey/headset_sec): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_sci/var/channels

proc name:
  source file: encryptionkey.dm,53
  usr: null
  src: the science radio encryption k... (/obj/item/encryptionkey/headset_sci)
  src.loc: null
  call stack:
the science radio encryption k... (/obj/item/encryptionkey/headset_sci): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_eng/var/channels

proc name:
  source file: encryptionkey.dm,35
  usr: null
  src: the engineering radio encrypti... (/obj/item/encryptionkey/headset_eng)
  src.loc: null
  call stack:
the engineering radio encrypti... (/obj/item/encryptionkey/headset_eng): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_rob/var/channels

proc name:
  source file: encryptionkey.dm,41
  usr: null
  src: the robotics radio encryption ... (/obj/item/encryptionkey/headset_rob)
  src.loc: null
  call stack:
the robotics radio encryption ... (/obj/item/encryptionkey/headset_rob): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_med/var/channels

proc name:
  source file: encryptionkey.dm,47
  usr: null
  src: the medical radio encryption k... (/obj/item/encryptionkey/headset_med)
  src.loc: null
  call stack:
the medical radio encryption k... (/obj/item/encryptionkey/headset_med): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_sci/var/channels

proc name:
  source file: encryptionkey.dm,53
  usr: null
  src: the science radio encryption k... (/obj/item/encryptionkey/headset_sci)
  src.loc: null
  call stack:
the science radio encryption k... (/obj/item/encryptionkey/headset_sci): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_medsci/var/channels

proc name:
  source file: encryptionkey.dm,59
  usr: null
  src: the medical research radio enc... (/obj/item/encryptionkey/headset_medsci)
  src.loc: null
  call stack:
the medical research radio enc... (/obj/item/encryptionkey/headset_medsci): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/headset_com/var/channels

proc name:
  source file: encryptionkey.dm,65
  usr: null
  src: the command radio encryption k... (/obj/item/encryptionkey/headset_com)
  src.loc: null
  call stack:
the command radio encryption k... (/obj/item/encryptionkey/headset_com): ()
: ()
runtime error: Undefined variable /obj/item/encryptionkey/heads/captain/var/channels

proc name:
  source file: encryptionkey.dm,71
runtime error: Undefined variable /obj/item/encryptionkey/heads/rd/var/channels

proc name:
  source file: encryptionkey.dm,77
runtime error: Undefined variable /obj/item/encryptionkey/heads/hos/var/channels

proc name:
  source file: encryptionkey.dm,83
runtime error: Undefined variable /obj/item/encryptionkey/heads/ce/var/channels

proc name:
  source file: encryptionkey.dm,89
runtime error: Undefined variable /obj/item/encryptionkey/heads/cmo/var/channels

proc name:
  source file: encryptionkey.dm,95
runtime error: Undefined variable /obj/item/encryptionkey/heads/hop/var/channels

proc name:
  source file: encryptionkey.dm,101
runtime error: Undefined variable /obj/item/encryptionkey/headset_cargo/var/channels

proc name:
  source file: encryptionkey.dm,107
runtime error: Undefined variable /obj/item/encryptionkey/headset_mining/var/channels

proc name:
  source file: encryptionkey.dm,113
runtime error: Undefined variable /obj/item/encryptionkey/headset_service/var/channels

proc name:
  source file: encryptionkey.dm,119
runtime error: Undefined variable /obj/item/encryptionkey/headset_com/var/channels

proc name:
  source file: encryptionkey.dm,65
runtime error: Undefined variable /obj/item/encryptionkey/headset_cent/var/channels

proc name:
  source file: encryptionkey.dm,126
runtime error: Undefined variable /obj/item/encryptionkey/heads/captain/var/channels

proc name:
  source file: encryptionkey.dm,71
runtime error: Undefined variable /obj/item/encryptionkey/ai/var/channels

proc name:
  source file: encryptionkey.dm,129
runtime error: Undefined variable /obj/item/encryptionkey/heads/captain/var/channels

proc name:
  source file: encryptionkey.dm,71
runtime error: Undefined variable /obj/item/encryptionkey/syndicate/var/channels

proc name:
  source file: encryptionkey.dm,16
runtime error: Undefined variable /obj/item/retractor/augment/var/materials

proc name:
  source file: tools.dm,16
runtime error: Undefined variable /obj/item/hemostat/augment/var/materials

proc name:
  source file: tools.dm,38
runtime error: Undefined variable /obj/item/cautery/augment/var/materials

proc name:
  source file: tools.dm,61
runtime error: Undefined variable /obj/item/surgicaldrill/augment/var/materials

proc name:
  source file: tools.dm,89
runtime error: Undefined variable /obj/item/scalpel/augment/var/materials

proc name:
  source file: tools.dm,130
runtime error: Undefined variable /obj/item/circular_saw/augment/var/materials

proc name:
  source file: tools.dm,177
runtime error: Undefined variable /obj/item/surgical_drapes/var/attack_verb

proc name:
  source file: tools.dm,188
578 global variables
World loaded at 16:18:48!
Crap! I'll run a test in the /tg source I have which is already in low memory mode.
I found the issue. Apparently the var def optimizations have to be made first, and only then can the init procs be done properly.

It took an ultra-complex project to show this problem. Basically you had a datum creating other datums in the init proc and the process of building those datums was causing crashes because their types weren't optimized yet.
Lummox JR resolved issue with message:
The init proc optimization was being done simultaneously with var def optimization, but this caused problems in complex games like SS13. Now the init procs are optimized after var defs.
I don't think this is entirely fixed. I had hoped that the rest of the weird errors cropping up in the log at later init stages were just a result of the first thing, but they persist when compiling and running on 1431. The specific thing we're seeing is runtimes of this sort:

[12:55:49] Runtime in food_mixtures.dm,6: Cannot execute .Copy().
  proc name: New (/datum/crafting_recipe/food/New)
  src: Filet migrawr (/datum/crafting_recipe/food/bearsteak)
  call stack:
  Filet migrawr (/datum/crafting_recipe/food/bearsteak): New()
  init subtypes(/datum/crafting_recipe (/datum/crafting_recipe), /list (/list))
  make datum references lists()
  world: New()

[12:55:51] Runtime in __techweb_helpers.dm,142: Cannot execute .Cut().
  proc name: calculate techweb nodes (/proc/calculate_techweb_nodes)
  src: null
  call stack:
  calculate techweb nodes()
  initialize all techweb nodes(0)
  Research (/datum/controller/subsystem/research): Initialize(717519)
  Master (/datum/controller/master): Initialize(10, 0)


which implies to me that these list variables are in some uninitialized state wherein they are not null (or the error would say "null.Cut()") but not lists either.

It should be reproducable on the same codebase you already have. The first error will look goofy in the log because our error handler tries to check the config which hasn't been initialized yet; the rest are formatted normally.

These two particular examples both involve "new"ing everything that "typesof" returns under a particular datum, relatively early on in world startup, that may have something to do with it?

The relevant code for the first error:


/datum/crafting_recipe
        ...
        var/parts[] = list() //type paths of items that will be placed in the result
        ...

//a later file

/datum/crafting_recipe/food
        var/real_parts

/datum/crafting_recipe/food/New()
        real_parts = parts.Copy()
In response to SpaceManiac
If you post a new report I can close it. I found a major issue last night in 1431 that has a fix ready to go.