Surprising #undef behavior

ID:2351771
 
Mar 9 2018, 1:00 am
BYOND Version:511
Operating System:Windows 10 Enterprise 64-bit
Web Browser:Firefox 58.0
Applies to:DM Language
Status: Deferred
This issue may be low priority or very difficult to fix, and has been put on the back burner for the time being.
Descriptive Problem Summary:
Calling a proc that uses a define that is undef'd before the proc call results in a compile error.

Numbered Steps to Reproduce Problem:
- Compile the following code
Code Snippet (if applicable) to Reproduce Problem:
#define THING 1
/proc/stuff()
    world.log << THING
    return "[THING]"
#undef THING
world/New()
    stuff()


Expected Results:
Since THING is only used inside stuff(), one would expect that #undefining THING after the proc has been declared would be fine.

Actual Results:
It is not fine. Moving the #undef AFTER the call to stuff() works, though. I have seen nothing in the DM reference that indicates this is the intended behavior.
Does the problem occur:
Every time? Or how often? Every time.
In other games? N/A
In other user accounts? N/A
On other computers? N/A

When does the problem NOT occur?
N/A
Did the problem NOT occur in any earlier versions? If so, what was the last version that worked? Unknown.
Mar 9 2018, 2:45 am
 

#define BLAH 1

/proc/definetest()
    return "[BLAH]"

#define BLAH 2

/client/verb/testdefine1()
    src << definetest()

#define BLAH 3

/client/verb/testdefine2()
    src << definetest()

Output:
2
2


It seems byond only compiles procs when they are first used.
Mar 9 2018, 8:53 am
This is actually rather surprising, because the parsing process uses a "lexer" that should be processing the preprocessor directives and the actual replacements in order. The proc compilation does indeed happen later, but the logic I'm seeing says the parsing of those procs counts first.

I'm gonna take a look and see why this is happening.
Mar 9 2018, 9:35 am (Edited on Mar 9 2018, 9:52 am)
Well this is a really weird one. It appears to be in some way "timing"-related, like the preprocessor directive is being encountered before it hits the "BLAH" because it's encountered while it's working on that statement. Truly strange. I'm still looking into this to see where the disconnect is.

[edit]
Yep, that's what's happening. The preprocessor directive is parsed and acted upon as soon as it's encountered, but that may happen while the parser is actually busy dealing with the previous token. The parser does a "look ahead" thing a lot where it moves on to the next token, and it's this looking ahead that's causing the problem.
Mar 9 2018, 1:53 pm
I have a plan to fix this, and it's a little involved.

1) When a directive is encountered while skipping code (e.g., from an #ifdef block), parse it immediately as we already do.
2) When a directive is encountered while not skipping code, return a new token type for preprocessor code.
3) Alter the token reading routines to act on the current token if it's a preprocessor directive, before moving onto the next token. If the next token is also a preprocessor directive, we can handle it immediately.
4) Alter all other routines that look at the next token to make sure preprocessor tokens are handled where appropriate. (This is the tough one.)

As you can see it's a bit projecty. However I think this is important to do, as I don't like the idea of preprocessor directives taking effect before prior code is properly parsed. The current behavior is just no good.
Mar 12 2018, 2:55 pm
Okay, this is officially way harder to fix than I allowed for. Trying to defer parsing of the preprocessor directives wreaks havoc with everything. After spending a ton of time on it, I've decided to let this drop for now and defer the issue.

The only place this comes up BTW is in embedded string expressions. Parsing string contents is deferred until it's time to create an expression, and that's the real reason why the #define sneaks in first.

There are two workarounds to this issue.

1) Don't let an embedded string be in the last line of real code (comments don't count) before the next preprocessor statement.

2) A variation of (1), just put in a line of code you know will do absolutely nothing, like /mob/suffix="".
Mar 12 2018, 2:55 pm
Lummox JR changed status to 'Deferred'