If you open that zipfile, you'll see a directory named 'dreamcatcher', with four things in it - 'test.dm', 'lex.l', 'parse.y', and 'makefile'. To build it, you'll need flex, a program for generating 'lexical analysers' on the basis of a regular-expression description of the tokens in the language, and bison, a program for generating parsers based on a grammar for the language in extended Bauckus-Naur form (Only LR grammars need apply).
If that was mostly greek to you, you may want to consult Wikipedia, or books on compiler writing/parsing at your local library.
Anyway, building: On a Linux system, install flex and bison, then load up a terminal, get into the directory, and enter 'make'. You'll get a program that can do... well, very little. Invoke it as 'dreamcatcher <test.dm', and it'll probably print some stuff out, followed by 'error'.
Not sure how to build it under Windows - figure it out yourself if you're that interested.
So, a quick look at what's in those files:
'lex.l' contains the Flex source for the DM lexical analyser. It's kind of ugly, and not terribly well written, but the same basic ideas will apply to the lexer I'll be writing for the new, improved, potentially-functional dreamcatcher. Flex files have a bunch of lines that look sort of like this:
\\.|\\\n {;}
That is, '<', alphabetic text, '>', some gobbledygook, a tab, '{', some C code, '}'.
The gobbledygook is a regular expression (If you're scratching you're head at what this one means, in the particular dialect in use in flex, it's 'A backslash, followed by any character [except a newline], or a backslash, followed by a newline'. The bit in '<' and '>' out the front is a state - this rule is only active when we're in the MULTICOMMENT state. And the code is code to be run when this rule is matched. In this case, it does nothing.
So this specific rule allows escaping of characters in multicomments (/* */ comments). The newline escape isn't strictly necessary, but it makes the rest of the multicomment rules cleaner.
'parse.y' contains the code for the parser. This is the bit that's fundamentally flawed. It turns out that in DM, it's impossible to distinguish between these two cases unless 'newline' is a lexical token (And this first-cut attempt doesn't have newline as a lexical token):
a/b
a
/b
There are some other grammatical rules that turn out to be simpler with a newline as a delimiter. In particular, a nasty hack to figure out whether / is being used as a path opening or a division operator is unnecessary.
Bison entries look sorta like this:
var: IDENTIFIER | var '.' IDENTIFIER;
That says that a 'var' is an IDENTIFIER, or, it's a var, followed by the character '.', followed by an IDENTIFIER. In short, it builds '.'-seperated lists of identifiers. Sorta like this:
src.contents.len
You may notice that some entries apparently have an extraneous '|' at the end:
proc_args: expression | proc_args ',' expression |;
That means that that production can be empty - that is, a 'proc_args' can be an expression, or a proc_args followed by a ',' followed by an expression, or it can be empty. These are all proc_args:
12 42, 23 "a", 7+5*3-(12<<5)
'test.dm' is just a very odd DM file with a series of test cases in them. It probably doesn't compile - they're completely isolated bits of code that I thought would be a useful test.
And, of course, 'makefile' is a makefile. Consult the internet for more information if you're really interested.</<5></5>
