ID:2809808
 
Getting to the bottom of the bugs in the last alpha release turned out to be quite the undertaking, but I think I accomplished a lot this week, even with Wednesday turning upside-down on me. I managed to get a new alpha out to the testers that night, and so far I'm waiting for new feedback.

In the meantime I've been looking into optimizing compilation times again, because... woof, trying to test the map editor with SS13 is a huge pain. Profiling tells me that one routine is taking up the lion's share of the time, and most of that time is parceled out to a few helper routines. I'm trying to come up with a solution to that, which could potentially be a massive improvement.

Part of my optimization work also included trying to speed up memory allocation in the compiler, because there are a lot of small structures that get allocated in large numbers. I built a memory pool template class to handle that load, in hopes of improving that. So far despite the work I put in on pooling memory, I'm not seeing any real improvement in performance. But I don't want to give up on that new code yet; on the whole I think it might actually be better for keeping memory use lower, and it has a lot of potential elsewhere in the engine.

I have some other ideas for optimizing that might work a lot better, in terms of a small-scale change that can affect a lot of function calls and therefore add up, but I'll discuss those in more detail elsewhere. Again though, I think the real battle is in the routines I identified earlier, where if I can find a way to avoid traversing the same nodes over and over and over, there could be some huge breakthroughs.

Thank you so much to all of you whose support makes this project possible. Without BYOND's Members and other donors I couldn't do this.

The next few weeks remain rather in flux, but I don't think things will get insane for at least another week. We'll see how all that shakes out in the time to come. Meanwhile I'm hoping some tests can be bumped up and will give us good news for a change. This is a sucky way to spend the summer. But for those of you who aren't living in a very early and protracted October, enjoy the sunshine and use that grill, and above all else: mak gam.
Always love to hear about optimizations! Just purchased a 1-year subscription to show my appreciation.

Keep grinding through!
After helping Grind Knight optimize code, was wondering if there would ever be a way to minimize proc-overhead.

Avoiding the overhead of traditional autotiling we were able to cut the time on a 1000x1000 map from never-ending to 1.6 seconds via

proc/Autotile_World()
var N,E,S,W,NE,NW,SE,SW,V
for(var/turf/map/grass_2_sand/t in world)
V = 0
N = locate(t.x,t.y+1,t.z); if(N?.type == t.type){V += 1}
S = locate(t.x,t.y-1,t.z); if(S?.type == t.type){V += 2}
E = locate(t.x+1,t.y,t.z); if(E?.type == t.type){V += 4}
W = locate(t.x-1,t.y,t.z); if(W?.type == t.type){V += 8}
//
NE = locate(t.x+1,t.y+1,t.z); if(NE?.type == t.type){V += 16}
NW = locate(t.x-1,t.y+1,t.z); if(NW?.type == t.type){V += 32}
SE = locate(t.x+1,t.y-1,t.z); if(SE?.type == t.type){V += 64}
SW = locate(t.x-1,t.y-1,t.z); if(SW?.type == t.type){V += 128}
//
t.icon_state = "[V]"
In response to Kozuma3
At the risk of derailing this thread, I would strongly recommend bitwise here to cut down on all the logical conditioning.

for(var/direction in list(NORTH,SOUTH,EAST,WEST))
for(var/atom/neighbor in get_step(src,direction))
if(neighbor.type != src.type) continue
if(neighbor != src) . |= direction

icon_state = "[.]"
In response to Crazah
Already works as needed and efficient too, I'd suggest defining var/atom/neighbor outside of the previous loop and creating a static list of directions globally.
In response to Kozuma3
Those locate() calls are way worse than using get_step(). Every time you access the x,y,z vars there's division. At a minimum you'd want to store them in local vars.
Being able to encapsulate such code in procs as needed for readability and for modularity isn't feasible as the over-head of proc calls builds up.

Was just curious if there were any plans on making the use of procs faster.
Moreover, if you run an at_join check for each tile in the world, you only need 4 directional checks for each tile. It's better to loop over each turf in a block, than to do for/in world type filtering on top of an 8-dir check for a limited number of tiles based on type. It's a more flexible system and a better approach overall.

@crazah the for in list approach isn't necessary and will induce a bunch of redundancy. You are looping over a sequence of NORTH, SOUTH, EAST, WEST, or 1, 2, 4, 8. You can simplify this with a standard for loop.

for(var/d=1;d<=8;d*=2)


for(var/d=NORTH;d<=WEST;d<<=1)


Take your pick.
This is by far the fastest and most efficient way to achieve autotiling. Now that I have solved that issue, my above comment about proc overhead is still available and open for opinions.

proc/Autotile_World()
for(var/turf/map/t in world)
if(!t.autotile){continue}
t.icon_state = "[initial(t.icon_state)] [\
(get_step(t,NORTH)?.type == t.type ? 1 : 0) + \
(get_step(t,SOUTH)?.type == t.type ? 2 : 0) + \
(get_step(t,EAST)?.type == t.type ? 4 : 0) + \
(get_step(t,WEST)?.type == t.type ? 8 : 0) + \
(get_step(t,NORTHEAST)?.type == t.type ? 16 : 0) + \
(get_step(t,NORTHWEST)?.type == t.type ? 32 : 0) + \
(get_step(t,SOUTHEAST)?.type == t.type ? 64 : 0) + \
(get_step(t,SOUTHWEST)?.type == t.type ? 128 : 0)]
"
In response to Kozuma3
Kozuma3 wrote:
This is by far the fastest and most efficient way to achieve autotiling. Now that I have solved that issue, my above comment about proc overhead is still available and open for opinions.

proc/Autotile_World()
> for(var/turf/map/t in world)
> if(!t.autotile){continue}
> t.icon_state = "[initial(t.icon_state)] [\
> (get_step(t,NORTH)?.type == t.type ? 1 : 0) + \
> (get_step(t,SOUTH)?.type == t.type ? 2 : 0) + \
> (get_step(t,EAST)?.type == t.type ? 4 : 0) + \
> (get_step(t,WEST)?.type == t.type ? 8 : 0) + \
> (get_step(t,NORTHEAST)?.type == t.type ? 16 : 0) + \
> (get_step(t,NORTHWEST)?.type == t.type ? 32 : 0) + \
> (get_step(t,SOUTHEAST)?.type == t.type ? 64 : 0) + \
> (get_step(t,SOUTHWEST)?.type == t.type ? 128 : 0)]
"


OMG this helps thank u so much!! <3 i hate mapping lol
If all the tiles are the same, autotiling is way faster.
Lummox JR wrote:
Part of my optimization work also included trying to speed up memory allocation in the compiler

Another thing you might try, if not done already, is string interning. It's usually a huge boon for compilers. Here's a decent Rust-focused post on it.
the only real recourse when you have sheer overhead from proc calls is to resort to macro abuse to keep the code readable.
In response to Hiead
String interning is something BYOND does already. The method used in AddWord() in the lexer is a tree, but the tree didn't do any rebalancing, so that's been fixed.