ID:50892
 

Dream Tutor: The Pitfalls of Savefiles

by Lummox JR

Savefiles are easy to work with in BYOND. But behind every simple feature is a whole universe of ways to run into problems using it. If your game implements persistent characters or scores, there are a few things you need to know.

What Is Saved?

When you save an object into a savefile, BYOND does a lot behind the scenes. If that object has a var that is not /tmp, and is not set to its initial value, then it will go into the file.

BYOND will not include the same object in a savefile more than once--all it has to do is save something one time. If you have two vars that have the same object as a value, the first one will save all the object's vital information, and the second is just a reference. It's like the savefile says, "Remember that sword I told you about earlier? That very same sword is the one this mob is using as a weapon."

Another important thing to know is that if there's a reference to a file in the cache, that file will be saved as well. That includes icon if it isn't set to its default value and isn't null.

What are some of the important vars to remember that get saved? The contents list is one--if you save a mob, everything the mob is carrying is saved too. If you save a turf, everything on it is saved. Any other lists will also get saved in this same way.

An atom's loc and verbs vars are not saved. If you want to preserve this info you'll have to save it yourself. There's a good reason loc is not saved--if it was, then loc.loc would also be saved, and so on until an area was reached. Then everything in the area--turfs, objs, mobs, and all--would have to be saved just to save one item.

The overlays and underlays lists also undergo a curious transformation--if they're not empty, they'll each be converted to a single icon, losing any pixel offsets and layer information. Later on we'll see why this can be a problem in a number of ways.

/tmp Is Your Friend

Not every var in a game is meant to be saved. Very often you'll want to keep track of something only while a character is in use. A common example would be a var used to temporarily lock a mob's movement while performing another action, or anything related to movement delay. It's important to keep that in mind, and plan accordingly.

var/tmp/nomove

Chances are you have a few vars already that should be /tmp. It's not too late to change them.

Not only will /tmp make your savefile smaller, they'll also eliminate potential bugs. Suppose nomove was not /tmp, and it was turned on right when a user logged out, so their character saved that way. When they load the character again they can't move. Oops!

BYOND's savefiles do you a huge favor by saving just about everything you'll ever need. Instead of having to remember to modify your savefile code every time you add a new var, you don't really have to do anything in order for that new info to show up in a savefile. So BYOND saves you a lot of work by saving just about everything except what you tell it not to. The problem is, well, BYOND saves just about everything except what you tell it not to. And since some vars are not meant to be saved, that can be a problem.

When you start using savefiles in your game, you're adding a kind of depth that single-session games don't have. Now instead of vars lasting just for the life of the game, a var can come back the next time you start a session or the next time a certain player logs in. For some vars this is a good thing, and for some it isn't. Wherever it isn't, use /tmp.

As we'll see later, there are all kinds of things that may be saved that shouldn't be, and this can cause some unexpected behavior if you don't keep that in mind. But /tmp isn't the only tool in your arsenal; sometimes there are cases where you want to save a little bit of info, but you don't want to save everything. We'll revisit that idea a little later.

Don't Save the Icons!

If you customize a character's icon at all, or basically change to any icon other than the mob's default, the icon must be saved in the savefile to be used again. A savefile can become quite accidentally bloated with a lot of extra data this way. (This obviously doesn't apply to cases where you only change icon_state.) To make things even weirder, as I mentioned earlier BYOND will save any overlays and underlays as a single icon each--which is undesirable not only because it can bloat the file, but because it won't preserve any info about layers and pixel offsets those overlays used.

What's the best way out of this predicament? Don't let these things save. Or save them, but strip them out of the file right afterward.

mob/Write(savefile/F)
..()
F["icon"] << null
F["overlays"] << null
F["underlays"] << null

But how, then, do you keep custom-colored icons or show equipment overlays? What you'll need in their place is some way of remembering which icons, which overlays, should be used. Let's say in your game each player can choose from one of three icon files for their character, and they can choose a hairstyle with a custom color, and their equipment gets overlaid. Let's make things easy and build the player's icon and overlays in special procs--then we can call these whenever we need to, like when a character is created or loaded.

var/list/character_icons = list("human" = 'human.dmi',
"elf" = 'elf.dmi',
"robot" = 'robot.dmi')

mob
var/character_icon = "human" // default value
var/hairstyle = "fro"
var/haircolor = "#333333"

mob/proc/BuildIcons()
var/icon/main = new(character_icons[character_icon || "human"])
var/icon/hair = new('hair.dmi', hairstyle)
// red component becomes custom color, green becomes highlight
hair.MapColors(haircolor, "#ffffff", "#000000", "#000000")
main.Blend(hair, ICON_OVERLAY)
icon = main

mob/proc/BuildEquipmentOverlays()
overlays.Cut()
var/obj/olay = new
for(var/obj/item/O in equipment)
if(O.has_overlay)
olay.icon = O.icon
olay.icon_state = "equipped"
olay.layer = (O.equipment_layer || MOB_LAYER+1)
overlays += olay

Hey, that's not too bad. We have a special proc for re-creating the main icon as needed, and one for totally redoing the equipment overlays. Since we're no longer saving the icon and overlays themselves to the savefile, we just need to call these procs after loading the character.

mob/Read(savefile/F)
..()
BuildIcons()
BuildEquipmentOverlays()

You may be wondering at this point why the code uses an associative list called character_icons. Wouldn't it be easier to just set character_icon='elf.dmi' directly instead of referring to the icon by a name and having to look it up? Actually no.

Here's something you may not have known about BYOND vars: When you're just running a game, a var that's set to one of your icon files such as character_icon='elf.dmi' is going to hold a cache reference. Basically this means BYOND thinks of 'elf.dmi' as (for example) the 14th item in the .rsc file. (If you used the \ref macro to take a look at this var, you'd see a value of "[0xc00000d]".) But if the engine only saved that info and you added some files to the game later, so that 'elf.dmi' was now the 17th file in the resources, then you'd have some funny-looking icons when you loaded the character again. For all you know the 14th file could now be a sound file, or a wall icon. BYOND gets around this problem by saving the entire icon file. When you loaded the character, the engine would notice that the file it was loading was the same as cache file #17.

Because we don't want BYOND saving raw icon data to bloat the file, this code makes sure that only simple text strings such as "elf", "long", and "#CCCC33" (gold, an appropriate hair color for an elf) are saved. The associative list takes one of those text strings, "elf", and finds the icon you actually wanted to use.

Keep Version Info

One thing your savefiles should always have is a var that says what version the file is. There's one very important reason for this: Old savefiles don't necessarily have to be wiped. Let's say you have an RPG where every character has certain stats, and later on you decide to change the stats so they're on a different scale. You may know that in the old version a strength of 20 is equivalent to 8 in the new version, but obviously if you load the old character as-is it'll be overpowered. The method most authors use to deal with this is the dreaded "pwipe", totally sacrificing all saved games for the sake of an update. Sometimes wipes are unavoidable, but usually they aren't, and whenever you do one your players will hate you.

But if you know the formula to convert an old character into a new one, all you need to do is:

  1. Read the version number from the savefile.
  2. Make adjustments that need to be done before the object is loaded.
  3. Finish loading the object.
  4. Make any adjustments that need to be done after the object loads.

That's it. Pretty simple, huh? And you can use the same version for the whole file, so if you need to update one character in a file that holds several of them, you can update the whole thing. For now though let's look at the easy approach.

// update this any time you need to change the savefile format
#define SAVEFILE_VERSION 8

mob/var/tmp/savefile_version // you don't need to load this--make it /tmp

mob/Write(savefile/F)
// save version info with the character
F["savefile_version"] << SAVEFILE_VERSION
..()
F["icon"] << null
F["overlays"] << null
F["underlays"] << null

mob/Read(savefile/F)
var/version
F["savefile_version"] >> version
if(isnull(version)) version = 0
if(version < 4) // back when icons were saved--oops
F["icon"] << null
F["overlays"] << null
F["underlays"] << null
// do the normal read
..()
var/resave
if(version < 8) // back when all the stats were huge...
strength = round(strength * 0.4, 1)
dexterity = round(dexterity * 0.4, 1)
wisdom = round(wisdom * 0.4, 1)
hp = round(hp * 0.25, 1)
max_hp = round(max_hp * 0.25, 1)
resave = 1
if(resave)
// re-save the character
Write(F)
BuildIcons() // now we set up the icons when we load
BuildEquipmentOverlays()

In this example game, for a few versions the files all had icons saved with them. There's no need for that anymore, so any older files should have that info purged. And as of version 8, there's been a change to the stat balance, the time to update that is when an older character is loaded.

The most important reason to use version numbers in your savefiles is that if your files end up having a critical problem that you discover later on, they can be fixed. You won't have to erase savefiles or manually edit every single one.

By the way, if you already have savefiles it's not too late to add version info. Just think of the version-less files as version 0, and save version 1 to all your new files from now on. If version info can't be loaded you'll just get a null value, so just change it to a 0 and go from there.

Groundhog Day

There's one giant pitfall in savefiles that even intermediate to advanced users have run across unexpectedly. It's called the "rollback bug", only it's not a bug in BYOND--it's a bug in the game. Here's the scenario that happens most often: You're in a game minding your own business, when another regular logs in. Suddenly you're transported to a place you haven't been in a while. All your stats are lower, recently acquired items are gone, and in every way it looks like the clock has rolled back on your character. What just happened?

What happened is that when the other player logged out the last time and their character saved, your mob came along for the ride. A copy of your character got accidentally saved in their file. As part of a normal save, a mob's key var is also saved, which would be your BYOND key. When the old copy of your character was loaded, the key var was loaded along with it, and you happened to be online. Remember that whenever mob.key is changed to a user's key while that user is logged in, that user is switched over to that new mob. Logout() is called for the mob they were using before, and Login() is called for the mob they just took over. So you've just been switched over to the old copy of your mob, and the one you were just using is either being deleted or is sitting "dead" in the middle of a field somewhere with a vacant expression and a pack of undead getting ready to loot your best new items.

How can this happen? Well, here's one easy way to cause the problem:

mob
var/mob/best_friend

Notice best_friend isn't /tmp--it will be saved. And because it's a direct reference, it will save a copy of the best friend's mob. The game author may not have wanted to use /tmp because this info is supposed to be saved--but it's not supposed to save the entire mob! Here's a viable alternative:

mob
var/tmp/best_friend
var/best_friend_key

proc/BestFriend(mob/new_best_friend)
if(new_best_friend)
best_friend = new_best_friend
best_friend_key = new_best_friend.ckey + ":" + new_best_friend.name
return best_friend
if(best_friend || !best_friend_key) return best_friend
// best_friend_key is in "[ckey]:[character name]" format
var/index = findText(best_friend_key, ":")
var/bf_ckey = copytext(best_friend_key, 1, index++)
var/bf_name = copytext(best_friend_key, index)
for(var/client/C)
if(C.ckey == bf_ckey)
if(C.mob && C.mob.name == bf_name)
best_friend = C.mob
return best_friend
return // user found, but wrong character; return null

Notice that the info we're saving now is just what's needed to find the mob--not the mob itself. That lets us keep persistent info without having to worry about accidentally saving two mobs in one file.

This by the way is another use for version controls. If you had already put the best_friend var in your savefiles, you could use version checking to safely take it out before actually loading either mob.

mob
Write(savefile/F)
...

if(version < 7)
F["best_friend"] << null // better to lose this info than load an old mob
..()

...

There are other ways a mob could end up in a savefile, though. The example above is a direct reference, but what if the reference is indirect?

obj/item
var/mob/original_owner

If you're holding this item in your inventory, the original owner is gonna join you in your savefile. How about this one?

mob
var/turf/home

This var looks totally innocuous, but it hides a vicious secret. To pry out that secret, picture this scenario: You and Joe are both playing the game, when Joe logs out and his character is saved. Unbeknownst to you at the time, you're standing on Joe's home turf while he happens to be gallivanting about fighting monsters. A week passes, and one fine Saturday afternoon you're out fighting monsters of your own when suddenly Joe logs in--and you're standing on the same turf he is, wondering how you got there and where your whole week of progress went. What happened? The whole turf was saved along with Joe--and so was everything on it.

The way out of this sort of problem is to save only the coordinates of the turf, but make the turf itself a /tmp var. Just as in the best friend example above, the key is to make sure the only info that can save is what you would need to reconstruct a working character.

The first example was a direct reference to another mob--the turf is an indirect reference, since the other mob is only referred to in the turf's contents. Indirect references are harder to spot, and they can pass through several levels of indirection before they're spotted. For instance, you might be carrying a special homing weapon that keeps track of who it last targeted. Or that item might keep track of the last place it was set down, so you could end up with a very rare situation where the obj you're carrying references a turf which references another mob (two levels of indirection). Items in a list can get saved this same way, and you might have lists for any number of reasons.

Not only are rollbacks hard to spot, but they're intermittent--more so if it takes a few levels of indirection to cause the glitch. Your game could go through months of heavy play before such a bug ever popping up.

Is there any good news? Yes, there is. First, if you're on the lookout for vars that can be saved but shouldn't--and ways they might end up being saved--then you can anticipate problems and design around them. That's half the battle. Second, this kind of issue comes up most commonly as games get more complex, not early in the game's life. If you've started with a firm foundation and you follow good design principles (like looking out for the rollback pitfall) whenever you add new features, it's entirely possible to avoid this kind of problem altogether. And if you ever do run into the problem, using version checking as described earlier can get you out of a jam.

Go Forth and Save

As you now know, using savefiles isn't simply a matter of plopping them into your game. Like any feature, there are simple ways to make it go wrong. There are two different ways to approach savefiles: Make the author manually save everything and take total control of the file format, or save just about everything and make the author carve out exceptions as needed. BYOND chooses the latter, easier option. You can think of this as the difference between building a house from scratch and using prefab materials to assemble it in a fraction of the time. Just because using prefab is easier, cheaper, and just as good quality doesn't mean you can't pay attention to how you're putting it together--you have to cut through walls to put in outlets, trim off excess to make corners fit snug, etc.

Like with a house, using the prefab solution is definitely the way to go in most cases, but you can't do it blindfolded. Be aware of the potential problems you could run into, and you'll find you won't run into them very much at all. Knowing what you know now, would you make a waiting_to_move_again var without declaring it as /tmp? Not on purpose, you wouldn't. And if you forget and do it anyway, you now have the knowledge you need to fix it: A simple version check can save the day. Even better, if you've already got a lot of weird bugs coming from your savefiles, now you can repair them without tearing everything down and starting from scratch.

Wow... Long... and I haven't even started reading :) TIME TO READ :)
LOl
Very nice. This'll help those space-conservative people, as well as those who experience "rolling back." Thanks for the info!
Pretty nice article, I always wondered what the rollback bug was.
Very nice article. I think you should also mention stuff about saving lists because I was fiddling around with savefiles today and found out that you can't directly save lists and you have to use your own directory (unless I was going about it wrong).

I have managed to avoid the rollback bug by declaring every variable I define for an object that can be saved as /tmp and then if it lookd like it needs to be saved, I take it out of the /tmp row by untabbing it.
Haven't read everything yet, but note savefiles don't have a Remove() proc and those calls are meant to be on the savefile's dir list instead.

EDIT: Overwriting the value with null works too, but what it was before is cleaner IMO. It just needed to be F.dir.Remove("icon","overlays","underlays") instead of F.Remove(...).
I might be mistaken but i think in your hair.MapColors(hairstyle, "#ffffff", "#000000", "#000000") the hairstyle should actually be haircolor in your example, unless im reading it wrong but thats how it looks like
Good catch; thanks.
mob/var/tmp/savefile_version // you don't need to load this--make it /tmp Defined but not used? I can't see anywhere in that example where you use this variable.
i am going through the tutorials one by one, when i am done is there a test to take to check my skills or is it straight to creating games
Taymoney7 wrote:
i am going through the tutorials one by one, when i am done is there a test to take to check my skills or is it straight to creating games

Straight to creating games.
Now that I think about it, couldn't you, as a fallback measure, prevent the rollback bug entirely by simply overriding mob/player/Write() and not letting it save players when it's not supposed to? Seems very possible and quite straightforward. You could even do something similar with Read() as well.
In theory yes, you could set a flag that says a mob is safe to save, and then check that during Write(). However Write() would still try to output something I believe, so like as not you'd end up with a bunch of blank generic mobs loaded onto a turf. Also anything that depended on a genuine mob reference, like who your party leader is, would still be broken. That's to say nothing of duplicate objs and spurious turf loads.

I suspect that in this case, making saves fault-tolerant would be a worse cure than just avoiding these kinds of mistakes in the first place. Little bits of brokenness would be introduced that would be very hard to diagnose or (in some cases) even detect properly. You'd basically be trading off one glitch for another, harder-to-find glitch.
It shouldn't be a problem to prevent blank mobs, but anyway. That'd be pretty great as a fallback measure, not main prevention method, as my post suggested.