Under the hood: Appearances and you

ID:1863944
 
Jun 1 2015, 11:42 pm (Edited on Apr 11 2017, 11:00 pm)
People have often wondered: When you add something to an overlays list, why can't you change it? The DM reference always used to say that the items in an overlays list are stored in a special internal format, but it didn't go into more detail than that. With the new appearance var in BYOND 508, it's time to shed a little light on this internal format that has, until now, been undocumented.

What is an appearance?

Every atom, or image, has an appearance. Its appearance is made up of all the vars that could possibly impact the way it's displayed. This includes:
  • icon, icon_state
  • layer
  • dir
  • name, desc, suffix
  • text
  • screen_loc
  • gender
  • density (not actually visual, but stored here anyway)
  • opacity
  • luminosity
  • invisibility
  • pixel offsets (movables handle these specially)
  • glide_size
  • mouse pointers, mouse_opacity
  • maptext
  • override (images only)
  • transform
  • color, alpha, blend_mode
  • overlays, underlays
  • verbs
  • plane (509)
  • appearance_flags (510)
Appearances are handled a lot like how DM handles strings. They're immutable and unique, which means when you change one var, you're actually creating a whole new appearance. Since they're unique, they can be shared. In this way, every single turf is not storing all of these vars, but instead can just say they use appearance #11.

Notice how verbs are in the mix? That's why you can alter an atom's verbs list: You're actually creating a new appearance that includes the verbs you want.

Understanding overlays

So now you know the special format that overlays and underlays use. These lists only store appearances, and in fact are not regular lists at all. They're an internal kind of ID-only list that conserves memory. And for the fun part: ID lists, like appearances, are also immutable and unique. This means that if you have two different mobs whose only overlay is a full health bar, their overlays lists will be the exact same list. (The verbs list is also an ID list, but for procs.)

Knowing this, what happens when you add an obj to another atom's overlays list? Or a type path? Or just an icon or an icon_state? All of those are legal to add to overlays.

Let's go through the process of adding an overlay to a mob.

mob/proc/HatMe()
    var/obj/O = new
    O.icon = 'hat.dmi'
    O.pixel_y = 16
    O.layer = FLOAT_LAYER
    overlays += O

The mob's overlays list does not contain this temporary obj O, but a copy of its appearance. This is what happens, in order:
  1. A new obj is created; it is assigned the default appearance used by a regular /obj. We'll call this appearance A.
  2. O's new icon means it needs a new appearance. A new one is created, or an old one is looked up, that is identical to A but has hat.dmi as the icon. Call this B. (What if /obj already had hat.dmi as the icon? No change, and B is A.)
  3. O's pixel offsets are changed. This does not affect its normal appearance, because movables store pixel offsets separately. (Why? To prevent "churn" in games where pixel offsets are the main thing that changes. This was before we had the step vars.) A pixel-offset appearance will only be calculated if it's needed.
  4. Changing O's layer changes its appearance again: from B to new appearance C. If B isn't being used anymore, it's deleted.
  5. Adding to src's overlays means we need a copy of O's current appearance, including pixel offsets. That offset appearance is calculated now, which gives us appearance D.
  6. A new ID list is created, or looked up, that matches src.overlays but has D added onto the end.
  7. src gets a new appearance. If its appearance was P, now it will be Q instead; Q is the same as P but uses the new overlays list.
  8. The proc ends. O is deleted because it has no more references. Appearance D is still in use, but if nothing else is using C, it's deleted too.
Appearances and ID lists get recycled. If an appearance is seen by any client, it won't be recycled until each client says they haven't seen it for a while. In this case, appearances B and C were never seen by the player at all, so they get recycled pretty much immediately unless they appear elsewhere on the map.

Appearances and animation

When you call animate(), you include all of the vars you want to change. It's clear now what this is doing: A new appearance is created, and animation just interpolates from one to another.

Each stage of an animation has a "from" appearance and a "to", which we'll call A and B for clarity. When Dream Seeker displays an animated atom, it looks up which stage it should be using at the current time index. Then it figures out how far along it must be from A to B, and compares the two appearances to see what's different. For anything that's different, it will interpolate smoothly if it can (like color, transform, etc.). The atom will always still have a "true" appearance, which is wherever its animation ended. If the animation doesn't loop forever, eventually it will finish there and the animation will be erased.

Appearances and savefiles

Wait a sec, you say: How does this factor into savefiles? Are they saving whole appearances? Nope, they are not.

When an atom is saved, its appearance-related vars are saved--if they differ from what's normal for that atom's type. That is, layer will show up in the savefile if you changed it, but otherwise it won't. That's the same for any other vars.

The overlays and underlays lists, though, don't fare so well. When the savefile format was first invented, it was given no "standard" way to save appearance info. As a result, savefiles do the simplest thing they can: merging all overlays into a single icon, and the same for underlays. When the savefile is loaded again, what might have been ten overlays loads up as one, which is just an icon and layer=FLOAT_LAYER.

For this reason, I always recommend that you clear overlays when saving, and rebuild them yourself after loading. It avoids bloat in both your savefiles and the icon cache, and it's just good maintenance.

Using the new appearance var

Version 508 introduces the atom.appearance var (also used by image), which can be used to grab a copy of an atom's current appearance, or update it in one shot. These are the only vars that will not be affected when you set an appearance:
  • density
  • dir
  • screen_loc
  • verbs
Here's an example of how that might be used:

mob/var/tmp/dg_reset

mob/verb/Doppelganger(mob/M in oview())
    if(magic < 10 || dg_reset) break
    magic -= 10

    // save the current appearance
    dg_reset = appearance
    // copy the spell target's appearance
    appearance = M.appearance
    spawn(600)
        appearance = dg_reset
        dg_reset = null

There are some subtleties to this of course. If used in a real game, you'd want to make sure to revert to the original appearance (if any) when saving. You might want to be able to cast the Doppelganger spell again before time runs out, which would involve just a little more sophistication to the timer. But the gist is that it's trivial to change your appearance with the spell; without having to remember a couple dozen different vars, instead you can change them all at once.

Or here's a weird one for you:

mob/proc/ChangeOverlayColor(olay, clr)
    if(!clr) clr="#fff"
    var/i = overlays.Find(olay)
    if(i)
        var/obj/O = new
        O.appearance = olay
        O.color = clr
        O.alpha = 255   // no fair including alpha in the color
        overlays[i] = O

The temporary obj O is created just to give us a canvas to work on for creating a new appearance. The olay argument is a the appearance we'd like to find in the mob's overlays list, then replace with a new one.

The appearance var can also be used in animate(). Very handy if you want to alter more than one var at once without a complicated proc call, and it saves both space and time. Fewer arguments to load means that animate() will be called sooner, and it also will have to create fewer intermediate appearances than it would by changing one var at a time.

Closing the hood

Now you know everything you need to know about appearances, one of the fundamental structures underlying the DM language.
Jun 2 2015, 12:41 am
Bravo. This is everything I've said about appearances over the last year in a single post and even a bunch of stuff I didn't actually know.

Though, my only complaint is that this article lacks information on what I call "appearance abuse", using the same object multiple times and triggering an appearance change between each iteration of a loop, where at the end of each iteration the object is added to the overlays, thus resulting in using only a single spare object to generate numerous overlays.

There's also the interesting case of accessing and inspecting appearances by casting them as atoms.

And there's also the use of "\ref" macro on overlays lists and appearances themselves to generate a uniqueid for anything that has a unique appearance, thus allowing you to hijack some of the behind the scenes magic that BYOND does for you.
Jun 2 2015, 9:31 pm
Very nice! Can't wait for BYOND 508!!

Keep up the good work.
Jun 3 2015, 2:48 am
I'm curious if you did any work on making it possible to save the visual data of an appearance through fcopy() or ftp()?
Jun 3 2015, 2:50 am
I'm curious if you did any work on making it possible to save the visual data of an appearance through fcopy() or ftp()?

Oh man. That needs a feature request.
Jun 3 2015, 9:36 am
At present no, there's no conversion from appearance to icon.
Jun 10 2015, 6:16 pm
I just read this and I have to say that this is an amazing addition to the new BYOND. I would have never guessed that appearances were how the engine handles things like overlays!
Jul 25 2015, 3:05 am (Edited on Jul 25 2015, 7:38 am)
In 508.1294, overlays[i] gives runtime error: cannot write to indexed value in this type of list

Is that a trick to replace the common procedure of "remove, change, re-add overlay" with "replace overlay"?
overlays -= olay
olay.color = clr
olay.alpha = 255
overlays += olay


Also, would it be practical to have a new proc set_appearance(atom, icon, icon_state, layer, ...) that sets all the provided values (similar to animate()) and only updates the appearance once, or would animate(atom, ..., time = 0) already do that?

Also, if an appearance-related variable is set to the value it already is (e.g. icon_state = icon_state), is the appearance rechecked?
Jul 25 2015, 6:35 pm
Although the setting thing isn't a bug, please make a bug report on that; that's behavior I think can and should be changed. (As a workaround, you can copy the overlays list, set an appearance in the copy, and then reassign the overlays list wholesale.)

I think having a way to set multiple appearance vars at once might be useful, except that there's really no mechanism for it. Even animate() doesn't strictly work that way; it calls the var write routine for each var. (There's a reason it all works that way under the hood.) I'd need some kind of catch-all routine for setting appearance vars en masse.

To answer the last question, the routine that makes the change will create a temporary appearance, change the var, and then it does a lookup-or-add on the temporary to assign it an ID. If the value hasn't changed, the lookup still happens. There may be a couple of routines (I forget) that short-circuit this and skip the lookup if there's definitely no change.
Jul 25 2015, 10:02 pm
I think having a way to set multiple appearance vars at once might be useful, except that there's really no mechanism for it. Even animate() doesn't strictly work that way; it calls the var write routine for each var. (There's a reason it all works that way under the hood.) I'd need some kind of catch-all routine for setting appearance vars en masse.

Maybe you should expose the /appearance type directly so we can start working with appearances a bit more directly. What you just mentioned would seriously mitigate appearance churn, which right now is one of the bigger issues with BYANB. (More or less like I did with StdLib, only actually representative of the type itself and not a pass-through interface. Again, consider every feature in StdLib a standing feature request from me to you.)

Maybe rather than actually making a new method to change appearances directly, you could look into simply keeping a global collection of all appearances that have had values changed indexed against a temporary structure that stores those changes. Then at the end of the frame, the server can just loop through the collection, generate the new appearances and do the normal appearance-foo. That way it's basically an optimization to existing games rather than something developers need to learn.
Jul 25 2015, 10:33 pm
The problem with optimizing appearance settings by keeping a temporary Appearance struct is that reconciling the changes would have to be done on every sleep, var read from the atom, or proc end. (Also all of the internal functions that change these values would have to be modified, but that's another issue.) The checking for that would, I believe, be worse for performance in the general case. Thinking of what those changes would entail, I think it'd be a huge overhaul of the current system--and it'd be even worse for turfs.

I do think it'd be nice to have a catch-all setter for the various appearance vars that could do them in bulk, bypassing the need for more than one lookup. If there were a built-in proc for that, then it might be possible (though I find the idea daunting) for the compiler to recognize multiple vars being set for the same object, and reorganize them into a single call. That could be potentially beneficial, and would obviously help routines like anmiate().