How do you subtract the alpha channel of an object using overlays and color matrices only?

ID:2585895
 
Jul 5 2020, 5:45 pm (Edited on Jul 5 2020, 5:51 pm)
(See the best response by Lummox JR.)
I'm trying to create an effect where when behind tall objects, you can see yourself. I've tried many techniques so far but the one that I really want to do is an overlay that subtracts the alpha value from an object below using the plane master system combined with a layer filter.

https://i.imgur.com/9XQsadn.png

/obj/plane_master/scenery
    plane = PLANE_SCENERY
    mouse_opacity = 0
    color = list(1,0,0,0, 0,1,0,0, 0,0,1,0, 0,0,0,1, 0,0,0,0)


/obj/plane_master/scenery/New(var/desired_loc)
    . = ..()
    var/icon/I = new/icon('icons/test.dmi',"reverse")
    filters += filter(type="layer",icon = I, blend_mode = BLEND_SUBTRACT )
    return .


I've been experimenting with blending operations, and so far I can't really get anything to work in terms of subtracting the alpha layer. I've been messing around with the color matrix to see if I can make it so that the red channel is the alpha channel, but I'm not very good at it.
Jul 5 2020, 5:51 pm
I'm a fool. There is actually an alpha mask filter that I completely missed which gives the intended effect.

//Scenery
/obj/plane_master/scenery
    plane = PLANE_SCENERY
    mouse_opacity = 0
    color = list(1,0,0,0, 0,1,0,0, 0,0,1,0, 0,0,0,1, 0,0,0,0)
    

/obj/plane_master/scenery/New(var/desired_loc)
    . = ..()
    var/icon/I = new/icon('icons/test.dmi',"reverse")
    filters += filter(type="alpha",icon = I, flags = MASK_INVERSE)
    return .
Jul 6 2020, 12:03 am
Best response
I don't think you want to subtract alpha, but multiply it by something. For that, simply setting atom.alpha to something like 128 would work great.