Map Data Extractor

by DarkCampainger
Extract only the relevant bits of data from your project for your mappers

About Map Data Extractor

ID:848097
 
Jul 1 2012, 10:00 pm
If you want to allow others to make maps for your game, but don't have a custom map editor, you might be tempted to give them the entire source code. However, a mapper only needs a small subset of your code to create maps for your game: the basic visual data of your mappable turfs, objs, and mobs.

This library allows you to extract all of the data for your mappable atoms and create a separate project just for your mappers in as little as a single line of code:

var/MapDataExtractor/mde = new/MapDataExtractor()


The default settings will analyze your project to automatically extract any atoms that have been used in included map files, and will exclude any types listed in the project's Object Tree Hidden Paths preference. You can also specify your own list of types to include and/or exclude.

The generated project contains only a single DM code file--which holds just the visual data of the included atoms--and a precompiled RSC file.

////////////////////////
//                    //
// MAP DATA EXTRACTOR //
//     VERSION 1.0    //
//                    //
//         BY         //
//   DARKCAMPAINGER   //
//                    //
//   REFERENCE FILE   //
//                    //
////////////////////////
//
//  > Extract mapping data and generate mapping project
//
         var/MapDataExtractor/mde = new/MapDataExtractor(whitelist, blacklist, projectName)
//
//     > Arguments:
//
//        - whitelist:   List of atom paths to include in generated map data.
//                          (children of the supplied paths ARE also included)
//                       If null, parse included map files for paths.
//                          (children of the located paths are NOT included)
//                       If list passed but first entry is null, parse included
//                          map files for paths and append to end of list.
//
//        - blacklist:   List of atom paths to exclude from generated map data.
//                          (children of the supplied paths ARE also excluded)
//                       If null, parse project preferences for 'HIDDENPATHS' paths.
//                          (children of the located paths ARE also excluded)
//                       If list passed but first entry is null, parse project
//                          preferences for hidden paths and append to end of list.
//
//        - projectName: Text to use as the name for the generated project / directory
//                          (Do *NOT* set to an existing folder, as it will be DELETED!)
//
//     > Action:
//
//          Remove any paths from whitelist that are in blacklist. Extract the visual data
//             for the remaining paths and their parent paths. In the [projectName]
//             directory: generate a DM code file containing the visual data, create a
//             DME environment file, and copy over the existing RSC file.


Update History:

Version 1.0 (July 3rd, 2012)

Initial release
Jul 3 2012, 1:01 am
This is simply amazing.
I've been searching for such a tool on BYOND for years.
It is gonna cut the world building process a ton for all developers. I hate searching for turfs/objs used in world and making a separate file for them myself, just so a map builder can be made, and I believe everybody would agree with that.
I only tested it with TOP_DOWN maps and it works flawlessly. I haven't tried with ISOMETRIC but I see no reason why it shouldn't.
Thank you!
Jul 3 2012, 1:14 am
You're welcome, I'm glad it'll be useful to you. If you run into any issues/bugs or have an idea for a feature, let me know! :)
Jul 3 2012, 9:52 am
Cloud Magic wrote:
That's really neat.

However, what if someone is afraid of their icons being stolen? You could create a flag that would essentially either downgrade the quality of each icon, or blur them up a bit, etc.

It doesn't copy the icon files.
It merely copies the generated RSC which includes the icons.
A tool called RSC extractor is used to steal the icons from RSC files. The RSC generated by Map Data Extractor and RSC downloaded when you play a game is same(I guess). Therefore your icons can still be stolen as when someone connects your game, they download the RSC along a few other resource files.

However, a feature to put watermark or something similar might be nice but nothing crucial.

Talking of features, I haven't tried the tool with special datum/atom types. In example, I have a special type for maptext, another one for zones and another for radiation. It would be awesome if those can also be preserved when using MDE.
Jul 3 2012, 12:25 pm
In response to Phoestre
Phoestre wrote:
Cloud Magic wrote:
That's really neat.

However, what if someone is afraid of their icons being stolen? You could create a flag that would essentially either downgrade the quality of each icon, or blur them up a bit, etc.

It doesn't copy the icon files.
It merely copies the generated RSC which includes the icons.
A tool called RSC extractor is used to steal the icons from RSC files. The RSC generated by Map Data Extractor and RSC downloaded when you play a game is same(I guess). Therefore your icons can still be stolen as when someone connects your game, they download the RSC along a few other resource files.

It copies the RSC file from your project to the mapping project. This means that currently it includes ALL of the icons from the game, regardless of whether or not they're needed for the map. Unfortunately, it is pitifully easy to extract icons from the project RSC files. However, the RSC files you download as a client when simply playing a game are encrypted to prevent extraction, so there's some protection there.

However, a feature to put watermark or something similar might be nice but nothing crucial.

Originally I planned to recompile the RSC with only the map-related icons included, to reduce the risk of other icons (players, attacks, HUD, ect) being included. However, the work to do so was becoming a bit silly, so I set that feature aside for the initial release. I think I've come up with a better way to go about it, by literally stripping out the extra icons from the RSC directly, but it will still include extra icon-states. Applying a blur or watermark would be an interesting option, so I may reconsider my original plan to support that (although, if you can't even trust a mapper with your icons, it's probably not worth having them on your team).

Talking of features, I haven't tried the tool with special datum/atom types. In example, I have a special type for maptext, another one for zones and another for radiation. It would be awesome if those can also be preserved when using MDE.

Strictly speaking, you can't place datums on the map, and you shouldn't be placing atoms or atom/movables. They shouldn't break the library, but their default values may not work as expected (things like layer, which differ based on the type), so you may end up with redundant code.
Jul 3 2012, 7:43 pm
In response to Phoestre
Phoestre wrote:
A tool called RSC extractor is used to steal the icons from RSC files. The RSC generated by Map Data Extractor and RSC downloaded when you play a game is same(I guess). Therefore your icons can still be stolen as when someone connects your game, they download the RSC along a few other resource files.

The RSC extractor no longer works apparently.
Jul 3 2012, 9:04 pm
In response to A.T.H.K
The RSC files you get as a client are encrypted to prevent extraction. However, the RSC files that are included in host files (and with the projects MDE generates) are not. This is an important distinction. However, ultimately someone can get to your icons if they really want to, even if they have to do it one printscreen at a time.
Jul 4 2012, 2:09 pm
Sorry for hijacking the thread but since when RSCs are encrypted? It seems like a major update I missed lol.
Jul 4 2012, 4:10 pm
In response to Phoestre
Phoestre wrote:
Sorry for hijacking the thread but since when RSCs are encrypted? It seems like a major update I missed lol.

Since the beginning of time?
Jul 4 2012, 4:34 pm (Edited on Jul 9 2012, 6:11 pm)
In response to A.T.H.K
It was added in version 468.1072. But again, only for client files, not hosts'.

<edit>
Okay guys, I don't think there's anything else to add about RSC files, so please don't derail this thread any further. If you have a serious question about them, post it in the main BYOND Discussion forums. Thanks.
Jul 27 2012, 9:44 am
I wouldn't be able to move the "MDE_MAP_PROJECT" Folder...would i?

I tried moving it outside the source and it needs to have the DMI with it. This makes it a lot less usefull since we're still forced to give our mapper the icons.
Jul 27 2012, 11:09 am
In response to Kidpaddle45
Kidpaddle45 wrote:
I wouldn't be able to move the "MDE_MAP_PROJECT" Folder...would i?

I tried moving it outside the source and it needs to have the DMI with it. This makes it a lot less usefull since we're still forced to give our mapper the icons.

Yes, you can. There shouldn't be any DMI files in the MDE_Map_Project folder, just a copy of your RSC file. The first couple of times you compile the project it will give you warnings about the icons only being found within the RSC, but those will go away. However, be aware that this method isn't full-proof, as the previous comments discussed.
Jul 27 2012, 12:27 pm
Hmmm well it seems now that icons that are in a folder inside the source aren't accepted. For example in this case,icons in my "Icons/Turfs" folder.

Screenshot: http://imageshack.us/photo/my-images/831/bug2oh.png/

Is this because i call the icon's directory when programming it? Like maybe i should've done icon='Tree.dmi' instead of icon='Icons/Turfs/Tree.dmi'?
Jul 27 2012, 6:05 pm
That's strange. I though it might be the paths, but I just tested it with an extremely long one ('Test/Test2/Test3/Test4/Test5/test.dmi') and it seems to be working fine for me.

Can you confirm that the RSC file (MDE_Map_Project.rsc) is present in your map project folder? Did you do a clean compile, or a normal compile?
Sep 1 2012, 12:01 pm
This seems like a very useful tool. I'll probably end up needing it in the near-future..

Though, I notice it doesn't work with special datums. I decided to go into the code and add parent_type to the list of saved vars, but that saves it for EVERYTHING.. XD
Just figured you should know..
Sep 1 2012, 1:38 pm
In response to RageFury33
Do you mean something like:
CustomDatum
    parent_type = /obj
    icon = 'LawnChair.dmi'


Thanks for pointing it out, it slipped my mind. I can add support for it, but it might be awhile before I get around to it.

If you want to do it yourself, you just have to add it to the list (like you did) and define it's default value in IsDefault():
        IsDefault(list/child, list/parent, attribute)

            // Special case for parent_type
            if(attribute == "parent_type")
                // Built-in parent_types
                if( (child["type"] == /obj && child["parent_type"] == /atom/movable) \
                 || (child["type"] == /mob && child["parent_type"] == /atom/movable) \
                 || (child["type"] == /area && child["parent_type"] == /atom) \
                 || (child["type"] == /turf && child["parent_type"] == /atom))
                    return 1

                // Custom parent_types
                if(GetPathParent("[child["type"]]") == "[child["parent_type"]]")
                    return 1
                else
                    return 0

            // ... rest of the process as-is


Apr 18 2013, 12:58 am
doesn't work if you have 2 turfs in the same location
Apr 18 2013, 6:43 am (Edited on Apr 18 2013, 6:51 am)
In response to Kisioj
Kisioj wrote:
doesn't work if you have 2 turfs in the same location

Hi Kisioj, can you elaborate? Do you mean that if you place one turf on top of another, the lower turf isn't extracted? I just tested it, and it seems to work properly, so if you could give me more details I would appreciate it.

One workaround would be to specifically tell MDE about that turf type so it can include it. You can both scan the map and manually specify types by passing a list with a null element first, followed by whatever types you want to manually include:
var/MapDataExtractor/mde = new/MapDataExtractor(whitelist = list(null, /turf/OtherType, /turf/AnotherType))


The included Demo.dm has some more information.
Apr 18 2013, 2:44 pm
In response to DarkCampainger
Ah, I'm sorry. Look's like I've been mistaken. I had map with one turf on top of another and it didn't work, but when I deleted one of them, it started working. I assumed MDE doesn't work when there are two turfs in the same location.
Just investigated your library again and I know circumstances when it won't work. For example try to place this mob on your demo's map:
mob
    AI
        New()
            ..()
            src.AI()

        proc
            AI()
                while(src)
                    sleep(50)

If you do this, your library won't ever finish and will just stop at line:
var/datum/D = new path(lab) //GenerateGraph(list/includeList) proc

As a result, MDE_Map_Project folder and the new .dm file won't be ever created.