Dynamic access to files in RSC/cache at runtime

Page: 1 2
ID:121083
 
Nov 16 2011, 8:28 pm
Applies to:DM Language
Status: Open
Issue hasn't been assigned a status value.
This has been semi-suggested several times before (see "Access resource files dynamically" here and the second paragraph of Lummox JR's second comment here), but there doesn't appear to be a formal request.

It would be helpful if files in the cache could be accessed at runtime by filename. This would allow for storing file names in savefiles, instead of saving the entire file or hard-coding an associative-list of file cache references ("turfs.dmi"='turfs.dmi', "dragon.dmi"='dragon.dmi', ect).

For my purposes, I'm working on a map saver/loader, and would like to be able save any non-initial icon values without saving the entire icon or forcing the user to put together and maintain the aforementioned cache reference list. It would also allow for developers to write a clean process to save overlay data without flattening and saving the icon.

It has been suggested that file() be given a second parameter to specify if it's from the cache, but I think file_rsc(filename) would also be appropriate (and in-line with fcopy/fcopy_rsc).

Simple example use-case:
mob
    icon='mob.dmi'
    verb
        SaveIcon()
            var/savefile/S = new/savefile("test.sav")
            S<<"[icon]"
        LoadIcon()
            var/savefile/S = new/savefile("test.sav")
            var/filename
            var/icon/I
            S>>filename
            I = file_rsc(filename)
            if(I)
                src.icon = I
            else
                src<<"File '[filename]' no longer in cache!"


In the above example, I assume file_rsc() would return null if the file could not be found in the cache. I think this makes more sense then throwing an error, because files disappearing from the cache may be a common occurrence (ie for old savefiles), and there's no real way for a developer to avoid the error.
Aug 16 2012, 7:01 pm
I'd absolutely love to see this feature since it could be great for games that use a lot of user-uploaded content.
Aug 16 2012, 7:03 pm
+1 this should of been available a long long time ago IMO ...
Sep 7 2013, 4:19 am (Edited on Sep 7 2013, 9:05 am)
This was merged in from a separate thread

My suggestion is fairly simple; add a function, say cfile() ("c[ache]file"), that when given a filename such as "icon.dmi", returns a cache reference if a file in the cache has the same name.

An example is that the two code segments below would be functionally identical:

MyOtherObject.icon = 'black.dmi' // Ensure black.dmi is in the .rsc
MyObject.icon = 'black.dmi'


MyOtherObject.icon = 'black.dmi' // Ensure black.dmi is in the .rsc
MyObject.icon = cfile("black.dmi")


in both cases, MyObject.icon == MyOtherObject.icon would be true.

If the file was not in cache, then this would react the same way as file() does when passed a file that does not exist.
Sep 7 2013, 5:37 am
In response to Topkasa
Doesn't fcopy_rsc() already do this?


mob
    verb
        cacheIconTest()
            var/obj/obj1 = new
            obj1.icon = 'bleh.dmi'
            
            world << "\ref[obj1.icon]"
            
            var/obj/obj2 = new
            obj2.icon = fcopy_rsc("bleh.dmi")
            world << "\ref[obj2.icon]"
Sep 7 2013, 6:07 am
In response to Topkasa
fcopy_rsc() requires that the original, non-cached file exists. It pulls that file into the .rsc except if it's already in.

I need different functionality; I need to get a reference to a file in the .rsc *without* needing the original file to be present.
Sep 7 2013, 6:22 am
In response to Topkasa
Can you provide a use-case?

alternatively, you can do something like this:

        cacheIconTest2()
            for(var/x in savedIcons)
                
                savedIcons["[x]"] = "\ref[x]"
                savedIcons -= x
            for(var/x in savedIcons)
                world << "[x] = [savedIcons[x]]"
        
//global or datum definition
var/savedIcons = list('bleh.dmi', 'icons.dmi', 'red.dmi')
Sep 7 2013, 6:34 am
In response to Topkasa
That is a workaround, and I am using it, but I would rather see a 'proper' solution to this.

My use-case is a map loader that parses a DMM-compatible spec into map data for dynamic loading. I need to be able to support things like the 'Black.dmi' in this line:

"a" = (/turf/Floor/Generic/Basic{icon = 'Project/Icons/UI/Black.dmi'; icon_state = "Black"},/area)


Keeping a massive array of Filename-CacheRef entries is doable, but clunky compared to a proper cache-file proc.
Nov 16 2015, 5:10 am
Currently, if we want to reference a file compiled in with the rsc (cache files), we have to use the single-quote syntax like so

var/contents = file2text('files/aFile.txt')

However, variables are not allowed within this path, meaning dynamic fetching of cached resources is impossible. My suggestion is that the file() proc be extended with an additional argument to specify "where" it gets the file from, like so

var/contents = file2text(file("files/aFile[someVar].txt", 1))

Where the second argument tells the proc to fetch from the resource cache rather than external files only. This preserves backwards compatibility (a setting of 0 or null or simply not set defaults to existing file() functionality) while keeping things "clean" by not adding another proc or whatever else.
Nov 16 2015, 5:11 am
In response to Wirewraith
I approve, there's little reason why this couldn't be done IMO.
Nov 16 2015, 8:53 am (Edited on Nov 16 2015, 10:46 am)
In response to Wirewraith
Just make an associative list with the name of the file = the file. Then you can get files dynamically.

I doubt this is feasible otherwise because the compiler doesn't know what files to include unless you have them in single quotes somewhere in the code.

Plus, I'm sure this has been requested and rejected before.
Nov 16 2015, 8:58 am
In response to Wirewraith
In the Unity game engine, you can create a folder called "Resources" and every resource in there can be loaded at runtime by name.

http://docs.unity3d.com/ScriptReference/Resources.html

It's essentially the same as what I said, but the associative list is generated by the engine so you don't have to type it out yourself.
Nov 16 2015, 9:03 am
In response to Wirewraith
Making an associative list of files is a terrible idea and defeats the purpose of this request. You can copy files into the resource cache. Having a resource folder like that is also pointless because you can access the filesystem outside of the resource files. Just because the compiler may not add said file to the resource pack doesn't make this request unfeasible because the server knows about all the files in the resource pack at all times.
Nov 16 2015, 10:44 am
In response to Somepotato
Somepotato wrote:
Having a resource folder like that is also pointless because you can access the filesystem outside of the resource files.
You can't access files that aren't included in the .rsc and aren't in the same folder as the host files, unless they're uploaded at run-time, in which case you would already have a reference to them.

Just because the compiler may not add said file to the resource pack doesn't make this request unfeasible because the server knows about all the files in the resource pack at all times.

When you said this
if we want to reference a file compiled in with the rsc (cache files), we have to use the single-quote syntax

I assumed that if you had the option to use variable file paths, you wouldn't be including the file into the rsc because you wouldn't be using the single-quotes anymore. If you don't use the single quotes to refer to a file, then it's not included in the .rsc when you compile. Because of this, I said that since you must have the file in single quotes somewhere in your code, you might as well put them in an associative list, which does support dynamic access.
// this list is defined at compile-time
var resources[] = list(
    "files/aFile123.txt" = 'files/aFile123.txt'
)

// e.g.
var someVar = 123
return resources["files/aFile[someVar].txt"] // this gets you 'files/aFile123.txt'
Nov 16 2015, 10:46 am
In response to Somepotato
Somepotato wrote:
because the server knows about all the files in the resource pack at all times.

This is only true for files explicitly referenced in single quotes. If you remove a reference to a file in your program, but it still exists in the RSC, the server has no clue it's there.
Nov 16 2015, 11:04 am
In response to Super Saiyan X
Super Saiyan X wrote:
Somepotato wrote:
because the server knows about all the files in the resource pack at all times.

This is only true for files explicitly referenced in single quotes. If you remove a reference to a file in your program, but it still exists in the RSC, the server has no clue it's there.
This is simply untrue
--anyway--
I can see the appeal of the associative lists, however this function would go hand in hand if flist would work with resource packs. The main concern with this request I think is that you can put things into a resource pack but its one way- you can't access them back.
Nov 16 2015, 11:08 am
In response to Somepotato
Somepotato wrote:
Super Saiyan X wrote:
Somepotato wrote:
because the server knows about all the files in the resource pack at all times.

This is only true for files explicitly referenced in single quotes. If you remove a reference to a file in your program, but it still exists in the RSC, the server has no clue it's there.
This is simply untrue
--anyway--
I can see the appeal of the associative lists, however this function would go hand in hand if flist would work with resource packs. The main concern with this request I think is that you can put things into a resource pack but its one way- you can't access them back.

Are you sure it's untrue? The server would index every file in the RSC, rather than just the ones it needs?

You can access anything in the rsc, btw, just not by file name. As long as it has an explicit reference in the code at least once.

Nov 16 2015, 11:09 am
In response to Super Saiyan X
Super Saiyan X wrote:
Somepotato wrote:
Super Saiyan X wrote:
Somepotato wrote:
because the server knows about all the files in the resource pack at all times.

This is only true for files explicitly referenced in single quotes. If you remove a reference to a file in your program, but it still exists in the RSC, the server has no clue it's there.
This is simply untrue
--anyway--
I can see the appeal of the associative lists, however this function would go hand in hand if flist would work with resource packs. The main concern with this request I think is that you can put things into a resource pack but its one way- you can't access them back.

Are you sure it's untrue? The server would index every file in the RSC, rather than just the ones it needs?

You can access anything in the rsc, btw, just not by file name. As long as it has an explicit reference in the code at least once.

Yeah, just because of how the rsc format works its more efficient that way unless it changed recently
Nov 16 2015, 12:55 pm
In response to Somepotato
Somepotato wrote:
Yeah, just because of how the rsc format works its more efficient that way unless it changed recently

On the other hand, last I checked the DMB itself does maintain a list of CRC + types that it actually references from the resource pack, so either way is plausible.
Nov 16 2015, 1:15 pm
In response to Wirewraith
Here is the thing, the server creates a dynamic rsc at run time,

So what if i want to load files into that rsc, so that clients get them on connection, but i don't know at compile time what they are because it involves indexing a folder based on a config or a rolled game mode type or something like that, where it changes round by round.

Or i just want my stuff to be more dynamic in general and not require pre-defining all of the paths.

In general some better control of rsc vs local file system wouldn't hurt.
Page: 1 2