Binary file loading

ID:134477
 
Jul 2 2006, 8:35 pm (Edited on Jul 2 2006, 8:41 pm)
This is obviously a low priority on the wishlist, but it would be fairly easy for BYOND to load binary files into a running world (and save binary files therefrom). Loading and saving binary files is predominantly useful when you want to use BYOND to edit and create files that have been produced by other programs on your computer. Naturally, the same security access restrictions of the standard Dream Seeker apply to loading and saving files as binary -- if you can't load it as text, you won't be able to load it as binary either.

Presently, it is not possible to load a binary file in BYOND: attempting to load a binary file as text will cause the string to terminate at the first null (0x00) character found in the binary file.

However, the simplest solution I can think of would simply be to load the file as an ASCII-printed hexidecimal conversion of the file. That is, a stream of binary data would be converted into a stream of ASCII characters corresponding to the hexadecimal content of the file.

For example, an otherwise unspectacular text file containing the text:
HELLO WORLD

...would be loaded from binary as the string:
"48454c4c4f20574f524c44"


A function would also naturally exist to write a hexadecimal string in the format above to a file, byte-by-byte (or, more likely, 32-bit integer by 32-bit integer, ensuring that the proper Endianness is preserved).

Naturally, this is quite a bit less efficient than having a way to read and write files directly as binary (a 200% memory footprint for any file loaded as binary), but is a simple enough tweak that it could be sneaked into a new release of BYOND with as little as ten minutes' work, and having any ability to load and save binary files is certainly a step in the right direction!


[edit]Suggested names for the above procs:

file2hex("filename.ext")
hex2file(hexstring, "filename.ext")
[/edit]
Jul 2 2006, 9:26 pm
Jtgibson wrote:
This is obviously a low priority on the wishlist, but

But nothing! I think it would make a great addition. In fact, it was just last night that I ran into the problem of the null-terminator, and while trying to figure out where the program was bunging up, Jon88 was kind enough to explain to me this binary-reading flaw.

However, the simplest solution I can think of would simply be to load the file as an ASCII-printed hexidecimal conversion of the file. That is, a stream of binary data would be converted into a stream of ASCII characters corresponding to the hexadecimal content of the file.

Would fixing the null-terminator truly be so hard and trivial? Would it interfere with any current BYOND functions? I personally think that since this problem is known and doesn't seem too complicated, it wouldn't make for a difficult fix, but that may be due to my lack of knowledge in the BYOND internals. Aside from that rant, way to go on providing a nice hack to get around any(if applicable) internal errors that would prevent the problem to be simply fixed regularly.

Hiead says, "I like this request."

Hiead
Jul 2 2006, 9:30 pm
In response to Hiead
Would fixing the null-terminator truly be so hard and trivial? Would it interfere with any current BYOND functions? I personally think that since this problem is known and doesn't seem too complicated, it wouldn't make for a difficult fix, but that may be due to my lack of knowledge in the BYOND internals. Aside from that rant, way to go on providing a nice hack to get around any(if applicable) internal errors that would prevent the problem to be simply fixed regularly.

As far as I'm aware, BYOND strings are stored as null-terminated strings in memory, so changing that would probably be pretty non-trivial; they would have to include length information with each string, upgrading the existing strings into heavier-weight classes or structs -- this would also interfere with the existing "string recycling" that goes on in memory.

(You said "trivial", but I'm assuming you mean "non-trivial", especially when juxtaposed next to "hard". ;-))
Jul 2 2006, 9:33 pm
In response to Jtgibson
Jtgibson wrote:
(You said "trivial", but I'm assuming you mean "non-trivial", especially when juxtaposed next to "hard". ;-))

Yeah, I meant "non-trivial." Good catch, that. ;)

Gee, what a way to work in the strings. So much for my idea of a "simple fix." :o

Hiead
Jul 4 2006, 6:45 am
Anything that lets me edit something besides text and savefiles with BYOND is good, and something that I've wanted for a long time.
Jul 5 2006, 5:26 am (Edited on Jul 5 2006, 1:23 pm)
That feature would be really, really nice. Among other things, it would make possible the runtime reading of .dmi files, in turn making possible the creation of icon.PixelColor(x,y), which has been on the wish list of many for quite a while now. [edit: It's already possible, as discussed in [link] and its responses. I just hate being mistaken on the same point twice! =P]
Jul 5 2006, 12:51 pm
In response to Wizkidd0123
Wizkidd0123 wrote:
That feature would be really, really nice. Among other things, it would make possible the runtime reading of .dmi files, in turn making possible the creation of icon.PixelColor(x,y), which has been on the wish list of many for quite a while now.

Actually, in [link], when I said "In fact, it was just last night that I ran into the problem of the null-terminator," I was actually referring to my attempt to do exactly that! Small world, eh?

Hiead
Jul 5 2006, 1:18 pm
In response to Hiead
Hiead wrote:
Wizkidd0123 wrote:
That feature would be really, really nice. Among other things, it would make possible the runtime reading of .dmi files, in turn making possible the creation of icon.PixelColor(x,y), which has been on the wish list of many for quite a while now.

Actually, in [link], when I said "In fact, it was just last night that I ran into the problem of the null-terminator," I was actually referring to my attempt to do exactly that! Small world, eh?

Actually, now that I think about it, it is currently possible to write icon.PixelColor(x,y) (see [link] and the responses that follow). For some reason, I'd never gotten to making it as per the help given to me in that thread, and had since completely forgot about it.

(Despite that, I still think Jtgibson's suggestion to be a good one =P)
Jul 5 2006, 1:29 pm
In response to Wizkidd0123
Wizkidd0123 wrote:
Actually, now that I think about it, it is currently possible to write icon.PixelColor(x,y) (see [link] and the responses that follow). For some reason, I'd never gotten to making it as per the help given to me in that thread, and had since completely forgot about it.

(Despite that, I still think Jtgibson's suggestion to be a good one =P)

Wow, that's certainly an interesting thread, and a pleasant insight into some more internal action(Thanks Shadowdarke for the info, and thanks Wiz for leading me to said info!). I think I'll start looking into that right now. >=)

Edit:
Additionally, I still support Jtgibson's post as well. ;)

Hiead
Jul 6 2006, 4:52 pm
In response to Jtgibson
Jtgibson wrote:
Would fixing the null-terminator truly be so hard and trivial? Would it interfere with any current BYOND functions? I personally think that since this problem is known and doesn't seem too complicated, it wouldn't make for a difficult fix, but that may be due to my lack of knowledge in the BYOND internals. Aside from that rant, way to go on providing a nice hack to get around any(if applicable) internal errors that would prevent the problem to be simply fixed regularly.

As far as I'm aware, BYOND strings are stored as null-terminated strings in memory, so changing that would probably be pretty non-trivial; they would have to include length information with each string, upgrading the existing strings into heavier-weight classes or structs -- this would also interfere with the existing "string recycling" that goes on in memory.

It's even worse, because BYOND also treats ASCII 255 as a special character, so it would be impossible to properly handle strings even with the null terminator accounted for. A completely new string format would be needed, one which would store the string's length, full text, number of special inserts, and their positions and types.

Lummox JR