Multi-dimensional list problem

ID:141588
 
Jan 14 2009, 10:54 am
Code:
var/global/Minerals= new/list(world.maxx, world.maxy, world.maxz)


Problem description:
When I declare this 3 dimensional list, the game won't start, I assume it's because the list is 300 x 300 x 14, is there some other way I can have a 3 dimensional array as big as my map? This may be a 'big list' but compared to arrays in other languages, this list is small and should be very possible. All I want to put in each one is some text, nothing fancy. A fix or another way to do this would be much appreciated.

Also, I've tried declaring it as Minerals[300][300][14], using constant integers instead of world.maxx, etc. But it didn't make a difference.
Jan 14 2009, 12:32 pm
 
var/global/Minerals= new/list(world.maxx, world.maxy, world.maxz)


You appear to have no clue what you're doing. You're declaring a normal list there, not a multi-dimensional one! There's a good article in the Dream maker's guild on them, I think with was called "[someone]'s programming tutorial III" or something like that.

Minerals[300][300][14]


Also, you cannot have over 2^16-1 lists at compile time. For example, your list would contain over 1 million lists! You can do this at runtime (like in world/New()) but I don't recommend it! Over 1000000 lists would use a lot of RAM. Try to find alternatives; what exactly are you trying to do?
Jan 14 2009, 12:51 pm
In response to Jeff8500
I just want to have a list of strings that's the same dimensions at the game's world, hence the world.maxx, etc.

I suppose I could use an extra variable in every /turf but I want to be able to save/load the whole thing
Jan 14 2009, 1:14 pm
In response to Liens
Sorry, but that first sentence was rather difficult to understand. Is English your native language?

You want to keep track of every turf in the world? If so, the best way would be to give things an extra variable, like you said. You could still save things, too (it would involve looping through all the turfs, creating lag, but it would be worth it in the long run).
Jan 14 2009, 1:21 pm
In response to Jeff8500
I'm not sure if you're trying to help me, or insult me.

I have a feeling that giving each turf an extra string variable, will be the easiest way to go about this.
Jan 14 2009, 1:28 pm
In response to Liens
Liens wrote:
I'm not sure if you're trying to help me, or insult me.

We don't do that around here. Well, most of us don't. We try to be as nice and as straight-forward as possible. Unless, obviously, the OP doesn't treat us the same way.

I have a feeling that giving each turf an extra string variable, will be the easiest way to go about this.

It is, it's a lot better than having those multidimensional lists you were trying to accomplish.
Jan 14 2009, 3:17 pm
In response to Andre-g1
It seems that a /savefile is far too slow, is there a faster way to save each text var from the turfs? I've heard about .dmp files but I don't really know what they're for.
Jan 14 2009, 3:27 pm
In response to Liens
.dmp is the old extension for map files, which are now .dmm files (when 4.0 was released, they changed the extension because .dmp is a crash dump file; the format is still the same though). It will use infinitely more CPU to read and write a map file; what are you trying to save exactly? I would recommend saving some important strings over entire turfs!

And I wasn't insulting you; your one sentence literally made no sense, and I was curious whether English was your native language. It helps to know those things, as you can usually discern whether someone is a ripper etc. by how they type.
Jan 14 2009, 3:54 pm
In response to Jeff8500
English is my native language and I believe I have fairly decent English skills.

var/savefile/F=new("Save")
var/count= 0
for(var/turf/T in world)
    count+= 1
    F["[count]"]<< T.textvar


That's my code for saving the text variable, but it's so slow.. When I ran it single player, it caused dream seeker to keep running in the background after I'd closed it, trying to save stuff. It did the first 1.5MB of the savefile fine (In a few seconds), then after dream seeker 'closed' (it was still invisibly running in the background) it started going up a few KB per 2-3 seconds while maxing out one core of my CPU.
Jan 14 2009, 9:54 pm
In response to Liens
I was totally going to suggest stretching your saving out over time and throttle it to not use up more or less than a certain amount of your cpu... However, I also noticed once the savefile passes a certain point the speed it saves at drops draaastically.

I tried just saving every turf at once in one savefile... This caused the process to slow to a halt after passing a few Z ranges of 100x100. I thought it might be too much for one savefile, so I split them into one Z range per file. The first few files are made quickly enough, then after that the progress slows to a halt, again. I thought perhaps it was because one proc was handling it, so split it up into separate sequential procs, still the same problem. Even separate verbs called after a very large (as much as 3 minutes delay) still kept slowing over time.

This is a BYOND bug IMO.

BTW: The text files were created instantly even when the savefiles took minutes to generate.

Here is an example of one of the codes I used.
"
                                        if(world.cpu<=40)
                                                CounterLimit*=1.05
                                                world<<"([CounterLimit])

"
                S.ExportText("/",TXTFile)
Jan 14 2009, 10:29 pm
In response to Liens
A few things to consider:

* Once the world is saved once, you'd only need to save the turfs that have changed. This means that initially the world takes a long time to save, and after that it should be very very quick.

* Consider spawning off saves into clumps as well, as AJX suggested.

You're going to hit a freeze-point whether you use savefiles, your own save format or MySQL (which is also supported) if you try and save this many things all at once. The solution is to either do that once and only save changes in the future, or spawn out the saving so that it occurs over time at a reasonable rate. Or both.

Note that if you do use MySQL, you're going to be saving slower initially as you generate queries to save turfs, limiting you to about 20-50 turfs per second. Which means about a 4 hour save for 1 million turfs. You load faster than with savefiles though, and queries reduce your search-time compared to savefiles to next to nothing (0.00001s in most cases for tables with, say, 100000 members).
Jan 14 2009, 10:45 pm
In response to Alathon
He can also only save 1 turf per typepath(default one with initial textstring) and then add the ones that were changed.
Jan 15 2009, 1:14 am
In response to AJX
Yeah I wonder why that happens.. I mean, it's only a few MB.. that's nothing to my comp..

You say text files don't do this? Hmm well I only want to save one variable from each turf in the world (Not the whole turf, just one text variable from it, which contains 1 word), how would I write that to a text file?
Jan 15 2009, 2:11 am (Edited on Jan 15 2009, 2:20 am)
In response to Liens
The problem is the way I was creating the text files was with ExportText which comes from a savefile. If the problem is creating the savefile in the first place then exporting the text from there is a bit silly isn't it? As far as actually using txt files themselves as a savefile, that you would have to create your own parser for... Nothing incredibly difficult, just a bit time consuming. One of the more advanced programmers probably has an example of this lying around. (I searched the HUB but nothing came up)

EDIT: Well, I was going to write up a simple parser for you but I discovered the only reason the ExportText() to a text file was instant was because the savefile creation process was already done. When it comes down to it there is actually no speed difference in writing strait to a text file with text2file() than using savefiles, and both seem to progressively slow over time even when using separate files... :S I'd wait till Lummox or Tom stops on by to say something about this, they'll be able to shed some light on why it slows down and what a better approach would be.
Jan 15 2009, 2:32 am
In response to Liens
Here's a way to stop it from lagging so much, like you say it does.

[link]
Jan 15 2009, 6:15 am
In response to AJX
It's not a bug, it just causes massive amounts of lag because BYOND has to turn all your one million turfs into text and add them to the save file. Whenever you do something 1 million times, it's bound to cause lag.
Jan 15 2009, 6:40 am
In response to Jeff8500
Actually, it was a bug and seems to be fixed now ;)
Jan 15 2009, 6:52 am
In response to Schnitzelnagler
Oh, it appears it was running even slower than usual then!
Jan 15 2009, 4:38 pm
Perhaps finding an alternate solution to whatever you're trying to do might help? It seems like there might be a better way to do this. It'd help if you gave more information on what you're trying to do, exactly, besides for saving every turf.