Hi!
At the moment the only ways to communicate from a DLL is through the call() proc (which is limited to strings) and by faking a world.Export() call in the DLL (which is limited by the issue #1405894.
Possible solutions:
- Allow the return value to be something other than a string. Preferably the ability to send through entire associative lists.
- Fix world.Export() some more so it no longer has any delays when receiving commands.
- Expose a "raw" API to DLL's to call BYOND (global?) procs directly, or to call a single fixed proc (/world/DllCall or something?).
Is any of this feasible in the short- to medium-term? I think I could contribute a DLL that would mitigate a lot of the engine limitations, but without a fast, reliable two-way communication I'm limited by what I can accomplish.
Thanks in advance.
ID:1881814
 Jun 25 2015, 3:45 pm
|
|||||||
| |||||||
Oh, ok. I understand. I was only wondering if there'd be some quick fix that could be applied here to make this possible.
Regarding world.Export I'm not referring to sending information, rather I'm referring to receiving it. From what I'm reading there is no delay when receiving Export calls (through world/Import), is this correct? If so this issue could be moot. :) What I'm looking into is sending information from an external application (running on the same host) to a BYOND world by manually spoofing a world.Export call (sending to 127.0.0.1) & I'm worried this won't run smoothly / be delayed somehow. | |
Receiving will be handled quickly, but I don't recall if it happens immediately.
| |
This issue mentions that call() is slow and I've been able to confirm this. Even without the loop call() can be slower.
I think this is because call() waits for a response. My request, if this is trivial and won't take up much of your time, is if it could be made possible for call to run asynchronously. That is, not wait for a response but pass the call on to the DLL and return immediately. It would solve problems for situations where you want to pass information to a DLL but don't need a response. Do you think this is feasible on the short-term? Sorry if I'm pushy with this, I'm trying to come up with the easiest solution. ;) | |
For a game that requires extensive, constant mysql queries, slow call() calls are a huge headache.
| |
In the short-term I'm really not planning to go near this code. Medium-term possibly.
| |
Hey lum, making call speedy and async, developers could do new and exciting things that wouldn't require bothering Lummox about coding, especially If there were a way to properly receive data more complex than strings to get around ***slow*** internal string conversion procs. (currently the best workaround for the latter is some hackyness involving spoofing world.export calls)
| |
As far as receiving information using world.export, setup a system that returns something immediately and then the server waits for something in world.import, when whatever program/webserver/etc is done processing it can send what it needs.
| |
world.Export still has ridiculous delays? I thought this was fixed. Guess not..
| |
Regarding call being synchronous, that is what I'd expect to be the case. If I required something I was executing within a call()d DLL to be asynchronous, I'd thread it in the DLL so whatever bulk work the DLL was doing didn't block BYOND. It seems like a fairly reasonable workaround, as they come. You then poll out for a response at your convenience.
With respect to MySQL specifically, you've still got the internal API to call on (which isn't driven by call as I understand it) via Dantom.DB. That internal API will be synchronous ... but then you'd expect it to be. If there's a specific game here that's suffering in this department, I'd be interested to see what some query profiling (and similarly executing the same queries via a more native CLI on the same machine as BYOND) yields. While not meaning to be overly critical of any BYOND devs, poorly considered schema/query structure and plain old I/O latency / DB through-put tend to be bigger culprits of poor DB response in application code more often, than framework layers. I'll have a test of the world.Export() scenario again. I do recall on the last test that you had this strange apparent 100 ms delay going on pretty much regardless of what you were exporting to (previously before a certain fix it had been 200 ms), that almost felt like BYOND was waiting until the next tick to actually action the procedure again after Export() had returned. | |
In almost every case of a return from a waiting procedure, the return proc will execute on the next tick. There may be ways to speed that up eventually and do some work "early" while it's waiting.
| |
I suspect that is probably the thing that needs investigating the medium-term then, as I don't doubt that's the root cause of the complaints on world.Export().
Does call() also exhibit this wait behaviour on the caller? Naively, I wouldn't have imagined it doing so, as unlike say world.Export(), you're not assuming some 'long' I/O that would shoot the scheduler's ability to deliver on all the other actionable procs that tick. | |
A DLL call does not sleep the calling proc; it waits for the DLL to respond.
| |
I guess you could have a call(), then have the dll programmed to immediately reply with a generic reply that gets gc'd and soft deleted to free up byond to do other things that tick, and have another processor just sitting there in the background waiting promiscuously for world.export calls?
| |
Well it depends really where the bottleneck stands, as to what you'd do. Are there some particular use-cases people are looking at here?
| |
This bug report is also directly related to the thread here: http://www.byond.com/forum/?post=1872728
In our case, we're using MySQL to manage server clusters and transfer savefiles between them. We went with MySQL because our worlds are intended to share savefiles so any player can play on any server, to avoid the common occurrence of smaller ORPGs having servers that die off player-wise, forcing players to start a new character on a different world. We're not sure how to do this in any other databases available to DM (or if it's even possible) so MySQL is essential to what we're going for here. If anyone with experience in MySQL wants to take a look at the code we're using and point out any potential flaws that could speed things up, that'd be appreciated. As it stands, a single transfer can cause a CPU spike of 30% and upwards, making our current approach one we can't really pursue until transfers are optimized. | |
I'm a bit confused, how is the MySQL usage related to call()? The MySQL integration people traditionally use is via libmysql and BYOND's direct integration with that, as opposed to call().
Regarding the SQL aspect itself, I'll comment on the linked thread. | |
if you want it to be asynchronous you can have server startup make a thread and have dll calls communicate with that thread.
making call async by itself would be messy internally, and since you can do it already... | |
A way to call a proc from the C/++ side would be all thats necessary to get around this.
| |
NullQuery wrote:
I can't see any way for this to work without adding great complexity to the DLL call setup.
The Export() thing has been an annoyance for some time. I definitely want to fix that and think it would have the greatest impact, but it's been very hard to construct a reliable test case.
Having no delays at all in world.Export() will never be a possibility; it is vital that the proc sleeps while it waits for an answer. The possibility of calling the return proc before the next tick (assuming a response arrives quickly) may be open, but that's not a simple change.
Speeding up Export() is the only thing I consider viable in the medium term, though again I need some kind of test case to work with.
Exposing something to schedule a proc call on a running server might be feasible, but boy does that sound like a lot of work for something that would only impact a few games.