ReplaceAll_With_() vs dd_replacetext()

Page: 1 2
ID:151831
 
Jan 16 2009, 8:03 am
My own replace text function is 10 times slower than deadron's:
//mine
proc/ReplaceAll_With_(var/txt="",var/target="",var/replacement="")
    var/endOftxt = lentext(txt)
    var/endOftarget = lentext(target)
    var/i = endOftxt - endOftarget
    while(i>1)
        if(copytext(txt,i, i+endOftarget)==target)
            txt = copytext(txt,1,i) + replacement + copytext(txt,i+endOftarget,0)
        i-- 
    return txt



//deadron's

dd_replaceText(text, search_string, replacement_string)
        var/list/textList = dd_text2List(text, search_string)
        return dd_list2text(textList, replacement_string)


which calls...

dd_text2List(text, separator)
        var/textlength      = lentext(text)
        var/separatorlength = lentext(separator)
        var/list/textList   = new /list()
        var/searchPosition  = 1
        var/findPosition    = 1
        var/buggyText
        while (1)                                                            // Loop forever.
            findPosition = findText(text, separator, searchPosition, 0)
            buggyText = copytext(text, searchPosition, findPosition)        // Everything from searchPosition to findPosition goes into a list element.
            textList += "[buggyText]"                                       // Working around weird problem where "text" != "text" after this copytext().

            searchPosition = findPosition + separatorlength                 // Skip over separator.
            if (findPosition == 0)                                           // Didn't find anything at end of string so stop here.
                return textList
            else
                if (searchPosition > textlength)                            // Found separator at very end of string.
                    textList += ""                                          // So add empty element.
                    return textList


Then deadron's function calls...

dd_list2text(list/the_list, separator)
        var/total = the_list.len
        if (total == 0)                                                      // (deadron's comment) Nothing to work with.
            return

        var/newText = "[the_list[1]]"                                        // (deadron's comment) Treats any object/number as text also.
        var/count
        for (count = 2, count <= total, count++)
            if (separator) newText += separator
            newText += "[the_list[count]]"
        return newText


I'm really baffled. My function is tiny! How is it so much slower? Is the minus operation really that slow? Or is it because I'm using copytext()?

Note to thread moderator: I placed this in design philosophy because it seemed like a design question (e.g. I don't have a problem with my code because it works, I just wanted to make it better).
Jan 16 2009, 8:22 am
Why not something a lot simpler but with exactly the same concept, I have no speed problems with this though:

proc
        Hw_Replace_Text(string,look_for,replace)
            var/a = findtext(string,look_for)
            while(a)
                string = copytext(string,1,a)+ replace + copytext(string,a+length(look_for))
                a = findtext(string,look_for,a+length(replace))
            return string
Jan 16 2009, 8:56 am
It's likely because you're making heavy use of copytext(). You're using it to match the text you're looking for and find it that way (when you should really be using findtext() instead), then to do the actual replacing. Deadron only uses copytext() for the latter.
Jan 16 2009, 9:03 am
In response to Kaioken
Kaioken wrote:
It's likely because you're making heavy use of copytext(). You're using it to match the text you're looking for and find it that way (when you should really be using findtext() instead), then to do the actual replacing. Deadron only uses copytext() for the latter.

Ahhh, I see.
Jan 16 2009, 5:03 pm
I've taken to using call() and external libraries for my string handling, it's much faster and allows for more goodies with less work.

For a Windows DLL you can use this:
#include <string>
using namespace std;


extern "C" __declspec(dllexport) char *str_replace(int argc, char* argv[]){
    if(argc != 3) return 0;    // This just kills the function if not enough arguments were provided.
    // The following three lines will set three string objects to the values of the three function arguments
    string str_string = argv[0];
    string str_needle = argv[1];
    string str_replace = argv[2];
    // A string object is just like an obj, it contains various functions and variables to ease things up.
    // Here we'll check of any of the arguments were empty strings, and if so we'll return 0 (false)
    if(str_string.empty()||str_needle.empty()) return 0;
    int pos = 0; // This is the current position of the search
    while(str_string.find(str_needle,pos)!=string::npos){ // And this is a loop ;)
        int n_pos = str_string.find(str_needle,pos);
        str_string.replace(n_pos,str_needle.size(),str_replace); // We call STL's replace() function and let it work for us.
        pos = n_pos + str_replace.size(); // And we set the position to passed the last one so we don't get infinite here.
        if(pos > str_string.length()) pos = str_string.length(); // Make sure it doesn't go TOO high.
    }
    return strdup(str_string.c_str()); // And we return the nice memory-managed version of our final string.
}

extern "C" __declspec(dllexport) char *str_insert(int argc,char* argv[]){
    if(argc != 3) return 0;
    string str = argv[0];
    string insert = argv[1];
    int pos = atoi(argv[2]); // Sets the pos variable to an int (number) and converts the third argument from a string to a number.
    //atoi = all to int
    if(str.empty()||insert.empty()||pos<0) return 0;
    if(pos > str.length()) pos = str.length(); 
    if(pos < 0) pos = 0;
    str.insert(pos,insert); // See, STL rocks, no loops or anything.
    return strdup(str.c_str());
}


For a Linux SO file you can use this:
#include <cstdio>
#include <string>
using namespace std;

extern "C" {
        char *str_insert(int argc,char *argv[]);
        char *str_replace(int argc,char *argv[]);
};

char *str_insert(int argc,char *argv[]){
    if(argc != 3) return 0;
    string str = argv[0];
    string insert = argv[1];
    int pos = atoi(argv[2]); // Sets the pos variable to an int (number) and converts the third argument from a string to a number.
    //atoi = all/anything to int
    if(str.empty()||insert.empty()||pos<0) return 0;
    if(pos > str.length()) pos = str.length(); 
    if(pos < 0) pos = 0;
    str.insert(pos,insert); // See, STL rocks, no loops or anything.
    return strdup(str.c_str());
}

char *str_replace(int argc,char *argv[]){
    if(argc != 3) return 0;    // This just kills the function if not enough arguments were provided.
    // The following three lines will set three string objects to the values of the three function arguments
    string str_string = argv[0];
    string str_needle = argv[1];
    string str_replace = argv[2];
    // A string object is just like an obj, it contains various functions and variables to ease things up.
    // Here we'll check of any of the arguments were empty strings, and if so we'll return 0 (false)
    if(str_string.empty()||str_needle.empty()) return 0;
    int pos = 0; // This is the current position of the search
    while(str_string.find(str_needle,pos)!=string::npos){ // And this is a loop ;)
        int n_pos = str_string.find(str_needle,pos);
        str_string.replace(n_pos,str_needle.size(),str_replace); // We call STL's replace() function and let it work for us.
        pos = n_pos + str_replace.size(); // And we set the position to passed the last one so we don't get infinite here.
        if(pos > str_string.length()) pos = str_string.length(); // Make sure it doesn't go TOO high.
    }
    return strdup(str_string.c_str()); // And we return the nice memory-managed version of our final string.
}
Jan 16 2009, 6:59 pm
In response to Nadrew
That would be a great way to do it, but I'm stuck using non-dll functions because dlls can't be accessed by anyone except the host.
Jan 16 2009, 7:24 pm
In response to Rockinawsome
Eh... in that sense all code can be accessed only by the host, so a DM proc is no different...? Seeing as all processing is done on the server, which manages the program and its strings and all, you don't need something like that to run client-side. Unless you meant something different.
Jan 16 2009, 8:10 pm
In response to Kaioken
Kaioken wrote:
Eh... in that sense all code can be accessed only by the host, so a DM proc is no different...? Seeing as all processing is done on the server, which manages the program and its strings and all, you don't need something like that to run client-side. Unless you meant something different.

I know what you're saying, but the dlls can't be accessed by any client. I would say to try my xbox 360 controller lib for an example, but I charge 5 bucks for it. But I think there is a binary file manipulator lib out there that someone wrote that uses a dll if you want to see what I'm attempting to say.
Jan 16 2009, 8:15 pm
In response to Rockinawsome
In this case it doesn't really matter, you're not actually trying to do anything with a client, you're manipulating a string that's server-side. I've used this method plenty myself, works pretty good. Just create a function that does the call() for you and returns the value, you can then use it much like dd_replacetext().

my_string = str_replace("hello","howdy")


During testing results were about four to eight times quicker than dd_replacetext() especially on larger strings.
Jan 16 2009, 8:45 pm
In response to Nadrew
I'll have to check out your technique then when I have more sleep. Thanks for the info.

I'm having a problem mentally picturing how the two dlls are used so differently though if the client initiates the request for the replacetext function.

To make up an example:

client/verb/Some_Verb()
   src.Xbox360Controller.Initialize()


versus

client/verb/SomeVerb()
   ReplaceText()


In this example, is it because ReplaceText doesn't belong to client?

Jan 16 2009, 8:51 pm
In response to Rockinawsome
There is no client-side processing on BYOND, except for a few minor graphical things (I think). It doesn't matter if the client initiates the request; the server does the processing.
Jan 16 2009, 8:58 pm
In response to Popisfizzy
O I see my mistake. The controller functions belong solely to the host, where the host controllers reside. This dll runs into no problems though because nothing needs to be controlled.

Excepting a possible problem because I suspect that it would kick up a prompt asking if the dll can be used by byond in safe mode if asked for by the client, which could only be confirmed manually by the hoster.
Jan 17 2009, 5:02 am
In response to Rockinawsome
Uhm... seems like you're still kinda missing it. There isn't anything like a connection or dependency of any running code on any players. All code always runs server-side, no matter what, and players don't affect it; there's no difference between execution that was initiated by a New() proc or by a verb, other than that usr will be set when a verb is ran.
IIRC currently you'll always get permission popups when DM is about to call a DLL, unless the game is ran in trusted mode.
Jan 17 2009, 10:23 am
In response to Rockinawsome
The thing about your controller setup is that it's grabbing data directly from the client (gamepad input), input that BYOND never normally gets so it relies on being initiated on the client machine, which BYOND doesn't support (for obvious reasons).

The string handler stuff is working on data the client has already passed to the server, it's operating solely on the server side of things. This isn't actually operating on anything client-side at all, the function is changing stuff stored on the server (the strings).


So the moral: Your controller thing relies on actual client-side data which requires the DLL to be running from the client machine. The string handling setup only deals with stuff on the server, so only the host needs the DLL.


Also, a good way of dealing with the security prompts is loading the DLL files when the world starts, then the host can deal with them once when the world starts and not have to worry about it. External libraries like DLL and SO files are kept loaded after their first reference, making subsequent calls quicker and not requiring of permission.
Jan 17 2009, 10:56 am
In response to Nadrew
Nadrew wrote:
The thing about your controller setup is that it's grabbing data directly from the client (gamepad input), input that BYOND never normally gets so it relies on being initiated on the client machine, which BYOND doesn't support (for obvious reasons).


The string handler stuff is working on data the client has already passed to the server, it's operating solely on the server side of things. This isn't actually operating on anything client-side at all, the function is changing stuff stored on the server (the strings).

Putting it this way helps me form a better mental image of what's going on. Thanks.

So the moral: Your controller thing relies on actual client-side data which requires the DLL to be running from the client machine. The string handling setup only deals with stuff on the server, so only the host needs the DLL.

Yes. Exactly. I was actually trying to get at this; this is what I understood after your's and popisfizzy's posts.
Also, a good way of dealing with the security prompts is loading the DLL files when the world starts, then the host can deal with them once when the world starts and not have to worry about it. External libraries like DLL and SO files are kept loaded after their first reference, making subsequent calls quicker and not requiring of permission.

This is a good point, however, what if I created an installer with something like NSIS (like installshield, but free), and had the dll loaded into BYOND's directory. I remember reading something about that. Doesn't that keep the dlls from needing a confirm on the popup?
Jan 17 2009, 10:58 am
In response to Rockinawsome
I believe a security warning will always appear unless the world is running in trusted mode.
Jan 17 2009, 11:26 am
In response to Nadrew
I don't suppose there's any talk of a trusted file list or a script or option to launch certain worlds with trusted mode enabled by default?
Jan 17 2009, 11:37 am
In response to Rockinawsome
There was some discussion of features like a DLL file whitelist some time ago. I suppose it's on The List. As for launching certain worlds with trusted mode by default, perhaps you should expand on that idea and see if it wouldn't completely defeat the purpose of the safety modes.
Jan 17 2009, 11:44 am
In response to Kaioken
Kaioken wrote:
There was some discussion of features like a DLL file whitelist some time ago. I suppose it's on The List. As for launching certain worlds with trusted mode by default, perhaps you should expand on that idea and see if it wouldn't completely defeat the purpose of the safety modes.

Well, what I meant was that if the person who downloaded and hosted the game could make trusted mode a permanent option, not for the programmer to have such rights.
Jan 17 2009, 12:07 pm
In response to Rockinawsome
Like a setting a user can enable client-side to always run that game on trusted mode? Yeah, I see what you mean. I dunno if it'd be a considerable benefit though, since games don't host themselves automatically and a user needs to do some actions to start that, so he might as well set the mode along the way.
You may already be able to do something similar, if Dream Daemon can take a command-line parameter for the mode to run the game at, you could set up a shortcut for running a trusted server of a given game. If not, startup() should be able to do this, so you could cook something with it, though it'd be kinda roundabout.
Page: 1 2