BYOND and dynamic libraries

ID:151742
 
Jun 14 2009, 7:16 am
I'm curious about how BYOND works with pointers on the heap returned from dynamic libraries (DLL, SO). As return values are strings, and often you don't know the necessary size of the string buffer before the function body, it makes sense that you will often have need of heap-allocated strings. My question specifically, then, is if one needs to free the allocated memory inside the library (such as during a DLL_DETACH), or if BYOND claims ownership of the pointer and frees it when it is no longer needed. If the former, is it not likely then that you could consider using DLLs to be a "memory leak" as memory use compounds as you make successive calls to the DLL and doesn't free until program exit? Consider:
// C++ code (mostly due to the use of "new" instead of "malloc"):
extern "C" __declspec(dllexport) char *repeatString(int argc, char *argv[]) // DM-Args: string, repeat
{
    int count = atoi(argv[1]);
    long size = strlen(argv[0])*count + 1; // len*repeat + '\0'
    char *message = new char[size];
    if(message)
    {
        message[0] = '\0';
        while(count > 0)
        {
            strcat_s(message, size, argv[0]);
            --count;
        }
        return message;
    } else return "-1";
}

// DM code:
mob/verb/Repeat_String()
    var/s = input(src, "What string?") as null|text
    if(s)
        var/i = input(src, "How many repeats?") as num
        src << call("some.dll", "repeatString")(s, num2text(i))


Now, of course I could create a single-sized (n) buffer on the stack, compare the sizes of the strings and either cap off the string at n-1 characters, or report an error if it exceeds that size, but that makes the function far less flexible in the long run.

I'm just curious because, although this repeatString example isn't indicative of what I'm doing, it demonstrates the point with similar effectiveness to the program at hand I'm working on.
Jun 14 2009, 7:24 am
I'd made the reasonable assumption that BYOND takes ownership of the memory I've allocated and returned via that pointer, on the grounds the string has presumably become part of it's memory management framework. It seems as though libraries have not reasonable way to determine when it's safe to free that memory.
Jun 14 2009, 11:27 am
In response to Stephen001
Stephen001 wrote:
I'd made the reasonable assumption that BYOND takes ownership of the memory I've allocated and returned via that pointer, on the grounds the string has presumably become part of it's memory management framework. It seems as though libraries have not reasonable way to determine when it's safe to free that memory.

To further thought on this assumption, how would BYOND differentiate between a string returned that was heap-allocated and one that was stack-allocated. For example, comparing the snippet in my previous post to the snippet in Tom's [link], which was:
#include <string.h>
// This is an example of an exported function.
// windows requires __declspec(dllexport) to be used to 
// declare public symbols
extern "C" __declspec(dllexport) char *merge(int n, char *v[])
{
    static char buf[100]; 
    *buf=0;
    for(int i=0;i<n;i++) {
        strcat(buf,v[i]);
    }
    return buf;
}


I'm curious how BYOND might be able to differentiate between the buff pointer from Tom's code and the message pointer from mine and know which one is safe to free. Maybe you could free it like so:
void deletePtr(char **ptr) // call deletePtr(&myPtr)
{
   try
   {
      delete[] (*ptr);
      *ptr = 0;
   } catch(...)
   {
   }
}

But I'm not entirely sure if you can catch an exception in this way to achieve the desired result.
Jun 14 2009, 11:48 am
In response to Stephen001
Stephen001 wrote:
I'd made the reasonable assumption that BYOND takes ownership of the memory I've allocated and returned via that pointer

Well after a bit of testing, I've decided that there is indeed a leak here. Using the C++-exported function from the original post, and the following DM proc with the string "My country 'tis of the. " at 50 repeats:
mob/verb/Repeat()
    var/s = input(src, "What string?") as null|text
    if(s)
        var/i = input(src, "How many repeats?") as num
        src << call("repeat.dll", "repeatString")(s, num2text(i))
        for(var/j in 1 to 10000)
            call("repeat.dll", "repeatString")(s, num2text(i))

Each successive run of the verb increases the memory footprint of dreamseeker.exe by about 13kb (per Task Manager; I think my actual increase is 12,768). That's no good!
Jun 14 2009, 1:21 pm
It's not a bug. A developer's DLL is simply supposed to do its own cleaning up of anything it creates or allocates.
Jun 14 2009, 3:24 pm
In response to Kaioken
Kaioken wrote:
It's not a bug. A developer's DLL is simply supposed to do its own cleaning up of anything it creates or allocates.

It's not a bug indeed, but probably undesirable behavior. The DLL stays attached until program exit. That means that there's no decent way (as far as I'm aware) to free the memory in the meantime. Whether you free it on DLL_DETACH or not, the memory you allocate will stay in use for the program's lifespan, which means that successive calls to functions that allocate memory on the heap will compound the memory usage. The only workaround seems to be to never allocate anything on the heap, which is pretty limiting in terms of functionality and flexibility.
Jun 14 2009, 3:26 pm
In response to Kuraudo
As stated before, BYOND's DLL support isn't really meant to provide you with a large array of functionality. It's meant for BASIC things that don't do a lot of processing.
Jun 14 2009, 3:32 pm
In response to Nadrew
I'm pretty much done with a C++-based regular expression library which allocates its return value (essentially a list2params-esque return, to be parsed via params2list post-call) on the heap. It would be nice if that memory were freed after use.