GNUstep and valgrind

classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

GNUstep and valgrind

amon
Perhaps someone can tell me if my guess on the following is
correct. After an afternoon of trying to figure out where the
large number of leaks reported by valgrind were coming from, I
went all the way back to basics. What it seems to show is that
the GNUStep run time allocates a bunch of stuff at start up and
never clears it. Here are the two test cases. The first is an
empty main(), which one would expect should come up clean, and
it does:

        /* Build with
            gcc -I /usr/include/GNUstep test2.m -lm -lobjc \
                        -lgnustep-base -o test2
          [DMA 20180220]
        */
        #import <Foundation/Foundation.h>
        main () {
        }
        /* This claims there are leaks in the above program:
            valgrind --leak-check=yes --show-leak-kinds=definite ./test2
          [DMA 20180313]
        */

        ~/Work $ valgrind --leak-check=yes --show-leak-kinds=definite ./test2
        ==5211== Memcheck, a memory error detector
        ==5211== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et
al.
        ==5211== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright
info
        ==5211== Command: ./test2
        ==5211==
        ==5211==
        ==5211== HEAP SUMMARY:
        ==5211==     in use at exit: 0 bytes in 0 blocks
        ==5211==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
        ==5211==
        ==5211== All heap blocks were freed -- no leaks are possible
        ==5211==
        ==5211== For counts of detected and suppressed errors, rerun with: -v
        ==5211== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

The second does nothing but create a pool and release it, and
holy mackerel does it ever leak:

        /* Build with
            gcc -I /usr/include/GNUstep test2.m -lm -lobjc \
                        -lgnustep-base -o test2
          [DMA 20180220]
        */
        #import <Foundation/Foundation.h>
        main () {
          NSAutoreleasePool *pool;
          pool = [[NSAutoreleasePool alloc] init];
          [pool release];
        }
        /* This claims there are leaks in the above program:
            valgrind --leak-check=yes --show-leak-kinds=definite ./test2
          [DMA 20180313]
        */

I only included the very tail end of a huge volume of data.

        ==5232== 2,720 (48 direct, 2,672 indirect) bytes in 3 blocks are
definitely lost in loss record 452 of 471
        ==5232==    at 0x4C2AB80: malloc (vg_replace_malloc.c:296)
        ==5232==    by 0x4E439B8: objc_malloc (in
/usr/lib/x86_64-linux-gnu/libobjc.so.4.0.0)
        ==5232==    by 0x4E426B4: ??? (in
/usr/lib/x86_64-linux-gnu/libobjc.so.4.0.0)
        ==5232==    by 0x4E428CB: ??? (in
/usr/lib/x86_64-linux-gnu/libobjc.so.4.0.0)
        ==5232==    by 0x4E43050: __objc_exec_class (in
/usr/lib/x86_64-linux-gnu/libobjc.so.4.0.0)
        ==5232==    by 0x40101D9: call_init.part.0 (dl-init.c:78)
        ==5232==    by 0x40102C2: call_init (dl-init.c:36)
        ==5232==    by 0x40102C2: _dl_init (dl-init.c:126)
        ==5232==    by 0x4001299: ??? (in /lib/x86_64-linux-gnu/ld-2.19.so)
        ==5232==
        ==5232== LEAK SUMMARY:
        ==5232==    definitely lost: 1,842 bytes in 110 blocks
        ==5232==    indirectly lost: 4,704 bytes in 294 blocks
        ==5232==      possibly lost: 4,646 bytes in 132 blocks
        ==5232==    still reachable: 431,594 bytes in 7,837 blocks
        ==5232==         suppressed: 0 bytes in 0 blocks
        ==5232== Reachable blocks (those to which a pointer was found) are not
shown.
        ==5232== To see them, rerun with: --leak-check=full
--show-leak-kinds=all
        ==5232==
        ==5232== For counts of detected and suppressed errors, rerun with: -v
        ==5232== ERROR SUMMARY: 149 errors from 149 contexts (suppressed: 0
from 0)

Is this indeed the GNUstep runtime doing this or have I
not set up or not interpreted valgrid correctly?
--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Ivan Vučica
I think this came up a few years ago with resolution of "it's not a leak unless it grows in ram usage" -- it's fine (and faster) if some resources are cleaned up by the OS at process termination time. Think singletons and such.

If you observe RAM use growing and leaking beyond this, it's a problem. I think it might be possible to mark certain allocations as safe, not sure how.

Valgrind can tell you where memory was allocated, too. Is it giving you useful information in this case?

I realize you have safety concerns. Nonetheless, it's probably better to have working code and observe if "leaks" are leaks, i.e. growing over time.

Sent from my iPhone

> On 13 Mar 2018, at 23:53, amon <[hidden email]> wrote:
>
> Perhaps someone can tell me if my guess on the following is
> correct. After an afternoon of trying to figure out where the
> large number of leaks reported by valgrind were coming from, I
> went all the way back to basics. What it seems to show is that
> the GNUStep run time allocates a bunch of stuff at start up and
> never clears it. Here are the two test cases. The first is an
> empty main(), which one would expect should come up clean, and
> it does:
>
>    /* Build with
>        gcc -I /usr/include/GNUstep test2.m -lm -lobjc \
>            -lgnustep-base -o test2
>      [DMA 20180220]
>    */
>    #import <Foundation/Foundation.h>
>    main () {
>    }
>    /* This claims there are leaks in the above program:
>        valgrind --leak-check=yes --show-leak-kinds=definite ./test2
>      [DMA 20180313]
>    */
>
>    ~/Work $ valgrind --leak-check=yes --show-leak-kinds=definite ./test2
>    ==5211== Memcheck, a memory error detector
>    ==5211== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
>    ==5211== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
>    ==5211== Command: ./test2
>    ==5211==
>    ==5211==
>    ==5211== HEAP SUMMARY:
>    ==5211==     in use at exit: 0 bytes in 0 blocks
>    ==5211==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
>    ==5211==
>    ==5211== All heap blocks were freed -- no leaks are possible
>    ==5211==
>    ==5211== For counts of detected and suppressed errors, rerun with: -v
>    ==5211== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
>
> The second does nothing but create a pool and release it, and
> holy mackerel does it ever leak:
>
>    /* Build with
>        gcc -I /usr/include/GNUstep test2.m -lm -lobjc \
>            -lgnustep-base -o test2
>      [DMA 20180220]
>    */
>    #import <Foundation/Foundation.h>
>    main () {
>      NSAutoreleasePool *pool;
>      pool = [[NSAutoreleasePool alloc] init];
>      [pool release];
>    }
>    /* This claims there are leaks in the above program:
>        valgrind --leak-check=yes --show-leak-kinds=definite ./test2
>      [DMA 20180313]
>    */
>
> I only included the very tail end of a huge volume of data.
>
>    ==5232== 2,720 (48 direct, 2,672 indirect) bytes in 3 blocks are definitely lost in loss record 452 of 471
>    ==5232==    at 0x4C2AB80: malloc (vg_replace_malloc.c:296)
>    ==5232==    by 0x4E439B8: objc_malloc (in /usr/lib/x86_64-linux-gnu/libobjc.so.4.0.0)
>    ==5232==    by 0x4E426B4: ??? (in /usr/lib/x86_64-linux-gnu/libobjc.so.4.0.0)
>    ==5232==    by 0x4E428CB: ??? (in /usr/lib/x86_64-linux-gnu/libobjc.so.4.0.0)
>    ==5232==    by 0x4E43050: __objc_exec_class (in /usr/lib/x86_64-linux-gnu/libobjc.so.4.0.0)
>    ==5232==    by 0x40101D9: call_init.part.0 (dl-init.c:78)
>    ==5232==    by 0x40102C2: call_init (dl-init.c:36)
>    ==5232==    by 0x40102C2: _dl_init (dl-init.c:126)
>    ==5232==    by 0x4001299: ??? (in /lib/x86_64-linux-gnu/ld-2.19.so)
>    ==5232==
>    ==5232== LEAK SUMMARY:
>    ==5232==    definitely lost: 1,842 bytes in 110 blocks
>    ==5232==    indirectly lost: 4,704 bytes in 294 blocks
>    ==5232==      possibly lost: 4,646 bytes in 132 blocks
>    ==5232==    still reachable: 431,594 bytes in 7,837 blocks
>    ==5232==         suppressed: 0 bytes in 0 blocks
>    ==5232== Reachable blocks (those to which a pointer was found) are not shown.
>    ==5232== To see them, rerun with: --leak-check=full --show-leak-kinds=all
>    ==5232==
>    ==5232== For counts of detected and suppressed errors, rerun with: -v
>    ==5232== ERROR SUMMARY: 149 errors from 149 contexts (suppressed: 0 from 0)
>
> Is this indeed the GNUstep runtime doing this or have I
> not set up or not interpreted valgrid correctly?
> --
> +---------------------------------------------------------------+
> |   Dale Amon                  Immortal Data                    |
> |   CEO             Midland International Air and Space Port    |
> | [hidden email]       "Data Systems for Deep Space and Time"     |
> +---------------------------------------------------------------+
>
> _______________________________________________
> Discuss-gnustep mailing list
> [hidden email]
> https://lists.gnu.org/mailman/listinfo/discuss-gnustep

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

amon
In reply to this post by amon
Thanks for the confirmation that I was really seeing what I
thought I was seeing. Not too worried about a one off loss
at startup to the run time.

The biggest issue is it makes it hard to find where your own
problems start.

One that I am a bit worried about is NSLog(). I see a number
of items showing up in calls from my code that end in a
malloc deep inside of it.


+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

David Wetzel
Hi!

Try

 scan-build40 -enable-checker osx.cocoa.RetainCount -enable-checker osx.cocoa.NSAutoreleasePool -enable-checker unix.Malloc gmake

If you have llvm. Your version number might be different.

David

Von meinem iPhone gesendet

> Am 13.03.2018 um 20:50 schrieb amon <[hidden email]>:
>
> The biggest issue is it makes it hard to find where your own
> problems start.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

amon
In reply to this post by amon
> scan-build40 -enable-checker osx.cocoa.RetainCount -enable-checker
> osx.cocoa.NSAutoreleasePool -enable-checker unix.Malloc gmak
+e
>  If you have llvm. Your version number might be different.

That doesn't look like a Linux Mint gcc command line to me...
which is what I have. Well, I also have an Ubuntu LTS
system. There are differences in what happens under different
systems... I once spent a month narrowing down why gcc and
clang each had fatal but different idiosyncracies on the
platform I was using. So I might get different results on
my platform than what you would get on yours.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

amon
Just for the enjoyment and edification to anyone who is
watching, the answer to my question about NSLog() is 'no'.
It is not leaking. valgrind gives an LPL response, but
running my own tester shows no loss of pool space after
running it 100 times:

   ZoneInfoPrint(NSZoneFromPointer( (void*) tmp));
   for (i=0; i<100; i++) {NSLog(@"Does this suck storage?");}
   ZoneInfoPrint(NSZoneFromPointer( (void*) tmp));

        Total           =    1794048 bytes
        Used            =    1154304 bytes
        Free            =     639744 bytes
                100 NSLog messages...
        Zone Storage Info
        Total           =    1794048 bytes
        Used            =    1154304 bytes
        Free            =     639744 bytes

So I think I am convinced I can suppress this in valgrind.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Richard Frith-Macdonald-9
In reply to this post by amon


> On 13 Mar 2018, at 23:53, amon <[hidden email]> wrote:
>
> Perhaps someone can tell me if my guess on the following is
> correct. After an afternoon of trying to figure out where the
> large number of leaks reported by valgrind were coming from, I
> went all the way back to basics. What it seems to show is that
> the GNUStep run time allocates a bunch of stuff at start up and
> never clears it.

Yes, there's a whole lot of stuff cached ... not just on startup but also on demand (ie the first time the program uses some feature).

If you look at the NSObject extensions documentation (or the GNUstep/NSObject+GNUstepBase.h header) you will see festures for handling cleanup of that sort of thing on process exit.
Be aware though that, because it's normally a bad idea to explicitly free stuff on exit, cleanup is not normally enabled (with the +setShouldCleanUp: method or the GNUSTEP_SHOULD_CLEAN_UP environment variable) so it's a relatively untested feature and is definitely incomplete (many classes that could support cleanup of their cached information just don't implement it).
If you are interested in contributing additional cleanup code it would be welcome for those cases where people want to use valgrind on processes that terminate.
Mostly I deal with code that runs 24x7 so cleanup of cached memory is not an issue and I only worry about actual leaks,  but it's reassuring to have valgrind tell you that a process frees all memory before exit anyway.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

amon
In reply to this post by amon
I've been continuing my slog up the learning curve of valgrind
and the like. I've also found the tool I wrote to be quite useful.

At this point I am seeing one key issue behind much of my
storage issues. It's the autorelease pool. I'm in an environment
where pretty much anything that goes into it might as well be
a leak, because there is never a release. Systems are intended
to run for days, months, years and that means I've got all sorts
of crud vanishing into the pool that I have no control over.

In many cases there seems to be no choice but to use the methods
that return autoreleased storage because there is no instance
method varient of them. This is particularly the case in NSString
where all the useful string initializers are Class methods only.
So I'm between a rock and a hard place there.

And please, if you haven't been in this discussion with me
before, note that I am working on embedded real time systems.
There is no GUI. There is no event loop where everything can
be released each iteration. It just sits there and does what
it has been commanded to do and listens over links to parent
and child hosts/processes forever. I have to control the
allocations pretty much manually with retain's and releases.

I can think of some hacks though... what if there were a pool
drain method that did not decrement the retain count unless
it was one already? Or if there were a method that let you
remove an object from the pool? Seems like it would be at least
possible to subclass NSAutoreleasePool and add such behavior.
After all, that's why the CS Gods gave us OO!

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Fred Kiefer
Hi Anon,

I don’t want to stop you in your plans to subclass/rewrite NSAutoReleasePool if you think you have to do this, just go ahead. But there are other less problematic ways to achieve the same results.

1) David would probably point you to the usage of ARC but as far as I understand you are using gcc and this is no option for your.

2) In Objective C it is a popular concept to use intermediate auto release pools. That is if you have a bit of code that creates a huge amount of short lived objects, wrap that code in its own auto release pool and most of this objects will be gone when the pool gets cleaned up. This technic gets used a lot in loops where each iteration may have its own auto release pool. This is how the gui code handles events, each event processing is included in its own auto release pool. Even if you application has no gui, there will be some sort of main loop and you could use this technic there.

3) In most cases you don’t have to use the class methods to create objects. They are mostly short cuts to make live easier. In many cases there are similar init methods that take the same arguments. So instead of [NSString stringWithFormat: XXXX] you could be using [[NSString alloc] initWithFormat: XXXX] this is more clumsy but gives you full control over the live time of your object.
 
Hope this helps,
Fred

PS: Did you ever give a try to the memory diagnosis function I pointed you to in my previous mail? This is not gui specific it is implemented on the base level. Gui just offers a simple way to view this information.


> Am 16.03.2018 um 01:07 schrieb amon <[hidden email]>:
>
> I've been continuing my slog up the learning curve of valgrind
> and the like. I've also found the tool I wrote to be quite useful.
>
> At this point I am seeing one key issue behind much of my
> storage issues. It's the autorelease pool. I'm in an environment
> where pretty much anything that goes into it might as well be
> a leak, because there is never a release. Systems are intended
> to run for days, months, years and that means I've got all sorts
> of crud vanishing into the pool that I have no control over.
>
> In many cases there seems to be no choice but to use the methods
> that return autoreleased storage because there is no instance
> method varient of them. This is particularly the case in NSString
> where all the useful string initializers are Class methods only.
> So I'm between a rock and a hard place there.
>
> And please, if you haven't been in this discussion with me
> before, note that I am working on embedded real time systems.
> There is no GUI. There is no event loop where everything can
> be released each iteration. It just sits there and does what
> it has been commanded to do and listens over links to parent
> and child hosts/processes forever. I have to control the
> allocations pretty much manually with retain's and releases.
>
> I can think of some hacks though... what if there were a pool
> drain method that did not decrement the retain count unless
> it was one already? Or if there were a method that let you
> remove an object from the pool? Seems like it would be at least
> possible to subclass NSAutoreleasePool and add such behavior.
> After all, that's why the CS Gods gave us OO!
>
> --
> +---------------------------------------------------------------+
> |   Dale Amon                  Immortal Data                    |
> |   CEO             Midland International Air and Space Port    |
> | [hidden email]       "Data Systems for Deep Space and Time"     |
> +---------------------------------------------------------------+
>
> _______________________________________________
> Discuss-gnustep mailing list
> [hidden email]
> https://lists.gnu.org/mailman/listinfo/discuss-gnustep


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

David Chisnall-7
On 16 Mar 2018, at 08:51, Fred Kiefer <[hidden email]> wrote:
>
> 1) David would probably point you to the usage of ARC but as far as I understand you are using gcc and this is no option for your.

Note that even if you’re using GCC, if you’re using libobjc2 then you can call the ARC objc_retainAutoreleasedReturnValue function explicitly, rather than sending a -retain message to an autoreleased value.  This will pop the top object from the autorelease pool and transfer ownership of it to the caller.  The caller is then responsible for releasing it.

In a system with realtime requirements, autorelease pools are a really bad idea.  They will introduce pauses proportional to the total number of objects in the pool when they are destroyed.  Most realtime systems don’t allow allocating objects at all, so Objective-C is probably exactly the wrong choice for them.

David


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

amon
In reply to this post by amon


> Note that even if you’re using GCC, if you’re using libobjc2
> then you can call the ARC objc_retainAutoreleasedReturnValue
> function explicitly, rather than sending a -retain message to
> an autoreleased value.  This will pop the top object from the
> autorelease pool and transfer ownership of it to the caller.
> The caller is then responsible for releasing it.

> In a system with realtime requirements, autorelease pools are
> a really bad idea.  They will introduce pauses proportional
> to the total number of objects in the pool when they are
> destroyed.  Most realtime systems don’t allow allocating
> objects at all, so Objective-C is probably exactly the wrong
> choice for them.

That is of course true. The hard real time sections are
called from ObjC but once running they do carefully optimized
C code under Linux RT priority.

The rest of it is in other threads or processes which take
what time they can get and communicate with it. It works and
has been proven in quite real world circumstances. However,
the ObjC side never has a chance to sleep even if it is running
at lower priority.

The solution of multiple pools is already in use in those places
where there is a known start/finish to a particular operation.
That still leaves me with areas in which that is not feasible.
Also, I am looking for a way to make things more deterministic
so that I can write up the paper work that will doubtless be
required.

Your suggestion looks rather interesting. Now whereever I can,
I use [[Foo alloc] initmethod], but there are cases where things
like NSString do not have an appropriate init method or one that
is even close to what I need. In those cases, strictly internal
to a given method, I would use your method. Here's an example,
where arglist and cmdline are IVAR's of the object being init'd
(or re-init'd). Don't focus too much on this particular case, it
is just one example that I happened to run down yesterday.

             [arglist release];
   arglist = [[[NSMutableString stringWithCString: [cmdline cString]]
                componentsSeparatedByString: DELIM] retain];

This happens inside an init. arglist is release by the dealloc
method. However, NSMutableString insists on making it autoreleased.
I want to make it not do that. So would you say that if I

          objc_retainAutoreleasedReturnValue(arglist);

that will take that NSString out of the pool so that my dealloc
method will have the sole responsibility for the last release?

My goal is to get to this state of nirvana:

          // zone used = n bytes
          obj = [Foo new];
          // zone used = n+m bytes
          [obj release];
          // zone used = n bytes.

Where Foo could have a highly complex set of IVARS and be
used in complex ways before release.

Next question is whether ARC is available. I have fairly old
LTS (we only use LTS) Ubuntu and Mint systems that are pretty
much bog standard vendor out of the box single board systems. I
typically do add the tool chain to one so I can natively build
on an ARM machine, but never do any major mods beside that. It is
habit right now, but in the long run every change will have to be
tracked, explained and documented and a pain...

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Richard Frith-Macdonald-9


> On 16 Mar 2018, at 15:18, amon <[hidden email]> wrote:
>
>
>            [arglist release];
>  arglist = [[[NSMutableString stringWithCString: [cmdline cString]]
>               componentsSeparatedByString: DELIM] retain];
>
> This happens inside an init. arglist is release by the dealloc
> method. However, NSMutableString insists on making it autoreleased.
> I want to make it not do that.

Portable GNUstep code would look like this:

CREATE_AUTORELEASE_POOL(pool);
ASSIGN(arglist, [[[NSMutableString stringWithCString: [cmdline cString]] componentsSeparatedByString: DELIM]);
DESTROY(pool);

with no wasted/leaked memory.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Fred Kiefer


> Am 16.03.2018 um 16:32 schrieb Richard Frith-Macdonald <[hidden email]>:
>
>
>
>> On 16 Mar 2018, at 15:18, amon <[hidden email]> wrote:
>>
>>
>>           [arglist release];
>> arglist = [[[NSMutableString stringWithCString: [cmdline cString]]
>>              componentsSeparatedByString: DELIM] retain];
>>
>> This happens inside an init. arglist is release by the dealloc
>> method. However, NSMutableString insists on making it autoreleased.
>> I want to make it not do that.
>
> Portable GNUstep code would look like this:
>
> CREATE_AUTORELEASE_POOL(pool);
> ASSIGN(arglist, [[[NSMutableString stringWithCString: [cmdline cString]] componentsSeparatedByString: DELIM]);
> DESTROY(pool);
>
> with no wasted/leaked memory.

I have to disagree. This will get rid of the intermediate NSMutableString but later on when the surrounding Foo object gets released the arglist ivar will get released as well, but the component strings in that array will still remain in what ever autorelease pool is active at that time and will only get freed when that pool is cleaned up. Or at least this is my understanding of autorelease pools.

In the given example this could be resolved by adding another auto release pool:

        // zone used = n bytes
        CREATE_AUTORELEASE_POOL(pool);
        obj = [Foo new];
        // zone used = n+m bytes
        [obj release];
        DESTROY(pool);
        // zone used = n bytes.

This should result in the expected behaviour.
_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Richard Frith-Macdonald-9


> On 16 Mar 2018, at 15:44, Fred Kiefer <[hidden email]> wrote:
>
>
>
>> Am 16.03.2018 um 16:32 schrieb Richard Frith-Macdonald <[hidden email]>:
>>
>>
>>
>>> On 16 Mar 2018, at 15:18, amon <[hidden email]> wrote:
>>>
>>>
>>>          [arglist release];
>>> arglist = [[[NSMutableString stringWithCString: [cmdline cString]]
>>>             componentsSeparatedByString: DELIM] retain];
>>>
>>> This happens inside an init. arglist is release by the dealloc
>>> method. However, NSMutableString insists on making it autoreleased.
>>> I want to make it not do that.
>>
>> Portable GNUstep code would look like this:
>>
>> CREATE_AUTORELEASE_POOL(pool);
>> ASSIGN(arglist, [[[NSMutableString stringWithCString: [cmdline cString]] componentsSeparatedByString: DELIM]);
>> DESTROY(pool);
>>
>> with no wasted/leaked memory.
>
> I have to disagree. This will get rid of the intermediate NSMutableString but later on when the surrounding Foo object gets released the arglist ivar will get released as well, but the component strings in that array will still remain in what ever autorelease pool is active at that time and will only get freed when that pool is cleaned up.

That's (almost always) wrong.  Releasing a container does not put its contents into an autorelease pool, it releases them all. so the components get released/deallocated at the point when the array is deallocated.

Of course you can never guarantee that a class will behave normally (if you don't have access to its source code), but generally when an object is deallocated it releases all its instance variables.

In this instance I think it's fair to assume we are talking about the GNUstep NSMutableString class generating components in a GNUstep NSArray, so we know that we don't have some perverse implementation and the above three-line solution is all that's needed to ensure no leaked objects.

However, I agree that if someone had for instance re-implemented -componentsSeparatedByString: in a category of NSMutableString, and their re-implementation had created an NSArray subclass which, on deallocation, would put the array contents into an autorelease pool rather than releasing them, then you could get the situation you describe where the components would be placed in a pool at the point when the array ivar is released.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Richard Frith-Macdonald-9


> On 16 Mar 2018, at 16:53, Richard Frith-Macdonald <[hidden email]> wrote:
>
>
>
>> On 16 Mar 2018, at 15:44, Fred Kiefer <[hidden email]> wrote:
>>
>>
>>
>>> Am 16.03.2018 um 16:32 schrieb Richard Frith-Macdonald <[hidden email]>:
>>>
>>>
>>>
>>>> On 16 Mar 2018, at 15:18, amon <[hidden email]> wrote:
>>>>
>>>>
>>>>         [arglist release];
>>>> arglist = [[[NSMutableString stringWithCString: [cmdline cString]]
>>>>            componentsSeparatedByString: DELIM] retain];
>>>>
>>>> This happens inside an init. arglist is release by the dealloc
>>>> method. However, NSMutableString insists on making it autoreleased.
>>>> I want to make it not do that.
>>>
>>> Portable GNUstep code would look like this:
>>>
>>> CREATE_AUTORELEASE_POOL(pool);
>>> ASSIGN(arglist, [[[NSMutableString stringWithCString: [cmdline cString]] componentsSeparatedByString: DELIM]);
>>> DESTROY(pool);
>>>
>>> with no wasted/leaked memory.
>>
>> I have to disagree. This will get rid of the intermediate NSMutableString but later on when the surrounding Foo object gets released the arglist ivar will get released as well, but the component strings in that array will still remain in what ever autorelease pool is active at that time and will only get freed when that pool is cleaned up.
>
> That's (almost always) wrong.  Releasing a container does not put its contents into an autorelease pool, it releases them all. so the components get released/deallocated at the point when the array is deallocated.
>
> Of course you can never guarantee that a class will behave normally (if you don't have access to its source code), but generally when an object is deallocated it releases all its instance variables.
>
> In this instance I think it's fair to assume we are talking about the GNUstep NSMutableString class generating components in a GNUstep NSArray, so we know that we don't have some perverse implementation and the above three-line solution is all that's needed to ensure no leaked objects.
>
> However, I agree that if someone had for instance re-implemented -componentsSeparatedByString: in a category of NSMutableString, and their re-implementation had created an NSArray subclass which, on deallocation, would put the array contents into an autorelease pool rather than releasing them, then you could get the situation you describe where the components would be placed in a pool at the point when the array ivar is released.

Of course your example:

       // zone used = n bytes
       CREATE_AUTORELEASE_POOL(pool);
       obj = [Foo new];
       // zone used = n+m bytes
       [obj release];
       DESTROY(pool);
       // zone used = n bytes.

is perfect if what you care about the end result after finishing with the instance of Foo.

The other:

CREATE_AUTORELEASE_POOL(pool);
ASSIGN(arglist, [[[NSMutableString stringWithCString: [cmdline cString]] componentsSeparatedByString: DELIM]);
DESTROY(pool);

is what you'd do in a setter method of Foo, where the instance of Foo is long-lived and you want to ensure that you don't increase memory usage duriung that long life.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Fred Kiefer
In reply to this post by Richard Frith-Macdonald-9


> Am 16.03.2018 um 17:53 schrieb Richard Frith-Macdonald <[hidden email]>:
>
>
>
>> On 16 Mar 2018, at 15:44, Fred Kiefer <[hidden email]> wrote:
>>
>>
>>
>>> Am 16.03.2018 um 16:32 schrieb Richard Frith-Macdonald <[hidden email]>:
>>>
>>>
>>>
>>>> On 16 Mar 2018, at 15:18, amon <[hidden email]> wrote:
>>>>
>>>>
>>>>         [arglist release];
>>>> arglist = [[[NSMutableString stringWithCString: [cmdline cString]]
>>>>            componentsSeparatedByString: DELIM] retain];
>>>>
>>>> This happens inside an init. arglist is release by the dealloc
>>>> method. However, NSMutableString insists on making it autoreleased.
>>>> I want to make it not do that.
>>>
>>> Portable GNUstep code would look like this:
>>>
>>> CREATE_AUTORELEASE_POOL(pool);
>>> ASSIGN(arglist, [[[NSMutableString stringWithCString: [cmdline cString]] componentsSeparatedByString: DELIM]);
>>> DESTROY(pool);
>>>
>>> with no wasted/leaked memory.
>>
>> I have to disagree. This will get rid of the intermediate NSMutableString but later on when the surrounding Foo object gets released the arglist ivar will get released as well, but the component strings in that array will still remain in what ever autorelease pool is active at that time and will only get freed when that pool is cleaned up.
>
> That's (almost always) wrong.  Releasing a container does not put its contents into an autorelease pool, it releases them all. so the components get released/deallocated at the point when the array is deallocated.
>
> Of course you can never guarantee that a class will behave normally (if you don't have access to its source code), but generally when an object is deallocated it releases all its instance variables.
>
> In this instance I think it's fair to assume we are talking about the GNUstep NSMutableString class generating components in a GNUstep NSArray, so we know that we don't have some perverse implementation and the above three-line solution is all that's needed to ensure no leaked objects.
>
> However, I agree that if someone had for instance re-implemented -componentsSeparatedByString: in a category of NSMutableString, and their re-implementation had created an NSArray subclass which, on deallocation, would put the array contents into an autorelease pool rather than releasing them, then you could get the situation you describe where the components would be placed in a pool at the point when the array ivar is released.

Are we talking about this method:

- (NSArray*) componentsSeparatedByString: (NSString*)separator
{
  NSRange search;
  NSRange complete;
  NSRange found;
  NSMutableArray *array = [NSMutableArray array];

  search = NSMakeRange (0, [self length]);
  complete = search;
  found = [self rangeOfString: separator
                      options: 0
                        range: search
                       locale: nil];
  while (found.length != 0)
    {
      NSRange current;

      current = NSMakeRange (search.location,
        found.location - search.location);
      [array addObject: [self substringWithRange: current]];

      search = NSMakeRange (found.location + found.length,
        complete.length - found.location - found.length);
      found = [self rangeOfString: separator
                          options: 0
                            range: search
                           locale: nil];
    }
  // Add the last search string range
  [array addObject: [self substringWithRange: search]];

  // FIXME: Need to make mutable array into non-mutable array?
  return array;
}

Here the elements in the array get generated via the method substringWithRange: and this will return autoreleased objects. Which should lead to about the scenario I was talking about. And I thing this is the big issue for the original poster. Even if he manages to keep all the objects that his code generates properly non-autoreleased, there will be some internal objects that may be created autoreleased by library methods.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Fred Kiefer


> Am 16.03.2018 um 18:39 schrieb Fred Kiefer <[hidden email]>:
>
>
>
>> Am 16.03.2018 um 17:53 schrieb Richard Frith-Macdonald <[hidden email]>:
>>
>>
>>
>>> On 16 Mar 2018, at 15:44, Fred Kiefer <[hidden email]> wrote:
>>>
>>>
>>>
>>>> Am 16.03.2018 um 16:32 schrieb Richard Frith-Macdonald <[hidden email]>:
>>>>
>>>>
>>>>
>>>>> On 16 Mar 2018, at 15:18, amon <[hidden email]> wrote:
>>>>>
>>>>>
>>>>>        [arglist release];
>>>>> arglist = [[[NSMutableString stringWithCString: [cmdline cString]]
>>>>>           componentsSeparatedByString: DELIM] retain];
>>>>>
>>>>> This happens inside an init. arglist is release by the dealloc
>>>>> method. However, NSMutableString insists on making it autoreleased.
>>>>> I want to make it not do that.
>>>>
>>>> Portable GNUstep code would look like this:
>>>>
>>>> CREATE_AUTORELEASE_POOL(pool);
>>>> ASSIGN(arglist, [[[NSMutableString stringWithCString: [cmdline cString]] componentsSeparatedByString: DELIM]);
>>>> DESTROY(pool);
>>>>
>>>> with no wasted/leaked memory.
>>>
>>> I have to disagree. This will get rid of the intermediate NSMutableString but later on when the surrounding Foo object gets released the arglist ivar will get released as well, but the component strings in that array will still remain in what ever autorelease pool is active at that time and will only get freed when that pool is cleaned up.
>>
>> That's (almost always) wrong.  Releasing a container does not put its contents into an autorelease pool, it releases them all. so the components get released/deallocated at the point when the array is deallocated.
>>
>> Of course you can never guarantee that a class will behave normally (if you don't have access to its source code), but generally when an object is deallocated it releases all its instance variables.
>>
>> In this instance I think it's fair to assume we are talking about the GNUstep NSMutableString class generating components in a GNUstep NSArray, so we know that we don't have some perverse implementation and the above three-line solution is all that's needed to ensure no leaked objects.
>>
>> However, I agree that if someone had for instance re-implemented -componentsSeparatedByString: in a category of NSMutableString, and their re-implementation had created an NSArray subclass which, on deallocation, would put the array contents into an autorelease pool rather than releasing them, then you could get the situation you describe where the components would be placed in a pool at the point when the array ivar is released.
>
> Are we talking about this method:
>
> - (NSArray*) componentsSeparatedByString: (NSString*)separator
> {
>  NSRange search;
>  NSRange complete;
>  NSRange found;
>  NSMutableArray *array = [NSMutableArray array];
>
>  search = NSMakeRange (0, [self length]);
>  complete = search;
>  found = [self rangeOfString: separator
>                      options: 0
>                        range: search
>                       locale: nil];
>  while (found.length != 0)
>    {
>      NSRange current;
>
>      current = NSMakeRange (search.location,
> found.location - search.location);
>      [array addObject: [self substringWithRange: current]];
>
>      search = NSMakeRange (found.location + found.length,
> complete.length - found.location - found.length);
>      found = [self rangeOfString: separator
>  options: 0
>    range: search
>                           locale: nil];
>    }
>  // Add the last search string range
>  [array addObject: [self substringWithRange: search]];
>
>  // FIXME: Need to make mutable array into non-mutable array?
>  return array;
> }
>
> Here the elements in the array get generated via the method substringWithRange: and this will return autoreleased objects. Which should lead to about the scenario I was talking about. And I thing this is the big issue for the original poster. Even if he manages to keep all the objects that his code generates properly non-autoreleased, there will be some internal objects that may be created autoreleased by library methods.

I think that now I see the flaw in my argument. I was under the impression that an autorelease pool will only release contained objects if they have a reference count of one. But that is nonsense. The autorelease pool will release all contained objects and as usual the ones where the reference count drops to zero will be deallocated. The autorelease pool itself doesn’t care about the reference count. That way your example was correct, with the autorelease pool around the object creation they will only be retained by the array and go away when that is deallocated.

Sorry for the confusion.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

Richard Frith-Macdonald-9


> On 16 Mar 2018, at 17:58, Fred Kiefer <[hidden email]> wrote:
>
> I think that now I see the flaw in my argument. I was under the impression that an autorelease pool will only release contained objects if they have a reference count of one. But that is nonsense. The autorelease pool will release all contained objects and as usual the ones where the reference count drops to zero will be deallocated. The autorelease pool itself doesn’t care about the reference count. That way your example was correct, with the autorelease pool around the object creation they will only be retained by the array and go away when that is deallocated.

Exactly.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

amon
In reply to this post by amon
Big thanks to all who have been helping. I'm making steady
progress and now have a large chunk of the problem area
understood and have to roll those modes through the 10K lines
of code... that ought to take me a few days. :-)

After I get that chunk, I will probably come back and see
what can be done to attack whatever the next biggest issue that
is still standing... I saw the lightbulb go on when I showed
the split and join methods... I don't have a good answer there
yet either, but I'll come back to it after I've turned the
low hanging fruit into juice.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep and valgrind

amon
Just noticed this while researching... it does not appear to be
possible to create an NSHost object that is *not* autoreleased.
Is this true or did I miss something in the docs?

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
123