Use of NSZoneStat()

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Use of NSZoneStat()

amon
I am trying to get a handle on storage leaks and NSZoneStat()
would be ideal... except it doesn't work.

   NSZone *dftzone = NSDefaultMallocZone();
   NSZone *curzone = NSZoneFromPointer ( (void *) arglist);
   printf ("dft = %lx cur  = %lx\n",
          (long unsigned int) dftzone,
          (long unsigned int) curzone);

   struct NSZoneStats zs0 = NSZoneStats (dftzone);
   printf ("Bytes Used %d Free %d\n",
          (int) zs0.bytes_used,
          (int) zs0.bytes_free);

dft = 7ffcfdf26200 cur  = 7ffcfdf26200
bin/gpstest: Uncaught exception NSGenericException, reason: No
statistics for default zone

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

Gregory Casamento
Did you test this on Mac OS X?  What results are returned there?

On Thu, Mar 8, 2018 at 11:14 amon <[hidden email]> wrote:
I am trying to get a handle on storage leaks and NSZoneStat()
would be ideal... except it doesn't work.

   NSZone *dftzone = NSDefaultMallocZone();
   NSZone *curzone = NSZoneFromPointer ( (void *) arglist);
   printf ("dft = %lx cur  = %lx\n",
          (long unsigned int) dftzone,
          (long unsigned int) curzone);

   struct NSZoneStats zs0 = NSZoneStats (dftzone);
   printf ("Bytes Used %d Free %d\n",
          (int) zs0.bytes_used,
          (int) zs0.bytes_free);

dft = 7ffcfdf26200 cur  = 7ffcfdf26200
bin/gpstest: Uncaught exception NSGenericException, reason: No
statistics for default zone

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
--
Gregory Casamento
GNUstep Lead Developer / OLC, Principal Consultant
http://www.gnustep.org - http://heronsperch.blogspot.com
http://ind.ie/phoenix/

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

David Chisnall-5
In reply to this post by amon
On 8 Mar 2018, at 16:15, amon <[hidden email]> wrote:
>
> I am trying to get a handle on storage leaks and NSZoneStat()
> would be ideal... except it doesn't work.

The default malloc zone just calls malloc, so you can get the statistics from your underlying malloc implementation.  Most mallocs come with various debugging features - take a look at the man page.

The NSZone functionality is largely gone these days.  Cocoa has removed it completely (and never exposed NSZone as anything other than an opaque type, so you could never usefully define custom zones).

David


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

Ivan Vučica-2
In reply to this post by Gregory Casamento
What you're doing won't work, as far as I can tell.

Header:

/** Deprecated ...<br />
 *  Obtain statistics about the zone.  Implementation emphasis is on
 *  correctness, not speed.  Not defined by OpenStep or OS X.
 */
struct NSZoneStats
NSZoneStats (NSZone *zone);

Implementation:

static struct NSZoneStats
default_stats (NSZone *zone)
{
  struct NSZoneStats dummy = {0,0,0,0,0};

  /* We can't obtain statistics from the memory managed by malloc(). */
  [NSException raise: NSGenericException
      format: @"No statistics for default zone"];
  return dummy;
}

static NSZone default_zone =
{
  default_malloc, default_realloc, default_free, default_recycle,
  default_check, default_lookup, default_stats, 0, @"default", 0
};

NSZone*
NSDefaultMallocZone (void)
{
  return &default_zone;
}

struct NSZoneStats
NSZoneStats (NSZone *zone)
{
  if (!zone)
    zone = NSDefaultMallocZone();
  return (zone->stats)(zone);
}



On Thu, Mar 8, 2018 at 4:19 PM, Gregory Casamento <[hidden email]> wrote:
Did you test this on Mac OS X?  What results are returned there?

On Thu, Mar 8, 2018 at 11:14 amon <[hidden email]> wrote:
I am trying to get a handle on storage leaks and NSZoneStat()
would be ideal... except it doesn't work.

   NSZone *dftzone = NSDefaultMallocZone();
   NSZone *curzone = NSZoneFromPointer ( (void *) arglist);
   printf ("dft = %lx cur  = %lx\n",
          (long unsigned int) dftzone,
          (long unsigned int) curzone);

   struct NSZoneStats zs0 = NSZoneStats (dftzone);
   printf ("Bytes Used %d Free %d\n",
          (int) zs0.bytes_used,
          (int) zs0.bytes_free);

dft = 7ffcfdf26200 cur  = 7ffcfdf26200
bin/gpstest: Uncaught exception NSGenericException, reason: No
statistics for default zone

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
--
Gregory Casamento
GNUstep Lead Developer / OLC, Principal Consultant
http://www.gnustep.org - http://heronsperch.blogspot.com
http://ind.ie/phoenix/

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep



_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

Richard Frith-Macdonald-9
In reply to this post by amon


> On 8 Mar 2018, at 16:15, amon <[hidden email]> wrote:
>
> I am trying to get a handle on storage leaks and NSZoneStat()
> would be ideal... except it doesn't work.
>
>  NSZone *dftzone = NSDefaultMallocZone();
>  NSZone *curzone = NSZoneFromPointer ( (void *) arglist);
>  printf ("dft = %lx cur  = %lx\n",
>  (long unsigned int) dftzone,
>  (long unsigned int) curzone);
>
>  struct NSZoneStats zs0 = NSZoneStats (dftzone);
>  printf ("Bytes Used %d Free %d\n",
>  (int) zs0.bytes_used,
>  (int) zs0.bytes_free);
>
> dft = 7ffcfdf26200 cur  = 7ffcfdf26200
> bin/gpstest: Uncaught exception NSGenericException, reason: No statistics for default zone

NSZoneStat is only supposed to work for special zones you have created, not for the default zone.  It's for cirecumstances where you are carefully isolating your work in one or more non-standard zones.

If you are looking for general memory management, the default zone is implemented using malloc/free and you can substitute a different malloc library (eg tcmalloc) which will provide its own diagnostics.  This is not GNUstep specific stuff an you are better off asking in general (or C) programming forums.

If you are interested in object allocation/deallocation (rather than total heap memory), gnustep-base has a rich set of extensions to help with that (see NSDebug.h) as well as some extra stuff to help deal with autorelease pools.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

amon
In reply to this post by amon
No Mac issues. I only work with Linux, partly *because* of the
way Apple tries to hide things.

So if I read between the lines, would you be saying that the
default zone can be analyzed via malloc and the NSZoneStats()
would work for any other NSZone's?

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

amon
FYI, the targets for my code tend to be very small ARM units.
Working mostly with the Mirabox at the moment.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

amon
What I want it to run an object in a test harness, say 1000
times, and meter the storage usage each time through so I can
see if there are cases in which objects are leaking storage.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

Fred Kiefer
If this is for a graphical application I can recommend the MemoryPanel (https://github.com/gnustep/libs-gui/blob/master/Source/GSMemoryPanel.m) that is build into all GNUstep applications. Just open the about panel and click on the GNUstep icon to open this hidden feature.
This code uses GSDebugAllocationClassList, which has been pointed to be Richard and may be used by non graphical applications.

Hope this helps,
Fred

> Am 08.03.2018 um 18:29 schrieb amon <[hidden email]>:
>
> What I want it to run an object in a test harness, say 1000
> times, and meter the storage usage each time through so I can
> see if there are cases in which objects are leaking storage.


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

amon
In reply to this post by amon
No, not graphical. This is for down in the nitty gritty debug
level on machines which do not even have a screen.

Based on the hint, I found that the current malloc has gained
some tools that were not there the last time I had to do this,
and in fact it looks like mallinfo(3) might even be the base
upon which NSZoneStats() was built. So I might be able to whip
up a Linux ZoneStats() that uses NSZoneStats if zone != default
and glues in the values from mallinfo into an struct NSZoneStats,
which would give me a much more generally useful tool.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

amon
I wrote it and got it working this afternoon. If anyone wants
to take a look and perhaps give me some feedback, I have
attached the source.

The format is likely quite different than what you are used to,
but this is our house style.

You probably will have to tweak a path or two since it will not
be compiling where it thinks it should be.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep

ZoneInfo.h (1K) Download Attachment
ZoneInfo.m (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

David Chisnall-7
In reply to this post by amon
On 8 Mar 2018, at 17:57, amon <[hidden email]> wrote:
>
> Based on the hint, I found that the current malloc has gained
> some tools that were not there the last time I had to do this,
> and in fact it looks like mallinfo(3) might even be the base
> upon which NSZoneStats() was built. So I might be able to whip
> up a Linux ZoneStats() that uses NSZoneStats if zone != default
> and glues in the values from mallinfo into an struct NSZoneStats,
> which would give me a much more generally useful tool.

Note that mallinfo is not a ‘linux’ thing, it is a dlmalloc thing.  Most GNU/Linux distros use dlmalloc as their default malloc but Android, for example, does not, and a few musl-based distros also use jemalloc, as do some programs (e.g. Firefox) on all GNU/Linux platforms.  jemalloc also has a (somewhat richer) set of debugging features, but these are exposed via different interfaces.

On Darwin, NSZone is implemented on top of the malloc_zone functionality in their malloc, which provides a structure that looks quite similar to GNUstep’s NSZone structure (though has a few other functions, for example one for allocating a large number of identical blocks simultaneously).  I have never seen any code that uses malloc_zone, but I believe that Apple uses it internally for a few things where direct control over memory management is important for performance.

David


_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

amon
In reply to this post by amon
It may be the case that the malloc scene is confused and lacks
standards, but the ability to debug leaks with a simple test
like this has been a long standing and very serious problem
with Objective C, to the point that I have my doubts there is
any significant size program written in the language that is not
silently leaking like a sieve.

This solution at least works in all the environments that are
of interest to my company, and if others can figure out extensions
to this simple API to make it work with some of the other ad-hoc
malloc and pseudo-malloc API's, its all to the advantage of
everyone. Feel free to take the code I posted and extend as
needed, or to simply use it to help you track down your own
memory leaks.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

Richard Frith-Macdonald-9


> On 9 Mar 2018, at 17:08, amon <[hidden email]> wrote:
>
> It may be the case that the malloc scene is confused and lacks
> standards, but the ability to debug leaks with a simple test
> like this has been a long standing and very serious problem
> with Objective C, to the point that I have my doubts there is
> any significant size program written in the language that is not
> silently leaking like a sieve.

Very few ObjC programs have big issues with malloc/free since they simply don't use them directly very much ... heap memory is generally associated with objects and dealt with by the retain/release/autorelease mechanism.
There *are* lots of features for management and reporting of object lifetimes etc which provide much more detailed/useful information than NSZoneStat() about object allocation/deallocation.
But, I agree that having a nice report of malloc stats is really useful in the specific cases wher you are writing code that does lots of messing with heap memory not associated with objects.
_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

Ivan Vučica-2
In reply to this post by amon


On Fri, Mar 9, 2018, 17:25 amon <[hidden email]> wrote:
It may be the case that the malloc scene is confused and lacks
standards, but the ability to debug leaks with a simple test
like this has been a long standing and very serious problem
with Objective C, to

Apple has nice tools to debug ObjC leaks.

It would be a reason enough to run my stuff on OS X.

the point that I have my doubts there is
any significant size program written in the language that is not
silently leaking like a sieve.

I would also use the same tool as I would in C: valgrind.

It would be a reason enough to run my stuff on desktop Linux.

mallinfo() is an excellent TIL, however. Seems useful to trivially export use of standard malloc() into a time series database, without merely observing the process ram use externally, and while also not resorting to using wrappers.


This solution at least works in all the environments that are
of interest to my company, and if others can figure out extensions
to this simple API to make it work with some of the other ad-hoc
malloc and pseudo-malloc API's, its all to the advantage of
everyone. Feel free to take the code I posted and extend as
needed, or to simply use it to help you track down your own
memory leaks.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

amon
In reply to this post by amon
Just to re-iterate... most of those tools are of little help.
I am doing embedded systems that will run for long periods of
time and may have life-safety implications. The small boxes
are ARM based linux boxes. No GUI. All work is via an xterm
over an ssh with the target host running one or more ObjC based
daemons.

Yes, I can test some things on a desktop, but the religious mantra
of my industry is "Test as you fly, fly as you test."

Yes, there are multiple ways to manage storage, and over a period
of years, all of them may end up in a given large code body and
have unexpected interactions, not to mention human confusion
and error. You really do need an absolute ground truth test so
you can run a Class (or entire application) through its paces
and at the end have a file-able document showing
Free(starttime) == Free(stoptime). Although not there yet, I
would not find it surprising if I had to supply such information
to show that I meet the tough regulatory requirements of the
various potential users.

The ZoneInfo code does get me that information, which is an
invaluable aid. Admittedly, it could be better. I had forgotten
about valgrind... the last time I looked at it was probably
15+ years ago so it has probably improved greatly. A quick check
shows they are supporting Linux/ARM so now you've caused me to
have to read another manual!

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: Use of NSZoneStat()

Ivan Vučica-2
On Sat, Mar 10, 2018 at 6:08 PM, amon <[hidden email]> wrote:

>
> Just to re-iterate... most of those tools are of little help.
> I am doing embedded systems that will run for long periods of
> time and may have life-safety implications. The small boxes
> are ARM based linux boxes. No GUI. All work is via an xterm
> over an ssh with the target host running one or more ObjC based
> daemons.
>
> Yes, I can test some things on a desktop, but the religious mantra
> of my industry is "Test as you fly, fly as you test."


Acknowledged.

In addition to the requirement to test on your target ARM platform,
you may still want to run your software on desktops (and run various
debugging tools both on desktop and your target platform). :-)

I do understand you need definitive information on your target platform, though.

>
> Yes, there are multiple ways to manage storage, and over a period
> of years, all of them may end up in a given large code body and
> have unexpected interactions, not to mention human confusion
> and error. You really do need an absolute ground truth test so
> you can run a Class (or entire application) through its paces
> and at the end have a file-able document showing
> Free(starttime) == Free(stoptime). Although not there yet, I
> would not find it surprising if I had to supply such information
> to show that I meet the tough regulatory requirements of the
> various potential users.
>
> The ZoneInfo code does get me that information, which is an
> invaluable aid. Admittedly, it could be better. I had forgotten
> about valgrind... the last time I looked at it was probably
> 15+ years ago so it has probably improved greatly.


Absolutely appreciate the situation. It was not obvious that tough
regulations kick in, this makes things clearer.

>
> A quick check
> shows they are supporting Linux/ARM so now you've caused me to
> have to read another manual!

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

GNUstep error handling

amon
In reply to this post by amon
I've run across documentation in GNUstep that talks of NS_HANDLER,
etc, which I've never used. I presume this is something relatively
new from the Apple world as I don't remember it (or any error
handler) from NeXTstep and in fact I've pretty much always used
@try/@catch/@finally for about as long as I can remember and I
think before that I just rolled my own stack unwinder error
handler.

Are they just macro covers over the original syntax or do they
add some features?

I use a lot of @throw NSException's since daemon code is not
allowed to not work. It has to punt to a level that can retry,
no matter what goes wrong with the outside world. Hey, you never
know... a solar flare might have taken out the server I'm
taking data from!

Are there any overriding reasons to shift an old body of code
over to the new method?

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep error handling

Richard Frith-Macdonald-9


> On 13 Mar 2018, at 01:52, amon <[hidden email]> wrote:
>
> I've run across documentation in GNUstep that talks of NS_HANDLER,
> etc, which I've never used. I presume this is something relatively
> new from the Apple world as I don't remember it (or any error
> handler) from NeXTstep and in fact I've pretty much always used
> @try/@catch/@finally for about as long as I can remember and I
> think before that I just rolled my own stack unwinder error
> handler.
>
> Are they just macro covers over the original syntax or do they
> add some features?
>
> I use a lot of @throw NSException's since daemon code is not
> allowed to not work. It has to punt to a level that can retry,
> no matter what goes wrong with the outside world. Hey, you never
> know... a solar flare might have taken out the server I'm
> taking data from!
>
> Are there any overriding reasons to shift an old body of code
> over to the new method?

NS_HANDLER is the *original* exception handler, which predates @catch (it was the only exception handling mechanism for many years).
If you are only using platforms with modern compilers, you don't need it.
_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
Reply | Threaded
Open this post in threaded view
|

Re: GNUstep error handling

amon
In reply to this post by amon
> NS_HANDLER is the *original* exception handler, which predates
> @catch (it was the only exception handling mechanism for many
> years).
> If you are only using platforms with modern compilers, you don't
> need it.

Thanks, that is good to know. It unfortunately still shows up
in a lot of GNUstep documentation and the various documents
on the web are also a mixed bag.

--
+---------------------------------------------------------------+
|   Dale Amon                  Immortal Data                    |
|   CEO             Midland International Air and Space Port    |
| [hidden email]       "Data Systems for Deep Space and Time"     |
+---------------------------------------------------------------+

_______________________________________________
Discuss-gnustep mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/discuss-gnustep
12