How to estimate the size of a hoc object from Python?

[WeinaJi]

Hello @nrnhines

I am trying to optimize the memory usage of a Python process that creates ~100K hoc objects. Ideally it is better to get the size of each single hoc object and multiplies, so that I can estimate the overall memory consumption. The scenario is similar as follows.

The hoc object is defined bysimple.hoc

begintemplate mytemplate
public a, b
proc init() {
    a = $1
    b = $2
}
endtemplate mytemplate

begintemplate simple
objref process, template
proc init() {
    process = new syn1(0.5)
    template = new mytemplate(1,2)
}
endtemplate simple

which creates a point process syn1.mod and another hoc mytemplate

On the Python side, I am using sys.getsizeof to measure the size of the hoc object created:

from neuron import h
import sys
h.load_file("simple.hoc")

sec1 = h.Section('soma')
sec1.push()
simple_objs = h.simple()
process = simple_objs.process
template = simple_objs.template
print(sys.getsizeof(simple_objs))
print(sys.getsizeof(process))
print(sys.getsizeof(template))
h.pop_section()

The output is

72
72
72

The values seems not right, i guess because sys.getsizeof gets the memory consumption directly attributed to the object , not the memory of objects that it refers to.
If I change to measure the objects' attributes

print(sys.getsizeof(process.gmax))
print(sys.getsizeof(process.onset))
print(sys.getsizeof(template.a))
print(sys.getsizeof(template.b))

The output is

24
24
24
24

seems consistent with the size of python float sys.getsizeof(1.234).

Is there a better way to measure the size of a complex hoc object in Python or in Neuron? Thanks in advance!

[nrnhines]

Perhaps h.nrn_malinfo(1) can help. (missing from nrn.readthedocs.io but see the end of nrn/src/oc/symbol.cpp)
The return values just need a bit of thoughtful interpretation, e.g.

hines@michaels-macbook-pro-2 oc % cat temp.py
from neuron import h

def make_vectors(nvec, nsize):
    memsize_begin = h.nrn_mallinfo(1)
    print("memsize_begin ", memsize_begin)
    vectors = [h.Vector(nsize) for _ in range(nvec)]
    memsize = h.nrn_mallinfo(1) - memsize_begin
    print("{} vectors of size {} using {} bytes".format(nvec, nsize, memsize))
    del vectors

make_vectors(100, 0)
make_vectors(100, 10)
make_vectors(100, 100)
make_vectors(100, 1000)

hines@michaels-macbook-pro-2 oc % for i in 1 2 3 4 ; do python3.10 temp.py ; done
memsize_begin  67010560.0
100 vectors of size 0 using 0.0 bytes
memsize_begin  67010560.0
100 vectors of size 10 using 0.0 bytes
memsize_begin  67010560.0
100 vectors of size 100 using 81920.0 bytes
memsize_begin  67092480.0
100 vectors of size 1000 using 802816.0 bytes
memsize_begin  67600384.0
100 vectors of size 0 using 32768.0 bytes
memsize_begin  67633152.0
100 vectors of size 10 using 0.0 bytes
memsize_begin  67633152.0
100 vectors of size 100 using 81920.0 bytes
memsize_begin  67715072.0
100 vectors of size 1000 using 802816.0 bytes
memsize_begin  67960832.0
100 vectors of size 0 using 65536.0 bytes
memsize_begin  68026368.0
100 vectors of size 10 using 0.0 bytes
memsize_begin  68026368.0
100 vectors of size 100 using 81920.0 bytes
memsize_begin  68108288.0
100 vectors of size 1000 using 737280.0 bytes
memsize_begin  67010560.0
100 vectors of size 0 using 0.0 bytes
memsize_begin  67010560.0
100 vectors of size 10 using 0.0 bytes
memsize_begin  67010560.0
100 vectors of size 100 using 81920.0 bytes
memsize_begin  67092480.0
100 vectors of size 1000 using 802816.0 bytes
hines@michaels-macbook-pro-2 oc %

[WeinaJi]

Thanks @nrnhines !
With mallinfo I get the size of my hoc object 97 KB which is consistent with memray output 95.3 KiB

memsize_begin = h.nrn_mallinfo(1)
    a = h.simple().process
memsize_end = h.nrn_mallinfo(1)
print("mem %s" %(memsize_end-memsize_begin))