using a generator for membrane.collector.collect helped a lot, but the ponder process is still killed for too much memory consumption (albeit after much longer than it used to be).
so there are still some optimizations which can happen to make this not such a ridiculous memory strain.
take a look at:
http://guillaume.segu.in/blog/code/487/optimizing-memory-usage-in-python-a-case-study/
for profiling memory usage, look at heapy:
https://stackoverflow.com/questions/110259/which-python-memory-profiler-is-recommended
potentially use namedtuples?
https://docs.python.org/2/library/collections.html#collections.namedtuple
using a generator for
membrane.collector.collecthelped a lot, but the ponder process is still killed for too much memory consumption (albeit after much longer than it used to be).so there are still some optimizations which can happen to make this not such a ridiculous memory strain.
take a look at:
http://guillaume.segu.in/blog/code/487/optimizing-memory-usage-in-python-a-case-study/
for profiling memory usage, look at
heapy:https://stackoverflow.com/questions/110259/which-python-memory-profiler-is-recommended
potentially use
namedtuples?https://docs.python.org/2/library/collections.html#collections.namedtuple