Python is not bad ;-)

[Cecil Westerhof]

Op Thursday 30 Apr 2015 19:59 CEST schreef Christian Gollwitzer:

> Am 30.04.15 um 18:11 schrieb Cecil Westerhof:
>> Op Thursday 30 Apr 2015 16:03 CEST schreef Michael Torrie:
>>
>>> On 04/30/2015 01:07 AM, Cecil Westerhof wrote:
>>>> When I do that the computer is freezed a few times. That is a
>>>> little less nice. Does not happen with Clojure when it gets an
>>>> out of memory.
>>>
>>> A system freeze is probably due to thrashing by your operating
>>> system as a process (in this case Python) uses more and more
>>> memory, causing massive swapping. Clojure's heap, being a
>>> JVM-based language, is based on JVM settings, so it may be maxing
>>> out at just a couple of GB. Whereas Python will happily max out
>>> your swap if your program demands the memory.
>>
>> I just posted a message about that. This gets the problem also:
>> l = range(int(1E9))
>>
>> The problem is that after this other processes are swapped out and
>> have a bad performance. There is nothing that can be done about it?
>>
>> So there is a positive point for working with the JVM. :-D
>>
>
> No, you understood this the wrong way. The JVM has a limit on the
> max memory size on startup, which you can give by the -Xmx option on
> the Oracle machine, so for instance
>
> java -Xmx512M -jar myjar.jar
>
> limits the memory that your program may consume to 512 megs, until
> the JVM kills it. The standard limit is usually fairly low and
> probably below your real memory, so the java program does not have
> the chance to max out your memory and make your computer swap.

Well, I did play a little with that. My experience is that this is to
high. For example by lowering it, a certain program did not swap
anymore. Sound counter intuitive, but not that strange if you know the
inner workings of the JVM.


> CPython, au contraire, uses all memory it can get from the OS. The
> OS kills it if it uses too much. On Linux, you can set this limit
> yourself using ulimit. The analogue to the java call would therefore
> be something like
>
> ulimit -m 512M
> python mypython.py

I use startup scripts for the different Java (or Scala, or Clojure)
programs to set the memory size. I should do that also for my (or
anothers) Python scripts.


> If you set the ulimit to something smaller than your physical memory
> size, you also cause the program to be killed before it can use up

Nope, the program gets a MemoryError and will not be killed. Much
better I think.


> should set both limits to the same value and see how far you can
> get.

You are right.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof