[issue37762] IDLE very slow due a super long line output in chunks

[Terry J. Reedy]

Terry J. Reedy <tjreedy at udel.edu> added the comment:

One may copy and paste small chunks of code and output into a message. By 'demo script', I presume you mean the following.

import tensorflow as tf
mnist = tf.keras.datasets.mnist

(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(512, activation=tf.nn.relu),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test)

The boxes represent characters that cannot be displayed in the tk Text widget with the current font.  They represent whatever is printed in the standard shell, which you call 'normal', which implies that they are not control chars.  Since the length of the box run is 1 less than the length of the visible data, my guess is that the intended format is alternating lines something like
 9696/60000............................................ acc: 0.8691
==================================================================
except that the line separators are some non-ascii char.  (Note that the sequence of [] substitutes in one less than the sequence of printable ascii, which would make it impossible to set the shell width so that wrapped chunks line up.)

The problem is the absence of '\n' newlines, which I consider a bug in keras. The result is one line that grows in chunks to some humongous length.  This is eventually deadly to tk Text widgets.  The symptom is that chunks are added quickly at first, then more slowly, then to a crawl, and maybe a stop.  Here is a short test.

from __future__ import print_function  # To run with 2.7.
for i in range(100000):
    print('%6s--------------------------------------------' % i, end='')

On my Win10 machine with 3.8.0b3 and tk 8.6.9 (see Help => About IDLE for the latter), slow down is evident after 10000 chars (200 chunks) and crawling by 40000 chars.  It might be worse on Linux and Mac.

I added a note about auto-squeezing the long lines in chunks case to #35855.  I expect to close this as 3rd party, or as a duplicate of #1442493.

----------
title: IDLE very slow due to special characters -> IDLE very slow due a super long line output in chunks

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue37762>
_______________________________________