Return str to a callback raise a segfault if used in string formating

[Vincent Vande Vyvre]

Le 13/10/17 à 09:23, Chris Angelico a écrit :
> On Fri, Oct 13, 2017 at 4:46 PM, Vincent Vande Vyvre
> <vincent.vande.vyvre at telenet.be> wrote:
>> Simplified code:
>>
>> ---%<------------------------------------------
>> ...
>> ---%<------------------------------------------
>>
>> If I place self.callback() at the end of the func process that doesn't
>> change anything.
> First off, exactly what version of Python are you running this under?
> With a segfault, you need to be pretty specific - platform, version,
> word size, as much as you can gather.
>
> Secondly: Can you create a version of this that doesn't comment out
> part of the work? If you can make a script where, any time you run it,
> Python segfaults, that would be extremely helpful.
>
> Your code currently looks a bit weird. You create a thread, start it,
> and then immediately wait for it (join()). When you add a task, if
> it's the first task you've added, you process tasks, thus removing
> that task. So the class isn't actually doing anything, and logically,
> you could simply process the files directly in the loop. I'm guessing
> that something in there (probably the threading) is causing the crash,
> but without a complete and testable demo, it's hard to be sure.
>
> ChrisA

I'm using 3.6.1 in venv

It's not easy to write a runnable code because in this example the 
method process() is too simplified and the code don't reproduce the 
segfault.

The code is an image processing in two parts: the processing himself in 
CPython and an api in Python.

I have written the two parts.

In the real code this is the equivalent of the method process():
----------------------------------------------
     def unraw(self, index=0, dest=""):
         """Run the demosaication process.

         Args:
         index -- the index of the image or "all" if there's more than 
one image
                  into the file
         dest -- the absolute file name for the image decoded.  If a 
file with
                 the same name already exists, it will be overwritten.

         Raise IndexError if index >= self.image_count
         """
         if not self.is_raw:
             fname = os.path.basename(self.filename)
             raise TypeError("RAW file %s not supported!" % fname)

         if index >= self.data["image_count"]:
             raise IndexError("Index of image %s out of range(%s)"
                              %(index, self.data["image_count"]))

         multi = 0
         if index == "all":
             multi = 1
             index = 0

         if not dest:
             dest = self.filename

         target = os.path.splitext(dest)[0]
         res = self.raw.demosaicate(index, multi, target)    # This call 
the CPython code
         self.out_filename = self.raw.out_file

     def run_multitasks(self):
         def process(*args):
             fname, idx, oname = args
             self.identify()
             self.unraw(idx, oname)

         while self.tasks:
             t = Thread(target=process, args=self.tasks.pop(0))
             t.start()
             t.join()
             if self.callback:
                 self.callback(self.out_filename)
----------------------------------------------------------------

I use a list of 20 files and all files are converted, one by one, 
because the CPython part is not thread safe.

But the real interesting thing is the behaviour in the slot() function.

Examples:
  1 -------------------------------
def slot(name):
     if os.path.isfile(name):
         print("Created:", name, type(name))
         print("%s created" % name)

result:
add: /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2
Scaling with darkness 127, saturation 3712, and
multipliers 2.310656 1.000000 1.257116 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Created: 
/home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff 
<class 'str'>
Erreur de segmentation (core dumped)
----------------------------------

  2 --------------------------------
def slot(name):
     new = name[:]
     if os.path.isfile(name):
         print("Created:", name, type(name))
         print("New:", new)
         print("%s created" % new)

result:
add /home/vincent/Images/RAW_orig/RAW_CANON_1DSM2.CR2
Scaling with darkness 127, saturation 3712, and
multipliers 2.310656 1.000000 1.257116 1.000000
AHD interpolation...
Converting to sRGB colorspace...
Created: 
/home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff 
<class 'str'>
New: /home/vincent/CPython/py361_venv/pyunraw/unraws/RAW_CANON_1DSM2.tiff
Erreur de segmentation (core dumped)
----------------------------------

The most important, of course is the name of the new file created and I 
can be satisfied with that, I just need the file name, but it's not a 
reason to ignore the problem.

Vincent