[Tutor] Debug help

[Cameron Simpson]

On 22Jul2020 16:39, jim <jf_byrnes at comcast.net> wrote:
>On 7/19/20 6:57 PM, Cameron Simpson wrote:
>>On 19Jul2020 15:25, jim <jf_byrnes at comcast.net> wrote:
>>[...]
>>>Here is the part of the code that generates the error, the remainder of
>>>the script just manipulates the spreadsheet.
>>>
>>>    for stock in stocks:
>>>        # ~ count = 0
>>>        # ~ while count < total_num_of_stocks:
>>>        # ~ while count < 33:
>>>        while True:
>>>            # ~ count = count + 1
>>>            try:
>>>                # ~ print('Day = ' + day)
>>>                ticker = yf.Ticker(stock)
>>>                # ~ print('Ticker = ' + ticker)
[...]
>
>I am finally able to get back to this discussion. Thanks for the great 
>explanation of try/except. I was always confused when people talked 
>about only catch what you expect and can handle. I googled python 
>exceptions and found a list of them. It makes more sense now.

A random google found reference might not be ideal (not to mention that 
programmes can invent their own). Consider these links:

    The Builtin Exceptions
    https://docs.python.org/3/library/exceptions.html

    Exceptions in the Python execution model
    https://docs.python.org/3/reference/executionmodel.html#exceptions

    Errors and Exceptions from the Python Tutorial
    https://docs.python.org/3/tutorial/errors.html

>The error that promoted my question was:
>
>uno.RuntimeException: Couldn't convert Date
>2020-07-17    134858.5
>2020-07-17    134858.5
>Name: Close, dtype: float64 to a UNO type; caught exception: <class 
>'AttributeError'>: 'Series' object has no attribute 'getTypes', 
>traceback follows
>
>This comes from the module ooSheet, because it was given data in a 
>form it did not expect.   Can try/except be used to catch exemptions 
>from an imported module? In this case I would want the program to 
>stop, but I imagine there could be a case where I would want it to 
>continue.

Not easily. Sometimes not at all. This is because the place where 
something could most usefully be done is buried deep in the code. By the 
time the exception reaches you too much follow on logic has been 
skipped, and you won't have whatever data should have been returned. If 
you're getting a series of data from a generator, the generator will 
have exiting with the exception and all the following data will be 
missing, and so forth.

You can of course catch the exception and continue your own local 
control structure eg the next loop iteration.

Your choices depend on how the exception occurred and the internal 
structure of the module you're using.

Have a look at the stack trace - it will identify where the exception 
was raised. This includes the soruce code references!

Options include:

Call something lower level. Suppose you code is calling a function which 
returns spreadsheet like data, some table of values from the stock data.  
Likely that function itself calls lower level things to process the data 
and construct the array. You could do that work yourself, and call some 
per-row function, and catch the exception on a row-by-row basis, 
skipping (and reporting in detail) the bad row.

Pass in a special parser. Some libraries have a degree of extensibility 
in mind, and have a hook for a custom parser, i.e. a functioin to 
convert the raw data into whatever data type you intend to get. If 
there's such a hook, use it. Initially you can find out the default 
parser function, and write a small function which:
- calls the default with a try/except
- catches exceptions and reports the exception and the dfata which 
  raised it, and return maybe None

For example:

    # you'd need to find this, if it exists
    default_parser = module.something

    def my_parser(row):
        try:
            return default_parser(row)
        except Exception as e:
            print("default_parser fails on row %r with exception %s" % (row, e))
            return None

    data = ticker.fetch(parser=my_parser)

Obviously this requires the module to offer this facility.

Monkey patch! Modify the modul in place.

Monkey patching is where you alter a module after it is loaded. In your 
case, look at the module source code and find the low level function 
which raised the exception. This is available in the stack trace!

Let's suppose the low level thing is module.parse_datum(), to invent a 
name. You can do the custom parser with a monkey patch:

    import yf

    # you'd need to find this, if it exists
    original_parser = yf.parse_datum

    def my_parse_datum(datum):
        try:
            return original_parse_datum(datum)
        except Exception as e:
            print("yf.parse_datum fails on %r with exception %s" % (datum, e))
            # you could parse datum specially here, if you know what to 
            # handle
            return None

    # monkey patch here:
    yf.parse_datum = my_parse_datum

    ...
    yf.Ticker(...)
    ...

Now your special function is in the module where the original parser 
was, and _uses_ the original parse unless it fails.

If nothing else, this provides the data causing the trouble. And you can 
then write your own code to handle just that data when it shows up!

Cheers,
Cameron Simpson <cs at cskk.id.au>