[Tutor] Python 3: string to decimal conversion

[Peter Otten]

US wrote:

> Thank you both for suggesting a way to handle errors. I have run the
> suggested code. What I learned is that all the values (not only empty
> cells) seem to be invalid for decimal.Decimal function.
> 
> I tried the float() function instead of decimal.Decimal and got an
> error message: could not convert string to float: '($75.59)'.
> 
> I also checked the type of values in row[4]. All the values passed on
> to decimal.Decimal () function are indeed of string type...
> 
> Is there anything I can do to the formatting of the csv file to make
> it 'readable' for decimal.Decimal function?
> 
> Here is my updated code along with the output I got from running it:
> 
> ----------------------------------------------------
> code:
> import numpy as np
> import csv
> import timestring as ts
> import decimal
> 
> months= [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
> expenses = {x: decimal.Decimal() for x in months}
> income = {x: decimal.Decimal() for x in months}
> test = []
> 
> exp_cat = []
> income_cat = []
> 
> files =['export.csv']
> 
> with open("budgetfile.csv","wt") as fw:
>     writer = csv.writer(fw)
>     for file in files:
>         with open(file, newline ='' ) as csvfile:
>             records = csv.reader(csvfile)
>             print("Processing file {}. \n" .format(file))
>             header = next(records)
>             for row in records:
>                 print("row[4] =", repr(row[4]), "value type =",
>                 type(row[4]))
> 
>                 if not row[4]:
>                     pass
>                 else:
>                     try:
>                         expenses[ts.Date(row[0]).month] +=
> decimal.Decimal(row[4])
>                     except decimal.InvalidOperation as e:
>                         print("unexpected expenses value:  %r" %
>                         (row[4],))

> last 4 lines of output:
> [4] = '($10.00)' value type = <class 'str'>
> unexpected expenses value:  '($10.00)'
> row[4] = '($287.42)' value type = <class 'str'>
> unexpected expenses value:  '($287.42)'

Perhaps you should use a custom conversion routine like to_decimal() below:

class MissingValue(ValueError):
    pass


@contextlib.contextmanager
def expect(exc):
    """Ensure that an excption of type `exc` was raised.

    Helper for the doctests.
    """
    try:
        yield
    except exc:
        pass
    except:
        raise AssertionError(
            "Wrong exception type (expected {})".format(exc))
    else:
        raise AssertionError("Exception was not raised")


def to_decimal(s):
    """
    >>> with expect(MissingValue):
    ...     to_decimal("   ")
    >>> with expect(ValueError):
    ...     to_decimal("42")
    >>> to_decimal("$12.34")
    Decimal('12.34')
    >>> to_decimal("($34.56)")
    Decimal('-34.56')
    >>> with expect(ValueError):
    ...     to_decimal("foo")
    >>> with expect(ValueError):
    ...     to_decimal("$bar")
    >>> with expect(ValueError):
    ...     to_decimal("($baz)")
    """
    s = s.strip()  # remove leading/trailing whitespace
    if not s:
        raise MissingValue("Empty amount")
    if s.startswith("(") and s.endswith(")"):
        sign = -1
        s = s[1:-1]  # remove parens
    else:
        sign = 1
    if not s.startswith("$"):
        raise ValueError("No leading $ found")
    s = s[1:]  # remove $
    try:
        value = decimal.Decimal(s)
    except decimal.InvalidOperation as err:
        raise ValueError(err.args[0]) from None
    return sign * value


That way you can spell out explicitly what the allowed values may look like, 
and if (e. g.) you want to allow for grouping you can easily add another 
preprocessing step to remove the commas. Use it like so in your code:

...
for row in records:
    try:
        amount = to_decimal(row[4])
    except ValueError as err:
        print(err, file=sys.stderr)
    else:
        ...  # add amount to expenses
...