From dave at nk7z.net  Wed Mar  1 00:16:00 2023
From: dave at nk7z.net (Dave (NK7Z))
Date: Tue, 28 Feb 2023 21:16:00 -0800
Subject: [Tutor] Where does the program pointer go when...
In-Reply-To: <nmjtvhl331js3k6hnl8k7kk9mgsfh0k56q@4ax.com>
References: <32cdf823-8ee7-6c33-37bb-f4452919ba1f@nk7z.net>
 <nmjtvhl331js3k6hnl8k7kk9mgsfh0k56q@4ax.com>
Message-ID: <b42ee457-8a92-9b74-9f01-ca4b8fec8815@nk7z.net>

Dennis,

Thanks you for the information, you answered my timeout-- where does it 
go, question.  Plus the references to the RFCs...  THANK YOU!

I see you have a  ham ticket, as I do, this script logs into an AR 
cluster, and watches the incoming data stream...  Parses it, and alerts 
me to calls I have tagged to look for...

Your answers have been very helpful, again, thank you sir!!!

73, and thanks,
Dave (NK7Z)
https://www.nk7z.net
ARRL Volunteer Examiner
ARRL Technical Specialist, RFI
ARRL Asst. Director, NW Division, Technical Resources

On 2/28/23 20:38, Dennis Lee Bieber wrote:
> On Tue, 28 Feb 2023 10:26:58 -0800, "Dave (NK7Z)" <dave at nk7z.net> declaimed
> the following:
> 
>> I am trying to get my head around what is a timeout, and what happens
>> when one occurs.  I have a working script that does what I want, but I
>> am now working on trapping errors.  i.e. Ethernet dies, telnet server
>> shuts down, telnet server just stops sending data, telnet server starts
>> sending b'', etc.
>>
> 
> 	The library reference manual is a crucial resource.
> 
>> As a result of this effort, I find I have a failed understanding of what
>> happens when a TIMEOUT occurs.  I am fairly sure I don't even know WHAT
>> a TIMEOUT is...  See assumptions list later.
>>
> 
> """
> Telnet.read_until(expected, timeout=None)
> 
>      Read until a given byte string, expected, is encountered or until
> timeout seconds have passed.
> 
>      When no match is found, return whatever is available instead, possibly
> empty bytes. Raise EOFError if the connection is closed and no cooked data
> is available.
> """
> 
> 	Note: "return whatever is available"... Often one buffers the partial
> data and retries the operation, combining partials until the complete data
> has been transferred or some more serious error occurs.
> 
>>
>> Script above here initing all variables, etc...
>>
>>          try:
>>              result = tn.read_until(b"\r\n", TIMEOUT)
> 
> ...	which could mean you get stuff up to a <cr> but not the <lf>, or
> anything else that might have been sent.
> 
>>              if result == b'':
>>                  relogin()
>>                  configureCluster()
>>                  exitscript()
> 
> 	Uhm... why "relogin" if you are just going to exit the program?
> 
>>          except socket.error:  # If socket error-- exit with error.
>>              logdata = 'Socket Error in watchstream(), at main read loop.'
>>              logit(logdata)
>>              exitscript()
>>          except EOFError:
>>              failcode = 6
>>              logdata = 'EOF at main read loop.'
>>              logit(logdata)
>>              exitscript()
>>          except OSError:
>>              logdata = 'OS Error in watchstream().'
>>              logit(logdata)
>>              exitscript()
>>
>> Rest of script...
>>
>> I have at least two assumptions I want to check here:
>>
>> 1.  TIMEOUT is defined as when when the telnet server just stops
>> spending data, but the telnet connection still exists, and TIMEOUT is
>> exceeded.  i.e. the far side data collection died, but left the server
>> connected to my client.  Is this correct?
> 
> 	No... The far side just doesn't have data to send... Imagine that the
> far side is sending stuff being typed in by a user at a console (in
> non-buffered mode). The user walks away to get a cup of coffee, and your
> end times out with whatever the user last entered. Then they come back and,
> maybe, enter a few more characters and hit <enter>.
> 
>>
>> 2.  I assume the run pointer, (where the next instruction is run), just
>> drops down to the "Rest of script..." and the run continues, if TIMEOUT
>> is reached.  Is this correct?
>>
> 
> 	On timeout, the next statement executed will be the "if result == b'':"
>>
>> Any help would be appreciated, I am new to Python...  I have looked all
> 
> 	This is not really Python specific -- you probably need to study the
> BSD TCP socket protocol, and maybe telnet protocol(s)
> https://www.rfc-editor.org/rfc/rfc854
> https://www.rfc-editor.org/rfc/rfc5198
> 
>> over and have not located anything that tells me what a TIMEOUT is
>> exactly.  Is it if the telnet server drops, is it if the data flow
>> stops, but the telnet server is still connected, etc...
> 
> 	For .read_until(), a closed connection raises EOFError. It may not be
> immediately detected (at the socket layer, there may be a buffer containing
> multiple lines of text and your .read_until() needs to empty that buffer
> before an actual socket read is next attempted).
> 
> 

From wlfraed at ix.netcom.com  Wed Mar  1 12:10:50 2023
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Wed, 01 Mar 2023 12:10:50 -0500
Subject: [Tutor] Where does the program pointer go when...
References: <32cdf823-8ee7-6c33-37bb-f4452919ba1f@nk7z.net>
 <ttm750$efn$1@ciao.gmane.io> <cb2db637-5213-5618-6771-14faba71acb0@nk7z.net>
Message-ID: <cr0vvh9v6m3a1t0phg8pqeodrcnla3nqd9@4ax.com>

On Tue, 28 Feb 2023 19:46:23 -0800, "Dave (NK7Z)" <dave at nk7z.net> declaimed
the following:

>and move to something else...  Can't use SSH, as the servers I am 
>interacting with do not use SSH...  Any suggestions on what lib I should 
>move to in that case?
>

	https://peps.python.org/pep-0594/
"""
telnetlib 	3.11 (3.0*) 	3.13 	1997 	no 	telnetlib3, Exscript
"""
https://pypi.org/project/telnetlib3/
https://telnetlib3.readthedocs.io/en/latest/

	Note that this uses asyncio, which may be a whole 'nother kettle of
fish to figure out.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/


From PythonList at DancesWithMice.info  Tue Mar  7 21:44:25 2023
From: PythonList at DancesWithMice.info (dn)
Date: Wed, 8 Mar 2023 15:44:25 +1300
Subject: [Tutor] ANN: The long white computing cloud
Message-ID: <006e967b-4085-2c3c-cbad-33ade135cc04@DancesWithMice.info>

The long white computing cloud
- hybrid meeting, ie both in-person and virtual attendance
Wed 15 March, 1800 for 1830 (NZDT) = 0530 UTC


1 Cloud and Proud - Digital Sovereignty in Aotearoa
Doug Dixon, CEO of Catalyst Cloud, Aotearoa New Zealand's cloud provider.

2 Python in the cloud
How to get-started with the admin-stuff, and build a toy-application 
live-demo.
(what could possibly go wrong!?)


NB This will be a hybrid event and RSVPs are requested: 
https://www.meetup.com/nzpug-auckland/events/291789444/


All welcome!
-- 
Regards,
=dn

From sjeik_appie at hotmail.com  Sun Mar 19 05:30:03 2023
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sun, 19 Mar 2023 10:30:03 +0100
Subject: [Tutor] Inherit from SyntaxError?
Message-ID: <DB6PR01MB38956FC12DD3C897650E276283839@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   Hi,
   I'm writing a parser using Beazly's PLY package. I defined a custom
   exception (actually, two: one for the lexer and one for the parser). They
   inherit from SyntaxError. Along with a few other exceptions (e.g.
   MemoryError), I've always regarded SyntaxError as "special". Are there any
   disadvantages to inheriting from SyntaxError? I could also inherit from
   ValueError.
   Thanks!
   Albert-Jan

From alan.gauld at yahoo.co.uk  Sun Mar 19 10:48:56 2023
From: alan.gauld at yahoo.co.uk (Alan Gauld)
Date: Sun, 19 Mar 2023 14:48:56 +0000
Subject: [Tutor] Inherit from SyntaxError?
In-Reply-To: <DB6PR01MB38956FC12DD3C897650E276283839@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
References: <DB6PR01MB38956FC12DD3C897650E276283839@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
Message-ID: <tv77co$9ql$1@ciao.gmane.io>

On 19/03/2023 09:30, Albert-Jan Roskam wrote:

>    MemoryError), I've always regarded SyntaxError as "special". Are there any
>    disadvantages to inheriting from SyntaxError? I could also inherit from
>    ValueError.

I believe it should work OK but I also believe (possibly wrongly!)
that SyntaxError is only ever thrown by the compiler prior to
commencing execution of the code, so you would never normally
be catching it and making inheritance of dubious value. But if
it truly is intended to indicate a syntax error in your parsed
data then I don't think inheriting from SyntaxError will cause
any particular problems.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos




From cs at cskk.id.au  Sun Mar 19 18:06:00 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Mon, 20 Mar 2023 09:06:00 +1100
Subject: [Tutor] Inherit from SyntaxError?
In-Reply-To: <DB6PR01MB38956FC12DD3C897650E276283839@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
References: <DB6PR01MB38956FC12DD3C897650E276283839@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
Message-ID: <ZBeHSAB65HqCZWCa@cskk.homeip.net>

On 19Mar2023 10:30, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
>   I'm writing a parser using Beazly's PLY package. I defined a custom
>   exception (actually, two: one for the lexer and one for the parser). They
>   inherit from SyntaxError. Along with a few other exceptions (e.g.
>   MemoryError), I've always regarded SyntaxError as "special". Are there any
>   disadvantages to inheriting from SyntaxError? I could also inherit from
>   ValueError.

Sounds good to me. I've got a class which inherits from SyntaxError:

     class ParseError(SyntaxError):
       ''' A ParseError subclasses SyntaxError in order to change the initialiser.
           This object has an additional attribute .context for the relevant FileContext
           (which has a .parent attribute).
       '''

       def __init__(self, context, offset, message, *a):
         ''' Initialise a ParseError given a FileContext and the offset into `context.text`.
             Accept optional arguments `*a` after the `message`; if supplied these
             are embedded into `message` with %-formatting.
         '''
         if a:
           message = message % a
         self.msg = message
         self.filename = context.filename
         self.lineno = context.lineno
         self.text = context.text
         self.offset = offset
         self.context = context

I've had no problems, and it seems entirely sensible.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From sjeik_appie at hotmail.com  Tue Mar 21 13:07:19 2023
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Tue, 21 Mar 2023 18:07:19 +0100
Subject: [Tutor] Inherit from SyntaxError?
In-Reply-To: <ZBeHSAB65HqCZWCa@cskk.homeip.net>
Message-ID: <DB6PR01MB38952B3C6F051ED3A92C68E083819@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   On Mar 19, 2023 23:06, Cameron Simpson <cs at cskk.id.au> wrote:

     On 19Mar2023 10:30, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
     >?? I'm writing a parser using Beazly's PLY package. I defined a custom
     >?? exception (actually, two: one for the lexer and one for the parser).
     They
     >?? inherit from SyntaxError. Along with a few other exceptions (e.g.
     >?? MemoryError), I've always regarded SyntaxError as "special". Are
     there any
     >?? disadvantages to inheriting from SyntaxError? I could also inherit
     from
     >?? ValueError.

     Sounds good to me. I've got a class which inherits from SyntaxError:

     ???? class ParseError(SyntaxError):
     ?????? ''' A ParseError subclasses SyntaxError in order to change the
     initialiser.
     ?????????? This object has an additional attribute .context for the
     relevant FileContext
     ?????????? (which has a .parent attribute).
     ?????? '''

     ?????? def __init__(self, context, offset, message, *a):
     ???????? ''' Initialise a ParseError given a FileContext and the offset
     into `context.text`.
     ???????????? Accept optional arguments `*a` after the `message`; if
     supplied these
     ???????????? are embedded into `message` with %-formatting.
     ???????? '''
     ???????? if a:
     ?????????? message = message % a
     ???????? self.msg = message
     ???????? self.filename = context.filename
     ???????? self.lineno = context.lineno
     ???????? self.text = context.text
     ???????? self.offset = offset
     ???????? self.context = context

     I've had no problems, and it seems entirely sensible.

   ======
   Hi Alan, Cameron,
   Thanks! I now created one BaseParserError (inherits from SyntaxError) and
   a lexer + parser error (they each inherit from BaseParserError).
   So, is MemoryError the only really tricky exception? I mean, the
   interpreter might just die before it is able to recover from the
   MemoryError?
   Best wishes,
   Albert-Jan

From mats at wichmann.us  Tue Mar 21 14:09:04 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Tue, 21 Mar 2023 12:09:04 -0600
Subject: [Tutor] Inherit from SyntaxError?
In-Reply-To: <DB6PR01MB38952B3C6F051ED3A92C68E083819@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
References: <DB6PR01MB38952B3C6F051ED3A92C68E083819@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
Message-ID: <d132c474-5384-d3b6-eb23-5c502fc529ca@wichmann.us>

On 3/21/23 11:07, Albert-Jan Roskam wrote:
>     On Mar 19, 2023 23:06, Cameron Simpson <cs at cskk.id.au> wrote:
> 
>       On 19Mar2023 10:30, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
>       >?? I'm writing a parser using Beazly's PLY package. I defined a custom
>       >?? exception (actually, two: one for the lexer and one for the parser).
>       They
>       >?? inherit from SyntaxError. Along with a few other exceptions (e.g.
>       >?? MemoryError), I've always regarded SyntaxError as "special". Are
>       there any
>       >?? disadvantages to inheriting from SyntaxError? I could also inherit
>       from
>       >?? ValueError.
> 
>       Sounds good to me. I've got a class which inherits from SyntaxError:
> 
>       ???? class ParseError(SyntaxError):
>       ?????? ''' A ParseError subclasses SyntaxError in order to change the
>       initialiser.
>       ?????????? This object has an additional attribute .context for the
>       relevant FileContext
>       ?????????? (which has a .parent attribute).
>       ?????? '''
> 
>       ?????? def __init__(self, context, offset, message, *a):
>       ???????? ''' Initialise a ParseError given a FileContext and the offset
>       into `context.text`.
>       ???????????? Accept optional arguments `*a` after the `message`; if
>       supplied these
>       ???????????? are embedded into `message` with %-formatting.
>       ???????? '''
>       ???????? if a:
>       ?????????? message = message % a
>       ???????? self.msg = message
>       ???????? self.filename = context.filename
>       ???????? self.lineno = context.lineno
>       ???????? self.text = context.text
>       ???????? self.offset = offset
>       ???????? self.context = context
> 
>       I've had no problems, and it seems entirely sensible.
> 
>     ======
>     Hi Alan, Cameron,
>     Thanks! I now created one BaseParserError (inherits from SyntaxError) and
>     a lexer + parser error (they each inherit from BaseParserError).
>     So, is MemoryError the only really tricky exception? I mean, the
>     interpreter might just die before it is able to recover from the
>     MemoryError?

KeyboardInterrupt and SystemExit are a little tricky, though the way 
they're set up should cause few problems (they don't inherit from 
Exception, as you normally want to handle them separately and not catch 
them in a general catch of Exception).

MemoryError, *in theory* is raised if there's a chance of recovery. But 
there are situations, I presume, where external forces might get you - 
the operating system decides to kill Python.


From cs at cskk.id.au  Tue Mar 21 17:11:39 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Wed, 22 Mar 2023 08:11:39 +1100
Subject: [Tutor] Inherit from SyntaxError?
In-Reply-To: <DB6PR01MB38952B3C6F051ED3A92C68E083819@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
References: <DB6PR01MB38952B3C6F051ED3A92C68E083819@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
Message-ID: <ZBodiwpKfBvzryq/@cskk.homeip.net>

On 21Mar2023 18:07, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
>   Thanks! I now created one BaseParserError (inherits from 
>   SyntaxError)
>   and a lexer + parser error (they each inherit from BaseParserError).

Sounds good.

>   So, is MemoryError the only really tricky exception? I mean, the
>   interpreter might just die before it is able to recover from the
>   MemoryError?

You probably can't do anything about this. What kind of recovery would 
you do? And why?

Remember the rule: only catch things you have a meaningful action to 
catch (there are reare exceptions, usually around daemon like activity 
where you're running some task: catch and log the exception, and poceeed 
to the next task/request - you're not "recovering" or repairing here, 
just not terminating some outer framework).

What's tricky about a MemoryException? Just let it out, like a bunch of 
other "system" type exceptions. Can you describe what you want to do 
with MemoryError?

Cheers,
Cameron Simpson <cs at cskk.id.au>

From sjeik_appie at hotmail.com  Thu Mar 23 04:17:03 2023
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Thu, 23 Mar 2023 09:17:03 +0100
Subject: [Tutor] Inherit from SyntaxError?
In-Reply-To: <ZBodiwpKfBvzryq/@cskk.homeip.net>
Message-ID: <DB6PR01MB38956893C47BADF347302F0783879@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   On Mar 21, 2023 22:11, Cameron Simpson <cs at cskk.id.au> wrote:

     On 21Mar2023 18:07, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
     >?? Thanks! I now created one BaseParserError (inherits from
     >?? SyntaxError)
     >?? and a lexer + parser error (they each inherit from BaseParserError).

     Sounds good.

     >?? So, is MemoryError the only really tricky exception? I mean, the
     >?? interpreter might just die before it is able to recover from the
     >?? MemoryError?

     You probably can't do anything about this. What kind of recovery would
     you do? And why?

     Remember the rule: only catch things you have a meaningful action to
     catch (there are reare exceptions, usually around daemon like activity
     where you're running some task: catch and log the exception, and poceeed
     to the next task/request - you're not "recovering" or repairing here,
     just not terminating some outer framework).

     What's tricky about a MemoryException? Just let it out, like a bunch of
     other "system" type exceptions. Can you describe what you want to do
     with MemoryError?

   ====
   Hi Cameron,
   Sometimes catching MemoryError might come in handy with a "inside/outside
   RAM" strategy. In-memory is faster, but not scalable.
   For instance, something like:
   try:
   ? ? d = {}
   ? ? for record in records:
   ? ? ? ? d[record.id] = record.something
   except MemoryError:
   ? ? logger.warning("Using slower shelve route")
   ? ? d = shelve.open("/tmp/lookup.shelve", writeback=True)
   ? ? atexit.register(d.close)
   ? ? for record in records:
   ? ? ? ? d[record.id] = record.something
   If you know that "records" is always huge, this approach is needlessly
   complicated. But if it *might* be too big for some machines, it might be
   nice not having to choose the slow approach, just to be on the safe side.
   Best wishes,
   Albert-Jan

From mhysnm1964 at gmail.com  Thu Mar 23 05:14:47 2023
From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com)
Date: Thu, 23 Mar 2023 20:14:47 +1100
Subject: [Tutor] Python WX module program issues.
Message-ID: <003701d95d67$ebf818e0$c3e84aa0$@GMAIL.COM>

All,

 

I have created a simple Python 3.10 app using WX module. There is a main
frame and two horizontal panels. Panel1 has an edit object in it. Panel2 has
a list box. The issues I cannot resolve:
1. I cannot add any new content to the status bar from the class EditPanel. 

2. The RecordListPanel does not show any content and the database is
populated.

3.  F6 is meant to switch between the two panels and set the keyboard focus
to the next panel.

 

There is some issues with the saveDb method in the EditPanel Class which I
am still working on. If record key publisher has no values, it receives
"None". The Publisher table has unique records set to prevent duplication.
The same applies to authors, genre, narrator.

 

Can someone look at this code an let me know why point 1 to 3 isn't working?

 

 

import wx, os, codecs, chardet

from books_db import Book, Author, Narrator, Series, Publisher, Genre,
Session

from sqlalchemy.exc import IntegrityError

 

from books_db import Book, Author, Narrator, Series, Publisher, Genre,
Session

 

class RecordListPanel(wx.Panel):

    def __init__(self, parent):

        wx.Panel.__init__(self, parent=parent)

 

        self.list_ctrl = wx.ListCtrl(self, style=wx.LC_REPORT)

        self.list_ctrl.InsertColumn(0, 'Title', width=200)

        self.list_ctrl.InsertColumn(1, 'Author', width=200)

        self.list_ctrl.InsertColumn(2, 'Genre', width=200)

        self.list_ctrl.InsertColumn(3, 'Narrator', width=200)

        self.list_ctrl.InsertColumn(4, 'Publisher', width=200)

        self.list_ctrl.InsertColumn(5, 'Series', width=200)

        self.list_ctrl.InsertColumn(6, 'Series No.', width=100)

        self.populate_list_ctrl()

        #sizer = wx.BoxSizer(wx.VERTICAL)

        #sizer.Add(self.list_ctrl, 1, wx.EXPAND)

        #self.SetSizer(sizer)

 

    def populate_list_ctrl(self):

        session = Session()

        books = session.query(Book).all()

 

        for book in books:

            row = [book.title]

 

            # authors

            authors = ', '.join([author.author for author in book.authors])

            row.append(authors)

 

            # genres

            genres = ', '.join([genre.genre for genre in book.genres])

            row.append(genres)

 

           # narrators

            narrators = ', '.join([narrator.narrator for narrator in
book.narrators])

            row.append(narrators)

 

            row.append(book.publisher.publisher)

            row.append(book.series.series)

            row.append(str(book.series_no))

 

            self.list_ctrl.Append(row)

        session.close()

 

class EditPanel(wx.Panel):

    def __init__(self, parent):

        wx.Panel.__init__(self, parent=parent)

 

        # Initialize variables

        self.books = []

        self.current_book_index = 0

        self.record = {}

 

        # Create controls

        self.edit = wx.TextCtrl(self,
style=wx.TE_MULTILINE|wx.SUNKEN_BORDER)

        # Load the books

        self.load_books_paths()

        # Load the first book into the edit field

        self.load_book()

 

        #sizer = wx.BoxSizer(wx.VERTICAL)

        #sizer.Add(self.edit, 1, wx.EXPAND)

        #self.SetSizer(sizer)

 

    def load_books_paths(self):

        for root, dirs, files in os.walk("c:\\books"):

            for filename in files:

                if filename.endswith(".txt"):

                    filepath = os.path.join(root, filename)

                    self.books.append(filepath)

        print("loaded file paths ")

 

    def load_book(self):

        # Get the current book file path

        current_book = self.books[self.current_book_index]

        self.record['path'] = current_book

        with open(current_book, 'rb') as f:

            contents = f.read()

        encoding = chardet.detect(contents)['encoding']

 

        try:

            with codecs.open(current_book, "r", encoding=encoding) as f:

                contents = f.read()

        except UnicodeDecodeError:

            content = "Unknown file type for " + current_book

 

        # Load the contents of the file into the text box

        self.edit.SetValue(contents)

 

    def priorBook(self, event):

        self.record.clear() # initialize record.

        # Move to the prior  book and load it

        self.current_book_index -= 1

        if self.current_book_index < 0: # cannot pass 0.

            dlg = wx.MessageDialog(self, "Beginning of book path",
caption="Record Warning", style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)

            dlg.ShowModal()

            return 

        self.load_book()

        prior_file = self.books[self.current_book_index]

        self.GetTopWindow().SetStatusText(f"Prior    file: {prior_file}")

 

    def nextBook(self, event=None):

        self.record.clear() # initialize record.

        # Move to the next book and load it

        self.current_book_index += 1

        if self.current_book_index >= len(self.books):

            dlg = wx.MessageDialog(self, "end  of book path",
caption="Record Warning", style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)

            dlg.ShowModal()

            return 

        self.load_book()

        # Update status bar with path of next file

        next_file = self.books[self.current_book_index]

        self.GetTopWindow().SetStatusText(f"Next file: {next_file}")

 

    def titleSave(self, event):

        selected_text = self.edit.GetStringSelection()

        self.record['title'] = selected_text.strip()

        self.GetTopWindow().SetStatusText(f"title: {selected_text}")

 

    def authorSave(self, event):

        selected_text = self.edit.GetStringSelection()

        if 'author' not in self.record:

            self.record['author'] = []

        self.record['author'].append(selected_text.strip())

        self.GetTopWindow().SetStatusText(f"Author: {selected_text}")

 

    def aboutAuthorSave(self, event):

        selected_text = self.edit.GetStringSelection()

        if 'about_author' not in self.record:

            self.record['about_author'] = selected_text.strip()

        else:

            self.record['about_author'] = self.record['about_author'] + '\n'
+ selected_text.strip()

        self.GetTopWindow().SetStatusText("About Author saved")

 

    def descriptionSave(self, event):

        selected_text = self.edit.GetStringSelection()

        if 'description' not in self.record:

            self.record['description'] = selected_text.strip()

        else:

            self.record['description'] = self.record['description'] + "\n" +
selected_text.strip()

        self.GetTopWindow().SetStatusText("Description saved")

 

    def seriesSave(self, event):

        selected_text = self.edit.GetStringSelection()

        self.record['series'] = selected_text.strip()

        self.GetTopWindow().SetStatusText(f"Series: {selected_text}")

 

    def seriesNoSave(self, event):

        selected_text = self.edit.GetStringSelection()

        self.record['series_no'] = int(selected_text.strip())

        self.GetTopWindow().SetStatusText(f"Series Number: {selected_text}")

 

    def narratorSave(self, event):

        selected_text = self.edit.GetStringSelection()

        if 'narrator' not in self.record:

            self.record['narrator'] = []

        self.record['narrator'].append(selected_text.strip())

        self.GetTopWindow().SetStatusText(f"Narrator: {selected_text}")

 

    def miscSave(self, event):

        selected_text = self.edit.GetStringSelection()

        if 'misc' not in self.record:

            self.record['misc'] = selected_text.strip()

        else:

            self.record['misc'] = self.record['misc'] + "\n" +
selected_text.strip()

        self.GetTopWindow().SetStatusText("Misc saved")

 

    def genreSave(self, event):

        selected_text = self.edit.GetStringSelection()

        if 'genre' not in self.record:

            self.record['genre'] = []

        self.record['genre'].append(selected_text.strip())

        self.GetTopWindow().SetStatusText(f"Genre: {selected_text}")

 

    def publisherSave(self, event):

        selected_text = self.edit.GetStringSelection()

        self.record['publisher'] = selected_text.strip()

        self.GetTopWindow().SetStatusText(f"Publisher: {selected_text}")

 

    def durationSave(self, event):

        selected_text = self.edit.GetStringSelection()

        self.record['duration'] = selected_text.strip()

        self.GetTopWindow().SetStatusText(f"Duration saved:
{selected_text}")

 

    def releaseSave(self, event):

        selected_text = self.edit.GetStringSelection()

        self.record['release'] = selected_text.strip()

        self.GetTopWindow().SetStatusText(f"Release: {selected_text}")

 

    def copyrightSave(self, event):

        selected_text = self.edit.GetStringSelection()

        self.record['copyright'] = selected_text.strip()

        self.GetTopWindow().SetStatusText(f"Copyright: {selected_text}")

 

    def reviewDb(self, event):

    # review the dict

        print (self.record)

        msgString = ""

        for key, value in self.record.items():

            if isinstance(value, int):

                value = str(value)

            elif isinstance(value, list) and value:

                for v in value:

                    v += "\n"

            msgString += " ".join([key, value]) + "\n"

        dlg = wx.MessageDialog(self, msgString, caption="Record Content",
style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)

        dlg.ShowModal()

 

    def saveDb(self, event):

        if self.record.get('title') is None or self.record.get('author') is
None or self.record.get('description') is None:

            dlg = wx.MessageDialog(self, "Cannot save update database. Not
all required fields have been defined.", caption="Record Warning",
style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)

            dlg.ShowModal()

            return

        elif self.record.get('genre') is None:

            self.record.get['genre'] =  "None"

        elif   self.record.get('narrator') is None: 

            self.record['narrator'] = []

            self.record['narrator'].append("None")

        elif self.record.get('publisher') is None:

            self.record['publisher'] = "None"

        elif self.record.get('title') == '' or
self.record.get('description') == '':

            dlg = wx.MessageDialog(self, "Title and description   are
required before record can be  saved  to the database. ", caption="Record
Warning", style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)

            dlg.ShowModal()

            return

 

        # establish session with database.

        session = Session()

        try:

            # create objects

            publisher  =
session.query(publisher).filter_by(publisher=self.record.get('publisher')).f
irst()

            if publisher is None:

                publisher = Publisher(publisher=self.record.get('publisher',
''))

 

            # check if author exists

            authors = []

            for author_name in self.record.get('author', []):

                author =
session.query(Author).filter_by(author=author_name).first()

                if author is None:

                    author = Author(author=author_name,
about_author=self.record.get('about_author', ''))

                authors.append(author)

 

            # check if genre exists

            genres = []

            for genre_name in self.record.get('genre', []):

                genre =
session.query(Genre).filter_by(genre=genre_name).first()

                if genre is None:

                    genre = Genre(genre=genre_name)

                genres.append(genre)

 

            # check if narrator exists

            narrators = []

            for narrator_name in self.record.get('narrator', []):

                narrator =
session.query(Narrator).filter_by(narrator=narrator_name).first()

                if narrator is None:

                    narrator = Narrator(narrator=narrator_name)

                narrators.append(narrator)

 

            series = Series(series=self.record.get('series', ''))

            book = Book(title=self.record.get('title', ''),

                        path=self.record.get('path', ''),

                        release=self.record.get('release', ''),

                        duration=self.record.get('duration', ''),

                        copyright=self.record.get('copyright', ''),

                        description=self.record.get('description', ''),

                        misc=self.record.get('misc', ''),

                        series_no=self.record.get('series_no', 0))

 

            book.publisher = publisher

            book.authors = authors

            book.narrators = narrators

            book.genres = genres

            book.series = series

 

            # add objects to session

            session.add_all([book, publisher, series] + authors + narrators
+ genres)

            # commit changes

            session.commit()

            self.GetTopWindow().SetStatusText("Record saved successfully")

        except Exception as e:

            session.rollback()

            print(f"Error occurred: {e}")

            self.GetTopWindow().SetStatusText("Record did not saved
successfully")

        finally:

            session.close()

 

    def clearRecord(self, event):

        self.record.clear()

        self.GetTopWindow().SetStatusText("Record in memory cleared
successfully")

 

class MyApp(wx.App):

    def OnInit(self):

        # Create the main frame

        self.frame = wx.Frame(None, title="Library", size=(1920, 1080 ))

        self.frame.Maximize()

        sizer_main = wx.BoxSizer(wx.VERTICAL)

        # Create the two panels

        self.panel1 = EditPanel(self.frame)

        self.panel2 = RecordListPanel(self.frame)

        # Create the horizontal box sizers for the panels

        sizer_panel1 = wx.BoxSizer(wx.HORIZONTAL)

        sizer_panel2 = wx.BoxSizer(wx.HORIZONTAL)

 

 

        # Create the menubar

        menubar = wx.MenuBar()

        # Create the "Options" menu

        options_menu = wx.Menu()

        save_record_item = options_menu.Append(wx.ID_ANY, "Save
Record\tCtrl+S", "Save to database")

        self.Bind(wx.EVT_MENU,self.panel1.saveDb, save_record_item)

        next_file_item = options_menu.Append(wx.ID_ANY, "Next file\tCtrl+N",
"Load next file")

        prior_file_item = options_menu.Append(wx.ID_ANY, "Prior
file\tCtrl+P", "Load prior file")

        review_record_item = options_menu.Append(wx.ID_ANY, "Review
Record\tCtrl+r", "Review  record")

        self.Bind(wx.EVT_MENU,self.panel1.reviewDb, review_record_item)

        self.Bind(wx.EVT_MENU,self.panel1.nextBook, next_file_item)

        self.Bind(wx.EVT_MENU,self.panel1.priorBook, prior_file_item)

        clear_record_item = options_menu.Append(wx.ID_ANY, "Clear
Record\Alt+I", "Clear record in memory")

        self.Bind(wx.EVT_MENU,self.panel1.clearRecord, clear_record_item)

        options_menu.Append(wx.ID_EXIT, "Exit", "Exit the program")

        menubar.Append(options_menu, "&Options")

 

        # Create the "Save to record" menu

        save_menu = wx.Menu()

        title_item = save_menu.Append(wx.ID_ANY, "Title\tAlt+T", "Save book
title")

        self.Bind(wx.EVT_MENU,self.panel1.titleSave, title_item)

        author_item = save_menu.Append(wx.ID_ANY, "Author\tAlt+A", "Save
book author")

        self.Bind(wx.EVT_MENU,self.panel1.authorSave, author_item)

        series_item = save_menu.Append(wx.ID_ANY, "Series\tAlt+S", "Save
book Series")

        self.Bind(wx.EVT_MENU,self.panel1.seriesSave, series_item)

        seriesNo_item = save_menu.Append(wx.ID_ANY, "SeriesNo\tCtrl+3",
"Save Series No")

        self.Bind(wx.EVT_MENU,self.panel1.seriesNoSave, seriesNo_item)

        genre_item = save_menu.Append(wx.ID_ANY, "Genre\tAlt+G", "Save book
genre")

        self.Bind(wx.EVT_MENU,self.panel1.genreSave, genre_item)

        description_item = save_menu.Append(wx.ID_ANY, "Description\tAlt+D",
"Save book description")

        self.Bind(wx.EVT_MENU,self.panel1.descriptionSave, description_item)

        aboutAuthor_item = save_menu.Append(wx.ID_ANY, "About
Author\tAlt+B", "Save about author")

        self.Bind(wx.EVT_MENU,self.panel1.aboutAuthorSave, aboutAuthor_item)

        narrator_item = save_menu.Append(wx.ID_ANY, "Narrator\tAlt+N", "Save
book narrator")

        self.Bind(wx.EVT_MENU,self.panel1.narratorSave, narrator_item)

        release_item = save_menu.Append(wx.ID_ANY, "Release Date\tAlt+R",
"Save book release date")

        self.Bind(wx.EVT_MENU,self.panel1.releaseSave, release_item)

        duration_item = save_menu.Append(wx.ID_ANY, "Duration\tCtrl+D",
"Save book duration")

        self.Bind(wx.EVT_MENU,self.panel1.durationSave, duration_item)

        copyright_item = save_menu.Append(wx.ID_ANY, "Copyright\tAlt+C",
"Save book copyright")

        self.Bind(wx.EVT_MENU,self.panel1.copyrightSave, copyright_item)

        publisher_item = save_menu.Append(wx.ID_ANY, "Publisher\tAlt+P",
"Save book publisher")

        self.Bind(wx.EVT_MENU,self.panel1.publisherSave, publisher_item)

        misc_item = save_menu.Append(wx.ID_ANY, "Misc\tAlt+M", "Save book
misc")

        self.Bind(wx.EVT_MENU,self.panel1.miscSave, misc_item)

        menubar.Append(save_menu, "Sa&ve to record")

 

        self.frame.SetMenuBar(menubar)

 

        # Create the status bar

        self.frame.CreateStatusBar()

        #self.statusbar.SetFieldsCount(3)

        #self.statusbar.SetStatusWidths([-2, -1, -1])

 

# Set up the panel switching

        self.currentPanel = self.panel1

        self.panel1.SetFocus()

        self.frame.Bind(wx.EVT_KEY_DOWN, self.onKeyDown)

        self.frame.Show()

        self.SetTopWindow(self.frame)

        

        return True

 

    def onKeyDown(self, event):

        if event.GetKeyCode() == wx.WXK_F6:

            if self.currentPanel == self.panel1:

                self.currentPanel = self.panel2

                self.panel2.SetFocus()

            else:

                self.currentPanel = self.panel1

                self.panel1.SetFocus()

        else:

            event.Skip()

 

        # set focus to the current panel.

        self.currentPanel.SetFocus()

 

if __name__ == '__main__':

    # Create the wx

    app = MyApp()

 

    # Start the main event loop

    app.MainLoop()


From johnf at jfcomputer.com  Thu Mar 23 09:23:10 2023
From: johnf at jfcomputer.com (john fabiani)
Date: Thu, 23 Mar 2023 06:23:10 -0700
Subject: [Tutor] Python WX module program issues.
In-Reply-To: <003701d95d67$ebf818e0$c3e84aa0$@GMAIL.COM>
References: <003701d95d67$ebf818e0$c3e84aa0$@GMAIL.COM>
Message-ID: <3db331e7-b071-84fd-ce4f-0db42d758646@jfcomputer.com>

You need to join the wxPython forum ( https://discuss.wxpython.org).? 
I'm sure they can answer your questions.
Johnf

On 3/23/23 2:14 AM, mhysnm1964 at gmail.com wrote:
> All,
>
>   
>
> I have created a simple Python 3.10 app using WX module. There is a main
> frame and two horizontal panels. Panel1 has an edit object in it. Panel2 has
> a list box. The issues I cannot resolve:
> 1. I cannot add any new content to the status bar from the class EditPanel.
>
> 2. The RecordListPanel does not show any content and the database is
> populated.
>
> 3.  F6 is meant to switch between the two panels and set the keyboard focus
> to the next panel.
>
>   
>
> There is some issues with the saveDb method in the EditPanel Class which I
> am still working on. If record key publisher has no values, it receives
> "None". The Publisher table has unique records set to prevent duplication.
> The same applies to authors, genre, narrator.
>
>   
>
> Can someone look at this code an let me know why point 1 to 3 isn't working?
>
>   
>
>   
>
> import wx, os, codecs, chardet
>
> from books_db import Book, Author, Narrator, Series, Publisher, Genre,
> Session
>
> from sqlalchemy.exc import IntegrityError
>
>   
>
> from books_db import Book, Author, Narrator, Series, Publisher, Genre,
> Session
>
>   
>
> class RecordListPanel(wx.Panel):
>
>      def __init__(self, parent):
>
>          wx.Panel.__init__(self, parent=parent)
>
>   
>
>          self.list_ctrl = wx.ListCtrl(self, style=wx.LC_REPORT)
>
>          self.list_ctrl.InsertColumn(0, 'Title', width=200)
>
>          self.list_ctrl.InsertColumn(1, 'Author', width=200)
>
>          self.list_ctrl.InsertColumn(2, 'Genre', width=200)
>
>          self.list_ctrl.InsertColumn(3, 'Narrator', width=200)
>
>          self.list_ctrl.InsertColumn(4, 'Publisher', width=200)
>
>          self.list_ctrl.InsertColumn(5, 'Series', width=200)
>
>          self.list_ctrl.InsertColumn(6, 'Series No.', width=100)
>
>          self.populate_list_ctrl()
>
>          #sizer = wx.BoxSizer(wx.VERTICAL)
>
>          #sizer.Add(self.list_ctrl, 1, wx.EXPAND)
>
>          #self.SetSizer(sizer)
>
>   
>
>      def populate_list_ctrl(self):
>
>          session = Session()
>
>          books = session.query(Book).all()
>
>   
>
>          for book in books:
>
>              row = [book.title]
>
>   
>
>              # authors
>
>              authors = ', '.join([author.author for author in book.authors])
>
>              row.append(authors)
>
>   
>
>              # genres
>
>              genres = ', '.join([genre.genre for genre in book.genres])
>
>              row.append(genres)
>
>   
>
>             # narrators
>
>              narrators = ', '.join([narrator.narrator for narrator in
> book.narrators])
>
>              row.append(narrators)
>
>   
>
>              row.append(book.publisher.publisher)
>
>              row.append(book.series.series)
>
>              row.append(str(book.series_no))
>
>   
>
>              self.list_ctrl.Append(row)
>
>          session.close()
>
>   
>
> class EditPanel(wx.Panel):
>
>      def __init__(self, parent):
>
>          wx.Panel.__init__(self, parent=parent)
>
>   
>
>          # Initialize variables
>
>          self.books = []
>
>          self.current_book_index = 0
>
>          self.record = {}
>
>   
>
>          # Create controls
>
>          self.edit = wx.TextCtrl(self,
> style=wx.TE_MULTILINE|wx.SUNKEN_BORDER)
>
>          # Load the books
>
>          self.load_books_paths()
>
>          # Load the first book into the edit field
>
>          self.load_book()
>
>   
>
>          #sizer = wx.BoxSizer(wx.VERTICAL)
>
>          #sizer.Add(self.edit, 1, wx.EXPAND)
>
>          #self.SetSizer(sizer)
>
>   
>
>      def load_books_paths(self):
>
>          for root, dirs, files in os.walk("c:\\books"):
>
>              for filename in files:
>
>                  if filename.endswith(".txt"):
>
>                      filepath = os.path.join(root, filename)
>
>                      self.books.append(filepath)
>
>          print("loaded file paths ")
>
>   
>
>      def load_book(self):
>
>          # Get the current book file path
>
>          current_book = self.books[self.current_book_index]
>
>          self.record['path'] = current_book
>
>          with open(current_book, 'rb') as f:
>
>              contents = f.read()
>
>          encoding = chardet.detect(contents)['encoding']
>
>   
>
>          try:
>
>              with codecs.open(current_book, "r", encoding=encoding) as f:
>
>                  contents = f.read()
>
>          except UnicodeDecodeError:
>
>              content = "Unknown file type for " + current_book
>
>   
>
>          # Load the contents of the file into the text box
>
>          self.edit.SetValue(contents)
>
>   
>
>      def priorBook(self, event):
>
>          self.record.clear() # initialize record.
>
>          # Move to the prior  book and load it
>
>          self.current_book_index -= 1
>
>          if self.current_book_index < 0: # cannot pass 0.
>
>              dlg = wx.MessageDialog(self, "Beginning of book path",
> caption="Record Warning", style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)
>
>              dlg.ShowModal()
>
>              return
>
>          self.load_book()
>
>          prior_file = self.books[self.current_book_index]
>
>          self.GetTopWindow().SetStatusText(f"Prior    file: {prior_file}")
>
>   
>
>      def nextBook(self, event=None):
>
>          self.record.clear() # initialize record.
>
>          # Move to the next book and load it
>
>          self.current_book_index += 1
>
>          if self.current_book_index >= len(self.books):
>
>              dlg = wx.MessageDialog(self, "end  of book path",
> caption="Record Warning", style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)
>
>              dlg.ShowModal()
>
>              return
>
>          self.load_book()
>
>          # Update status bar with path of next file
>
>          next_file = self.books[self.current_book_index]
>
>          self.GetTopWindow().SetStatusText(f"Next file: {next_file}")
>
>   
>
>      def titleSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          self.record['title'] = selected_text.strip()
>
>          self.GetTopWindow().SetStatusText(f"title: {selected_text}")
>
>   
>
>      def authorSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          if 'author' not in self.record:
>
>              self.record['author'] = []
>
>          self.record['author'].append(selected_text.strip())
>
>          self.GetTopWindow().SetStatusText(f"Author: {selected_text}")
>
>   
>
>      def aboutAuthorSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          if 'about_author' not in self.record:
>
>              self.record['about_author'] = selected_text.strip()
>
>          else:
>
>              self.record['about_author'] = self.record['about_author'] + '\n'
> + selected_text.strip()
>
>          self.GetTopWindow().SetStatusText("About Author saved")
>
>   
>
>      def descriptionSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          if 'description' not in self.record:
>
>              self.record['description'] = selected_text.strip()
>
>          else:
>
>              self.record['description'] = self.record['description'] + "\n" +
> selected_text.strip()
>
>          self.GetTopWindow().SetStatusText("Description saved")
>
>   
>
>      def seriesSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          self.record['series'] = selected_text.strip()
>
>          self.GetTopWindow().SetStatusText(f"Series: {selected_text}")
>
>   
>
>      def seriesNoSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          self.record['series_no'] = int(selected_text.strip())
>
>          self.GetTopWindow().SetStatusText(f"Series Number: {selected_text}")
>
>   
>
>      def narratorSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          if 'narrator' not in self.record:
>
>              self.record['narrator'] = []
>
>          self.record['narrator'].append(selected_text.strip())
>
>          self.GetTopWindow().SetStatusText(f"Narrator: {selected_text}")
>
>   
>
>      def miscSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          if 'misc' not in self.record:
>
>              self.record['misc'] = selected_text.strip()
>
>          else:
>
>              self.record['misc'] = self.record['misc'] + "\n" +
> selected_text.strip()
>
>          self.GetTopWindow().SetStatusText("Misc saved")
>
>   
>
>      def genreSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          if 'genre' not in self.record:
>
>              self.record['genre'] = []
>
>          self.record['genre'].append(selected_text.strip())
>
>          self.GetTopWindow().SetStatusText(f"Genre: {selected_text}")
>
>   
>
>      def publisherSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          self.record['publisher'] = selected_text.strip()
>
>          self.GetTopWindow().SetStatusText(f"Publisher: {selected_text}")
>
>   
>
>      def durationSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          self.record['duration'] = selected_text.strip()
>
>          self.GetTopWindow().SetStatusText(f"Duration saved:
> {selected_text}")
>
>   
>
>      def releaseSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          self.record['release'] = selected_text.strip()
>
>          self.GetTopWindow().SetStatusText(f"Release: {selected_text}")
>
>   
>
>      def copyrightSave(self, event):
>
>          selected_text = self.edit.GetStringSelection()
>
>          self.record['copyright'] = selected_text.strip()
>
>          self.GetTopWindow().SetStatusText(f"Copyright: {selected_text}")
>
>   
>
>      def reviewDb(self, event):
>
>      # review the dict
>
>          print (self.record)
>
>          msgString = ""
>
>          for key, value in self.record.items():
>
>              if isinstance(value, int):
>
>                  value = str(value)
>
>              elif isinstance(value, list) and value:
>
>                  for v in value:
>
>                      v += "\n"
>
>              msgString += " ".join([key, value]) + "\n"
>
>          dlg = wx.MessageDialog(self, msgString, caption="Record Content",
> style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)
>
>          dlg.ShowModal()
>
>   
>
>      def saveDb(self, event):
>
>          if self.record.get('title') is None or self.record.get('author') is
> None or self.record.get('description') is None:
>
>              dlg = wx.MessageDialog(self, "Cannot save update database. Not
> all required fields have been defined.", caption="Record Warning",
> style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)
>
>              dlg.ShowModal()
>
>              return
>
>          elif self.record.get('genre') is None:
>
>              self.record.get['genre'] =  "None"
>
>          elif   self.record.get('narrator') is None:
>
>              self.record['narrator'] = []
>
>              self.record['narrator'].append("None")
>
>          elif self.record.get('publisher') is None:
>
>              self.record['publisher'] = "None"
>
>          elif self.record.get('title') == '' or
> self.record.get('description') == '':
>
>              dlg = wx.MessageDialog(self, "Title and description   are
> required before record can be  saved  to the database. ", caption="Record
> Warning", style=wx.OK|wx.CENTER, pos=wx.DefaultPosition)
>
>              dlg.ShowModal()
>
>              return
>
>   
>
>          # establish session with database.
>
>          session = Session()
>
>          try:
>
>              # create objects
>
>              publisher  =
> session.query(publisher).filter_by(publisher=self.record.get('publisher')).f
> irst()
>
>              if publisher is None:
>
>                  publisher = Publisher(publisher=self.record.get('publisher',
> ''))
>
>   
>
>              # check if author exists
>
>              authors = []
>
>              for author_name in self.record.get('author', []):
>
>                  author =
> session.query(Author).filter_by(author=author_name).first()
>
>                  if author is None:
>
>                      author = Author(author=author_name,
> about_author=self.record.get('about_author', ''))
>
>                  authors.append(author)
>
>   
>
>              # check if genre exists
>
>              genres = []
>
>              for genre_name in self.record.get('genre', []):
>
>                  genre =
> session.query(Genre).filter_by(genre=genre_name).first()
>
>                  if genre is None:
>
>                      genre = Genre(genre=genre_name)
>
>                  genres.append(genre)
>
>   
>
>              # check if narrator exists
>
>              narrators = []
>
>              for narrator_name in self.record.get('narrator', []):
>
>                  narrator =
> session.query(Narrator).filter_by(narrator=narrator_name).first()
>
>                  if narrator is None:
>
>                      narrator = Narrator(narrator=narrator_name)
>
>                  narrators.append(narrator)
>
>   
>
>              series = Series(series=self.record.get('series', ''))
>
>              book = Book(title=self.record.get('title', ''),
>
>                          path=self.record.get('path', ''),
>
>                          release=self.record.get('release', ''),
>
>                          duration=self.record.get('duration', ''),
>
>                          copyright=self.record.get('copyright', ''),
>
>                          description=self.record.get('description', ''),
>
>                          misc=self.record.get('misc', ''),
>
>                          series_no=self.record.get('series_no', 0))
>
>   
>
>              book.publisher = publisher
>
>              book.authors = authors
>
>              book.narrators = narrators
>
>              book.genres = genres
>
>              book.series = series
>
>   
>
>              # add objects to session
>
>              session.add_all([book, publisher, series] + authors + narrators
> + genres)
>
>              # commit changes
>
>              session.commit()
>
>              self.GetTopWindow().SetStatusText("Record saved successfully")
>
>          except Exception as e:
>
>              session.rollback()
>
>              print(f"Error occurred: {e}")
>
>              self.GetTopWindow().SetStatusText("Record did not saved
> successfully")
>
>          finally:
>
>              session.close()
>
>   
>
>      def clearRecord(self, event):
>
>          self.record.clear()
>
>          self.GetTopWindow().SetStatusText("Record in memory cleared
> successfully")
>
>   
>
> class MyApp(wx.App):
>
>      def OnInit(self):
>
>          # Create the main frame
>
>          self.frame = wx.Frame(None, title="Library", size=(1920, 1080 ))
>
>          self.frame.Maximize()
>
>          sizer_main = wx.BoxSizer(wx.VERTICAL)
>
>          # Create the two panels
>
>          self.panel1 = EditPanel(self.frame)
>
>          self.panel2 = RecordListPanel(self.frame)
>
>          # Create the horizontal box sizers for the panels
>
>          sizer_panel1 = wx.BoxSizer(wx.HORIZONTAL)
>
>          sizer_panel2 = wx.BoxSizer(wx.HORIZONTAL)
>
>   
>
>   
>
>          # Create the menubar
>
>          menubar = wx.MenuBar()
>
>          # Create the "Options" menu
>
>          options_menu = wx.Menu()
>
>          save_record_item = options_menu.Append(wx.ID_ANY, "Save
> Record\tCtrl+S", "Save to database")
>
>          self.Bind(wx.EVT_MENU,self.panel1.saveDb, save_record_item)
>
>          next_file_item = options_menu.Append(wx.ID_ANY, "Next file\tCtrl+N",
> "Load next file")
>
>          prior_file_item = options_menu.Append(wx.ID_ANY, "Prior
> file\tCtrl+P", "Load prior file")
>
>          review_record_item = options_menu.Append(wx.ID_ANY, "Review
> Record\tCtrl+r", "Review  record")
>
>          self.Bind(wx.EVT_MENU,self.panel1.reviewDb, review_record_item)
>
>          self.Bind(wx.EVT_MENU,self.panel1.nextBook, next_file_item)
>
>          self.Bind(wx.EVT_MENU,self.panel1.priorBook, prior_file_item)
>
>          clear_record_item = options_menu.Append(wx.ID_ANY, "Clear
> Record\Alt+I", "Clear record in memory")
>
>          self.Bind(wx.EVT_MENU,self.panel1.clearRecord, clear_record_item)
>
>          options_menu.Append(wx.ID_EXIT, "Exit", "Exit the program")
>
>          menubar.Append(options_menu, "&Options")
>
>   
>
>          # Create the "Save to record" menu
>
>          save_menu = wx.Menu()
>
>          title_item = save_menu.Append(wx.ID_ANY, "Title\tAlt+T", "Save book
> title")
>
>          self.Bind(wx.EVT_MENU,self.panel1.titleSave, title_item)
>
>          author_item = save_menu.Append(wx.ID_ANY, "Author\tAlt+A", "Save
> book author")
>
>          self.Bind(wx.EVT_MENU,self.panel1.authorSave, author_item)
>
>          series_item = save_menu.Append(wx.ID_ANY, "Series\tAlt+S", "Save
> book Series")
>
>          self.Bind(wx.EVT_MENU,self.panel1.seriesSave, series_item)
>
>          seriesNo_item = save_menu.Append(wx.ID_ANY, "SeriesNo\tCtrl+3",
> "Save Series No")
>
>          self.Bind(wx.EVT_MENU,self.panel1.seriesNoSave, seriesNo_item)
>
>          genre_item = save_menu.Append(wx.ID_ANY, "Genre\tAlt+G", "Save book
> genre")
>
>          self.Bind(wx.EVT_MENU,self.panel1.genreSave, genre_item)
>
>          description_item = save_menu.Append(wx.ID_ANY, "Description\tAlt+D",
> "Save book description")
>
>          self.Bind(wx.EVT_MENU,self.panel1.descriptionSave, description_item)
>
>          aboutAuthor_item = save_menu.Append(wx.ID_ANY, "About
> Author\tAlt+B", "Save about author")
>
>          self.Bind(wx.EVT_MENU,self.panel1.aboutAuthorSave, aboutAuthor_item)
>
>          narrator_item = save_menu.Append(wx.ID_ANY, "Narrator\tAlt+N", "Save
> book narrator")
>
>          self.Bind(wx.EVT_MENU,self.panel1.narratorSave, narrator_item)
>
>          release_item = save_menu.Append(wx.ID_ANY, "Release Date\tAlt+R",
> "Save book release date")
>
>          self.Bind(wx.EVT_MENU,self.panel1.releaseSave, release_item)
>
>          duration_item = save_menu.Append(wx.ID_ANY, "Duration\tCtrl+D",
> "Save book duration")
>
>          self.Bind(wx.EVT_MENU,self.panel1.durationSave, duration_item)
>
>          copyright_item = save_menu.Append(wx.ID_ANY, "Copyright\tAlt+C",
> "Save book copyright")
>
>          self.Bind(wx.EVT_MENU,self.panel1.copyrightSave, copyright_item)
>
>          publisher_item = save_menu.Append(wx.ID_ANY, "Publisher\tAlt+P",
> "Save book publisher")
>
>          self.Bind(wx.EVT_MENU,self.panel1.publisherSave, publisher_item)
>
>          misc_item = save_menu.Append(wx.ID_ANY, "Misc\tAlt+M", "Save book
> misc")
>
>          self.Bind(wx.EVT_MENU,self.panel1.miscSave, misc_item)
>
>          menubar.Append(save_menu, "Sa&ve to record")
>
>   
>
>          self.frame.SetMenuBar(menubar)
>
>   
>
>          # Create the status bar
>
>          self.frame.CreateStatusBar()
>
>          #self.statusbar.SetFieldsCount(3)
>
>          #self.statusbar.SetStatusWidths([-2, -1, -1])
>
>   
>
> # Set up the panel switching
>
>          self.currentPanel = self.panel1
>
>          self.panel1.SetFocus()
>
>          self.frame.Bind(wx.EVT_KEY_DOWN, self.onKeyDown)
>
>          self.frame.Show()
>
>          self.SetTopWindow(self.frame)
>
>          
>
>          return True
>
>   
>
>      def onKeyDown(self, event):
>
>          if event.GetKeyCode() == wx.WXK_F6:
>
>              if self.currentPanel == self.panel1:
>
>                  self.currentPanel = self.panel2
>
>                  self.panel2.SetFocus()
>
>              else:
>
>                  self.currentPanel = self.panel1
>
>                  self.panel1.SetFocus()
>
>          else:
>
>              event.Skip()
>
>   
>
>          # set focus to the current panel.
>
>          self.currentPanel.SetFocus()
>
>   
>
> if __name__ == '__main__':
>
>      # Create the wx
>
>      app = MyApp()
>
>   
>
>      # Start the main event loop
>
>      app.MainLoop()
>
> _______________________________________________
> Tutor maillist  -Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From cs at cskk.id.au  Thu Mar 23 19:06:35 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Fri, 24 Mar 2023 10:06:35 +1100
Subject: [Tutor] Inherit from SyntaxError?
In-Reply-To: <DB6PR01MB38956893C47BADF347302F0783879@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
References: <DB6PR01MB38956893C47BADF347302F0783879@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
Message-ID: <ZBzbezMwu+u/r08z@cskk.homeip.net>

On 23Mar2023 09:17, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
>   Sometimes catching MemoryError might come in handy with a
>   "inside/outside RAM" strategy. In-memory is faster, but not scalable.
>   For instance, something like:
>   try:
>       d = {}
>       for record in records:
>           d[record.id] = record.something
>   except MemoryError:
>       logger.warning("Using slower shelve route")
>       d = shelve.open("/tmp/lookup.shelve", writeback=True)
>       atexit.register(d.close)
>       for record in records:
>           d[record.id] = record.something
>   If you know that "records" is always huge, this approach is needlessly
>   complicated. But if it *might* be too big for some machines, it might
>   be nice not having to choose the slow approach, just to be on the safe
>   side.

Ah. Is the above taken from working code? I've always felt that when a 
MemoryError files things will be comprimised enough that a lot of stuff 
won't subsequently work.

Taking the above as the example, would you have enough memory to do 
logging or use shelve.open? Even if you explicitly discarded the 
incomplete "d" before trying that (bearing in mind that without invoking 
the garbage collector the memory might not yet be really freed). But 
this opinion is speculation. Have you had joy with the above approach?

Cheers,
Cameron Simpson <cs at cskk.id.au>

From alan.gauld at yahoo.co.uk  Fri Mar 24 06:12:38 2023
From: alan.gauld at yahoo.co.uk (Alan Gauld)
Date: Fri, 24 Mar 2023 10:12:38 +0000
Subject: [Tutor] Python WX module program issues.
In-Reply-To: <003701d95d67$ebf818e0$c3e84aa0$@GMAIL.COM>
References: <003701d95d67$ebf818e0$c3e84aa0$@GMAIL.COM>
Message-ID: <tvjt2m$11ce$1@ciao.gmane.io>

On 23/03/2023 09:14, mhysnm1964 at gmail.com wrote:

> I have created a simple Python 3.10 app using WX module.

Strictly speaking wxPython is not part of the standard
library so out of scope for the tutor list. There is a
dedicated wxPython forum that might be more suitable.

However, it's pretty common so we may have some expertise
to answer the questions. Although that is a lot of code
to plow through!


> frame and two horizontal panels. Panel1 has an edit object in it. Panel2 has
> a list box. The issues I cannot resolve:
> 1. I cannot add any new content to the status bar from the class EditPanel. 

Probably best to tackle these one at a time or else the discussions will
get very complicated.

The above is usually down to scoping. I assume the status bar is
contained in the outer main window? How does the child panel get
access to that main window? Is it the parent widget? Or the
parent of the parent? If you have access to the status bars
containing window then you should be able to update it.

The other thing that can confuse matters with status bars is
if another widget is also updating the status bar and overwrites
your panels output before you can see it.


> 2. The RecordListPanel does not show any content and the database is
> populated.

Can you populate it with a non-database list?
Can you print the database list items to the console?

The answers should determine whether the error lies in the
List update functions or in the database access finctions.

> 3.  F6 is meant to switch between the two panels and set the keyboard focus
> to the next panel.

And the problem is?
"isn't working" is a bit too vague to be helpful.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos




From sjeik_appie at hotmail.com  Fri Mar 24 11:56:29 2023
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Fri, 24 Mar 2023 16:56:29 +0100
Subject: [Tutor] Inherit from SyntaxError?
In-Reply-To: <ZBzbezMwu+u/r08z@cskk.homeip.net>
Message-ID: <DB6PR01MB3895E49FB72B7F579A80301F83849@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   On Mar 24, 2023 00:06, Cameron Simpson <cs at cskk.id.au> wrote:

     On 23Mar2023 09:17, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
     >?? Sometimes catching MemoryError might come in handy with a
     >?? "inside/outside RAM" strategy. In-memory is faster, but not
     scalable.
     >?? For instance, something like:
     >?? try:
     >?????? d = {}
     >?????? for record in records:
     >?????????? d[record.id] = record.something
     >?? except MemoryError:
     >?????? logger.warning("Using slower shelve route")
     >?????? d = shelve.open("/tmp/lookup.shelve", writeback=True)
     >?????? atexit.register(d.close)
     >?????? for record in records:
     >?????????? d[record.id] = record.something
     >?? If you know that "records" is always huge, this approach is
     needlessly
     >?? complicated. But if it *might* be too big for some machines, it
     might
     >?? be nice not having to choose the slow approach, just to be on the
     safe
     >?? side.

     Ah. Is the above taken from working code? I've always felt that when a
     MemoryError files things will be comprimised enough that a lot of stuff
     won't subsequently work.

     Taking the above as the example, would you have enough memory to do
     logging or use shelve.open? Even if you explicitly discarded the
     incomplete "d" before trying that (bearing in mind that without invoking
     the garbage collector the memory might not yet be really freed). But
     this opinion is speculation. Have you had joy with the above approach?

   Hi Cameron,
   I've always avoided this approach because I didn't know what behavior I
   could expect. But today I ran a test script and it worked. However,
   ionically, /tmp was quite small on this machine so the huge shelve caused
   a
   ... MemoryError. Oh, and uh, my colleague could no longer ssh to that
   machine. Oops. ? In the script below I am basically skipping the shelve
   step for that reason.

   albertjan at mycompany@mymachine:/data/home/albertjan at mycompany$ uname -a

   Linux mymachine.mycompany 4.15.0-207-generic #218-Ubuntu SMP Thu Feb 23
   23:36:05 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

   albertjan at mycompany@mymachine:/data/home/albertjan at mycompany$ python
   --version

   Python 3.7.6

   albertjan at mycompany@mymachine:/data/home/albertjan at mycompany$ cat test.py

   import atexit

   import gc

   import itertools

   import logging

   import shelve

   ?

   logger = logging.getLogger(__name__)

   logging.basicConfig(format="%(asctime)s: %(message)s", level=logging.INFO)

   ?

   records = itertools.cycle([1024 * 1024 * "x"])

   try:

   ??? logging.info("started RAM route")

   ??? d = {}

   ??? for id_, record in enumerate(records):

   ??????? d[str(id_)] = record[:]

   ??? logging.info("finished RAM route")

   except MemoryError:

   ??? d = {}

   ??? gc.collect()

   ??? d = shelve.open("/tmp/lookup.shelve", writeback=True)

   ??? atexit.register(d.close)

   ??? logging.info("started Shelve route")

   ??? for id_, record in enumerate(records):

   ??????? d[str(id_)] = record[:]

   ??????? break

   ??? logging.info("finished Shelve route")

   albertjan at mycompany@mymachine:/data/home/albertjan at mycompany$ python
   test.py

   2023-03-24 14:47:05,147: started RAM route

   2023-03-24 16:02:40,816: started Shelve route

   2023-03-24 16:02:40,820: finished Shelve route

From tariqkhasiri at gmail.com  Mon Mar 27 22:58:31 2023
From: tariqkhasiri at gmail.com (Tariq Khasiri)
Date: Mon, 27 Mar 2023 21:58:31 -0500
Subject: [Tutor] Help with two .shp file and python coding
Message-ID: <CAFy_oHAiYL4uiMZus18DjRt5gyA7i_sLjuOyzV8ZiAvoh7s_og@mail.gmail.com>

Hello everyone,

For a particular project, I need to collect the distance data from each US
county to the nearest intersection of interstate highway.

These are the publicly provided dataset where anyone has access to the
county shp files of usa and roads shp files.

this one for county polygons
https://public.opendatasoft.com/explore/dataset/us-county-boundaries/information/

this one for primary roads

https://catalog.data.gov/dataset/tiger-line-shapefile-2016-nation-u-s-primary-roads-national-shapefile/resource/d983eeef-21d9-4367-9d42-27a131ee72b8

```
import geopandas as gpd
from shapely.geometry import Point

# Load shapefiles
counties = gpd.read_file('path/to/counties.shp')
interstates = gpd.read_file('path/to/interstates.shp')

# Perform spatial join to find nearest interstate for each county
joined = gpd.sjoin_nearest(counties, interstates)

# Calculate distance between each county and its nearest interstate
def calculate_distance(row):
    county_point = row.geometry.x, row.geometry.y
    interstate_point = row.geometry_nearest.x, row.geometry_nearest.y
    return Point(county_point).distance(Point(interstate_point))

joined['distance'] = joined.apply(calculate_distance, axis=1)

# Save results to file
joined.to_file('path/to/output.shp')
```

do you think I can execute my goal successfully with the code snippet above
??

From wlfraed at ix.netcom.com  Tue Mar 28 08:31:12 2023
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Tue, 28 Mar 2023 08:31:12 -0400
Subject: [Tutor] Help with two .shp file and python coding
References: <CAFy_oHAiYL4uiMZus18DjRt5gyA7i_sLjuOyzV8ZiAvoh7s_og@mail.gmail.com>
Message-ID: <74m52i9nnt5i55dpgu5j1sr0bdhmrpfu14@4ax.com>

On Mon, 27 Mar 2023 21:58:31 -0500, Tariq Khasiri <tariqkhasiri at gmail.com>
declaimed the following:

>Hello everyone,
>
>For a particular project, I need to collect the distance data from each US
>county to the nearest intersection of interstate highway.
>

	I'd be concerned about the poorly defined requirement you state at the
start. Where IN the county is your zero reference, from which to determine
"nearest"?

	For example, if the zero reference is the SE corner of Kent county MI
(one of the nice /rectangular/ counties) then the nearest such intersection
would be I-96 and M-50/Alden Nash. From the NW corner, you'd have to follow
M-37 down through Kent City, Sparta, and into Comstock Park to find an
intersection with (again) I-96 -- and likely have to test a dozen
intersections of I-96 as that is the fringe of the Grand Rapids metro area,
and there are intersections every mile if not more often.

	It is a different matter if you mean "from each interstate intersection
to ITS nearest county border".

>do you think I can execute my goal successfully with the code snippet above
>??

	Have you tried it? I didn't see anything asking about help with Python,
nor with the shape files. 

	You presented a batch of code, and are basically asking us to say
Yay/Nay as to if it will do what you want. "Do what you want" is algorithm
design, and well thought out algorithms are language neutral -- you should
be able to take the algorithm and code it in Python, REXX, even COBOL.
Asking for what Python structure/language feature may be best for
representing part of the algorithm is something suitable, but I would not
think asking us to vet your application would be... As you can see from my
opening, there are different ways to interpret your requirement -- which
means different algorithms may be applied and argued to be correct.

	


From tariqkhasiri at gmail.com  Tue Mar 28 08:58:17 2023
From: tariqkhasiri at gmail.com (Tariq Khasiri)
Date: Tue, 28 Mar 2023 07:58:17 -0500
Subject: [Tutor] Help with two .shp file and python coding
In-Reply-To: <74m52i9nnt5i55dpgu5j1sr0bdhmrpfu14@4ax.com>
References: <CAFy_oHAiYL4uiMZus18DjRt5gyA7i_sLjuOyzV8ZiAvoh7s_og@mail.gmail.com>
 <74m52i9nnt5i55dpgu5j1sr0bdhmrpfu14@4ax.com>
Message-ID: <CAFy_oHBQxeXDRhFdjXwWNC_-5qCDxpFww1YbDUxQa3ZOwDBjZw@mail.gmail.com>

Thanks for clearing up the matter of what I was trying to ask.

measuring the distance from each interstate intersection to ITS nearest
county border - is my end goal.

I apologize for being vague.

Before doing so I wanted to make sure I'm on the right path since I have no
prior experience working with shp files with python code. So, before
jumping in I wanted to be assured that this is the right way to go forward
given the shp files.



On Tue, 28 Mar 2023 at 07:31, Dennis Lee Bieber <wlfraed at ix.netcom.com>
wrote:

> On Mon, 27 Mar 2023 21:58:31 -0500, Tariq Khasiri <tariqkhasiri at gmail.com>
> declaimed the following:
>
> >Hello everyone,
> >
> >For a particular project, I need to collect the distance data from each US
> >county to the nearest intersection of interstate highway.
> >
>
>         I'd be concerned about the poorly defined requirement you state at
> the
> start. Where IN the county is your zero reference, from which to determine
> "nearest"?
>
>         For example, if the zero reference is the SE corner of Kent county
> MI
> (one of the nice /rectangular/ counties) then the nearest such intersection
> would be I-96 and M-50/Alden Nash. From the NW corner, you'd have to follow
> M-37 down through Kent City, Sparta, and into Comstock Park to find an
> intersection with (again) I-96 -- and likely have to test a dozen
> intersections of I-96 as that is the fringe of the Grand Rapids metro area,
> and there are intersections every mile if not more often.
>
>         It is a different matter if you mean "from each interstate
> intersection
> to ITS nearest county border".
>
> >do you think I can execute my goal successfully with the code snippet
> above
> >??
>
>         Have you tried it? I didn't see anything asking about help with
> Python,
> nor with the shape files.
>
>         You presented a batch of code, and are basically asking us to say
> Yay/Nay as to if it will do what you want. "Do what you want" is algorithm
> design, and well thought out algorithms are language neutral -- you should
> be able to take the algorithm and code it in Python, REXX, even COBOL.
> Asking for what Python structure/language feature may be best for
> representing part of the algorithm is something suitable, but I would not
> think asking us to vet your application would be... As you can see from my
> opening, there are different ways to interpret your requirement -- which
> means different algorithms may be applied and argued to be correct.
>
>
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From threesomequarks at proton.me  Tue Mar 28 10:43:44 2023
From: threesomequarks at proton.me (ThreeBlindQuarks)
Date: Tue, 28 Mar 2023 14:43:44 +0000
Subject: [Tutor] Help with two .shp file and python coding
In-Reply-To: <CAFy_oHAiYL4uiMZus18DjRt5gyA7i_sLjuOyzV8ZiAvoh7s_og@mail.gmail.com>
References: <CAFy_oHAiYL4uiMZus18DjRt5gyA7i_sLjuOyzV8ZiAvoh7s_og@mail.gmail.com>
Message-ID: <kfMpQR9evOcmCzbwWaJFI_zKVoVDGDKPm_JrE8_VpeTaxXdsCtcuc_oRkC3NHa6UWf0tmUyiBKdQkiEq-LdiFwuEPw8brgCkaEocwjVTrOE=@proton.me>

Tariq,

Your entire program is written in a module called geopandas so to even understand it, we would need to find out what exactly various functions in that module you call are documented to do.

A little more explanation of what you are doing might be helpful.

Near as I can tell, you are reading in two files to make two structures in a pandas format. You join them based on some common columns in whatever manner gpd.sjoin_nearest() does so that each row now contains enough info for the next step.

You create a function that accepts one such row and extracts the coordinates of two parts and calculates a distance that I am guessing is a Euclidean distance as the crow flies, not as you drive on roads. 

You use that function on every row to create a new column.

Then you save the results in a file.

Does that do something? Who knows. The devil is in the details. The mysterious black box that does the initial join would have to uniquely match up counties and highways in a way that generates exactly the rows needed and exclude any that don't. We (meaning ME) have no assurance of what happens in there. Could you end up comparing all counties with their distance from somewhere on highway 1?

- Q


Sent with Proton Mail secure email.

------- Original Message -------
On Monday, March 27th, 2023 at 10:58 PM, Tariq Khasiri <tariqkhasiri at gmail.com> wrote:


> Hello everyone,
> 
> For a particular project, I need to collect the distance data from each US
> county to the nearest intersection of interstate highway.
> 
> These are the publicly provided dataset where anyone has access to the
> county shp files of usa and roads shp files.
> 
> this one for county polygons
> https://public.opendatasoft.com/explore/dataset/us-county-boundaries/information/
> 
> this one for primary roads
> 
> https://catalog.data.gov/dataset/tiger-line-shapefile-2016-nation-u-s-primary-roads-national-shapefile/resource/d983eeef-21d9-4367-9d42-27a131ee72b8
> 
> `import geopandas as gpd from shapely.geometry import Point # Load shapefiles counties = gpd.read_file('path/to/counties.shp') interstates = gpd.read_file('path/to/interstates.shp') # Perform spatial join to find nearest interstate for each county joined = gpd.sjoin_nearest(counties, interstates) # Calculate distance between each county and its nearest interstate def calculate_distance(row): county_point = row.geometry.x, row.geometry.y interstate_point = row.geometry_nearest.x, row.geometry_nearest.y return Point(county_point).distance(Point(interstate_point)) joined['distance'] = joined.apply(calculate_distance, axis=1) # Save results to file joined.to_file('path/to/output.shp')`
> 
> do you think I can execute my goal successfully with the code snippet above
> ??
> _______________________________________________
> Tutor maillist - Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From tariqkhasiri at gmail.com  Wed Mar 29 09:51:39 2023
From: tariqkhasiri at gmail.com (Tariq Khasiri)
Date: Wed, 29 Mar 2023 08:51:39 -0500
Subject: [Tutor] Help with two .shp file and python coding
In-Reply-To: <kfMpQR9evOcmCzbwWaJFI_zKVoVDGDKPm_JrE8_VpeTaxXdsCtcuc_oRkC3NHa6UWf0tmUyiBKdQkiEq-LdiFwuEPw8brgCkaEocwjVTrOE=@proton.me>
References: <CAFy_oHAiYL4uiMZus18DjRt5gyA7i_sLjuOyzV8ZiAvoh7s_og@mail.gmail.com>
 <kfMpQR9evOcmCzbwWaJFI_zKVoVDGDKPm_JrE8_VpeTaxXdsCtcuc_oRkC3NHa6UWf0tmUyiBKdQkiEq-LdiFwuEPw8brgCkaEocwjVTrOE=@proton.me>
Message-ID: <CAFy_oHA6ShgAQ+hgNtx4FwYLXyENuQBz0sea51qg7iimYtvKqQ@mail.gmail.com>

After following the steps

import geopandas as gpd
from shapely.geometry import Point

# Load shapefiles
counties = gpd.read_file('county.shp')
interstates = gpd.read_file('roads.shp')

# To resolve this warning, you can reproject one of the datasets to match
the CRS of the other. In this case, you could reproject the counties
dataset to match the CRS of the interstates dataset.

counties = counties.to_crs(interstates.crs)

# Perform spatial join to find nearest interstate for each county
joined = gpd.sjoin_nearest(interstates, counties, distance_col='distance')

#creating new data frame with necessary columns from the joined dataframe
of total 27 columns
new_joined = joined.loc[:, ['LINEARID', 'FULLNAME', 'RTTYP', 'geometry',
'index_right', 'statefp', 'countyfp', 'distance']]

print(new_joined.sample(10))

my data frames (10 observations) look like this

Now the rest of the code looks like this. But, I'm not sure how I need to
edit my code with the columns of new_joined data frame. On top of it I
need observation of the interstate highways. So from RTTYP column anything
that starts with *I- *, that's my needed observation. The rest of it is not
necessary since they are observations of highways or expressways. Only need
the interstate highway given my goal.

## the rest of the code needs to be edited

# Calculate distance between each county and its nearest interstate
def calculate_distance(row):
    county_point = row.geometry.x, row.geometry.y
    interstate_point = row.geometry_nearest.x, row.geometry_nearest.y
    return Point(county_point).distance(Point(interstate_point))

joined['distance'] = joined.apply(calculate_distance, axis=1)

# Save results to file
joined.to_file('output.shp')

##new_joined.sample 10 observation is in the following section

 LINEARID           FULLNAME RTTYP  \
11187  1104375149380          US Hwy 30     U
658    1105056901139              I- 96     I
4070    110465934760  Gerald R Ford Fwy     M
8925   1102218644426         Edens Expy     M
1254   1104492838322          US Hwy 23     U
3514   1101476091588              I- 90     I
3979    110431486317   Will Rogers Tpke     M
12155  1104762432455              I- 40     I
10332  1105088962603              I- 80     I
12086  1104755971958              I- 94     I

                                                geometry  index_right statefp  \
11187  LINESTRING (-105.52431 41.28870, -105.52369 41...         2773      56
658    LINESTRING (-83.08258 42.31966, -83.08260 42.3...          271      26
4070   LINESTRING (-85.78228 42.87707, -85.78357 42.8...         1083      26
8925   LINESTRING (-87.74634 41.96272, -87.74642 41.9...         1189      17
1254   LINESTRING (-83.99342 34.08687, -83.99330 34.0...          427      13
3514   LINESTRING (-75.97591 43.09333, -75.97613 43.0...           82      36
3979   LINESTRING (-95.27075 36.51022, -95.27031 36.5...         1474      40
12155  LINESTRING (-94.31317 35.45632, -94.31351 35.4...         1337      05
10332  LINESTRING (-121.54758 38.59867, -121.54713 38...          871      06
12086  LINESTRING (-96.28076 46.48266, -96.27988 46.4...          555      27

      countyfp  distance
11187      001       0.0
658        163       0.0
4070       139       0.0
8925       031       0.0
1254       135       0.0
3514       067       0.0
3979       035       0.0
12155      033       0.0
10332      113       0.0


On Tue, 28 Mar 2023 at 09:43, ThreeBlindQuarks <threesomequarks at proton.me>
wrote:

> Tariq,
>
> Your entire program is written in a module called geopandas so to even
> understand it, we would need to find out what exactly various functions in
> that module you call are documented to do.
>
> A little more explanation of what you are doing might be helpful.
>
> Near as I can tell, you are reading in two files to make two structures in
> a pandas format. You join them based on some common columns in whatever
> manner gpd.sjoin_nearest() does so that each row now contains enough info
> for the next step.
>
> You create a function that accepts one such row and extracts the
> coordinates of two parts and calculates a distance that I am guessing is a
> Euclidean distance as the crow flies, not as you drive on roads.
>
> You use that function on every row to create a new column.
>
> Then you save the results in a file.
>
> Does that do something? Who knows. The devil is in the details. The
> mysterious black box that does the initial join would have to uniquely
> match up counties and highways in a way that generates exactly the rows
> needed and exclude any that don't. We (meaning ME) have no assurance of
> what happens in there. Could you end up comparing all counties with their
> distance from somewhere on highway 1?
>
> - Q
>
>
> Sent with Proton Mail secure email.
>
> ------- Original Message -------
> On Monday, March 27th, 2023 at 10:58 PM, Tariq Khasiri <
> tariqkhasiri at gmail.com> wrote:
>
>
> > Hello everyone,
> >
> > For a particular project, I need to collect the distance data from each
> US
> > county to the nearest intersection of interstate highway.
> >
> > These are the publicly provided dataset where anyone has access to the
> > county shp files of usa and roads shp files.
> >
> > this one for county polygons
> >
> https://public.opendatasoft.com/explore/dataset/us-county-boundaries/information/
> >
> > this one for primary roads
> >
> >
> https://catalog.data.gov/dataset/tiger-line-shapefile-2016-nation-u-s-primary-roads-national-shapefile/resource/d983eeef-21d9-4367-9d42-27a131ee72b8
> >
> > `import geopandas as gpd from shapely.geometry import Point # Load
> shapefiles counties = gpd.read_file('path/to/counties.shp') interstates =
> gpd.read_file('path/to/interstates.shp') # Perform spatial join to find
> nearest interstate for each county joined = gpd.sjoin_nearest(counties,
> interstates) # Calculate distance between each county and its nearest
> interstate def calculate_distance(row): county_point = row.geometry.x,
> row.geometry.y interstate_point = row.geometry_nearest.x,
> row.geometry_nearest.y return
> Point(county_point).distance(Point(interstate_point)) joined['distance'] =
> joined.apply(calculate_distance, axis=1) # Save results to file
> joined.to_file('path/to/output.shp')`
> >
> > do you think I can execute my goal successfully with the code snippet
> above
> > ??
> > _______________________________________________
> > Tutor maillist - Tutor at python.org
> > To unsubscribe or change subscription options:
> > https://mail.python.org/mailman/listinfo/tutor
>

From tariqkhasiri at gmail.com  Wed Mar 29 10:19:14 2023
From: tariqkhasiri at gmail.com (Tariq Khasiri)
Date: Wed, 29 Mar 2023 09:19:14 -0500
Subject: [Tutor] Help with two .shp file and python coding
In-Reply-To: <CAFy_oHA6ShgAQ+hgNtx4FwYLXyENuQBz0sea51qg7iimYtvKqQ@mail.gmail.com>
References: <CAFy_oHAiYL4uiMZus18DjRt5gyA7i_sLjuOyzV8ZiAvoh7s_og@mail.gmail.com>
 <kfMpQR9evOcmCzbwWaJFI_zKVoVDGDKPm_JrE8_VpeTaxXdsCtcuc_oRkC3NHa6UWf0tmUyiBKdQkiEq-LdiFwuEPw8brgCkaEocwjVTrOE=@proton.me>
 <CAFy_oHA6ShgAQ+hgNtx4FwYLXyENuQBz0sea51qg7iimYtvKqQ@mail.gmail.com>
Message-ID: <CAFy_oHBdi2KAm=YzMM+t4+C7cGiCD=poHtmpJbKN86Z35ax=vg@mail.gmail.com>

apologies for not stating my goal of this coding clearly in my last email

The goal is to find the distance from each county to the *nearest
intersection of interstate highways*

On Wed, 29 Mar 2023 at 08:51, Tariq Khasiri <tariqkhasiri at gmail.com> wrote:

> After following the steps
>
> import geopandas as gpd
> from shapely.geometry import Point
>
> # Load shapefiles
> counties = gpd.read_file('county.shp')
> interstates = gpd.read_file('roads.shp')
>
> # To resolve this warning, you can reproject one of the datasets to match
> the CRS of the other. In this case, you could reproject the counties
> dataset to match the CRS of the interstates dataset.
>
> counties = counties.to_crs(interstates.crs)
>
> # Perform spatial join to find nearest interstate for each county
> joined = gpd.sjoin_nearest(interstates, counties, distance_col='distance')
>
> #creating new data frame with necessary columns from the joined dataframe
> of total 27 columns
> new_joined = joined.loc[:, ['LINEARID', 'FULLNAME', 'RTTYP', 'geometry',
> 'index_right', 'statefp', 'countyfp', 'distance']]
>
> print(new_joined.sample(10))
>
> my data frames (10 observations) look like this
>
> Now the rest of the code looks like this. But, I'm not sure how I need to
> edit my code with the columns of new_joined data frame. On top of it I
> need observation of the interstate highways. So from RTTYP column anything
> that starts with *I- *, that's my needed observation. The rest of it is
> not necessary since they are observations of highways or expressways. Only
> need the interstate highway given my goal.
>
> ## the rest of the code needs to be edited
>
> # Calculate distance between each county and its nearest interstate
> def calculate_distance(row):
>     county_point = row.geometry.x, row.geometry.y
>     interstate_point = row.geometry_nearest.x, row.geometry_nearest.y
>     return Point(county_point).distance(Point(interstate_point))
>
> joined['distance'] = joined.apply(calculate_distance, axis=1)
>
> # Save results to file
> joined.to_file('output.shp')
>
> ##new_joined.sample 10 observation is in the following section
>
>  LINEARID           FULLNAME RTTYP  \
> 11187  1104375149380          US Hwy 30     U
> 658    1105056901139              I- 96     I
> 4070    110465934760  Gerald R Ford Fwy     M
> 8925   1102218644426         Edens Expy     M
> 1254   1104492838322          US Hwy 23     U
> 3514   1101476091588              I- 90     I
> 3979    110431486317   Will Rogers Tpke     M
> 12155  1104762432455              I- 40     I
> 10332  1105088962603              I- 80     I
> 12086  1104755971958              I- 94     I
>
>                                                 geometry  index_right statefp  \
> 11187  LINESTRING (-105.52431 41.28870, -105.52369 41...         2773      56
> 658    LINESTRING (-83.08258 42.31966, -83.08260 42.3...          271      26
> 4070   LINESTRING (-85.78228 42.87707, -85.78357 42.8...         1083      26
> 8925   LINESTRING (-87.74634 41.96272, -87.74642 41.9...         1189      17
> 1254   LINESTRING (-83.99342 34.08687, -83.99330 34.0...          427      13
> 3514   LINESTRING (-75.97591 43.09333, -75.97613 43.0...           82      36
> 3979   LINESTRING (-95.27075 36.51022, -95.27031 36.5...         1474      40
> 12155  LINESTRING (-94.31317 35.45632, -94.31351 35.4...         1337      05
> 10332  LINESTRING (-121.54758 38.59867, -121.54713 38...          871      06
> 12086  LINESTRING (-96.28076 46.48266, -96.27988 46.4...          555      27
>
>       countyfp  distance
> 11187      001       0.0
> 658        163       0.0
> 4070       139       0.0
> 8925       031       0.0
> 1254       135       0.0
> 3514       067       0.0
> 3979       035       0.0
> 12155      033       0.0
> 10332      113       0.0
>
>
> On Tue, 28 Mar 2023 at 09:43, ThreeBlindQuarks <threesomequarks at proton.me>
> wrote:
>
>> Tariq,
>>
>> Your entire program is written in a module called geopandas so to even
>> understand it, we would need to find out what exactly various functions in
>> that module you call are documented to do.
>>
>> A little more explanation of what you are doing might be helpful.
>>
>> Near as I can tell, you are reading in two files to make two structures
>> in a pandas format. You join them based on some common columns in whatever
>> manner gpd.sjoin_nearest() does so that each row now contains enough info
>> for the next step.
>>
>> You create a function that accepts one such row and extracts the
>> coordinates of two parts and calculates a distance that I am guessing is a
>> Euclidean distance as the crow flies, not as you drive on roads.
>>
>> You use that function on every row to create a new column.
>>
>> Then you save the results in a file.
>>
>> Does that do something? Who knows. The devil is in the details. The
>> mysterious black box that does the initial join would have to uniquely
>> match up counties and highways in a way that generates exactly the rows
>> needed and exclude any that don't. We (meaning ME) have no assurance of
>> what happens in there. Could you end up comparing all counties with their
>> distance from somewhere on highway 1?
>>
>> - Q
>>
>>
>> Sent with Proton Mail secure email.
>>
>> ------- Original Message -------
>> On Monday, March 27th, 2023 at 10:58 PM, Tariq Khasiri <
>> tariqkhasiri at gmail.com> wrote:
>>
>>
>> > Hello everyone,
>> >
>> > For a particular project, I need to collect the distance data from each
>> US
>> > county to the nearest intersection of interstate highway.
>> >
>> > These are the publicly provided dataset where anyone has access to the
>> > county shp files of usa and roads shp files.
>> >
>> > this one for county polygons
>> >
>> https://public.opendatasoft.com/explore/dataset/us-county-boundaries/information/
>> >
>> > this one for primary roads
>> >
>> >
>> https://catalog.data.gov/dataset/tiger-line-shapefile-2016-nation-u-s-primary-roads-national-shapefile/resource/d983eeef-21d9-4367-9d42-27a131ee72b8
>> >
>> > `import geopandas as gpd from shapely.geometry import Point # Load
>> shapefiles counties = gpd.read_file('path/to/counties.shp') interstates =
>> gpd.read_file('path/to/interstates.shp') # Perform spatial join to find
>> nearest interstate for each county joined = gpd.sjoin_nearest(counties,
>> interstates) # Calculate distance between each county and its nearest
>> interstate def calculate_distance(row): county_point = row.geometry.x,
>> row.geometry.y interstate_point = row.geometry_nearest.x,
>> row.geometry_nearest.y return
>> Point(county_point).distance(Point(interstate_point)) joined['distance'] =
>> joined.apply(calculate_distance, axis=1) # Save results to file
>> joined.to_file('path/to/output.shp')`
>> >
>> > do you think I can execute my goal successfully with the code snippet
>> above
>> > ??
>> > _______________________________________________
>> > Tutor maillist - Tutor at python.org
>> > To unsubscribe or change subscription options:
>> > https://mail.python.org/mailman/listinfo/tutor
>>
>

From mk1853387 at gmail.com  Wed Mar 29 19:03:45 2023
From: mk1853387 at gmail.com (marc nicole)
Date: Thu, 30 Mar 2023 01:03:45 +0200
Subject: [Tutor] combinations of all rows and cols from a dataframe
Message-ID: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>

Hello everyone,

Given a dataframe like this:

2 6
8 5

I want to yield the following list of lists:
[  [[2],[6,5]],
[[2],[6]],
[[2],[5]],
[[8],[6,5]],
[[8],[6]],
[[8],[5]],
[[6],[2,8]],
[[6],[8]],
[[6],[2]],
[[5],[2,8]],
[[5],[2]],
[[5],[8]],
[[6,5],[2,8]]  ]

I have written the following (which doesn't yield the expected results)

import pandas as pd
> from itertools import combinations
> import numpy as np
> resList=[]
> resListTmp=[]
> resListTmp2=[]
> dataframe =
> pd.read_excel("C:\\Users\\user\\Desktop\\testData.xlsx",index_col=False,header=None)

for i in range(0, len(dataframe)+1):
>     for j in range(0, len(dataframe.columns)):
>         for k in range (0,len(dataframe)+1):
>             for xVals in list(combinations(dataframe.iloc[k:i,j], i)):
>                 if list(xVals) not in resListTmp:
>                     resListTmp.append(list(xVals))
>         resListTmp2.append(resListTmp)
>     resList.append(resListTmp2)
> print(resList)
>

What is wrong with my code?

From __peter__ at web.de  Thu Mar 30 04:37:17 2023
From: __peter__ at web.de (Peter Otten)
Date: Thu, 30 Mar 2023 10:37:17 +0200
Subject: [Tutor] combinations of all rows and cols from a dataframe
In-Reply-To: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>
References: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>
Message-ID: <0937eb01-411c-3bbb-dbf8-c78b49231c5b@web.de>

On 30/03/2023 01:03, marc nicole wrote:
> Hello everyone,
>
> Given a dataframe like this:
>
> 2 6
> 8 5
>
> I want to yield the following list of lists:
> [  [[2],[6,5]],
> [[2],[6]],
> [[2],[5]],
> [[8],[6,5]],
> [[8],[6]],
> [[8],[5]],
> [[6],[2,8]],
> [[6],[8]],
> [[6],[2]],
> [[5],[2,8]],
> [[5],[2]],
> [[5],[8]],
> [[6,5],[2,8]]  ]
>
> I have written the following (which doesn't yield the expected results)
>
> import pandas as pd
>> from itertools import combinations
>> import numpy as np
>> resList=[]
>> resListTmp=[]
>> resListTmp2=[]
>> dataframe =
>> pd.read_excel("C:\\Users\\user\\Desktop\\testData.xlsx",index_col=False,header=None)
>
> for i in range(0, len(dataframe)+1):
>>      for j in range(0, len(dataframe.columns)):
>>          for k in range (0,len(dataframe)+1):
>>              for xVals in list(combinations(dataframe.iloc[k:i,j], i)):
>>                  if list(xVals) not in resListTmp:
>>                      resListTmp.append(list(xVals))
>>          resListTmp2.append(resListTmp)
>>      resList.append(resListTmp2)
>> print(resList)
>>
>
> What is wrong with my code?

I think you need to move the initialization of the temporary list into
the respective loop, but was unable to get it to work.

My second attempt was to start with the combinations from one column:

 >>> def colcomb(column):
	result = []
	for i in range(len(column)):
		for c in combinations(column, i+1):
			result.append(list(c))
	return result

 >>> colcomb([2, 8])
[[2], [8], [2, 8]]

But what could be the next step? I'm unable to infer it from your sample
result which seems to be somewhere between

 >>> a = [[2, 8], [6, 5]]  # transposed to avoid pandas/numpy
 >>> from itertools import product
 >>> list(product(*(colcomb(col) for col in a)))
[([2], [6]), ([2], [5]), ([2], [6, 5]), ([8], [6]), ([8], [5]), ([8],
[6, 5]), ([2, 8], [6]), ([2, 8], [5]), ([2, 8], [6, 5])]

and

 >>> x = list(product(*(colcomb(col) for col in a)))
 >>> x + [y[::-1] for y in x]
[([2], [6]), ([2], [5]), ([2], [6, 5]), ([8], [6]), ([8], [5]), ([8],
[6, 5]), ([2, 8], [6]), ([2, 8], [5]), ([2, 8], [6, 5]), ([6], [2]),
([5], [2]), ([6, 5], [2]), ([6], [8]), ([5], [8]), ([6, 5], [8]), ([6],
[2, 8]), ([5], [2, 8]), ([6, 5], [2, 8])]

Perhaps you can try and tell us what you want in plain English?


From tariqkhasiri at gmail.com  Wed Mar 29 20:55:11 2023
From: tariqkhasiri at gmail.com (Tariq Khasiri)
Date: Wed, 29 Mar 2023 19:55:11 -0500
Subject: [Tutor] Making unique identifier from repeated observation data
Message-ID: <CAFy_oHAjoOixoT1NzGik5VPvKpb1my9crHM1QX9Q+jZtDt_UQA@mail.gmail.com>

My data frame looks like the following. Here, countyfp stands for usa
county code, statefp stands for statefip ( state code of USA). Fullname
stands for the name of the interstate highway. Geometry is the geographical
position of that interstate highway. Here, the distance variable is the key
for each interstate highway observation. The distance variable stands up
for the distance between each county and its nearest intersection of
interstate highways

I have multiple records when there are multiple interstates that intersect
a county, because they are all equidistant. Now, for each unique county I
only want to keep the minimum distance from the nearest intersection of the
interstate highway. How can I keep the observations making the combined
(countyfp and statefp) the unique identifier? The first column is index.

My objective is to find the minimum distance from each county to its the
nearest intersection of interstate highway.


FULLNAME geometry statefp countyfp distance 0 I- 75 POLYGON ((-83.88076
41.08036, -83.88076 41.081... 39 063 0.000000 111 I- 75 POLYGON ((-84.39719
40.78658, -84.39720 40.787... 39 003 0.000000 2301 I- 75 POLYGON
((-84.45562 40.55325, -84.45562 40.553... 39 011 0.000000 0 I- 75 POLYGON
((-83.88076 41.08036, -83.88076 41.081... 39 063 0.000000 111 I- 75 POLYGON
((-84.39719 40.78658, -84.39720 40.787... 39 003 0.000000 812 I- 75 POLYGON
((-83.70964 41.52185, -83.70950 41.522... 39 173 0.000000 1639 I- 75
POLYGON ((-83.47674 40.90496, -83.47666 40.917... 39 175 0.197390 0 I- 75
POLYGON ((-83.88076 41.08036, -83.88076 41.081... 39 063 0.000000 672 I- 75
POLYGON ((-84.34216 41.00514, -84.34222 41.012... 39 137 0.030141 812 I- 75
POLYGON ((-83.70964 41.52185, -83.70950 41.522... 39 173 0.000000 0 I- 75
POLYGON ((-83.88076 41.08036, -83.88076 41.081... 39 063 0.000000 812 I- 75
POLYGON ((-83.70964 41.52185, -83.70950 41.522... 39 173 0.000000 1 I- 95
POLYGON ((-77.58878 38.50404, -77.58295 38.508... 51 179 0.000000 101 I- 95
POLYGON ((-77.78589 38.26993, -77.78357 38.271... 51 177 0.000000 1140 I-
95 POLYGON ((-77.53268 38.30850, -77.53267 38.308... 51 630 0.000000 1 I-
95 POLYGON ((-77.58878 38.50404, -77.58295 38.508... 51 179 0.000000 304 I-
95 MULTIPOLYGON (((-77.57948 38.62765, -77.57955 ... 51 153 0.000000 1036
I- 95 POLYGON ((-77.26754 38.33410, -77.26721 38.333... 51 099 0.122636
2422 I- 95 POLYGON ((-76.89028 38.11468, -76.89335 38.112... 51 193
0.407545 1 I- 95 POLYGON ((-77.58878 38.50404, -77.58295 38.508... 51 179
0.000000

From threesomequarks at proton.me  Wed Mar 29 21:01:06 2023
From: threesomequarks at proton.me (ThreeBlindQuarks)
Date: Thu, 30 Mar 2023 01:01:06 +0000
Subject: [Tutor] combinations of all rows and cols from a dataframe
In-Reply-To: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>
References: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>
Message-ID: <X5LDkQUATpueoHItabIb43Uq1dfqXi6N8qILctDLnc9LImWAn8rp0a14iauARB_o6EA9spvI1S6Z5OIB7ftC3ggzM8dsGjM3Y-SpKDx9TuA=@proton.me>


Marc,

Before answering, we need to really understand what you want and whether you need to handle more cases than you specify.

You did not show the output of your code for comparison.

My reading of what you want is you have a 2x2 grouping of numbers with a LHS and a RHS. You want what looks like this:

For each row take the LHS argument (2 and later 8) and pair it with any combination of the RHS of ALL the rows except null. I mean you want a list of combinations of 6,5 as just [6], just [5] and the combo of [6,5] but NOT the symmetric [5,6].

But then you flip it without flipping it and seem to want a similar output for 

6 2
5 8

This new LHS is to be dealt with the same way by taking each row and doing the restricted combinations of everything in the LHS column.

The above is fairly easy to do if limited to your 2x2 example, perhaps with other numbers. Write a function that implements it on your 2x2 then call it again after exchanging the two columns and combine.

The code you show is complex and deeply nested with four loops. It looks like it is trying to solve a more complex case than 2x2 and I have not studied it. I suspect there is an easier and more pythonic way to do it, albeit I have no real notion why you need to massage your data in this way.

I would actually consider a two-pass version as another tack. You can take your data.frame and add a column or more that contains the required permutations, perhaps trimming out the ones you do not seem to want.

A second pass might make multiple rows out of each one, using each of the remaining permutations.

But I remain unsure of what you expect in other than the 2x2 case.

Good luck.



Sent with Proton Mail secure email.

------- Original Message -------
On Wednesday, March 29th, 2023 at 7:03 PM, marc nicole <mk1853387 at gmail.com> wrote:


> Hello everyone,
> 
> Given a dataframe like this:
> 
> 2 6
> 8 5
> 
> I want to yield the following list of lists:
> [ [[2],[6,5]],
> [[2],[6]],
> [[2],[5]],
> [[8],[6,5]],
> [[8],[6]],
> [[8],[5]],
> [[6],[2,8]],
> [[6],[8]],
> [[6],[2]],
> [[5],[2,8]],
> [[5],[2]],
> [[5],[8]],
> [[6,5],[2,8]] ]
> 
> I have written the following (which doesn't yield the expected results)
> 
> import pandas as pd
> 
> > from itertools import combinations
> > import numpy as np
> > resList=[]
> > resListTmp=[]
> > resListTmp2=[]
> > dataframe =
> > pd.read_excel("C:\\Users\\user\\Desktop\\testData.xlsx",index_col=False,header=None)
> 
> 
> for i in range(0, len(dataframe)+1):
> 
> > for j in range(0, len(dataframe.columns)):
> > for k in range (0,len(dataframe)+1):
> > for xVals in list(combinations(dataframe.iloc[k:i,j], i)):
> > if list(xVals) not in resListTmp:
> > resListTmp.append(list(xVals))
> > resListTmp2.append(resListTmp)
> > resList.append(resListTmp2)
> > print(resList)
> 
> 
> What is wrong with my code?
> _______________________________________________
> Tutor maillist - Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From goranikac65 at gmail.com  Thu Mar 30 08:58:12 2023
From: goranikac65 at gmail.com (Goran Ikac)
Date: Thu, 30 Mar 2023 14:58:12 +0200
Subject: [Tutor] My function correct_num()
Message-ID: <CAM53q1QDQJd2-=dHCWWhBYNsoDTD8C2UeL2b_j7V-wKL_PhiXg@mail.gmail.com>

Hi, I wish a nice day to every pythonist out there!
I'm a newbie, still learning the basics. For an exercise, I wrote a
function *correct_num() *to avoid ridiculous presentation of some numeric
calculations in Python (e.g. *print(.1 + .1 + .1) *outputs
*0.30000000000000004*):

# written by ike 2023

def correct_num(num: float) -> float:
    """
    The function takes any number as the only argument.
    If the number is an integer, or if it is a float with up to *six *
decimal
    digits, the function returns the same number.
    If the number is a float with more than *six *decimal digits,
    the function returns that number with up to one digit less precision.
    That way, it corrects the errors made by the computer's numerical
    calculations.

    >>> .1 + .1 + .1
    0.30000000000000004
    >>> correct_num(.1 + .1 + .1)
    0.3
    >>> 24.95 - 9.98
    14.969999999999999
    >>> correct_num(24.95 - 9.98)
    14.97
    >>> correct_num(123)
    123
    >>> correct_num(123.0)
    123.0
    >>> correct_num(-123.0)
    -123.0
    >>> correct_num(0.123456)
    0.123456
    >>> correct_num(0.1234567)
    0.123457
    >>> correct_num(0.123456789)
    0.12345679
    >>> correct_num(-0.123456789)
    -0.12345679
    >>> correct_num(5.12345678e-5)
    5.1e-05
    >>> correct_num(5.17345678e-5)
    5.2e-05
    >>> correct_num(5.173e-6)
    5e-06
    >>> correct_num(5.173e-7)
    1e-06
    >>> >>> correct_num(5.173e-8)
    0.0

    """

    if num == int(num):             # If the number is an integer,
        return num                      # return the same number.

    numstr = str(num)

    if 'e' in numstr:               # If num is written in scientific
notation
        numstr = '{:.*7*f}'.format(num)

    # print('numstring ==', numstr)     # This is a control line.

    if '.' in numstr:               # If the numstr represents a float,
        # determine the number of num's decimal digits
        dec_digits = len(numstr[(numstr.index('.') + 1):])
        # If the limitation in how computers do arithmetic doesn't affect
num:
        if dec_digits < *7*:
            return num                  # return the same number.

    return round(num, dec_digits - 1)


Now, I'm happy with the function's work, but I don't know what number of
decimal digits to leave alone, e.g. what number of decimal digits are
certainly OK). I've decided it to be *six *decimal digits:
        if dec_digits < *7*:
            return num                  # return the same number.
but it was by pure intuition. Does anybody know what is the right number of
decimal digits to leave as they were returned by Python numeric
calculations?
And, of course, I'd be thankful for any comment about my code.
Also, can anybody, please, correct my English in the docstring?
Watch out! I'm riding to catch up with you, big boys and girls.

From mk1853387 at gmail.com  Thu Mar 30 07:41:09 2023
From: mk1853387 at gmail.com (marc nicole)
Date: Thu, 30 Mar 2023 13:41:09 +0200
Subject: [Tutor] combinations of all rows and cols from a dataframe
In-Reply-To: <0937eb01-411c-3bbb-dbf8-c78b49231c5b@web.de>
References: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>
 <0937eb01-411c-3bbb-dbf8-c78b49231c5b@web.de>
Message-ID: <CAGJtH9RujsfwxtQNCMC3YCrgzWzsTQviHUmZxbqmC2t85RU+gg@mail.gmail.com>

@Peter what you did is more or less what i was looking for except that i
see "duplicate" tuples e.g.,   ([2], [6, 5]) &  ([6, 5], [2]) which are
unwanted in the final output

Le jeu. 30 mars 2023 ? 10:38, Peter Otten <__peter__ at web.de> a ?crit :

> On 30/03/2023 01:03, marc nicole wrote:
> > Hello everyone,
> >
> > Given a dataframe like this:
> >
> > 2 6
> > 8 5
> >
> > I want to yield the following list of lists:
> > [  [[2],[6,5]],
> > [[2],[6]],
> > [[2],[5]],
> > [[8],[6,5]],
> > [[8],[6]],
> > [[8],[5]],
> > [[6],[2,8]],
> > [[6],[8]],
> > [[6],[2]],
> > [[5],[2,8]],
> > [[5],[2]],
> > [[5],[8]],
> > [[6,5],[2,8]]  ]
> >
> > I have written the following (which doesn't yield the expected results)
> >
> > import pandas as pd
> >> from itertools import combinations
> >> import numpy as np
> >> resList=[]
> >> resListTmp=[]
> >> resListTmp2=[]
> >> dataframe =
> >>
> pd.read_excel("C:\\Users\\user\\Desktop\\testData.xlsx",index_col=False,header=None)
> >
> > for i in range(0, len(dataframe)+1):
> >>      for j in range(0, len(dataframe.columns)):
> >>          for k in range (0,len(dataframe)+1):
> >>              for xVals in list(combinations(dataframe.iloc[k:i,j], i)):
> >>                  if list(xVals) not in resListTmp:
> >>                      resListTmp.append(list(xVals))
> >>          resListTmp2.append(resListTmp)
> >>      resList.append(resListTmp2)
> >> print(resList)
> >>
> >
> > What is wrong with my code?
>
> I think you need to move the initialization of the temporary list into
> the respective loop, but was unable to get it to work.
>
> My second attempt was to start with the combinations from one column:
>
>  >>> def colcomb(column):
>         result = []
>         for i in range(len(column)):
>                 for c in combinations(column, i+1):
>                         result.append(list(c))
>         return result
>
>  >>> colcomb([2, 8])
> [[2], [8], [2, 8]]
>
> But what could be the next step? I'm unable to infer it from your sample
> result which seems to be somewhere between
>
>  >>> a = [[2, 8], [6, 5]]  # transposed to avoid pandas/numpy
>  >>> from itertools import product
>  >>> list(product(*(colcomb(col) for col in a)))
> [([2], [6]), ([2], [5]), ([2], [6, 5]), ([8], [6]), ([8], [5]), ([8],
> [6, 5]), ([2, 8], [6]), ([2, 8], [5]), ([2, 8], [6, 5])]
>
> and
>
>  >>> x = list(product(*(colcomb(col) for col in a)))
>  >>> x + [y[::-1] for y in x]
> [([2], [6]), ([2], [5]), ([2], [6, 5]), ([8], [6]), ([8], [5]), ([8],
> [6, 5]), ([2, 8], [6]), ([2, 8], [5]), ([2, 8], [6, 5]), ([6], [2]),
> ([5], [2]), ([6, 5], [2]), ([6], [8]), ([5], [8]), ([6, 5], [8]), ([6],
> [2, 8]), ([5], [2, 8]), ([6, 5], [2, 8])]
>
> Perhaps you can try and tell us what you want in plain English?
>
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From mk1853387 at gmail.com  Thu Mar 30 16:14:41 2023
From: mk1853387 at gmail.com (marc nicole)
Date: Thu, 30 Mar 2023 22:14:41 +0200
Subject: [Tutor] Get all possible ranges of columns across a dataframe's
Message-ID: <CAGJtH9T7rP_RdUWeaR3HHwD8e4-K_=-Ohh-ajpwjr33B0Lw79Q@mail.gmail.com>

I have this dataframe:

    12 2 17
    16 4 16
    2 19 11


I want to get, accross its columns, the following output

    [
> [12,16,2],[2,4,19],[17,16,11],[[12,16,2],[2,4,19]],[[2,4,19],[17,16,11]],[[12,16,2],[2,4,19],[17,16,11]]
>  ]


I have this code which yield the first 3 possibilities only:


>
>
>
>
>
>
>
> *from itertools import combinations        resultTmp2 = []        for j in
> range(1, len(dataframe.columns) + 1):          resultTmp = []          for
> xVal in list(combinations(dataframe.iloc[:len(dataframe) + 1,j-1],
>  len(dataframe)  )):             resultTmp.append(list(xVal))
> resultTmp2.append(resultTmp)        print(resultTmp2)*


How to update my code so that it yields correct mentioned output?

From alexkleider at gmail.com  Thu Mar 30 19:39:25 2023
From: alexkleider at gmail.com (Alex Kleider)
Date: Thu, 30 Mar 2023 16:39:25 -0700
Subject: [Tutor] My function correct_num()
In-Reply-To: <CAM53q1QDQJd2-=dHCWWhBYNsoDTD8C2UeL2b_j7V-wKL_PhiXg@mail.gmail.com>
References: <CAM53q1QDQJd2-=dHCWWhBYNsoDTD8C2UeL2b_j7V-wKL_PhiXg@mail.gmail.com>
Message-ID: <CAMCEyD7O=JOkVNOyhem3v49skPoc15P4tex6d7LzG57auq1wsw@mail.gmail.com>

String formatting might be a better way to deal with your issue.

On Thu, Mar 30, 2023, 4:36 PM Goran Ikac <goranikac65 at gmail.com> wrote:

> Hi, I wish a nice day to every pythonist out there!
> I'm a newbie, still learning the basics. For an exercise, I wrote a
> function *correct_num() *to avoid ridiculous presentation of some numeric
> calculations in Python (e.g. *print(.1 + .1 + .1) *outputs
> *0.30000000000000004*):
>
> # written by ike 2023
>
> def correct_num(num: float) -> float:
>     """
>     The function takes any number as the only argument.
>     If the number is an integer, or if it is a float with up to *six *
> decimal
>     digits, the function returns the same number.
>     If the number is a float with more than *six *decimal digits,
>     the function returns that number with up to one digit less precision.
>     That way, it corrects the errors made by the computer's numerical
>     calculations.
>
>     >>> .1 + .1 + .1
>     0.30000000000000004
>     >>> correct_num(.1 + .1 + .1)
>     0.3
>     >>> 24.95 - 9.98
>     14.969999999999999
>     >>> correct_num(24.95 - 9.98)
>     14.97
>     >>> correct_num(123)
>     123
>     >>> correct_num(123.0)
>     123.0
>     >>> correct_num(-123.0)
>     -123.0
>     >>> correct_num(0.123456)
>     0.123456
>     >>> correct_num(0.1234567)
>     0.123457
>     >>> correct_num(0.123456789)
>     0.12345679
>     >>> correct_num(-0.123456789)
>     -0.12345679
>     >>> correct_num(5.12345678e-5)
>     5.1e-05
>     >>> correct_num(5.17345678e-5)
>     5.2e-05
>     >>> correct_num(5.173e-6)
>     5e-06
>     >>> correct_num(5.173e-7)
>     1e-06
>     >>> >>> correct_num(5.173e-8)
>     0.0
>
>     """
>
>     if num == int(num):             # If the number is an integer,
>         return num                      # return the same number.
>
>     numstr = str(num)
>
>     if 'e' in numstr:               # If num is written in scientific
> notation
>         numstr = '{:.*7*f}'.format(num)
>
>     # print('numstring ==', numstr)     # This is a control line.
>
>     if '.' in numstr:               # If the numstr represents a float,
>         # determine the number of num's decimal digits
>         dec_digits = len(numstr[(numstr.index('.') + 1):])
>         # If the limitation in how computers do arithmetic doesn't affect
> num:
>         if dec_digits < *7*:
>             return num                  # return the same number.
>
>     return round(num, dec_digits - 1)
>
>
> Now, I'm happy with the function's work, but I don't know what number of
> decimal digits to leave alone, e.g. what number of decimal digits are
> certainly OK). I've decided it to be *six *decimal digits:
>         if dec_digits < *7*:
>             return num                  # return the same number.
> but it was by pure intuition. Does anybody know what is the right number of
> decimal digits to leave as they were returned by Python numeric
> calculations?
> And, of course, I'd be thankful for any comment about my code.
> Also, can anybody, please, correct my English in the docstring?
> Watch out! I'm riding to catch up with you, big boys and girls.
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>

From wlfraed at ix.netcom.com  Thu Mar 30 21:47:25 2023
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 30 Mar 2023 21:47:25 -0400
Subject: [Tutor] My function correct_num()
References: <CAM53q1QDQJd2-=dHCWWhBYNsoDTD8C2UeL2b_j7V-wKL_PhiXg@mail.gmail.com>
Message-ID: <tqdc2i5lfifc2feho9e9ghfvqsnfqselam@4ax.com>

On Thu, 30 Mar 2023 14:58:12 +0200, Goran Ikac <goranikac65 at gmail.com>
declaimed the following:

>Hi, I wish a nice day to every pythonist out there!
>I'm a newbie, still learning the basics. For an exercise, I wrote a
>function *correct_num() *to avoid ridiculous presentation of some numeric
>calculations in Python (e.g. *print(.1 + .1 + .1) *outputs
>*0.30000000000000004*):
>

	Since 0.1 is not power-of-two, it is a repeating binary value. Your
"ridiculous presentation" is Python attempting to display as much accuracy
as it can for the value.

	You can format the output easily, without invoking some long
function...

>>> x = .1 + .1 + .1
>>> print("%s" % x)
0.30000000000000004
>>> print("%f" % x)
0.300000
>>> print("%4.2f" % x)
0.30
>>> 
>>> print("%4.2e" % x)
3.00e-01
>>> 

{There are other ways to format output but the old "string interpolation"
just feels natural to me -- similar to C's printf() family.}

<SNIP>

>Now, I'm happy with the function's work, but I don't know what number of
>decimal digits to leave alone, e.g. what number of decimal digits are
>certainly OK). I've decided it to be *six *decimal digits:
>        if dec_digits < *7*:
>            return num                  # return the same number.
>but it was by pure intuition. Does anybody know what is the right number of
>decimal digits to leave as they were returned by Python numeric
>calculations?

	SINGLE precision (32-bit) floating point is commonly considered to be
good for 7 significant digits (significant means the digits before AND
after the decimal point.

>>> print("%13.6e" % x)
 3.000000e-01
>>> print("%13.6e" % -x)
-3.000000e-01
>>> 

Note that I've set the field width to allow for negative sign, the decimal
point, and the 4-character exponent.

	DOUBLE precision (64-bit) floating point is typically considered good
for 15 significant digits (Python uses double precision).

>>> print("%21.14e" % x)
 3.00000000000000e-01
>>> print("%21.14e" % -x)
-3.00000000000000e-01
>>> print("%23.16e" % x)
 3.000000000000000e-01
>>> print("%23.16e" % x)
 3.0000000000000004e-01
>>> print("%23.16e" % -x)
-3.0000000000000004e-01
>>> 

Python appears to be using 17 significant digits for its default output.


From wlfraed at ix.netcom.com  Thu Mar 30 21:51:43 2023
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 30 Mar 2023 21:51:43 -0400
Subject: [Tutor] combinations of all rows and cols from a dataframe
References: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>
 <0937eb01-411c-3bbb-dbf8-c78b49231c5b@web.de>
 <CAGJtH9RujsfwxtQNCMC3YCrgzWzsTQviHUmZxbqmC2t85RU+gg@mail.gmail.com>
Message-ID: <h1fc2i14f6ftulat3veb999mprh8ibpgue@4ax.com>

On Thu, 30 Mar 2023 13:41:09 +0200, marc nicole <mk1853387 at gmail.com>
declaimed the following:

>@Peter what you did is more or less what i was looking for except that i
>see "duplicate" tuples e.g.,   ([2], [6, 5]) &  ([6, 5], [2]) which are
>unwanted in the final output

	So... you may want to drop the tuples of lists, and go for sets of
tuples? And a set for the overall result system. Sets don't allow
duplicates, for the most part.


From wlfraed at ix.netcom.com  Thu Mar 30 22:02:43 2023
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 30 Mar 2023 22:02:43 -0400
Subject: [Tutor] Get all possible ranges of columns across a dataframe's
References: <CAGJtH9T7rP_RdUWeaR3HHwD8e4-K_=-Ohh-ajpwjr33B0Lw79Q@mail.gmail.com>
Message-ID: <36fc2ipnpcmi4a7plln2rq91qsa632q4j2@4ax.com>

On Thu, 30 Mar 2023 22:14:41 +0200, marc nicole <mk1853387 at gmail.com>
declaimed the following:

>I want to get, accross its columns, the following output
>

	This sounds a lot like just a new variation of your previous question.
The similarity makes them seem to be a lot like home work assignments. The
main purpose of this forum to assist with questions about Python, not to
solve home work problems -- or designing algorithms.

	
>>
>> *from itertools import combinations        resultTmp2 = []        for j in
>> range(1, len(dataframe.columns) + 1):          resultTmp = []          for
>> xVal in list(combinations(dataframe.iloc[:len(dataframe) + 1,j-1],
>>  len(dataframe)  )):             resultTmp.append(list(xVal))
>> resultTmp2.append(resultTmp)        print(resultTmp2)*
>

	Please use a client that doesn't strip line endings and/or leading
spaces... That may mean NOT USING the Google/Gmail web access!

	That mess is unusable. Python requires white space (indentation) to
determine code blocks, and the above is completely not executable. It is
also incomplete -- where is dataframe defined?

	Walk through the code you have using /paper/ to track what each name is
bound to at each point of the code. That may show you where your algorithm
is deficient.




From wlfraed at ix.netcom.com  Thu Mar 30 22:23:43 2023
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 30 Mar 2023 22:23:43 -0400
Subject: [Tutor] My function correct_num()
References: <CAM53q1QDQJd2-=dHCWWhBYNsoDTD8C2UeL2b_j7V-wKL_PhiXg@mail.gmail.com>
 <tqdc2i5lfifc2feho9e9ghfvqsnfqselam@4ax.com>
Message-ID: <jtgc2i98jae3lknna0h904qm6m9jefv1jk@4ax.com>


Mea Culpa

On Thu, 30 Mar 2023 21:47:25 -0400, Dennis Lee Bieber
<wlfraed at ix.netcom.com> declaimed the following:


>>>> print("%21.14e" % x)
> 3.00000000000000e-01
>>>> print("%21.14e" % -x)
>-3.00000000000000e-01
>>>> print("%23.16e" % x)
> 3.000000000000000e-01

	Whoops, ignore that one -- I'd edited in the PythonWin console before
duplicating it to a fresh line.

>>>> print("%23.16e" % x)
> 3.0000000000000004e-01
>>>> print("%23.16e" % -x)
>-3.0000000000000004e-01
>>>> 
>


From roel at roelschroeven.net  Fri Mar 31 04:06:25 2023
From: roel at roelschroeven.net (Roel Schroeven)
Date: Fri, 31 Mar 2023 10:06:25 +0200
Subject: [Tutor] combinations of all rows and cols from a dataframe
In-Reply-To: <CAGJtH9RujsfwxtQNCMC3YCrgzWzsTQviHUmZxbqmC2t85RU+gg@mail.gmail.com>
References: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>
 <0937eb01-411c-3bbb-dbf8-c78b49231c5b@web.de>
 <CAGJtH9RujsfwxtQNCMC3YCrgzWzsTQviHUmZxbqmC2t85RU+gg@mail.gmail.com>
Message-ID: <ba85b178-711a-2e43-f8a4-6cded43ccd70@roelschroeven.net>

Op 30/03/2023 om 13:41 schreef marc nicole:
> @Peter what you did is more or less what i was looking for except that i
> see "duplicate" tuples e.g.,   ([2], [6, 5]) &  ([6, 5], [2]) which are
> unwanted in the final output
There where duplicates in your example output too: you had both [[2], 
[6]] and [[6], [2]], and [[2], [5]] and [[5], [2]]. That's probably one 
of the reasons people asked you to state the requirements clearly.

-- 
"Ever since I learned about confirmation bias, I've been seeing
it everywhere."
         -- Jon Ronson


From __peter__ at web.de  Fri Mar 31 07:55:17 2023
From: __peter__ at web.de (Peter Otten)
Date: Fri, 31 Mar 2023 13:55:17 +0200
Subject: [Tutor] combinations of all rows and cols from a dataframe
In-Reply-To: <ba85b178-711a-2e43-f8a4-6cded43ccd70@roelschroeven.net>
References: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>
 <0937eb01-411c-3bbb-dbf8-c78b49231c5b@web.de>
 <CAGJtH9RujsfwxtQNCMC3YCrgzWzsTQviHUmZxbqmC2t85RU+gg@mail.gmail.com>
 <ba85b178-711a-2e43-f8a4-6cded43ccd70@roelschroeven.net>
Message-ID: <889619fb-5193-0660-4027-d97e5a11cb57@web.de>

On 31/03/2023 10:06, Roel Schroeven wrote:

> Op 30/03/2023 om 13:41 schreef marc nicole:
>> @Peter what you did is more or less what i was looking for except that i
>> see "duplicate" tuples e.g.,?? ([2], [6, 5]) &? ([6, 5], [2]) which are
>> unwanted in the final output

> There where duplicates in your example output too: you had both [[2],
> [6]] and [[6], [2]], and [[2], [5]] and [[5], [2]]. That's probably one
> of the reasons people asked you to state the requirements clearly.

As Roel says. I gave you a possible solution with no duplicates and
another with all duplicates, but your reference solution has some
duplicates.

From threesomequarks at proton.me  Fri Mar 31 10:32:17 2023
From: threesomequarks at proton.me (ThreeBlindQuarks)
Date: Fri, 31 Mar 2023 14:32:17 +0000
Subject: [Tutor] combinations of all rows and cols from a dataframe
In-Reply-To: <889619fb-5193-0660-4027-d97e5a11cb57@web.de>
References: <CAGJtH9QJ+hyci_mYe9JsvZrXAELfx-zeXDi-pipqh0MmxZARkg@mail.gmail.com>
 <0937eb01-411c-3bbb-dbf8-c78b49231c5b@web.de>
 <CAGJtH9RujsfwxtQNCMC3YCrgzWzsTQviHUmZxbqmC2t85RU+gg@mail.gmail.com>
 <ba85b178-711a-2e43-f8a4-6cded43ccd70@roelschroeven.net>
 <889619fb-5193-0660-4027-d97e5a11cb57@web.de>
Message-ID: <8gtrsRPezeMgZS-u7ORDDXogL2dWMZKuCmBrwpkIV3cOi3OyxxrxQNnlFb6M6MTA4JczC1hzmSJ8UEdIwK8DxUYKX2AIiOlUyPYmvVESIoI=@proton.me>


I ask people posting questions to put themselves into the position of the ones being asked about it and try to give enough info.

There are many reasons people give just a little info, including not wanting us to see it is homework, or not being noticed by others near them as asking for help and so on.

We now have two somewhat related questions on the table and both are not complete. I doubt the original problem is actually as simple as to take this specific 2x2 object and create a nested list structure containing exactly these contents. There is a trivial and useless solution to such one-time problems consisting of a print statement of the typed-in answer.

A serious question might look like: given a structure like a Dataframe with M rows and N columns, with both being at least 2 in length (or whatever) then make all combinations ... and make a list of that output.

Then provide one or a few examples that can be used to see if the program written to solve it works. If needed, provide criteria needed to see if the answer that may look right, is still wrong in the general case.

As an example, the first question looked like it wanted duplicates that were mirror images of each other (or perhaps just contained the same members in different orders) removed. But maybe it wanted a specific one kept as in the sorted in order one. One thought was to use a python set as an intermediate data structure to consolidate duplicates. But that has lots of considerations and may require some care AND it has an obvious flaw in that multiple copies of the same number may be swallowed. I won't say more except that the lack of clearer and even somewhat abstract requirements makes it hard to know what a proper solution might be.

If someone had provided some motivation of a real-world problem, who knows? What if I had a question about say a group of friends who entered a room in pairs shown as two columns and later walked out sometimes alone and sometimes in groups of two or three or perhaps more and you want to know which could happen. Sounds like all combinations? But what if on the way out, the guards checked and only allowed groups out that contained no more than one person from a group while allowing any number from the other group, as a way to foster some sort of interaction. You could have either only one person from column A or only one from column B. I MADE THIS EXAMPLE UP, but this could be an example that helps people understand what you are hoping to do and maybe create several test cases and see if the algorithm handles them.

As currently stated, no motivation has been given why anyone wants to do this. Sadly, this is not uncommon especially in homework assignments.

But as noted, the goal here is not doing homework for people but rather helping people to learn to help themselves. A narrower question might have been that your algorithm returned doubles, so how do you remove the doubles. Or maybe the algorithm is returning empty lists and you wonder how to change it to avoid that.

I won't go on by trying to solve the problem but want to point out the subject of these messages is about a Dataframe and wanting combinations of rows and columns. That may be a bit deceptive as it looks like one possible idea here is not at all about dataframes. It sounds like the question is given two (possibly more) collections of unrelated data except perhaps all of the same size, take one collection at a time and then connect each item with some kind of partial permutation of the second/other  (or maybe more) collections and then finally consolidate the results into a list containing sublists as needed.

If the above makes sense, and perhaps it is nonsense, then it might suggest a different approach in which the data read in as a Dataframe is rapidly copied into other data structures and manipulated.

The volunteers here will work with questions asked even if they are not already perfectly presented, of course, but it can be way more effective if they know what they are being asked up front. And I find that the person asking can often figure out their own answer in the process of trying to explain it well to others. 

- Q







Sent with Proton Mail secure email.

------- Original Message -------
On Friday, March 31st, 2023 at 7:55 AM, Peter Otten <__peter__ at web.de> wrote:


> On 31/03/2023 10:06, Roel Schroeven wrote:
> 
> > Op 30/03/2023 om 13:41 schreef marc nicole:
> > 
> > > @Peter what you did is more or less what i was looking for except that i
> > > see "duplicate" tuples e.g., ([2], [6, 5]) & ([6, 5], [2]) which are
> > > unwanted in the final output
> 
> > There where duplicates in your example output too: you had both [[2],
> > [6]] and [[6], [2]], and [[2], [5]] and [[5], [2]]. That's probably one
> > of the reasons people asked you to state the requirements clearly.
> 
> 
> As Roel says. I gave you a possible solution with no duplicates and
> another with all duplicates, but your reference solution has some
> duplicates.
> _______________________________________________
> Tutor maillist - Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor

From phillor9 at gmail.com  Fri Mar 31 21:21:09 2023
From: phillor9 at gmail.com (Phil)
Date: Sat, 1 Apr 2023 11:21:09 +1000
Subject: [Tutor] Pen plotter data format
Message-ID: <2c80cfbb-4c87-9b77-96b6-41f120549316@gmail.com>

This is probably an off-topic question but I cannot think of another 
group that may offer any help. All of the Python software is self-written.

Briefly, my laptop reads a gcode file and extracts the x any y 
coordinates into two lists (one x and one y). The pen-up and pen_down 
commands are not currently saved. The laptop then sends the x and y 
coordinates to the micro-controller one at a time until the lists are 
empty. The controller expects and waits for three bytes of x values and 
then three bytes of y values in that order.

I'm thinking that a better approach might to send an x byte, for 
example, like this "x, 123" and have the micro-controller separate the 
value. Currently, the micro-controller only accepts data and moves the 
motors and doesn't perform any processing.

Something like this might be better:

if cmd starts with 'x';
 ??? strip off the coord value
 ??? run the motor to that coord

if cmd starts with 'y':
 ??? etc

if cmd starts with 'z':
 ??? lift or lower pen

I'm still not sure how the laptop should handle the 'z' (pen-up) 
commands because they don't fit neatly into the same x and y pair 
pattern. Maybe the laptop should just parse the gcode file one line at a 
time and then send the complete line to the micro-controller for 
processing until the end of the file is reached? That would mean that 
all sent bytes would have to be the same length but I could add padding 
as I'm currently doing.

I don't know what format pen plotters normally receive their data in and 
I'm hoping that someone may know. An Internet search hasn't helped so far.







-- 
Regards,
Phil


From wlfraed at ix.netcom.com  Fri Mar 31 22:12:25 2023
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Fri, 31 Mar 2023 22:12:25 -0400
Subject: [Tutor] Pen plotter data format
References: <2c80cfbb-4c87-9b77-96b6-41f120549316@gmail.com>
Message-ID: <ci3f2ih9or9t10c9sm9sgaklfs6vp6bq87@4ax.com>

On Sat, 1 Apr 2023 11:21:09 +1000, Phil <phillor9 at gmail.com> declaimed the
following:


>I don't know what format pen plotters normally receive their data in and 
>I'm hoping that someone may know. An Internet search hasn't helped so far.

	What search terms have you tried? Note that most of these use HP-GL
control language, NOT the CNC g-code. If you are trying to create an actual
pen plotter, and not a 3-D printer, writing a parser for HP-GL may be
desirable.


http://support.ricoh.com/bb_v1oi/pub_e/oi_view/0001036/0001036829/view/printer/unv/0125.htm
https://medium.com/quarterstudio/an-intro-to-pen-plotters-29b6bd4327ba
https://chiplotle.readthedocs.io/en/latest/chapters/plotters/index.html
http://sites.music.columbia.edu/cmc/chiplotle/manual/chapters/tutorial/intro.html
<https://www.raspberry-pi-geek.com/Archive/2015/11/A-Python-interface-to-a-large-format-pen-plotter/(offset)/2>

	Note that the absolute minimum command set appears to be: IN
(initialize device -- likely means zeroing the paper position if the paper
also moves), SPn (Select Pen n), PUx,y (Pen Up, move to (x,y)), PDx,y (Pen
Down, move to (x,y)). All sent as ";" delimited text strings (but not
needed as a termination, it seems from the last link).


From cs at cskk.id.au  Fri Mar 31 21:54:51 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Sat, 1 Apr 2023 12:54:51 +1100
Subject: [Tutor] Pen plotter data format
In-Reply-To: <2c80cfbb-4c87-9b77-96b6-41f120549316@gmail.com>
References: <2c80cfbb-4c87-9b77-96b6-41f120549316@gmail.com>
Message-ID: <ZCeO6359VTG71N6o@cskk.homeip.net>

On 01Apr2023 11:21, Phil <phillor9 at gmail.com> wrote:
>Briefly, my laptop reads a gcode file and extracts the x any y 
>coordinates into two lists (one x and one y). The pen-up and pen_down 
>commands are not currently saved. The laptop then sends the x and y 
>coordinates to the micro-controller one at a time until the lists are 
>empty. The controller expects and waits for three bytes of x values and 
>then three bytes of y values in that order.

Why in threes?

Is this approach deficient because there's no delimiters, so if you're 
out of sync you misinterpret x as y etc?

>I'm thinking that a better approach might to send an x byte, for 
>example, like this "x, 123"

I'd be inclined to drop the comma.

>and have the micro-controller separate the value.

It's running software of your own, in Python, receiving the data?

>Currently, the micro-controller only accepts data and moves the motors 
>and doesn't perform any processing.
>
>Something like this might be better:
>
>if cmd starts with 'x';
>??? strip off the coord value
>??? run the motor to that coord
>
>if cmd starts with 'y':
>??? etc
>
>if cmd starts with 'z':
>??? lift or lower pen

Sounds ok.

>I'm still not sure how the laptop should handle the 'z' (pen-up) 
>commands because they don't fit neatly into the same x and y pair 
>pattern.

I happened to look at HPGL recently for a job application and IIRC it 
has pen-up motions (to x,y) and pen-down motions (to x,y). (And pick a 
pen colour, etc etc etc.)

>Maybe the laptop should just parse the gcode file one line at a time 
>and then send the complete line to the micro-controller for processing 
>until the end of the file is reached? That would mean that all sent 
>bytes would have to be the same length but I could add padding as I'm 
>currently doing.

I think you need to describe what's going on with the micro-controller a 
bit more. Is there a reason you can just progressively read the G-Code 
and stream instructions to the micro-controller as parsed? Do you need 
to wait for the micro-controller to physically perform the motions?

>I don't know what format pen plotters normally receive their data in 
>and I'm hoping that someone may know. An Internet search hasn't helped 
>so far.

Once upon a time a lot of HP plotters used HPGL. I wrote a (very) basic 
parser for the above mentioned job application. The language spec reads 
to me like something written after the printers were in the wild, as a 
formalisation of the protocol which people had implemented as they went; 
a bit ad hoc and weird.

No docs for your plotter? Or have you complete control?

Cheers,
Cameron Simpson <cs at cskk.id.au>

From cs at cskk.id.au  Fri Mar 31 23:19:22 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Sat, 1 Apr 2023 14:19:22 +1100
Subject: [Tutor] Pen plotter data format
In-Reply-To: <ci3f2ih9or9t10c9sm9sgaklfs6vp6bq87@4ax.com>
References: <ci3f2ih9or9t10c9sm9sgaklfs6vp6bq87@4ax.com>
Message-ID: <ZCeiukMarfiKNKvt@cskk.homeip.net>

On 31Mar2023 22:12, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>	Note that the absolute minimum command set appears to be: IN
>(initialize device -- likely means zeroing the paper position if the paper
>also moves), SPn (Select Pen n), PUx,y (Pen Up, move to (x,y)), PDx,y (Pen
>Down, move to (x,y)). All sent as ";" delimited text strings (but not
>needed as a termination, it seems from the last link).

Yeah, the termination is pretty weird. But is Phil generating HP-GL or 
trying to parse it? Generating is much easier because you can be nice 
and regular. Parsing has to handle some flexible termination.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From phillor9 at gmail.com  Fri Mar 31 23:54:23 2023
From: phillor9 at gmail.com (Phil)
Date: Sat, 1 Apr 2023 13:54:23 +1000
Subject: [Tutor] Pen plotter data format
In-Reply-To: <ZCeO6359VTG71N6o@cskk.homeip.net>
References: <2c80cfbb-4c87-9b77-96b6-41f120549316@gmail.com>
 <ZCeO6359VTG71N6o@cskk.homeip.net>
Message-ID: <dc7de798-6d14-dcd4-9cd3-2d9a76d3ed38@gmail.com>


On 1/4/23 11:54, Cameron Simpson wrote:

Thank you Cameron and also Dennis for the links. I should get some 
inspiration from them.

>
> Why in threes?
The original plan was to build a plotter with at least a 100mm x 100mm 
drawing surface. However, due to the parts that I have on hand plus some 
parts that are in the mail somewhere that area has been reduced so I 
will now only need to pass a two digit coordinate byte.
>
> Is this approach deficient because there's no delimiters, so if you're 
> out of sync you misinterpret x as y etc?
I haven't included a method that prevents getting out of sync but that's 
not a problem.
>
>> I'm thinking that a better approach might to send an x byte, for 
>> example, like this "x, 123"
>
> I'd be inclined to drop the comma.
The comma is part of a gcode line but it's easily removed.
>
>> and have the micro-controller separate the value.
>
> It's running software of your own, in Python, receiving the data?
Yes.
>
>> Currently, the micro-controller only accepts data and moves the 
>> motors and doesn't perform any processing.
>>
>> Something like this might be better:
>>
>> if cmd starts with 'x';
>> ??? strip off the coord value
>> ??? run the motor to that coord
>>
>> if cmd starts with 'y':
>> ??? etc
>>
>> if cmd starts with 'z':
>> ??? lift or lower pen
>
> Sounds ok.

The more I think about the more I think so too. The sweat is dripping 
off me at the moment so I'll leave further experimenting until tonight.


> I happened to look at HPGL recently for a job application and IIRC it 
> has pen-up motions (to x,y) and pen-down motions (to x,y). (And pick a 
> pen colour, etc etc etc.)

Dennis mention HPGL too. I'll look into that as well.


> I think you need to describe what's going on with the micro-controller 
> a bit more. Is there a reason you can just progressively read the 
> G-Code and stream instructions to the micro-controller as parsed? Do 
> you need to wait for the micro-controller to physically perform the 
> motions?

After the micro-controller has received a byte and moved the motor it 
send an acknowledgement back to the laptop signalling that it's ready 
for the next byte.

>
>
>> I don't know what format pen plotters normally receive their data in 
>> and I'm hoping that someone may know. An Internet search hasn't 
>> helped so far.

> No docs for your plotter? Or have you complete control?

I'm building a plotter from scratch using Lego parts and I'm not using 
the Lego software. It's just a project to keep me out of mischief and I 
doubt the plotter will ever serve a useful function. I'll be pleased if 
it produces a reasonable plot.

-- 

Regards,
Phil


From phillor9 at gmail.com  Fri Mar 31 23:59:19 2023
From: phillor9 at gmail.com (Phil)
Date: Sat, 1 Apr 2023 13:59:19 +1000
Subject: [Tutor] Pen plotter data format
In-Reply-To: <ZCeiukMarfiKNKvt@cskk.homeip.net>
References: <ci3f2ih9or9t10c9sm9sgaklfs6vp6bq87@4ax.com>
 <ZCeiukMarfiKNKvt@cskk.homeip.net>
Message-ID: <441532a4-aa6f-8081-e6db-f520ebdbbf9e@gmail.com>


On 1/4/23 13:19, Cameron Simpson wrote:
>
> Yeah, the termination is pretty weird. But is Phil generating HP-GL or 
> trying to parse it? Generating is much easier because you can be nice 
> and regular. Parsing has to handle some flexible termination.

I'm generating the gcode from SVG files. I've settled on inkscape to 
generate the images.

-- 

Regards,
Phil