From phillor9 at gmail.com Fri Feb 9 20:58:11 2024 From: phillor9 at gmail.com (Phil) Date: Sat, 10 Feb 2024 11:58:11 +1000 Subject: [Tutor] Importing a class from a sub-directory Message-ID: <247c2e2b-5282-42a8-b9a1-821d15303357@gmail.com> Thank you for reading this and this is a problem that I've struggled with for quite some time. The simple fix was to copy myclass into whatever project directory that I was working on at the time, but of course, this means that I have to maintain multiple copies of the same class. Say I have myclass.py saved in ~/Python and I want to import myclass into myproject.py that is saved in ~/Python/Tkinter/myproject. How do I do that? I have tried countless possibilities including: from ..myclass import Myclass I've created a file named "__init__.py and saved myclass.py into that file I've tried from "~/Python/myclass" import Myclass but that's a syntax error I've scoured the Internet looking for a solution and read several tutorials and even consulted my AI friends and I'm still at a lose to solve this seemingly simple problem. Using the tree command I have confirmed that I have the following directory structure: ~/Python |__ myclass.py |__ Tkinter ??? |_ myproject.py -- Regards, Phil From phillor9 at gmail.com Fri Feb 9 22:18:27 2024 From: phillor9 at gmail.com (Phil) Date: Sat, 10 Feb 2024 13:18:27 +1000 Subject: [Tutor] Regarding importing from a sub-directory Message-ID: <97e6bfe7-8a7c-448a-9b37-fdc13ad602c0@gmail.com> There's no need Alan, to post either of my previous messages to the list. I have discovered the answer to the problem. It's IDLE that's the problem. VS Code does not have a problem importing from a sub_directory. -- Regards, Phil From alan.gauld at yahoo.co.uk Sat Feb 10 04:19:20 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 10 Feb 2024 09:19:20 +0000 Subject: [Tutor] Regarding importing from a sub-directory In-Reply-To: <97e6bfe7-8a7c-448a-9b37-fdc13ad602c0@gmail.com> References: <97e6bfe7-8a7c-448a-9b37-fdc13ad602c0@gmail.com> Message-ID: On 10/02/2024 03:18, Phil wrote: > There's no need Alan, to post either of my previous messages to the > list. I have discovered the answer to the problem. It's IDLE that's the > problem. VS Code does not have a problem importing from a sub_directory. > Since I took you off moderation a while back I no longer see the messages so they all get posted, including this one! :-) However the IDE is not the answer to the problem since they are only used in development. What you need to check is that the import works when you run the code directly in Python, ie without any IDE. I suspect the real solution is for you to modify the PYTHONPATH or sys.path values to include ~/Python. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From phillor9 at gmail.com Sat Feb 10 15:53:10 2024 From: phillor9 at gmail.com (Phil) Date: Sun, 11 Feb 2024 06:53:10 +1000 Subject: [Tutor] Regarding importing from a sub-directory In-Reply-To: References: <97e6bfe7-8a7c-448a-9b37-fdc13ad602c0@gmail.com> Message-ID: On 10/2/24 19:19, Alan Gauld via Tutor wrote: > Since I took you off moderation a while back I no longer see the > messages so they all get posted, including this one! :-) > > However the IDE is not the answer to the problem since they are > only used in development. What you need to check is that the > import works when you run the code directly in Python, > ie without any IDE. > > I suspect the real solution is for you to modify the PYTHONPATH > or sys.path values to include ~/Python. This is another of my messages that I wish I could have cancelled. I found the problem shortly after posting my plea for help and it's IDLE. The library is found if I run my project from the command line and VS Code has no problem finding the library. I've had problems with IDLE from time to time in the past not running Tkinter code correctly or at all and I think that's due to IDLE itself being a Tkinter project. I seldom ever use IDLE but it's the default IDE for Python projects (of course I should change that setting) and it's handy for a quick edit. In this case it wasn't quick and I spent hours chasing a red herring. By the way, my PYTHONPATH does? include ~/Python and I did experiment with setting the sys.path in my project which later turned out be be unnecessary anyway. Thank you again Alan for your time. -- Regards, Phil From cs at cskk.id.au Sat Feb 10 19:52:13 2024 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 11 Feb 2024 11:52:13 +1100 Subject: [Tutor] Regarding importing from a sub-directory In-Reply-To: References: Message-ID: On 11Feb2024 06:53, Phil wrote: >On 10/2/24 19:19, Alan Gauld via Tutor wrote: >>I suspect the real solution is for you to modify the PYTHONPATH >>or sys.path values to include ~/Python. > >This is another of my messages that I wish I could have cancelled. > >I found the problem shortly after posting my plea for help and it's >IDLE. The library is found if I run my project from the command line >and VS Code has no problem finding the library. IMO, cancelling the message isn't a great choice. Better to just followup to it saying: "solved! this is how I fixed my problem" and describing what you did. That way people can benefit from what you learnt and possibly suggest a better approach if there is one. There's no shame in encountering a problem and initially being stumped by it. From alan.gauld at yahoo.co.uk Sun Feb 11 03:59:48 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 11 Feb 2024 08:59:48 +0000 Subject: [Tutor] Regarding importing from a sub-directory In-Reply-To: References: <97e6bfe7-8a7c-448a-9b37-fdc13ad602c0@gmail.com> Message-ID: On 10/02/2024 20:53, Phil wrote: > I've had problems with IDLE from time to time in the past not running > Tkinter code correctly or at all and I think that's due to IDLE itself > being a Tkinter project. There used to be several issues with IDLE but since a major restructure/repair project for Python 3.6/7(?) it has been pretty solid. And the issue with running Tkinter projects was fixed even further back, I think with Python 3.0! But it is odd that it didn't pick up the path properly. However, IDLE has never been intended as a full IDE it was always a demo program to showcase what you could do with Tkinter (along with the ill-fated Grail web browser). For serious work VS Code, Eclipse, etc are definitely better choices. I guess the real learning point here is that with any issues of that type (or indeed in general) it's always best to check what happens running from the command line, just to eliminate any environmental/tooling possibilities. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From jjhartley at gmail.com Wed Feb 28 13:02:50 2024 From: jjhartley at gmail.com (James Hartley) Date: Wed, 28 Feb 2024 12:02:50 -0600 Subject: [Tutor] file accessibility across directories? Message-ID: I am having issues with accessing files across different directories. The file structure I am using is simple: +-project | +- .git +-src | | | +- foo.py | +- bar.py | +- tests | | | +- test_foo.py | +- utils | +- baz.py Test files within project/tests reference classes successfully in foo.py through Pytest as: from src.foo import Foo Pytest is actually executed from the ./project directory successfully as: python -m pytest -v tests/test_foo.py baz.py had been in the src directory, where it successfully referenced class Foo as: from foo import Foo Now baz.py has been moved to utils/baz.py. I would think that this would require the import statement to changed to: from src.foo import Foo ...but I receive a "ModuleNotFoundError: No module named 'src'" error. I have tried adding an empty src/__init__.py file which doesn't change the error received. I always invoke the Python interpreter from the project directory. Always. The ./project/tests directory does not have a __init__.py file present. To date, I have not required this file anywhere, so I have not created it. Any insights shared which can help me resolve this simple issue would be appreciated. From PythonList at DancesWithMice.info Wed Feb 28 14:17:34 2024 From: PythonList at DancesWithMice.info (dn) Date: Thu, 29 Feb 2024 08:17:34 +1300 Subject: [Tutor] file accessibility across directories? In-Reply-To: References: Message-ID: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> James, On 29/02/24 07:02, James Hartley wrote: > I am having issues with accessing files across different directories. The > file structure I am using is simple: > > +-project > | > +- .git > +-src > | | > | +- foo.py > | +- bar.py > | > +- tests > | | > | +- test_foo.py > | > +- utils > | > +- baz.py > > Test files within project/tests reference classes successfully in foo.py > through Pytest as: > > from src.foo import Foo > > Pytest is actually executed from the ./project directory successfully as: > > python -m pytest -v tests/test_foo.py > > baz.py had been in the src directory, where it successfully referenced > class Foo as: > > from foo import Foo > > Now baz.py has been moved to utils/baz.py. I would think that this would > require the import statement to changed to: > > from src.foo import Foo > > ...but I receive a "ModuleNotFoundError: No module named 'src'" error. > > I have tried adding an empty src/__init__.py file which doesn't change the > error received. I always invoke the Python interpreter from the project > directory. Always. > > The ./project/tests directory does not have a __init__.py file present. To > date, I have not required this file anywhere, so I have not created it. > > Any insights shared which can help me resolve this simple issue would be > appreciated. Please clarify the command leading to the error: - running pytest and some unmentioned test file (which contains what code) testing baz.py - running ~/project/utils/baz.py as a script from the project-directory - running a some unmentioned project-script which calls baz.py successfully, before baz.py fails - something else -- Regards, =dn From jjhartley at gmail.com Wed Feb 28 17:39:11 2024 From: jjhartley at gmail.com (James Hartley) Date: Wed, 28 Feb 2024 16:39:11 -0600 Subject: [Tutor] file accessibility across directories? In-Reply-To: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> References: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> Message-ID: While in the project directory: $ pwd /home/me/project $ cat src/foo.py class Foo: pass $ cat utils/baz.py from src.foo import Foo $ python utils/baz.py Traceback (most recent call last): File "./baz.py", line 1 in from src.foo import Foo ModuleNotFoundError: No module named 'src' Conversation opened. 4 messages. All messages read. Conversation opened. 4 messages. All me import issues python/tutor at python.org James Hartley Jan 9, 2024, 12:06?PM to tutor Skip to content Using Gmail with screen readers 14 of 941 import issues python/tutor at python.org James Hartley wrote: > James, > > > On 29/02/24 07:02, James Hartley wrote: > > I am having issues with accessing files across different directories. > The > > file structure I am using is simple: > > > > +-project > > | > > +- .git > > +-src > > | | > > | +- foo.py > > | +- bar.py > > | > > +- tests > > | | > > | +- test_foo.py > > | > > +- utils > > | > > +- baz.py > > > > Test files within project/tests reference classes successfully in foo.py > > through Pytest as: > > > > from src.foo import Foo > > > > Pytest is actually executed from the ./project directory successfully as: > > > > python -m pytest -v tests/test_foo.py > > > > baz.py had been in the src directory, where it successfully referenced > > class Foo as: > > > > from foo import Foo > > > > Now baz.py has been moved to utils/baz.py. I would think that this would > > require the import statement to changed to: > > > > from src.foo import Foo > > > > ...but I receive a "ModuleNotFoundError: No module named 'src'" error. > > > > I have tried adding an empty src/__init__.py file which doesn't change > the > > error received. I always invoke the Python interpreter from the project > > directory. Always. > > > > The ./project/tests directory does not have a __init__.py file present. > To > > date, I have not required this file anywhere, so I have not created it. > > > > Any insights shared which can help me resolve this simple issue would be > > appreciated. > > > Please clarify the command leading to the error: > > - running pytest and some unmentioned test file > (which contains what code) testing baz.py > > - running ~/project/utils/baz.py as a script from the project-directory > > - running a some unmentioned project-script which calls baz.py > successfully, before baz.py fails > > - something else > > -- > Regards, > =dn > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From mats at wichmann.us Wed Feb 28 19:06:13 2024 From: mats at wichmann.us (Mats Wichmann) Date: Wed, 28 Feb 2024 17:06:13 -0700 Subject: [Tutor] file accessibility across directories? In-Reply-To: References: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> Message-ID: On 2/28/24 15:39, James Hartley wrote: > While in the project directory: > $ pwd > /home/me/project > $ cat src/foo.py > class Foo: > pass > $ cat utils/baz.py > from src.foo import Foo > $ python utils/baz.py > Traceback (most recent call last): > File "./baz.py", line 1 in > from src.foo import Foo > ModuleNotFoundError: No module named 'src' When you run a script, the directory of the script is added to sys.path. Not the directory you were in when you ran it. You can print out sys.path in utils/baz.py to convince yourself of this. There's no src in that directory (utils). From PythonList at DancesWithMice.info Wed Feb 28 20:45:00 2024 From: PythonList at DancesWithMice.info (dn) Date: Thu, 29 Feb 2024 14:45:00 +1300 Subject: [Tutor] file accessibility across directories? In-Reply-To: References: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> Message-ID: On 29/02/24 13:06, Mats Wichmann wrote: > On 2/28/24 15:39, James Hartley wrote: >> While in the project directory: >> $ pwd >> /home/me/project >> $ cat src/foo.py >> class Foo: >> ???? pass >> $ cat utils/baz.py >> from src.foo import Foo >> $ python utils/baz.py >> Traceback (most recent call last): >> ?? File "./baz.py", line 1 in >> ???? from src.foo import Foo >> ModuleNotFoundError: No module named 'src' > > > When you run a script, the directory of the script is added to sys.path. > Not the directory you were in when you ran it.? You can print out > sys.path in utils/baz.py to convince yourself of this.? There's no src > in that directory (utils). Whereas pytest works the other way around: run from the project-directory because its (internal) 'process of discovery' will find test* files by searching the entire directory-tree... (hence the questions in earlier reply) -- Regards, =dn From marastboud at gmail.com Thu Feb 29 06:26:01 2024 From: marastboud at gmail.com (Am Ra) Date: Thu, 29 Feb 2024 05:26:01 -0600 Subject: [Tutor] Running Command Prompt In Python Message-ID: Hi, I am trying to activate and run command prompt in python but I am stuck in one section of the code. Below I will explain what I have done so far and what issue I am dealing with. Importing os, and using os.system to connect to Command Prompt. [image: image.png] Then, dmel922.exe will be activated to generate an empty csv file from a raw dataset. [image: image.png] Up to here it is working as it is supposed to. However, it goes on a pause and not adding any data to the empty csv file because it is waiting on the user input which is ?Enter y to Continue?. As we see in the picture below, the Command Prompt is asking the user to press y to continue. As soon as, I press y in the Command Prompt it will continue and finish the task. [image: image.png] I just cannot figure out how to write a code in python to answer that ?Enter y to Continue? prompt or bypass that so that the system continues and finishes its task. Please advise. Thank you very much. Sincerely, Mike From mats at wichmann.us Thu Feb 29 10:17:23 2024 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 29 Feb 2024 08:17:23 -0700 Subject: [Tutor] Running Command Prompt In Python In-Reply-To: References: Message-ID: On 2/29/24 04:26, Am Ra wrote: > Hi, > I am trying to activate and run command prompt in python but I am stuck in > one section of the code. Below I will explain what I have done so far and > what issue I am dealing with. > > Importing os, and using os.system to connect to Command Prompt. > [image: image.png] > > Then, dmel922.exe will be activated to generate an empty csv file from a > raw dataset. > [image: image.png] > > Up to here it is working as it is supposed to. However, it goes on a pause > and not adding any data to the empty csv file because it is waiting on the > user input which is ?Enter y to Continue?. As we see in the picture below, > the Command Prompt is asking the user to press y to continue. As soon as, I > press y in the Command Prompt it will continue and finish the task. > [image: image.png] > > I just cannot figure out how to write a code in python to answer that > ?Enter y to Continue? prompt or bypass that so that the system continues > and finishes its task. > > Please advise. Thank you very much. Note: we can't see any of your pictures, the mailing list strips them. Just paste the text, it's much easier for people to work with anyway. A lot depends on how the Windows executable (that we know nothing about beyond what little you've said) is coded. Some programs do cute things like flush any pending input before issuing a prompt for a response, which would defeat the below. If you use subprocess.run to call the program, you have the ability to connect to the input pipe, so you can send it some text. Whether it will accept that input in the way you hope we cannot tell. At a conceptual level, then: from subprocess import run, PIPE p = run(command, stdout=PIPE, input="y\n", shell=True, text=True) # do something with p.stdout, e.g. check that the question was what you thought # do something with p.returncode, e.g. check it exited successfully # shell=True if you really wanted a cmd shell, if you read the subprocess # docs you'll see that's often not the case. From mats at wichmann.us Thu Feb 29 11:54:52 2024 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 29 Feb 2024 09:54:52 -0700 Subject: [Tutor] file accessibility across directories? In-Reply-To: References: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> Message-ID: <3c8673a2-1e52-4c0f-95b3-c663cc478091@wichmann.us> On 2/28/24 18:45, dn via Tutor wrote: > On 29/02/24 13:06, Mats Wichmann wrote: >> On 2/28/24 15:39, James Hartley wrote: >>> While in the project directory: >>> $ pwd >>> /home/me/project >>> $ cat src/foo.py >>> class Foo: >>> ???? pass >>> $ cat utils/baz.py >>> from src.foo import Foo >>> $ python utils/baz.py >>> Traceback (most recent call last): >>> ?? File "./baz.py", line 1 in >>> ???? from src.foo import Foo >>> ModuleNotFoundError: No module named 'src' >> >> >> When you run a script, the directory of the script is added to >> sys.path. Not the directory you were in when you ran it.? You can >> print out sys.path in utils/baz.py to convince yourself of this. >> There's no src in that directory (utils). > > Whereas pytest works the other way around: run from the > project-directory because its (internal) 'process of discovery' will > find test* files by searching the entire directory-tree... > (hence the questions in earlier reply) Right... pytest takes a common situation and helps you out, in this case (a) by discovering tests and (b) fiddling paths so those tests can import cleanly. If you don't get that help by using (for example) a test framework, you have to take care of it yourself. This import fails because for an absolute import (src.foo - doesn't start with dots), the starting directory has to be in the search path (sys.path), but it isn't. You can fix that by adding to sys.path in the scripts in utils, or by using PYTHONPATH. For example, this should work: PYTHONPATH=. python utils/baz.py You would think you could use relative paths. However, if you attempt a relative import like: from ..src.foo import Foo it doesn't work, because the way the script was invoked, Python doesn't think it's part of a package (invoked as a script, the name is "__main__", and contains no dots, thus you must not be in a package), and relative paths are package-hierarchy-relative, not filesystem-relative, so you'll get one of Python's famous error messages: ImportError: attempted relative import with no known parent package However, you could run baz as a module, like: python -m utils.baz because now the name of the running module is "utils.baz", so it looks like a package rooted at a level above "utils", and that directory will be in sys.path, so "from src/foo import Foo" should work. It just seems a bit ugly to have to invoke a "top level script" as a module to get imports to work. Python imports are... challenging. (yes, I'm assuming answers to the question @dn asked, so this could be wrong if I made the wrong guess) From mats at wichmann.us Thu Feb 29 12:14:33 2024 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 29 Feb 2024 10:14:33 -0700 Subject: [Tutor] file accessibility across directories? In-Reply-To: <3c8673a2-1e52-4c0f-95b3-c663cc478091@wichmann.us> References: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> <3c8673a2-1e52-4c0f-95b3-c663cc478091@wichmann.us> Message-ID: On 2/29/24 09:54, Mats Wichmann wrote: correction: > because now the name of the running module is "utils.baz", so it looks > like a package rooted at a level above "utils", and that directory will > be in sys.path, so "from src/foo import Foo" should work. sigh, type rather than paste and you'll get it wrong... should have been from src.foo import Foo From jjhartley at gmail.com Thu Feb 29 12:16:30 2024 From: jjhartley at gmail.com (James Hartley) Date: Thu, 29 Feb 2024 11:16:30 -0600 Subject: [Tutor] file accessibility across directories? In-Reply-To: <3c8673a2-1e52-4c0f-95b3-c663cc478091@wichmann.us> References: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> <3c8673a2-1e52-4c0f-95b3-c663cc478091@wichmann.us> Message-ID: This has been a useful discussion (at least for me). Thank you all for your candor. >From these comments, it sounds like playing with SYSPATH is a common task. I have some database tables defined for SQLAlchemy which I have imported into some Pytest tests. This works fine, but to import the same table definitions into scripts found in other directories suffers from the problems prompting this thread. It appears that I will have to modify SYSPATH to make one (preferrable) definition work in all locations. Thanks again for your comments! On Thu, Feb 29, 2024 at 10:57?AM Mats Wichmann wrote: > On 2/28/24 18:45, dn via Tutor wrote: > > On 29/02/24 13:06, Mats Wichmann wrote: > >> On 2/28/24 15:39, James Hartley wrote: > >>> While in the project directory: > >>> $ pwd > >>> /home/me/project > >>> $ cat src/foo.py > >>> class Foo: > >>> pass > >>> $ cat utils/baz.py > >>> from src.foo import Foo > >>> $ python utils/baz.py > >>> Traceback (most recent call last): > >>> File "./baz.py", line 1 in > >>> from src.foo import Foo > >>> ModuleNotFoundError: No module named 'src' > >> > >> > >> When you run a script, the directory of the script is added to > >> sys.path. Not the directory you were in when you ran it. You can > >> print out sys.path in utils/baz.py to convince yourself of this. > >> There's no src in that directory (utils). > > > > Whereas pytest works the other way around: run from the > > project-directory because its (internal) 'process of discovery' will > > find test* files by searching the entire directory-tree... > > (hence the questions in earlier reply) > > > Right... pytest takes a common situation and helps you out, in this case > (a) by discovering tests and (b) fiddling paths so those tests can > import cleanly. If you don't get that help by using (for example) a test > framework, you have to take care of it yourself. > > This import fails because for an absolute import (src.foo - doesn't > start with dots), the starting directory has to be in the search path > (sys.path), but it isn't. You can fix that by adding to sys.path in the > scripts in utils, or by using PYTHONPATH. For example, this should work: > > PYTHONPATH=. python utils/baz.py > > You would think you could use relative paths. However, if you attempt a > relative import like: > > from ..src.foo import Foo > > it doesn't work, because the way the script was invoked, Python doesn't > think it's part of a package (invoked as a script, the name is > "__main__", and contains no dots, thus you must not be in a package), > and relative paths are package-hierarchy-relative, not > filesystem-relative, so you'll get one of Python's famous error messages: > > ImportError: attempted relative import with no known parent package > > However, you could run baz as a module, like: > > python -m utils.baz > > because now the name of the running module is "utils.baz", so it looks > like a package rooted at a level above "utils", and that directory will > be in sys.path, so "from src/foo import Foo" should work. It just seems > a bit ugly to have to invoke a "top level script" as a module to get > imports to work. > > Python imports are... challenging. > > (yes, I'm assuming answers to the question @dn asked, so this could be > wrong if I made the wrong guess) > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From PythonList at DancesWithMice.info Thu Feb 29 14:40:45 2024 From: PythonList at DancesWithMice.info (dn) Date: Fri, 1 Mar 2024 08:40:45 +1300 Subject: [Tutor] file accessibility across directories? In-Reply-To: References: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> <3c8673a2-1e52-4c0f-95b3-c663cc478091@wichmann.us> Message-ID: James, On 1/03/24 06:16, James Hartley wrote: > This has been a useful discussion (at least for me). Thank you all for > your candor. This is a common concern/problem. However, you outlined it well. Plus, @Mat's expansion is a very good discussion of the interplay between Python (the interpreter) and one's directory structure. Using "python -m ..." is often a good way around. Thus, it's worth others reading (and learning)... > From these comments, it sounds like playing with SYSPATH is a common task. > I have some database tables defined for SQLAlchemy which I have imported > into some Pytest tests. This works fine, but to import the same table > definitions into scripts found in other directories suffers from the > problems prompting this thread. It appears that I will have to modify > SYSPATH to make one (preferrable) definition work in all locations. Yes and No! If you are both customer and supplier, then you probably won't have a problem - either at coding-time or later in life. If however, you distribute the code to others, these aspects take-on a whole new (and 'exciting') dimension! BTW the sub-dir layout is 'standard' for some of the 'installers' used for Python distribution. A first question though: if something has been put into a 'utility' directory - which makes it sound as if it will be import-ed from different parts of the code (even different projects), then why is it being executed as a script? Once the utility routines have been exercised and tested (routines either located in the project/test sub-dir or in project/utils/tests) their (only) access should be by import (from project/src routines)? For further thought (as if there isn't enough swirling your brain - and bearing in-mind that I haven't used SQLAlchemy for a while) is that DRY is a concern for schema as much as it is for Python-code. There's no point in testing one and running another. Oops! Accordingly, a separate file (and/or potentially project sub-dir) which holds the SQL details and access methods, (hah! and here it comes...) accessible from both code and test modules. > Thanks again for your comments! > > On Thu, Feb 29, 2024 at 10:57?AM Mats Wichmann wrote: > >> On 2/28/24 18:45, dn via Tutor wrote: >>> On 29/02/24 13:06, Mats Wichmann wrote: >>>> On 2/28/24 15:39, James Hartley wrote: >>>>> While in the project directory: >>>>> $ pwd >>>>> /home/me/project >>>>> $ cat src/foo.py >>>>> class Foo: >>>>> pass >>>>> $ cat utils/baz.py >>>>> from src.foo import Foo >>>>> $ python utils/baz.py >>>>> Traceback (most recent call last): >>>>> File "./baz.py", line 1 in >>>>> from src.foo import Foo >>>>> ModuleNotFoundError: No module named 'src' >>>> >>>> >>>> When you run a script, the directory of the script is added to >>>> sys.path. Not the directory you were in when you ran it. You can >>>> print out sys.path in utils/baz.py to convince yourself of this. >>>> There's no src in that directory (utils). >>> >>> Whereas pytest works the other way around: run from the >>> project-directory because its (internal) 'process of discovery' will >>> find test* files by searching the entire directory-tree... >>> (hence the questions in earlier reply) >> >> >> Right... pytest takes a common situation and helps you out, in this case >> (a) by discovering tests and (b) fiddling paths so those tests can >> import cleanly. If you don't get that help by using (for example) a test >> framework, you have to take care of it yourself. >> >> This import fails because for an absolute import (src.foo - doesn't >> start with dots), the starting directory has to be in the search path >> (sys.path), but it isn't. You can fix that by adding to sys.path in the >> scripts in utils, or by using PYTHONPATH. For example, this should work: >> >> PYTHONPATH=. python utils/baz.py >> >> You would think you could use relative paths. However, if you attempt a >> relative import like: >> >> from ..src.foo import Foo >> >> it doesn't work, because the way the script was invoked, Python doesn't >> think it's part of a package (invoked as a script, the name is >> "__main__", and contains no dots, thus you must not be in a package), >> and relative paths are package-hierarchy-relative, not >> filesystem-relative, so you'll get one of Python's famous error messages: >> >> ImportError: attempted relative import with no known parent package >> >> However, you could run baz as a module, like: >> >> python -m utils.baz >> >> because now the name of the running module is "utils.baz", so it looks >> like a package rooted at a level above "utils", and that directory will >> be in sys.path, so "from src/foo import Foo" should work. It just seems >> a bit ugly to have to invoke a "top level script" as a module to get >> imports to work. >> >> Python imports are... challenging. >> >> (yes, I'm assuming answers to the question @dn asked, so this could be >> wrong if I made the wrong guess) >> >> _______________________________________________ >> Tutor maillist - Tutor at python.org >> To unsubscribe or change subscription options: >> https://mail.python.org/mailman/listinfo/tutor >> > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -- Regards, =dn From jjhartley at gmail.com Thu Feb 29 15:17:19 2024 From: jjhartley at gmail.com (James Hartley) Date: Thu, 29 Feb 2024 14:17:19 -0600 Subject: [Tutor] file accessibility across directories? In-Reply-To: References: <681593ab-5e2e-45b0-8a83-58e6cebe58c9@DancesWithMice.info> <3c8673a2-1e52-4c0f-95b3-c663cc478091@wichmann.us> Message-ID: You are raising an interesting question, and I have not studied the standard distribution structure of packages. My intent was to separate scripts used to generate a database from the src directory largely to emphasize that these are stand-alone scripts which are to be executed in "isolation" -- even if they are dependent upon code in imported from src. If this is contrary to standard practices, I'm happy to rethink the matter. Thank you for your comments. On Thu, Feb 29, 2024 at 1:42?PM dn via Tutor wrote: > James, > > On 1/03/24 06:16, James Hartley wrote: > > This has been a useful discussion (at least for me). Thank you all for > > your candor. > > This is a common concern/problem. However, you outlined it well. Plus, > @Mat's expansion is a very good discussion of the interplay between > Python (the interpreter) and one's directory structure. Using "python -m > ..." is often a good way around. Thus, it's worth others reading (and > learning)... > > > > From these comments, it sounds like playing with SYSPATH is a common > task. > > I have some database tables defined for SQLAlchemy which I have imported > > into some Pytest tests. This works fine, but to import the same table > > definitions into scripts found in other directories suffers from the > > problems prompting this thread. It appears that I will have to modify > > SYSPATH to make one (preferrable) definition work in all locations. > > Yes and No! > > If you are both customer and supplier, then you probably won't have a > problem - either at coding-time or later in life. > > If however, you distribute the code to others, these aspects take-on a > whole new (and 'exciting') dimension! > > BTW the sub-dir layout is 'standard' for some of the 'installers' used > for Python distribution. > > > A first question though: if something has been put into a 'utility' > directory - which makes it sound as if it will be import-ed from > different parts of the code (even different projects), then why is it > being executed as a script? Once the utility routines have been > exercised and tested (routines either located in the project/test > sub-dir or in project/utils/tests) their (only) access should be by > import (from project/src routines)? > > > For further thought (as if there isn't enough swirling your brain - and > bearing in-mind that I haven't used SQLAlchemy for a while) is that DRY > is a concern for schema as much as it is for Python-code. There's no > point in testing one and running another. Oops! Accordingly, a separate > file (and/or potentially project sub-dir) which holds the SQL details > and access methods, (hah! and here it comes...) accessible from both > code and test modules. > > > > > Thanks again for your comments! > > > > On Thu, Feb 29, 2024 at 10:57?AM Mats Wichmann wrote: > > > >> On 2/28/24 18:45, dn via Tutor wrote: > >>> On 29/02/24 13:06, Mats Wichmann wrote: > >>>> On 2/28/24 15:39, James Hartley wrote: > >>>>> While in the project directory: > >>>>> $ pwd > >>>>> /home/me/project > >>>>> $ cat src/foo.py > >>>>> class Foo: > >>>>> pass > >>>>> $ cat utils/baz.py > >>>>> from src.foo import Foo > >>>>> $ python utils/baz.py > >>>>> Traceback (most recent call last): > >>>>> File "./baz.py", line 1 in > >>>>> from src.foo import Foo > >>>>> ModuleNotFoundError: No module named 'src' > >>>> > >>>> > >>>> When you run a script, the directory of the script is added to > >>>> sys.path. Not the directory you were in when you ran it. You can > >>>> print out sys.path in utils/baz.py to convince yourself of this. > >>>> There's no src in that directory (utils). > >>> > >>> Whereas pytest works the other way around: run from the > >>> project-directory because its (internal) 'process of discovery' will > >>> find test* files by searching the entire directory-tree... > >>> (hence the questions in earlier reply) > >> > >> > >> Right... pytest takes a common situation and helps you out, in this case > >> (a) by discovering tests and (b) fiddling paths so those tests can > >> import cleanly. If you don't get that help by using (for example) a test > >> framework, you have to take care of it yourself. > >> > >> This import fails because for an absolute import (src.foo - doesn't > >> start with dots), the starting directory has to be in the search path > >> (sys.path), but it isn't. You can fix that by adding to sys.path in the > >> scripts in utils, or by using PYTHONPATH. For example, this should > work: > >> > >> PYTHONPATH=. python utils/baz.py > >> > >> You would think you could use relative paths. However, if you attempt a > >> relative import like: > >> > >> from ..src.foo import Foo > >> > >> it doesn't work, because the way the script was invoked, Python doesn't > >> think it's part of a package (invoked as a script, the name is > >> "__main__", and contains no dots, thus you must not be in a package), > >> and relative paths are package-hierarchy-relative, not > >> filesystem-relative, so you'll get one of Python's famous error > messages: > >> > >> ImportError: attempted relative import with no known parent package > >> > >> However, you could run baz as a module, like: > >> > >> python -m utils.baz > >> > >> because now the name of the running module is "utils.baz", so it looks > >> like a package rooted at a level above "utils", and that directory will > >> be in sys.path, so "from src/foo import Foo" should work. It just seems > >> a bit ugly to have to invoke a "top level script" as a module to get > >> imports to work. > >> > >> Python imports are... challenging. > >> > >> (yes, I'm assuming answers to the question @dn asked, so this could be > >> wrong if I made the wrong guess) > >> > >> _______________________________________________ > >> Tutor maillist - Tutor at python.org > >> To unsubscribe or change subscription options: > >> https://mail.python.org/mailman/listinfo/tutor > >> > > _______________________________________________ > > Tutor maillist - Tutor at python.org > > To unsubscribe or change subscription options: > > https://mail.python.org/mailman/listinfo/tutor > > -- > Regards, > =dn > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From alan.gauld at yahoo.co.uk Thu Feb 29 16:52:06 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 29 Feb 2024 21:52:06 +0000 Subject: [Tutor] Running Command Prompt In Python In-Reply-To: References: Message-ID: On 29/02/2024 11:26, Am Ra wrote: > Importing os, and using os.system to connect to Command Prompt. os.system is nearly always the wrong approach. os.system is really only worthwhile if its a simple throwaway command. For example I use it for an app that manages a list of images and I have a Show menu option that displays the image in the systems file viewer. os.system is fine for that. But for anything where you want to either interact with the command or to view its output you should use subsystem. Either Popen or one of its utility functions. > [image: image.png] The server strips binary files for security reasons, please cut 'n paste the text directly into the email. > Then, dmel922.exe will be activated to generate an empty csv file from a > raw dataset. If it's really empty would it not be simpler to just create it directly from Python? Or is there more going on in the background? > Up to here it is working as it is supposed to. However, it goes on a pause > and not adding any data to the empty csv file... Ah, OK that was only half the story! :-) > ...because it is waiting on the > user input which is ?Enter y to Continue?. As we see in the picture below, > the Command Prompt is asking the user to press y to continue. As soon as, I > press y in the Command Prompt it will continue and finish the task. Subprocess is your friend in this scenario. Read the module docs carefully, they are quite detailed and include several examples. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos