From robertvstepp at gmail.com Thu Jul 1 20:02:14 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Thu, 1 Jul 2021 19:02:14 -0500 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: References: Message-ID: OK, I have just re-read everyone's posts to this thread. Thank you very much for the detailed replies! I'm still mulling over all of this and probably will for some time. I don't think I yet understand all of the nuances of the information presented. Probably won't truly settle in until I struggle with enough code in enough varied circumstances.. One thing I realized during the re-read is that my relatively naive use of the word "data" needs to be more precise. I gather that "data" might have the following meanings, depending on context: 1) User-generated data external to my program. This assumes that my program is designed to enable the user to process his data. So she needs to give my program the path to this data, either through command line arguments when invoking my program or via a mechanism provided by my program, probably some sort of file selector widget. 2) My program generates data from the user's use of it. This means that either the data will be stored in one of the OS-provided standard locations (such as in Windows %appdata%) or allow the user a choice as to where to store it, in which case my program must allow the user a way of providing that path, which then gets stored in my program's preferences or configuration file in one of the OS-standard locations. 3) Information my program needs to be properly configured is normally stored in an OS-provided location. 4) User preferences and other user-initiated customization of my program. Again stored in an appropriate configuration file in an OS-approved location. I think that covers most of what was discussed in this thread, though dn provided some more complex scenarios to contemplate in his usual inimitable way! In each of these instances my program should have no need (normally) of using __file__ - based tricks. One thing that is mildly troubling is that if I wish to write cross-platform applications then I must go to the bother of detecting the user's OS and stashing "data" appropriately for that OS. But that should not normally be too troublesome. Cameron brought up virtual environments for segregating one's programs and their dependencies during development. Another topic I have been putting off. I guess I will have to investigate that soon for some of the programs I am currently imaging doing. It is hard enough keeping up with the pace of Python updates and having to reinstall the packages I use for my personal use! Question: I am about to embark on writing a relatively small program that I will use to further explore the use of SQLite for storing data, something I keep dabbling with off and on. As you know SQLite stores each database as a single file. In this thread's data location discussion, where should this database file be stored? And for more substantial database programs than SQLite, how would that normally be handled if this database program is integrated into the overall application structure? I guess that is all for now. Again, thanks for the very good, detailed help! boB Stepp From cs at cskk.id.au Thu Jul 1 21:37:08 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 2 Jul 2021 11:37:08 +1000 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: References: Message-ID: On 01Jul2021 19:02, boB Stepp wrote: >Cameron brought up virtual environments for segregating one's programs >and their dependencies during development. [...] >It is hard enough keeping >up with the pace of Python updates and having to reinstall the >packages I use for my personal use! I usually don't update Python whenever a release happens. Do it at need (new feature or bugfix or security fix). Or, of course, when a brew or OS update yanks the old Python out from under me and force a venv rebuild. One nice thing about a venv (well any pip-based install really) is that you can stick your required module names in a file (with optional version requirements) and just load them back up if you have a fresh venv. Conventionally that file is called 'requirenets.txt", but you can use any name (I use venv-requirements.txt in my personal dev env). Then for a fresh install you go: ...../bin/python -m pip install -r requirements.txt and it does all the hard work for you. >Question: I am about to embark on writing a relatively small program >that I will use to further explore the use of SQLite for storing data, >something I keep dabbling with off and on. As you know SQLite stores >each database as a single file. In this thread's data location >discussion, where should this database file be stored? IMO, because it is a file, wherever you would store any other kind of user app data. I have a few SQLite dbs myself. I keep a personal directory named "var" in my home directory and typically choose: ~/var/appname.sqlite or similar. On Windows, I gather you've got a conventional per-user-app-data area, which I'd use instead. Your objective is (a) keep the data suitable partitioned from other data, so probably per-user per-app and (b) a fairly simple easy to remember naming convention. Finally, remember Heuer's Razor: If it can't be turned off, it's not a feature. - Karl Heuer Let the user provide an environmnet variable or command line switch to specify the data location. If your app has several state files, eg an sqlite db and some other things, that might be a folder name where several things live. >And for more >substantial database programs than SQLite, how would that normally be >handled if this database program is integrated into the overall >application structure? To take an example of PostgreSQL, you would usually have a shared database server (local to your machine or not), and individual databases for various apps. For "personal" app data I'd typically involve both the user's login name and the app name in the "database name", eg "appname_cameron" perhaps. Or "cameron_appname". Just to partition off the user's app data from other users' app data. Cheers, Cameron Simpson From mats at wichmann.us Fri Jul 2 11:11:13 2021 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 2 Jul 2021 09:11:13 -0600 Subject: [Tutor] Multiprocessing In-Reply-To: <31ab6d9a51b343c08a9e8c4dbaa9fa32@liverpool.ac.uk> References: <31ab6d9a51b343c08a9e8c4dbaa9fa32@liverpool.ac.uk> Message-ID: <4faee0a1-30f0-358f-a381-f075a4257a97@wichmann.us> On 6/30/21 9:00 AM, Pugliese, Francesco wrote: > Dear all, > > I am Francesco and currently PhD at the University of Liverpool. > > I am trying to use the multiprocessing library in python to performance some structural analyses using Opensees (please see the attached file for reference). I am coding like this: > > > if __name__=="__main__": > p = Pool(8) > p.map(fun, range(1,10001), chunksize = 1) > p.terminate() > p.join() > > > After completing all analyses, the system does not close the pool but remains stuck (frozen) without doing anything. Could please help me out to understand where I am making mistakes? multiprocessing is full of traps. as an experiment maybe try setting the start method to "spawn" and see if the behavior changes. the spawn approach is considered a bit slower than the LInux default, but it's definitely safer; and since you're creating a pool it's all one-time startup cost and shouldn't be an issue. From mats at wichmann.us Fri Jul 2 12:59:20 2021 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 02 Jul 2021 10:59:20 -0600 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: <5n8udg1hcimme45un3chg850msa4ficqi0@4ax.com> References: <5n8udg1hcimme45un3chg850msa4ficqi0@4ax.com> Message-ID: On July 2, 2021 10:06:02 AM MDT, Dennis Lee Bieber wrote: >On Thu, 1 Jul 2021 19:02:14 -0500, boB Stepp >declaimed the following: > >>One thing that is mildly troubling is that if I wish to write >>cross-platform applications then I must go to the bother of detecting >>the user's OS and stashing "data" appropriately for that OS. But that >>should not normally be too troublesome. Take a look at the applies package on pypi for some help with this. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From mats at wichmann.us Fri Jul 2 13:54:10 2021 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 02 Jul 2021 11:54:10 -0600 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: References: <5n8udg1hcimme45un3chg850msa4ficqi0@4ax.com> Message-ID: <990A5EAF-0613-4687-99C0-94044D99BA87@wichmann.us> On July 2, 2021 10:59:20 AM MDT, Mats Wichmann wrote: >On July 2, 2021 10:06:02 AM MDT, Dennis Lee Bieber > wrote: >>On Thu, 1 Jul 2021 19:02:14 -0500, boB Stepp >>declaimed the following: >> >>>One thing that is mildly troubling is that if I wish to write >>>cross-platform applications then I must go to the bother of detecting >>>the user's OS and stashing "data" appropriately for that OS. But >that >>>should not normally be too troublesome. > >Take a look at the applies package on pypi for some help with this. Appdirs. Sorry, autocorrect got it. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From eryksun at gmail.com Fri Jul 2 19:46:45 2021 From: eryksun at gmail.com (Eryk Sun) Date: Fri, 2 Jul 2021 18:46:45 -0500 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: <5n8udg1hcimme45un3chg850msa4ficqi0@4ax.com> References: <5n8udg1hcimme45un3chg850msa4ficqi0@4ax.com> Message-ID: On 7/2/21, Dennis Lee Bieber wrote: > > {Interesting: R seems to use both %userprofile% and > %userprofile%\documents} Note that different apps used different "home" > directories -- Linux would just be ~/.file to find the current user home. In Windows, almost all of the Unix home-directory conventions are either wrong or at least non-conventional. In the case of "\Documents", it's fundamentally wrong. The user's documents directory is a known folder that's relocatable. It may even be set to a directory on a remote share. One can't assume that the default location in the profile directory is correct or even exists. Instead, use the known-folder API to get the correct path. For example, query the path of the user's documents folder via SHGetKnownFolderPath(FOLDERID_Documents, ...). > As I believe I mentioned, many programs ported from Linux to Windows > DON'T bother to detect and user %appdata%. They store their files in (what > in Linux) hidden "dot" files... (the leading . doesn't make them hidden on > Windows). Creating dot files and directories in the user's profile directory goes against the platform convention to use the per-user local or roaming application-data directory. Personally, when applications create these dot files it annoys me because there's no convention in Windows shells to hide them. I can at least manually hide dot directories. For dot files, on the other hand, I can't even manually hide them since the hidden attribute is one of the special file attributes that has to be conserved when overwriting a file, else the open fails with access denied. From robertvstepp at gmail.com Fri Jul 2 23:55:14 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Fri, 2 Jul 2021 22:55:14 -0500 Subject: [Tutor] Proper SQLite cursor handling? Message-ID: I have the start of a class to create solitaire game objects: class SolitaireGame: """Representation of a solitaire game.""" def __init__(self, db_cursor: sqlite3.Cursor, game_name: str) -> None: """Create or open a solitaire game.""" self.cur = db_cursor self.game_name = game_name My current thought is to create only a single cursor object and use it throughout the program's lifetime. Is this a sensible thing to do? In the above class skeleton my intent is to retrieve the game information from the database using the passed in cursor. During the course of a program session the user may have any number of different solitaire games open, so each of these game objects would have its own reference to the _same_ cursor object. I have an uneasy feeling that this might cause issues, but I don't know enough (yet) and am still at the pondering point for this class. Eventually I will write tests and try things out to see what happens, but one of you might save me some painful explorations and time! TIA! boB Stepp From cs at cskk.id.au Sat Jul 3 00:38:37 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Sat, 3 Jul 2021 14:38:37 +1000 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 02Jul2021 22:55, boB Stepp wrote: >I have the start of a class to create solitaire game objects: > >class SolitaireGame: > """Representation of a solitaire game.""" > > def __init__(self, db_cursor: sqlite3.Cursor, game_name: str) -> None: > """Create or open a solitaire game.""" > self.cur = db_cursor > self.game_name = game_name > >My current thought is to create only a single cursor object and use it >throughout the program's lifetime. Is this a sensible thing to do? I'd have thought generally not. Cursors tend to be ephemeral, made for a specific operation and then discarded. Usually I would pass in the db connection and make cursors at need. It looks like you cannot commit from a cursor. Well, I guess there is cursor.connection.commit, but that's really going back to the connection to do that work. In fact, it looks like you can do a number of cursor operations directly from the connection (a Cursor gets made behind the scenes apparently). Eg this example from the sqlite3 module docs: con = sqlite3.connect(":memory:") # Create the table con.execute("create table person(firstname, lastname)") # Fill the table con.executemany("insert into person(firstname, lastname) values (?, ?)", persons) # Print the table contents for row in con.execute("select firstname, lastname from person"): print(row) Of course, if you switch dbs to something else that nonstandard approach would need to be ported to use cursors anyway... >In the above class skeleton my intent is to retrieve the game >information from the database using the passed in cursor. During the >course of a program session the user may have any number of different >solitaire games open, so each of these game objects would have its own >reference to the _same_ cursor object. I have an uneasy feeling that >this might cause issues, but I don't know enough (yet) and am still at >the pondering point for this class.C I'd be concerned about methods which call other methods. Supposing you have a method which performs some query and yields stuff during the query by caling fetchone() a lot. And something's consuming those and making other method calls - the single cursor is tied up with the fetchone() and doing other things with it would be bad or nonsensical. I would expect to want a cursor per method call (well, per db operation). Cheers, Cameron Simpson From alan.gauld at yahoo.co.uk Sat Jul 3 04:18:32 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 3 Jul 2021 09:18:32 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 03/07/2021 04:55, boB Stepp wrote: > I have the start of a class to create solitaire game objects: > > class SolitaireGame: > """Representation of a solitaire game.""" > > def __init__(self, db_cursor: sqlite3.Cursor, game_name: str) -> None: > """Create or open a solitaire game.""" > self.cur = db_cursor > self.game_name = game_name > > My current thought is to create only a single cursor object and use it > throughout the program's lifetime. Is this a sensible thing to do? You might get away with it but in general its a terrible idea. cursors hold a reference to the result of a query, you may not process all of a cursors data in one iteration of a method. If another method then performs another query it will lose the reference to the results from the first query. > In the above class skeleton my intent is to retrieve the game > information from the database using the passed in cursor. During the > course of a program session the user may have any number of different > solitaire games open, so each of these game objects would have its own > reference to the _same_ cursor object. At the very least you should have a separate cursor per game and then its up to you to ensure it never gets clobbered by another method within the game (maybe have a cursor_busy flag or something. but its still kind of defeating the purpose of cursors! Better to pass in a reference to the connection in the init() and then have methods create cursors as needed. You can, of course, just always store all of the data in a cursor in a local list of tuples using fetchall() but that only works on small databases. The whole point of cursors is that they avoid the need to store humungous amounts of data locally! > this might cause issues, but I don't know enough Think of the cursor as a pointer into the database result set (which is, significantly, stored on the database and not in your application. This is less important with SQLite but in a client/server DB makes a massive difference!) The cursor is not that big, it just marks the location of the current row of interest. But the total data result set could be gigabytes big... Because it's potentially so big you would normally process the result in batches or one row at a time. This could be in a long running loop(maybe in a thread) or as part of an idle(or timer) event in an event-driven architecture. Now, in your specific scenario: 1) Game data tends not to be too big and you can store the entire cursor content locally. 2) SQLite databases are a single file and you usually have it on the local machine so storage is the same wherever you keep it! 3) SQLite can perform cursor operations directly from the connection so you probably don't even need a cursor at all if you plan on reading all the data each time. So, in general, shared cursors are bad but in your specific case you probably don't even need one, let alone a shared one! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Sat Jul 3 04:31:05 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 3 Jul 2021 09:31:05 +0100 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: References: Message-ID: On 02/07/2021 01:02, boB Stepp wrote: > One thing that is mildly troubling is that if I wish to write > cross-platform applications then I must go to the bother of detecting > the user's OS and stashing "data" appropriately for that OS. But that > should not normally be too troublesome. For industrial grade programs the answer is yes, yes and yes again. OS approved folder location conventions are different on every major OS: Windows, MacOS, Linux, VMS, IBM OSxxx etc. Even different variants of Unix can have different preferred locations for config settings. Most commercial multi-platform programs will either have a complex initialization function that detects OS and sets the key folder locations appropriately or, more common in compiled languages like C, Java etc, they have a separate platform specific file/module that sets the locations directly for each OS and then at build time incorporate the required settings file. But there is no standard you can follow. To be a good citizen on each platform you need to do platform specific stuff. And the preferred location can change between OS versions, especially on Windows! So you need to use the OS API there to find it. When you add the possibility of the user having a config that over-rides the OS default too, it gets even more messy. It is a non-trivial exercise. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PyTutor at DancesWithMice.info Sat Jul 3 05:02:54 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sat, 3 Jul 2021 21:02:54 +1200 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: <87a16d41-7c53-a2d8-4c3e-28dcf900538a@DancesWithMice.info> On 03/07/2021 15.55, boB Stepp wrote: > I have the start of a class to create solitaire game objects: > > class SolitaireGame: > """Representation of a solitaire game.""" > > def __init__(self, db_cursor: sqlite3.Cursor, game_name: str) -> None: > """Create or open a solitaire game.""" > self.cur = db_cursor > self.game_name = game_name > > My current thought is to create only a single cursor object and use it > throughout the program's lifetime. Is this a sensible thing to do? > In the above class skeleton my intent is to retrieve the game > information from the database using the passed in cursor. During the > course of a program session the user may have any number of different > solitaire games open, so each of these game objects would have its own > reference to the _same_ cursor object. I have an uneasy feeling that > this might cause issues, but I don't know enough (yet) and am still at > the pondering point for this class. Eventually I will write tests and > try things out to see what happens, but one of you might save me some > painful explorations and time! Plenty of DB advice, elsewhere. As a general rule, it is a good idea to keep I/O separate from 'processing'. Given that you are learning DB interactions, probably even more advantageous. Why? You can learn, develop, and test each phase independently, ie - you can develop the DB routines quite separately - you can develop the game class using manually-prepared data that won't need to come from a DB, nor return to one ...and after all that, when you're ready: - you can switch between file-stores/DBMS-es whenever the whim takes you (without affecting the mechanics of the game) In other words, try something like: class SolitaireGame: """Representation of a solitaire game.""" def __init__(self, game_parameters ) -> None: """Create or open a solitaire game.""" self.name = game_parameters.name or retrieval function or ... etc now, to run a game, the three steps are self-documenting: game_parameters = get_game_data_from_DB( db_defns ) game = SolitaireGame( game_name, other_game_data ) store_game_results_in_DB( game.results_for_DB() ) NB the DB interaction(s) has been illustrated as two functions, for ease of comprehension. I'd argue that a class (with two I/O methods) would be a better choice, because its namespace maintains "state". See (object) illustration within __init__(). -- -- Regards, =dn From alan.gauld at yahoo.co.uk Sat Jul 3 06:13:00 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 3 Jul 2021 11:13:00 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: <87a16d41-7c53-a2d8-4c3e-28dcf900538a@DancesWithMice.info> References: <87a16d41-7c53-a2d8-4c3e-28dcf900538a@DancesWithMice.info> Message-ID: On 03/07/2021 10:02, dn via Tutor wrote: > now, to run a game, the three steps are self-documenting: > > game_parameters = get_game_data_from_DB( db_defns ) > game = SolitaireGame( game_name, other_game_data ) > store_game_results_in_DB( game.results_for_DB() ) >From an OOP perspective I'd change that to: parameters = GameParameters( db_defns ) game = SolitaireGame( parameters ) game.store() -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From robertvstepp at gmail.com Sat Jul 3 13:40:40 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 3 Jul 2021 12:40:40 -0500 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: References: Message-ID: On Sat, Jul 3, 2021 at 3:32 AM Alan Gauld via Tutor wrote: > > On 02/07/2021 01:02, boB Stepp wrote: > > > One thing that is mildly troubling is that if I wish to write > > cross-platform applications then I must go to the bother of detecting > > the user's OS and stashing "data" appropriately for that OS. But that > > should not normally be too troublesome. [...] > When you add the possibility of the user having a config that > over-rides the OS default too, it gets even more messy. > It is a non-trivial exercise. I said, "...But that should not normally be too troublesome." I am now a troubled man with a worried mind! ~(:>)) boB Stepp From PyTutor at DancesWithMice.info Sat Jul 3 20:47:27 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sun, 4 Jul 2021 12:47:27 +1200 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: <87a16d41-7c53-a2d8-4c3e-28dcf900538a@DancesWithMice.info> Message-ID: <5296b31e-304a-c15f-3a6f-75a7e3c3c305@DancesWithMice.info> On 03/07/2021 22.13, Alan Gauld via Tutor wrote: > On 03/07/2021 10:02, dn via Tutor wrote: > >> now, to run a game, the three steps are self-documenting: >> >> game_parameters = get_game_data_from_DB( db_defns ) >> game = SolitaireGame( game_name, other_game_data ) >> store_game_results_in_DB( game.results_for_DB() ) > > From an OOP perspective I'd change that to: > > parameters = GameParameters( db_defns ) > game = SolitaireGame( parameters ) > game.store() Does this mean that within game.store() the code will call a method within parameters (GameParameters) to perform the necessary o/p to backing-store? It will work (see below). Is it 'good design'/'best practice'? An alternative (re-stating the original outline) might be something like: parameters = GameParameters( db_defns ) game = SolitaireGame( parameters ) parameters.store_results( game.store() ) In this case, game.store() extracts/exports relevant parameter-values from the game. These are then handed across an interface/'API'/"contract" to parameters.store_results() which performs the transfer of values being carried-forward into the next game/play, onto the 'DB'/backing-store. There is room for some debate about which of these two approaches is 'best' - or a better approach to the decision: which is preferable for this application? (it's nothing personal) The reasons for (my) preferring the (second outline) above is that each object (GameParameters and SolitaireGame) has its own (separate) set of methods, and the two only 'meet' across the two 'mirror' interfaces, ie a 'Clean Architecture'. SolitaireGame plays Solitaire. GameParameters handles persistence. Whereas, the former case requires that SolitaireGame not only take in the data (parameters) but either: - understand that the parameters object to include a store_results/store_parameters method, or - will directly store the parameters itself (which would mean that both objects need to know about I/O for persistence - whither 'separation of concerns'?). The disadvantage of distinct "separation" is that whilst the I/O method can change without affecting the game-object, adding additional parameters will require evident changes to both. Changes to 'the database' (or the API) are seldom insignificant within projects, and are often reasons why an application will 'step up' a "major" version-number (https://en.wikipedia.org/wiki/Software_versioning) - because all 'old' stores require conversion/migration before re-use. Perhaps without such separation it might be possible to 'bury' changes and make them seem less significant ("major")? On the other hand, in a commercial situation, "separation" enables the work to be split between different people/teams. In this example, we could ask our DB-specialist to create GameParameters, whilst our games-expert gets-busy with SolitaireGame. The two will have to 'put their heads together' to design and agree the interface both up-front and in an Agile cyclic-update manner thereafter. Which brings me to... If achieving working-code is where the objective/satisfaction lies. Either of the above, or indeed the paired-functions (used as a more simple illustration, earlier in the thread), will 'do the job'. If the author is likely to be the only user, then should changes (even 'catastrophic changes') need to be made, 'no puppies will die'. Alternately, (and rightly or wrongly) I take @boB's penchant for learning-by-experimenting as a key component of any response - and in this case (per previous discussion-threads 'here') his "learning objectives" in this project seem to encompass Python, coding techniques, storage options, and application-design. Accordingly, the earlier suggestion that separating I/O from game-play would enable various (and independent) iterations of both game-complexity and storage-medium - all the while (post iteration nr1), having a working-application. In this project's learning-path we might see some merit in starting-out with a dict to hold key:value pairs of parameters. Zero persistence, but with a trade-off of easy addition/alteration. Accordingly, one may concentrate on the workings of SolitaireGame, and find that over-time the requirements for I/O (interface) seem to naturally 'fall out' and make themselves obvious. In the early days, the dict can be easily expanded and altered. When SolitaireGame starts to feel 'stable', there will be time and 'spare effort' to apply to the I/O object/methods. A dict will lead naturally to JSON, and thereafter JSON could lead to MongoDB. Thus, a convenient expansion plan/learning-path. Not that it would be difficult to re-form the dict into a relational schema either... -- Regards, =dn From PyTutor at DancesWithMice.info Sat Jul 3 23:24:40 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sun, 4 Jul 2021 15:24:40 +1200 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: References: Message-ID: On 04/07/2021 05.40, boB Stepp wrote: > On Sat, Jul 3, 2021 at 3:32 AM Alan Gauld via Tutor wrote: >> >> On 02/07/2021 01:02, boB Stepp wrote: >> >>> One thing that is mildly troubling is that if I wish to write >>> cross-platform applications then I must go to the bother of detecting >>> the user's OS and stashing "data" appropriately for that OS. But that >>> should not normally be too troublesome. > > [...] > >> When you add the possibility of the user having a config that >> over-rides the OS default too, it gets even more messy. >> It is a non-trivial exercise. > > I said, "...But that should not normally be too troublesome." I am > now a troubled man with a worried mind! ~(:>)) Well, we can't have that! Let's see if I can trivialise your thoughts by complicating matters... Sound-track (https://www.youtube.com/watch?v=fX5USg8_1gA) to ease your troubled mind, is "Layla" by Eric Clapton! Follow that up with a little Myles Kennedy (https://www.tekstowo.pl/piosenka,myles_kennedy,worried_mind.html) Some people, when confronted with a problem, think ?I know, I'll use regular expressions.? Now they have two problems. - (attributed to) Jamie Zawinski Consider (as one of our colleagues is wont to ask): "The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming." Donald Knuth, Computer Programming as an Art (1974) - in this case might "efficiency" be represented as trying to make something work under all OpSys - other than the one you happen to be using, today. Might it be better (use of your (learning-) time) to simplify this consideration, and move-on? aka Is it part of the MVP of your project? (in this case, perhaps the "learning" value takes place inside your own head, cf amongst potential clients) Thus, how about building a function which sets the applicable/default directory? Today it can return a constant. Tomorrow you can make the working application more flexible/powerful... When you feel like you're walking a tight-rope, see also "Technical Debt"... There now, don't you feel so much better! (?) Musical accompaniment to your whirring thoughts: https://www.youtube.com/watch?v=FgcGOWaTPdU [not to be used by folk affected by a suicide, and more applicable to those of us old-enough to remember the TV show it introduced] Web.Refs: https://www.explainxkcd.com/wiki/index.php/208:_Regular_Expressions https://en.wikiquote.org/wiki/Donald_Knuth debated at https://news.ycombinator.com/item?id=1671458 https://stackify.com/premature-optimization-evil/ http://passionateaboutoss.com/premature-optimisation/ https://www.agilealliance.org/glossary/mvp/ https://en.wikipedia.org/wiki/Technical_debt https://www.agilealliance.org/introduction-to-the-technical-debt-concept https://www.projectpractical.com/technical-debt/ -- Regards, =dn From manpritsinghece at gmail.com Sun Jul 4 00:16:32 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Sun, 4 Jul 2021 09:46:32 +0530 Subject: [Tutor] Python program to extract numeric suffix from a string Message-ID: Dear sir, Assume a problem to extract a numeric suffix from a string . String is st = "abcd12ghi456" The numeric suffix of this string is "456" in case if there is no suffix, the program should print " no suffix found" The code i have written is given below : def return_suffix(st): for x, y in enumerate(reversed(st)): if not y.isdigit(): return st[-x:] if x else "No digit suffix found" st1 = "abcd1gh123" ans = return_suffix(st1) print(ans) # gives the right answer as '123' st1 = "abcd1gh" ans = return_suffix(st1) print(ans) # gives the right answer as 'No digit suffix found' I feel I have used enumerate and reversed both which makes the program a little difficult to read . Can this be written in a more efficient way ? Need your guidance. Regards Manprit Singh From manpritsinghece at gmail.com Sun Jul 4 02:17:04 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Sun, 4 Jul 2021 11:47:04 +0530 Subject: [Tutor] Python program to extract numeric suffix from a string In-Reply-To: References: Message-ID: Dear sir, Correcting the function written for the problem : Problem to extract a numeric suffix from a string def ret(st): for x, y in enumerate(reversed(st)): if not y.isdigit(): return st[-x:] if x else "No digit suffix found" return st Anything more readable in comparison to this function be done ? Kindly guide Regards Manprit Singh On Sun, Jul 4, 2021 at 9:46 AM Manprit Singh wrote: > Dear sir, > > Assume a problem to extract a numeric suffix from a string . > String is st = "abcd12ghi456" > The numeric suffix of this string is "456" in case if there is no suffix, > the program should print " no suffix found" > The code i have written is given below : > > def return_suffix(st): > for x, y in enumerate(reversed(st)): > if not y.isdigit(): > return st[-x:] if x else "No digit suffix found" > > st1 = "abcd1gh123" > ans = return_suffix(st1) > print(ans) # gives the right answer as '123' > > st1 = "abcd1gh" > ans = return_suffix(st1) > print(ans) # gives the right answer as 'No digit suffix found' > > I feel I have used enumerate and reversed both which makes the program a > little difficult to read . Can this be written in a more efficient way ? > > Need your guidance. > > Regards > Manprit Singh > From manpritsinghece at gmail.com Sun Jul 4 02:28:09 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Sun, 4 Jul 2021 11:58:09 +0530 Subject: [Tutor] Python program to extract numeric suffix from a string In-Reply-To: References: Message-ID: Dear sir, Again working and found something little better which will give speed improvements : The function(To extract a numeric suffix) written below: def ret_suffix(st): if st.isdigit(): return st else: for x, y in enumerate(reversed(st)): if not y.isdigit(): return st[-x:] if x else "No digit suffix found" Need your comments on this function . Kindly give comments . Regards Manprit Singh On Sun, Jul 4, 2021 at 11:47 AM Manprit Singh wrote: > Dear sir, > > Correcting the function written for the problem : > Problem to extract a numeric suffix from a string > > def ret(st): > for x, y in enumerate(reversed(st)): > if not y.isdigit(): > return st[-x:] if x else "No digit suffix found" > return st > > Anything more readable in comparison to this function be done ? Kindly > guide > > Regards > Manprit Singh > > > > > On Sun, Jul 4, 2021 at 9:46 AM Manprit Singh > wrote: > >> Dear sir, >> >> Assume a problem to extract a numeric suffix from a string . >> String is st = "abcd12ghi456" >> The numeric suffix of this string is "456" in case if there is no suffix, >> the program should print " no suffix found" >> The code i have written is given below : >> >> def return_suffix(st): >> for x, y in enumerate(reversed(st)): >> if not y.isdigit(): >> return st[-x:] if x else "No digit suffix found" >> >> st1 = "abcd1gh123" >> ans = return_suffix(st1) >> print(ans) # gives the right answer as '123' >> >> st1 = "abcd1gh" >> ans = return_suffix(st1) >> print(ans) # gives the right answer as 'No digit suffix found' >> >> I feel I have used enumerate and reversed both which makes the program a >> little difficult to read . Can this be written in a more efficient way ? >> >> Need your guidance. >> >> Regards >> Manprit Singh >> > From phillor9 at gmail.com Sun Jul 4 03:17:39 2021 From: phillor9 at gmail.com (Phil) Date: Sun, 4 Jul 2021 17:17:39 +1000 Subject: [Tutor] Really basic won't run error Message-ID: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> I've been working on this idea for most of the day and have finally reduced the problem to a wxPython demo example but it won't run. No doubt the reason is obvious but I cannot see it. import wx class ImagePanel(wx.Panel): ??? def __init__(self, parent): ??????? wx.Panel.__init__(self, parent, -1)#None, title="Bitmap test") ??????? wx.StaticText(self, -1, "This is a wx.StaticBitmap.", (45, 15)) ??????? bmp = 'test.jpg'.GetBitmap() ??????? #mask = wx.Mask(bmp, wx.BLUE) ??????? #bmp.SetMask(mask) ??????? #StaticBitmap(self, -1, bmp, (80, 50), (bmp.GetWidth(), bmp.GetHeight())) ??????? StaticBitmap(self, -1, bmp, (80, 150)) if __name__ == "__main__": ??? app = wx.App() ??? frame = ImagePanel().Show() ??? app.MainLoop() Traceback (most recent call last): ? File "/usr/lib/python3.8/idlelib/run.py", line 559, in runcode ??? exec(code, self.locals) ? File "/home/phil/Python/wxpython_bitmap.py", line 38, in ??? frame = MyFrame().Show() TypeError: __init__() missing 1 required positional argument: 'parent' -- Regards, Phil From PyTutor at DancesWithMice.info Sun Jul 4 05:16:02 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sun, 4 Jul 2021 21:16:02 +1200 Subject: [Tutor] Really basic won't run error In-Reply-To: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> References: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> Message-ID: <9a8cba7b-8fee-fe8d-f90c-15b115fda493@DancesWithMice.info> On 04/07/2021 19.17, Phil wrote: > I've been working on this idea for most of the day and have finally > reduced the problem to a wxPython demo example but it won't run. No > doubt the reason is obvious but I cannot see it. > > import wx > > class ImagePanel(wx.Panel): > ??? def __init__(self, parent): > ??????? wx.Panel.__init__(self, parent, -1)#None, title="Bitmap test") > > ??????? wx.StaticText(self, -1, "This is a wx.StaticBitmap.", (45, 15)) > > ??????? bmp = 'test.jpg'.GetBitmap() > ??????? #mask = wx.Mask(bmp, wx.BLUE) > ??????? #bmp.SetMask(mask) > ??????? #StaticBitmap(self, -1, bmp, (80, 50), (bmp.GetWidth(), > bmp.GetHeight())) > > ??????? StaticBitmap(self, -1, bmp, (80, 150)) > > if __name__ == "__main__": > ??? app = wx.App() > ??? frame = ImagePanel().Show() > ??? app.MainLoop() > > Traceback (most recent call last): > ? File "/usr/lib/python3.8/idlelib/run.py", line 559, in runcode > ??? exec(code, self.locals) > ? File "/home/phil/Python/wxpython_bitmap.py", line 38, in > ??? frame = MyFrame().Show() > TypeError: __init__() missing 1 required positional argument: 'parent' Reading the err.msgs we are first pointed to "line 38" which attempts to first "instantiate" and then "Show" an "ImagePanel", with the comment that a "positional argument" is missing (omitted). Previously in the code, class ImagePanel is defined. Its __init__() declares a "parent" parameter. This is missing from line 38. It has been a while since I played with wxPython. Should the "app" object, instantiated on line 37, be provided as an argument to ImagePanel? -- Regards, =dn From PyTutor at DancesWithMice.info Sun Jul 4 05:19:19 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sun, 4 Jul 2021 21:19:19 +1200 Subject: [Tutor] Python program to extract numeric suffix from a string In-Reply-To: References: Message-ID: On 04/07/2021 18.28, Manprit Singh wrote: > Dear sir, > Again working and found something little better which will give speed > improvements : > The function(To extract a numeric suffix) written below: ... How about publishing these speed-comparisons, in order that others may learn/benefit? -- Regards, =dn From phillor9 at gmail.com Sun Jul 4 06:28:48 2021 From: phillor9 at gmail.com (Phil) Date: Sun, 4 Jul 2021 20:28:48 +1000 Subject: [Tutor] Really basic won't run error In-Reply-To: <9a8cba7b-8fee-fe8d-f90c-15b115fda493@DancesWithMice.info> References: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> <9a8cba7b-8fee-fe8d-f90c-15b115fda493@DancesWithMice.info> Message-ID: <8feb4e2a-01a2-a687-2c14-0f4ab02668bc@gmail.com> On 4/7/21 7:16 pm, dn via Tutor wrote: > Reading the err.msgs we are first pointed to "line 38" which attempts to > first "instantiate" and then "Show" an "ImagePanel", with the comment > that a "positional argument" is missing (omitted). > > Previously in the code, class ImagePanel is defined. Its __init__() > declares a "parent" parameter. This is missing from line 38. > > It has been a while since I played with wxPython. Should the "app" > object, instantiated on line 37, be provided as an argument to ImagePanel? Thank you dn for your reply. At the moment I don't know what lines 37 and 38 are referring to, possibly some wxPython class constructor code. It's time to give this away for the day. -- Regards, Phil From roel at roelschroeven.net Sun Jul 4 07:07:59 2021 From: roel at roelschroeven.net (Roel Schroeven) Date: Sun, 4 Jul 2021 13:07:59 +0200 Subject: [Tutor] Python program to extract numeric suffix from a string In-Reply-To: References: Message-ID: <45793498-41ac-9f39-f603-78f1c35b3b74@roelschroeven.net> Alternative: def ret_suffix(st): ??? without_number_suffix = st.rstrip('0123456789') ??? if without_number_suffix == st: ??????? return "No digit suffix found" ??? return st[len(without_number_suffix):] Manprit Singh schreef op 4/07/2021 om 6:16: > Dear sir, > > Assume a problem to extract a numeric suffix from a string . > String is st = "abcd12ghi456" > The numeric suffix of this string is "456" in case if there is no suffix, > the program should print " no suffix found" > The code i have written is given below : > > def return_suffix(st): > for x, y in enumerate(reversed(st)): > if not y.isdigit(): > return st[-x:] if x else "No digit suffix found" > > st1 = "abcd1gh123" > ans = return_suffix(st1) > print(ans) # gives the right answer as '123' > > st1 = "abcd1gh" > ans = return_suffix(st1) > print(ans) # gives the right answer as 'No digit suffix found' > > I feel I have used enumerate and reversed both which makes the program a > little difficult to read . Can this be written in a more efficient way ? > > Need your guidance. > > Regards > Manprit Singh > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -- "'How to Stop Worrying and Learn to Love the Internet': 1) everything that?s already in the world when you?re born is just normal; 2) anything that gets invented between then and before you turn thirty is incredibly exciting and creative and with any luck you can make a career out of it; 3) anything that gets invented after you?re thirty is against the natural order of things and the beginning of the end of civilisation as we know it until it?s been around for about ten years when it gradually turns out to be alright really. Apply this list to movies, rock music, word processors and mobile phones to work out how old you are." -- Douglas Adams From alan.gauld at yahoo.co.uk Sun Jul 4 07:59:15 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 4 Jul 2021 12:59:15 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: <5296b31e-304a-c15f-3a6f-75a7e3c3c305@DancesWithMice.info> References: <87a16d41-7c53-a2d8-4c3e-28dcf900538a@DancesWithMice.info> <5296b31e-304a-c15f-3a6f-75a7e3c3c305@DancesWithMice.info> Message-ID: On 04/07/2021 01:47, dn via Tutor wrote: > On 03/07/2021 22.13, Alan Gauld via Tutor wrote: >> On 03/07/2021 10:02, dn via Tutor wrote: >> >>> now, to run a game, the three steps are self-documenting: >>> >>> game_parameters = get_game_data_from_DB( db_defns ) >>> game = SolitaireGame( game_name, other_game_data ) >>> store_game_results_in_DB( game.results_for_DB() ) >> >> From an OOP perspective I'd change that to: >> >> parameters = GameParameters( db_defns ) >> game = SolitaireGame( parameters ) >> game.store() > > > Does this mean that within game.store() the code will call a method > within parameters (GameParameters) to perform the necessary o/p to > backing-store? It will work (see below). Is it 'good design'/'best > practice'? No, my assumption is that the parameters objects sole responsibility is to identify the parametrs for the current installation/user. Thus things like the default locations(as per the other running thread) and maybe some global values such as high scores, other players etc. Among this would be the database connection(or reference thereto) The game is then responsible for load()ing the data at init)() and store()ing the data before closedown. After all, only the game should know what data is in the object. Exposing the game's internal data to an external object would be against all the principles of OOP. As Peter Coad repeatedly says in his OOP book "Objects should do it to themselves" In fact I'd probvably reduce the three lines above to just 2: game = SolitaireGame( GameParameters( db_defns ) ).play() game.store() The parametrs are only required during the games lifetime so there's no need to hold on to a reference outside the game (unless they are shared across multiple game instances of course) And the store() method only needs to be called after the game has completed. > An alternative (re-stating the original outline) might be something like: > > parameters = GameParameters( db_defns ) > game = SolitaireGame( parameters ) > parameters.store_results( game.store() ) > > In this case, game.store() extracts/exports relevant parameter-values > from the game. These are then handed across an > interface/'API'/"contract" to parameters.store_results() which performs > the transfer of values being carried-forward into the next game/play, > onto the 'DB'/backing-store. But that then makes another object responsible for the handling of the games state. That's a bad smell OOPwise. It may be that the Game uses some kind of data adaptor object internally to do its storage but that should not be exposed at the higher level application code, it should be internal to the game. Only the game knows about its data and it alone is responsible for it. > There is room for some debate about which of these two approaches is > 'best' - or a better approach to the decision: which is preferable for > this application? Always. But if viewing through the OOP prism there are fewer allowed approaches. If we permit hybrid design paradigms then all sorts of options present themselves. > The reasons for (my) preferring the (second outline) above is that each > object (GameParameters and SolitaireGame) has its own (separate) set of > methods, and the two only 'meet' across the two 'mirror' interfaces, ie > a 'Clean Architecture'. SolitaireGame plays Solitaire. GameParameters > handles persistence. That's a different interpretation of GameParamer's responsibilities than I had in mind. But splitting an objects data handling across multiple objects is very bad OOP practice. it means changes in one object need to be reflected in another. > Whereas, the former case requires that SolitaireGame not only take in > the data (parameters) but either: > - understand that the parameters object to include a > store_results/store_parameters method, or > - will directly store the parameters itself > (which would mean that both objects need to know about I/O for > persistence - whither 'separation of concerns'?). I would always take the second approach. Only the game should know about the game data. The parameters only knows where to find it plus any shared data between games 9but even that could/should be in the database. > Perhaps without such separation it might be possible to 'bury' changes > and make them seem less significant ("major")? Thats the OOP theory, any changes to schema(ie data) should only impact the object that manages that data. > On the other hand, in a commercial situation, "separation" enables the > work to be split between different people/teams. In this example, we > could ask our DB-specialist to create GameParameters, whilst our > games-expert gets-busy with SolitaireGame. The two will have to 'put > their heads together' to design and agree the interface both up-front > and in an Agile cyclic-update manner thereafter. Which brings me to... Clearly defined interfaces are always the key to project management division of labour. The Game object could itself be split by teams or the game (and all other objects in the system) could utilize a database adaptor object to do its storage. The database adaptor object in turn could be the responsibility of another team - a database specialist team. > If achieving working-code is where the objective/satisfaction lies. > Either of the above, or indeed the paired-functions (used as a more > simple illustration, earlier in the thread), will 'do the job'. Absolutely. you can produce code that "does the job" using any approach. You may wind up with spaghetti but it will work - for a while at least. But you can apply structured design, OOP, Functional programming or any other approach and get a working (and maintainable!) result. Which approach suits the problem domain (and team skills) best is a project specific decision. > author is likely to be the only user, then should changes (even > 'catastrophic changes') need to be made, 'no puppies will die'. True, but a lot of midnight oil may be burned! (Been there, bought the tee shirt!) > storage options, and application-design. Accordingly, the earlier > suggestion that separating I/O from game-play would enable various (and > independent) iterations of both game-complexity and storage-medium - all > the while (post iteration nr1), having a working-application. The question is whether the separation requires separate objects or whether it is only a matter of separate methods in the same object. In OOP every object is responsible for the lifecycle of its own data and thus incorporates the creation, update, storage and removal of its own data. This is one of the biggest differences in approach between OOP and more traditional structured design techniques where separation is achieved by high level functions that take responsibility for everyone's data. These two approaches lead to very different code structures (and very different performance characteristics depending on the nature of the data!). > In this project's learning-path we might see some merit in starting-out > with a dict to hold key:value pairs of parameters. Zero persistence, That was very much my expectation of the parameters object. It's main role was navigating the morass of OS/User locations and config options. Once it has achieved that it simply hands a set of references to the Game which uses them to load the requisite state. > concentrate on the workings of SolitaireGame, and find that over-time > the requirements for I/O (interface) seem to naturally 'fall out' and > make themselves obvious. Most games are interactive and by their nature will lend themselves to an MVC architecture. The Model object is what we are discussing, the interaction would be in the view and controller objects. The model's only I/O is therefore the management of state data. Any user or network interaction would be down to the controller(input) and view(output) > In the early days, the dict can be easily expanded and altered. When > SolitaireGame starts to feel 'stable', there will be time and 'spare > effort' to apply to the I/O object/methods. A dict will lead naturally > to JSON, and thereafter JSON could lead to MongoDB. Thus, a convenient > expansion plan/learning-path. Not that it would be difficult to re-form > the dict into a relational schema either... But that should all be internal to the Game (and/or its data adaptor). The adaptor may need to change if we move from SQLite to Mongo say, but the game's need to store its state remains the same. I suspect we may be saying the same things but from slightly different assumptions about architecture.... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Sun Jul 4 08:43:06 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 4 Jul 2021 13:43:06 +0100 Subject: [Tutor] Python program to extract numeric suffix from a string In-Reply-To: References: Message-ID: On 04/07/2021 05:16, Manprit Singh wrote: > Dear sir, > > Assume a problem to extract a numeric suffix from a string . You need to define the suffix more precisely. What happens if the name without suffix, ends in numbers? How do you determine where the name ends and the suffix starts? Usually we include a separator (a - or . say) or use a fixed length. Assuming in this case that the name can not end in a number.... > String is st = "abcd12ghi456" > The numeric suffix of this string is "456" in case if there is no suffix, > the program should print " no suffix found" > The code i have written is given below : > > def return_suffix(st): > for x, y in enumerate(reversed(st)): Please use meaningful names, it makes the code much easier to read. Even n(dex) and c(har0 would be better than x,y but index,char would be better still! > if not y.isdigit(): > return st[-x:] if x else "No digit suffix found" > I feel I have used enumerate and reversed both which makes the program a > little difficult to read . Can this be written in a more efficient way ? Ease of reading and efficiency are very different things. Which is more important to you? Personally I go for ease of reading unless I know I need speed. But why use reversed? Why not just iterate in reverse. Also, it's not good practice to return different data types depending on the success. It would be better to either return an invalid suffix(-1 say) or raise an exception. I've opted to return the suffix as a string and an empty string if none found. def f(s): if (not s) or s.isdigit(): return s # empty or no name if not s[-1].isdigit(): return "" # no suffix n = -2 while s[n].isdigit(): n -= 1 return s[n+1:] Avoids the enumerate() and reversed() and is arguably readable. No idea how it performs compared to the original. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From phillor9 at gmail.com Sun Jul 4 19:46:43 2021 From: phillor9 at gmail.com (Phil) Date: Mon, 5 Jul 2021 09:46:43 +1000 Subject: [Tutor] Really basic won't run error In-Reply-To: References: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> <9a8cba7b-8fee-fe8d-f90c-15b115fda493@DancesWithMice.info> <8feb4e2a-01a2-a687-2c14-0f4ab02668bc@gmail.com> Message-ID: <57c8f99b-0598-e321-d8cf-5c3017e41f7f@gmail.com> On 5/7/21 6:26 am, Dennis Lee Bieber wrote: > It's your own file... But you seem to have trimmed the example > WITHOUT posting the error message as reported by the sample. In the trimmed > example it is line 18. Thank you Dennis for your reply. I've never been able to run any of the demo examples outside of demo.py. Occasionally, I've been able to take a line or two from a demo example and add it to my template and have it work or find a snippet in the documentation but mostly it takes hours of Internet searching to get something working. I should have mentioned that I had removed "parent" in two places to eliminate the errors but since the code ran but didn't display anything I thought that "parent" must be required. -- Regards, Phil From robertvstepp at gmail.com Sun Jul 4 21:33:37 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Sun, 4 Jul 2021 20:33:37 -0500 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: <87a16d41-7c53-a2d8-4c3e-28dcf900538a@DancesWithMice.info> <5296b31e-304a-c15f-3a6f-75a7e3c3c305@DancesWithMice.info> Message-ID: I have read all of the posts in this thread. I have been convinced of the dangers of using a single cursor! I have spent most of my time pondering the overall structure of my code. I have especially been attentive to dn and Alan's discussions on this. At the end of this email is the current state of the code. It is *not* meant to be in any sort of runnable shape. However, I think it more than adequately describes what I am attempting and how I have structured things. Comments, suggestions but no thrown rotting vegetables are welcome! I am trying to follow the MVC design pattern. I have not focused at all on the View(s) at this time. My main focus has been on the SolitaireGame and Controller classes along with who is responsible for the database and its access. The Controller is a bit hodge-podge, but is meant to coordinate the currently known minimal functionality I am planning. Questions: 1) Am I doing the Controller properly? 2) You will note that SolitaireGame will detect a new game situation where the user must enter needed data. Alan has emphasized that SolitaireGame is solely responsible for its data. But getting user input is normally handled by the Display and Controller. How do I "properly" acquire the needed data from the user? 3) Am I handling the database stuff properly so far? 4) I haven't made it this far yet, but part of each game will be a record of each hand played. The current table "games" will refer to these tables, one per game. The fields in each of these tables, named by "game_name" will be: hand_number (primary key), date_played, time_recorded and hand_score. Of course for every new game this table will have to be created. If it exists (not a new game) this data needs to be acquired by the SolitaireGame object. I want these game tables to be able to extract statistical data to help compare various strategies for playing the solitaire games. I don't have any specific questions yet, but wanted to make you aware of this additional component which I haven't mentioned previously. 5) I am about to start writing tests and then flesh out the code. How does one test stuff reliant on databases? Thoughts that come to mind are having special test databases accessed by changing the database path to a testing database directory. Creating in-memory databases as a mock. Something else? 6) The UI will be the last thing I worry about and will probably develop iteratively from a command line display to possibly a curses-based display (Because I want to make use of Alan's book!), and finally to a tkinter UI. How does one test UIs? I can see relatively easily testing UI-generated function calls, but mimicking mouse clicks in an automated fashion? I suppose one could figure out a way to pass in mouse coordinates and trigger a mouse or keyboard event? Anyway, awaiting your commentary. Cheers! boB Stepp =========================================================== """Program to keep track of solitaire scores. Definitions: Foundations: Target area of four piles arranged by suits. Object of most games is to get as many cards as possible into this area as these cards count positively towards one's hand score. Reserve: Most games start with a fixed number of cards in this pile, which the player endeavors to deplete, as in most solitaire games any remaining cards in this pile count negatively against one's hand score. Hand: One round of a solitaire game. """ import sqlite3 from typing import Tuple def open_db() -> sqlite3.Connection: """Create/open database and return connection object.""" # TODO: Implement more robust DB_PATH handling. DB_PATH = "C:/Projects/solitaire_scorekeeper/solitaire.db" con = sqlite3.connect(DB_PATH) # Create games table on first use of program. cur = con.cursor() cur.execute( """ CREATE TABLE IF NOT EXISTS games ( game_id INTEGER PRIMARY KEY, game_name TEXT UNIQUE, strategy TEXT, num_reserve_cards INTEGER, num_foundation_cards INTEGER, reserve_card_value INTEGER, foundation_card_value INTEGER, total_score INTEGER DEFAULT 0 ); """ ) cur.close() con.commit() return con class SolitaireGame: """Representation of a solitaire game.""" def __init__(self, game_name: str, db_connection: sqlite3.Connection) -> None: """Create or open a solitaire game.""" self.game_name = game_name self.con = db_connection ( self.game_id, _, # Already have game_name value. self.strategy, self.num_reserve_cards, self.num_foundation_cards, self.reserve_card_value, self.foundation_card_value, self.total_score, ) = self._get_game_data() self.max_hand_score = self.num_foundation_cards * self.foundation_card_value self.min_hand_score = self.num_reserve_cards * self.reserve_card_value def _get_game_data(self) -> Tuple[int, str, str, int, int, int, int, int]: """Retrieve game data from database.""" cur = self.con.cursor() cur.execute( f""" SELECT * FROM games WHERE game_name = {self.game_name}; """ ) game_data = cur.fetchall()[0] if not game_data: # Must be new game, so no game_data present. # Get user input to initialize new game data. # Be sure to write user-inputted data to games table! pass return game_data class Controller: """Coordinator of actions between Display and SolitaireGame.""" def __init__(self) -> None: """Instantiate controller and initialize database.""" self.con = open_db() self.open_games = {} self.active_game = "" def new_game(self): """Create new solitaire game.""" """ # Ask user for game_name: str. # Name must be unique. # Validate name: BEWARE MALICIOUS USER INPUT!!! # Call SolitaireGame. """ def open_game(self): """Open existing solitaire game.""" # Call SolitaireGame. def record_game(self): """Record game score.""" # Call appropriate method of SolitaireGame. def exit_program(self): """Shut down program safely.""" # Ensure all db commits done. # Ensure db properly closed. def Application(): """Run program.""" controller = Controller() # Program event loop. if __name__ == "__main__": app = Application() From arthomas152 at comcast.net Sun Jul 4 16:42:26 2021 From: arthomas152 at comcast.net (arthomas152 at comcast.net) Date: Sun, 4 Jul 2021 16:42:26 -0400 Subject: [Tutor] Connecting to a new mysql database Message-ID: <007901d77115$1a254120$4e6fc360$@comcast.net> This works: import mysql.connector mydb = mysql.connector.connect( host='localhost', user='root', password='xxxxxxxxxxxxxxx', database='sakila') mycursor = mydb.cursor() mycursor.execute("Show tables;") myresult = mycursor.fetchall() for x in myresult: print(x) mydb.close() But, when I try to open a new mysql database, I receive the message "unknown database." Mysql reads the database without a problem. Do I need to register my database somehow? Windows 10, Python 3.9, and MySQL 8.0. The software for both are up to date. I am stymied. Thanks for any help you can give me. Arthur Thomas Hold a book in your hand and you're a pilgrim at the gates of a new city. Elizabeth Langer From riskxexbiz at aol.com Sun Jul 4 19:55:56 2021 From: riskxexbiz at aol.com (riskxexbiz at aol.com) Date: Sun, 4 Jul 2021 23:55:56 +0000 (UTC) Subject: [Tutor] Python Question References: <2097543254.877902.1625442956875.ref@mail.yahoo.com> Message-ID: <2097543254.877902.1625442956875@mail.yahoo.com> I have a form asking for some information and I want to send an error message if the information is filled out incorrectly.? See below form: Here is the python code I am using but it is not working correctly: ? ? ? ? # Ensure investor type is either conservative, moderate or aggressive? ? ? ? if investor != ["conservative" or "moderate" or "aggressive"]:? ? ? ? ? ? error_statement = "Investor Type Must Be Either Conservative, Moderate or Aggressive!"? ? ? ? ? ? return render_template("information.html", error_statement=error_statement, investor=investor) ? ? ? ? # Ensure investment amount is at least 50000? ? ? ? if investment < 50000:? ? ? ? ? ? error_statement = "Investment Amount Must Be At Least 50000!"? ? ? ? ? ? return render_template("information.html", error_statement=error_statement, investment=investment) name is "investor"? placeholder is "Investor Type"name is "investment"? placeholder is "Investment Amount" Thanks for any help you can give me!Larry From alan.gauld at yahoo.co.uk Mon Jul 5 04:04:58 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 5 Jul 2021 09:04:58 +0100 Subject: [Tutor] Connecting to a new mysql database In-Reply-To: <007901d77115$1a254120$4e6fc360$@comcast.net> References: <007901d77115$1a254120$4e6fc360$@comcast.net> Message-ID: On 04/07/2021 21:42, arthomas152 at comcast.net wrote: > This works: > > import mysql.connector > > mydb = mysql.connector.connect( > host='localhost', > user='root', > password='xxxxxxxxxxxxxxx', > database='sakila') > mycursor = mydb.cursor() > mycursor.execute("Show tables;") > myresult = mycursor.fetchall() > for x in myresult: > print(x) > mydb.close() > > > > But, when I try to open a new mysql database, I receive the message "unknown > database." Its always best to show us the code that does NOT work. And to include a full cut n paste of the error message. For example, I'm not sure what you mean by "open a new database" Do you mean you are trying to create a database from Python? Or do you mean accessing a newly created database with no data in it yet? How is the database being created? What are you trying to do with it? Real code that breaks will hopefully answer those questions. > Mysql reads the database without a problem. Do I need to register > my database somehow? This suggests that the database is already created? So what exactly are you doing with MySQL to "read" it? Again the actual SQL commands you are using (and output?) would help. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Mon Jul 5 04:11:24 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 5 Jul 2021 09:11:24 +0100 Subject: [Tutor] Python Question In-Reply-To: <2097543254.877902.1625442956875@mail.yahoo.com> References: <2097543254.877902.1625442956875.ref@mail.yahoo.com> <2097543254.877902.1625442956875@mail.yahoo.com> Message-ID: On 05/07/2021 00:55, riskxexbiz--- via Tutor wrote: > Here is the python code I am using but it is not working correctly: Please always post in plain text otherwise the mail system mangles the code as below... > ? ? ? ? # Ensure investor type is either conservative, moderate or aggressive? ? ? ? if investor != ["conservative" or "moderate" or "aggressive"]:? ? ? ? ? ? error_statement = "Investor Type Must Be Either Conservative, Moderate or Aggressive!"? ? ? ? ? ? return render_template("information.html", error_statement=error_statement, investor=investor) Picking out one line: > if investor != ["conservative" or "moderate" or "aggressive"]: This does not do what you want. Python evaluates the bit inside the list as [(True or True or True)] so your code is read by Python as: if investor != [True] Instead you probably want to use the 'in' operator: if investor not in ["conservative", "moderate", "aggressive"]: However, investor must match the test strings exactly so it's usually a good idea to make the case consistent and remove any extra whitespace: if investor.strip().lower() not in ["conservative", "moderate", "aggressive"]: (wrapping is just to fit in the email message!) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Mon Jul 5 04:17:23 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 5 Jul 2021 09:17:23 +0100 Subject: [Tutor] Really basic won't run error In-Reply-To: <57c8f99b-0598-e321-d8cf-5c3017e41f7f@gmail.com> References: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> <9a8cba7b-8fee-fe8d-f90c-15b115fda493@DancesWithMice.info> <8feb4e2a-01a2-a687-2c14-0f4ab02668bc@gmail.com> <57c8f99b-0598-e321-d8cf-5c3017e41f7f@gmail.com> Message-ID: On 05/07/2021 00:46, Phil wrote: > On 5/7/21 6:26 am, Dennis Lee Bieber wrote: >> It's your own file... But you seem to have trimmed the example >> WITHOUT posting the error message as reported by the sample. In the trimmed >> example it is line 18. > > Thank you Dennis for your reply. > > I've never been able to run any of the demo examples outside of demo.py. > Occasionally, I've been able to take a line or two from a demo example > and add it to my template and have it work or find a snippet in the > documentation but mostly it takes hours of Internet searching to get > something working. When writing code with a new framework it usually pays to go through the basic tutorials sufficiently that you understand how it works. If you don't know what the various parameters are for then you are almost certainly going to spend a lot of your life hunting for solutions. My old teacher used to call it "poke and hope", it's a very slow and inefficient way to work. Spending a day (or two) working through a detailed tutorial that explains how the framework does things will save a lot of time hunting for some random code that works (possibly by accident!) Don't just type (or paste) in the examples, pay attention to the explanations. If the tutorial doesn't explain its not a good tutorial, find another. Learning by example is only good for getting started. You need understanding too. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From cs at cskk.id.au Mon Jul 5 04:18:10 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Mon, 5 Jul 2021 18:18:10 +1000 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 04Jul2021 20:33, boB Stepp wrote: >3) Am I handling the database stuff properly so far? Just to this, there's some stuff to criticise here. > ( > self.game_id, > _, # Already have game_name value. > self.strategy, > self.num_reserve_cards, > self.num_foundation_cards, > self.reserve_card_value, > self.foundation_card_value, > self.total_score, > ) = self._get_game_data() This is quite fragile. If the columns in your game table change, even in their order, this will assign the wrong things to the various attributes. You can have sqlite3 return a Row object instead of a bare tuple of values for each row. There's an example in the module docs in the "Row Objects" section. Adapting it, after you connect to the database, add: self.con.row_factory = sqlite3.Row Then the row fetch methods return Row insteance, which still act like tuples but also have attributes named after the columns. So the assignment above would become: game_row = self._get_game_data() self.game_id = game_row.game_id self.strategy = game_row.strategy self.num_reserve_cards = game_row.num_reserve_cards self.num_foundation_cards = game_row.num_foundation_cards self.reserve_card_value = game_row.reserve_card_value self.foundation_card_value = game_row.foundation_card_value self.total_score = game_row.total_score > def _get_game_data(self) -> Tuple[int, str, str, int, int, int, > int, int]: This would return an sqlite3.Row now. > """Retrieve game data from database.""" > cur = self.con.cursor() > cur.execute( > f""" > SELECT * FROM games WHERE game_name = {self.game_name}; > """ > ) Normally you try not to SELECT * because it returns all the column values, which makes things hairy if you add columns - instead you'd select exactly what you want and nothing more. For example, you're not using the game_name. However, with a Row object being returned you can pick what you want out of the row as above. Still, it is better practice to SELECT just the columns you want. If nothing else, this reduces what gets returned. In larger programmes and databases that can be important. Also, very importantly, this is the wrong way to insert values into an SQL query. The format field {self.game_name} will just get the bare game name inserted raw into the SQL, which will likely be a syntax error or be misread as a column name. See this document: https://xkcd.com/327/ The execute method accepts a dict for things to fill in in the SQL. There's an example in the docs: cur.execute("select * from people where name_last=:who and age=:age", {"who": who, "age": age}) That will place syntacticly correct values at the ":who" and ":age" placeholders in the SQL string, using the values from the dict. Do that instead. > game_data = cur.fetchall()[0] You ccould just fetchone(). > if not game_data: This test is too late. game_data won't be empty or false, you'll get an exception during the fetch. You could go: game_rows = cur.fetchall() if not game_rows: # new game else: # we expect exactly one row, so the assignment below expects # exactly one element in game_rows game_data, = game_rows Cheers, Cameron Simpson From alan.gauld at yahoo.co.uk Mon Jul 5 07:17:11 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 5 Jul 2021 12:17:11 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: <87a16d41-7c53-a2d8-4c3e-28dcf900538a@DancesWithMice.info> <5296b31e-304a-c15f-3a6f-75a7e3c3c305@DancesWithMice.info> Message-ID: On 05/07/2021 02:33, boB Stepp wrote: > ... I am trying to follow the MVC design pattern. MVC is a great principle but unless you have an existing MVC framework it is quite hard to implement completely. In particular you need some kind of protocol (eg publish/subscribe or broadcast) to keep model and view synchronised. But if you stick with the basic concepts it's a good strategy. but like most things OOP it works more effectively when things scale up. In simple programs it can add extra effort. > I have not focused at all on the View(s) at this time. My main focus > has been on the SolitaireGame and Controller classes along with who is > responsible for the database and its access. The Controller is a bit > hodge-podge, but is meant to coordinate the currently known minimal > functionality I am planning. Controllers tend to be confusing. In essence their job is simple: capture some input and send it on to either the view or the model or both as necessary. But to do that often requires that they know more than they should about the makeup of the model and view! There is usually a fair amount of coupling between controller and view, hopefully less so with the model. > Questions: > > 1) Am I doing the Controller properly? Probably not unless you are building a full MVC framework. But I wouldn't stress over it! But I haven't looked at the code yet... > 2) You will note that SolitaireGame will detect a new game situation > where the user must enter needed data. Alan has emphasized that > SolitaireGame is solely responsible for its data. But getting user > input is normally handled by the Display and Controller. How do I > "properly" acquire the needed data from the user? The controller reads data but it does nothing with it other than identify a destination object. Often a controller sends the data to the view which updates the display (and the model if necessary) There will usually be one controller per UI "screen" and that may encompass multiple view and their associated model objects. One approach to controllers is that they should coordinate the activities of the various views and models within the screen/window. The other is that the controlled embodies the actual workflow of the application, In either case controllers are most useful where there are multiple objects involved. Having now looked at the code I don't think a separate controller is necessary in your app since it looks like you will only have one view and one model (unless maybe you have multiple games running at once?) One problem with MVC is that there are many variants of the pattern. The Smalltalk approach (the original MVC) has the concept of "pluggable controllers" where you register a controller with a view or model and they use a standard API to communicate. But most web frameworks don't do that. And many GUIs incorporate the controller into the view to produce a Model-View architecture instead. This is often easier for smaller projects. > 3) Am I handling the database stuff properly so far? > > 4) I haven't made it this far yet, but part of each game will be a > record of each hand played. The current table "games" will refer to > these tables, one per game. Can you elaborate by what you mean by games? Do you mean each type of game(clock, spyder, canfield, etc) will have its own table or do you mean each game session? (game1,game2 etc.) If the latter I'd expect it all to be a single table. But if the former you may have different attributes per game type and multiple tables makes sense. > The fields in each of these tables, named > by "game_name" will be: hand_number (primary key), date_played, > time_recorded and hand_score. Of course for every new game this table > will have to be created. If the fields are the same just use a single table and add a game_id field. The more tables you create the more complex the queries become. SQL is designed to filter data out of a large mixed collection. > If it exists (not a new game) this data > needs to be acquired by the SolitaireGame object. I want these game > tables to be able to extract statistical data to help compare various > strategies for playing the solitaire games. Do you want the tables to hold the statistical data or can you calculate it on demand? The latter means it is always current. Storing the stats at the time of creation implies it is only relevant to a historical date or that you need to update every row when the stats change. You can do that with a database trigger but "there be dragons..." Especially for performance. > 5) I am about to start writing tests and then flesh out the code. > How does one test stuff reliant on databases? Thoughts that come to > mind are having special test databases accessed by changing the > database path to a testing database directory. Creating in-memory > databases as a mock. Something else? All of the above. Most big projects I've worked on we created a specific test database that could be easily reloaded. It was carefully designed to have data of all the different scenarios we might need(including broken values) A mock is often used in unit tests, especially if the DB has a higher level API - mocking SQL itself it much harder! > 6) The UI will be the last thing I worry about and will probably > develop iteratively from a command line display to possibly a > curses-based display (Because I want to make use of Alan's book!), and > finally to a tkinter UI. How does one test UIs? There are libraries and tools for generating events at the OS level and you can drive these with scripts. But its very time consuming to do thoroughly. I'd only go that far if it was a commercial project with a long expected lifetime. Mostly, I just play with the GUI by hand... Or better, get somebody else to do so! > in mouse coordinates and trigger a mouse or keyboard event? At a basic level most GUIs include a function/method for pushing events onto the event queue. You can use that to simulate many things but it requires you to think about events at a very low level (think mouse movements, keystrokes etc) I wouldn't recommend it except for a few specific scenarios where a few events are all that's required. > =========================================================== > """Program to keep track of solitaire scores. > > Definitions: > Foundations: Target area of four piles arranged by suits. Object of most > games is to get as many cards as possible into this area as these cards > count positively towards one's hand score. > Reserve: Most games start with a fixed number of cards in this pile, which > the player endeavors to deplete, as in most solitaire games any > remaining cards in this pile count negatively against one's hand score. > Hand: One round of a solitaire game. > """ > > import sqlite3 > from typing import Tuple > > > def open_db() -> sqlite3.Connection: > """Create/open database and return connection object.""" > # TODO: Implement more robust DB_PATH handling. > DB_PATH = "C:/Projects/solitaire_scorekeeper/solitaire.db" > con = sqlite3.connect(DB_PATH) > > # Create games table on first use of program. Personally I always put database creation into a separate utility program, otherwise it clutters up your code with a lot of stuff that only ever runs once(or very rarely). By all means check the database exists and is populated with the requisite tables but leave the DDL stuff to a separate program. > cur = con.cursor() > cur.execute( > """ > CREATE TABLE IF NOT EXISTS games ( > game_id INTEGER PRIMARY KEY, > game_name TEXT UNIQUE, > strategy TEXT, > num_reserve_cards INTEGER, > num_foundation_cards INTEGER, > reserve_card_value INTEGER, > foundation_card_value INTEGER, > total_score INTEGER DEFAULT 0 > ); > """ I'd strongly recommend adding some constraints, at the very least NOT NULL where appropriate. They are a good way of catching errors. > ) > cur.close() > con.commit() > return con > > > class SolitaireGame: > """Representation of a solitaire game.""" > > def __init__(self, game_name: str, db_connection: > sqlite3.Connection) -> None: > """Create or open a solitaire game.""" > self.game_name = game_name > self.con = db_connection > ( > self.game_id, > _, # Already have game_name value. > self.strategy, > self.num_reserve_cards, > self.num_foundation_cards, > self.reserve_card_value, > self.foundation_card_value, > self.total_score, > ) = self._get_game_data() Since get_game_data is a method of self why doesn't it populate the attributes? I don't see any point in the big tuple. > self.max_hand_score = self.num_foundation_cards * > self.foundation_card_value > self.min_hand_score = self.num_reserve_cards * self.reserve_card_value > > def _get_game_data(self) -> Tuple[int, str, str, int, int, int, int, int]: > """Retrieve game data from database.""" > cur = self.con.cursor() > cur.execute( > f""" > SELECT * FROM games WHERE game_name = {self.game_name}; > """ > ) > game_data = cur.fetchall()[0] > if not game_data: > # Must be new game, so no game_data present. > # Get user input to initialize new game data. > # Be sure to write user-inputted data to games table! > pass I don't think this is a good idea. Getting the user involved in initialization is usually a bad idea. I'd be more inclined to just return empty data. The game can then prompt the user (by sending a message to the controller) to provide the data. > return game_data As mentioned above, I think this method should populate the attributes itself. Passing large collections around inside the object is just a maintenance issue waiting to happen. > class Controller: > """Coordinator of actions between Display and SolitaireGame.""" > > def __init__(self) -> None: > """Instantiate controller and initialize database.""" > self.con = open_db() > self.open_games = {} > self.active_game = "" > > def new_game(self): > """Create new solitaire game.""" > """ > # Ask user for game_name: str. > # Name must be unique. > # Validate name: BEWARE MALICIOUS USER INPUT!!! > # Call SolitaireGame. > """ > > def open_game(self): > """Open existing solitaire game.""" > # Call SolitaireGame. > > def record_game(self): > """Record game score.""" > # Call appropriate method of SolitaireGame. > > def exit_program(self): > """Shut down program safely.""" > # Ensure all db commits done. > # Ensure db properly closed. > This feels more like the application than a controller. its not really coordinating anything. I'm inclined to say just move the code into the application. > > def Application(): > """Run program.""" > > controller = Controller() > # Program event loop. Especially since the application does nothing but instantiate the controller! The controller is the application. What's missing is how the application actually runs. You need some kind of view to launch the thing. It might just be a CLI initially but the controller/application needs to present something to the user. Otherwise all you have is a very transient connection to a database. If you were using a GUI you would start the GUI running and pass the controller for it to communicate with the game. If a CLI you need to create your own loop mechanism within the CLI view code. And if a web app you need to send the first screen to the browser and connect the controller to the web framework request handlers. But something needs to initiate event handling. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From cs at cskk.id.au Mon Jul 5 19:10:22 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 6 Jul 2021 09:10:22 +1000 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 05Jul2021 12:17, Alan Gauld wrote: >On 05/07/2021 02:33, boB Stepp wrote: >> 4) I haven't made it this far yet, but part of each game will be a >> record of each hand played. The current table "games" will refer to >> these tables, one per game. > >Can you elaborate by what you mean by games? >Do you mean each type of game(clock, spyder, canfield, etc) >will have its own table or do you mean each game session? >(game1,game2 etc.) > >If the latter I'd expect it all to be a single table. >But if the former you may have different attributes >per game type and multiple tables makes sense. We may have a nomenclature problem here. It looks like boB is making a single table, with a _row_ per game, at least far as the list-of-games goes. >> The fields in each of these tables, named >> by "game_name" will be: hand_number (primary key), date_played, >> time_recorded and hand_score. Of course for every new game this table >> will have to be created. > >If the fields are the same just use a single table and add a >game_id field. The more tables you create the more complex >the queries become. SQL is designed to filter data out of >a large mixed collection. Again, I think bobB has a single db table with many rows, as you'd expect. >> 5) I am about to start writing tests and then flesh out the code. >> How does one test stuff reliant on databases? Thoughts that come to >> mind are having special test databases accessed by changing the >> database path to a testing database directory. Creating in-memory >> databases as a mock. Something else? > >All of the above. Most big projects I've worked on we created a >specific test database that could be easily reloaded. It was >carefully designed to have data of all the different scenarios >we might need(including broken values) The scenario Alan's describing here implies making the "oriiginal" test db (or just maintaining one) with the test situations and having a dump of it lying around. During testing you load the dump into a fresh scratch database and test, not affecting the original. I've been working in a Django dev env recently, and the test suites make a new empty db per test run, and the test creates the scenario to be tested (eg makes some database entries with the scenario for the test). I think this is arranged to be cheap yet isolated by doing tests inside a transaction: start transaction, set up db for the test, run test, roll the transaction back leaving the db as it was before. That way the db can be made once for the whole test run. >A mock is often used in unit tests, especially if the DB >has a higher level API - mocking SQL itself it much harder! Also, SQLite has a ":memory:" db backend instead of a file, meaning you can cmake a new in-memory db with no need to make a file on disc in some scratch area or with a funny name. Cheers, Cameron Simpson From phillor9 at gmail.com Mon Jul 5 19:48:47 2021 From: phillor9 at gmail.com (Phil) Date: Tue, 6 Jul 2021 09:48:47 +1000 Subject: [Tutor] Really basic won't run error In-Reply-To: References: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> <9a8cba7b-8fee-fe8d-f90c-15b115fda493@DancesWithMice.info> <8feb4e2a-01a2-a687-2c14-0f4ab02668bc@gmail.com> <57c8f99b-0598-e321-d8cf-5c3017e41f7f@gmail.com> Message-ID: <56126e9a-55b4-eb79-1674-ab901ab198f4@gmail.com> On 6/7/21 2:34 am, Dennis Lee Bieber wrote: Thank you Alan and Dennis for your time and comments. I do have three wxPython books but they are dated and I cannot relate their content with wxPython's manual and the demo code from demo.py. Online tutorials, though helpful, are mostly dated as well. However, I do take on your points and I will investigate the book links as well. Dennis, I note that your solution is to use a frame instead of a panel. I did eventually go back to my standard template which also uses a frame rather than a panel and managed to display my JPEG file a few hours after posting my reply yesterday. Thank you both once again for you time. -- Regards, Phil From alan.gauld at yahoo.co.uk Mon Jul 5 19:56:15 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 6 Jul 2021 00:56:15 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 06/07/2021 00:10, Cameron Simpson wrote: >> All of the above. Most big projects I've worked on we created a >> specific test database that could be easily reloaded. It was >> carefully designed to have data of all the different scenarios >> we might need(including broken values) > > The scenario Alan's describing here implies making the "oriiginal" test > db (or just maintaining one) with the test situations and having a dump > of it lying around. During testing you load the dump into a fresh > scratch database and test, not affecting the original. Exactly so. Usually you can run an entire suite of tests before having to reload. (Since the test database is itself several hundred MB in size (still very small compared to the 3-4 terabytes of the production one!) > I've been working in a Django dev env recently, and the test suites make > a new empty db per test run, and the test creates the scenario to be > tested (eg makes some database entries with the scenario for the test). > I think this is arranged to be cheap yet isolated by doing tests inside > a transaction: I've worked on projects using that approach too, but the danger lies in the fact that there is no spurious data lying around in the database, since you tend to only create the rows you need. If a query is overly inclusive you don't realize that it's harvesting more than it should because there are no other rows, or at least only the "expected" other rows. You have much less chance of collecting data accidentally. You are also less likely to spot performance hot-spots because you only have the basic data, not any huge tables - but arguably you would construct a performance-test database specifically for that purpose. But if the database is a fully loaded one and your errant queries do start returning many more rows than expected it becomes obvious sooner rather than later. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From learn2program at gmail.com Mon Jul 5 20:05:52 2021 From: learn2program at gmail.com (Alan Gauld) Date: Tue, 6 Jul 2021 01:05:52 +0100 Subject: [Tutor] Fwd: Connecting to a new mysql database In-Reply-To: <000901d771de$5caeb1b0$160c1510$@comcast.net> References: <000901d771de$5caeb1b0$160c1510$@comcast.net> Message-ID: <08be6a69-0ffd-8e73-243c-810480b620d5@yahoo.co.uk> Forwarding to tutor. Please use ReplyAll or ReplyList when responding to tutor messages. Otherwse it just comes to me! Alan g. -------- Forwarded Message -------- When I run this code, the table names are returned. > import mysql.connector > > mydb = mysql.connector.connect( > host='localhost', > user='root', > password='xxxxxxxxxxxxxxx', > database='sakila') > mycursor = mydb.cursor() > mycursor.execute("Show tables;") > myresult = mycursor.fetchall() > for x in myresult: > print(x) > mydb.close() When I run this code, I receive error messages. > import mysql.connector > > mydb = mysql.connector.connect( > host='localhost', > user='root', > password='xxxxxxxxxxxxxxx', > database='GloCultur') > mycursor = mydb.cursor() > mycursor.execute("Show tables;") > myresult = mycursor.fetchall() > for x in myresult: > print(x) > mydb.close() Here are the results? C:\Users\Arthur Thomas\AppData\Local\Microsoft\WindowsApps\python3.9.exe" D:/GloCulture/Glo_opensesame.py Traceback (most recent call last): File "D:\GloCulture\Glo_opensesame.py", line 3, in mydb = mysql.connector.connect( File "C:\Users\Arthur Thomas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\mysql\connector\__init__.py", line 273, in connect return MySQLConnection(*args, **kwargs) File "C:\Users\Arthur Thomas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\mysql\connector\connection.py", line 107, in __init__ self.connect(**kwargs) File "C:\Users\Arthur Thomas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\mysql\connector\abstracts.py", line 1003, in connect self._open_connection() File "C:\Users\Arthur Thomas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\mysql\connector\connection.py", line 352, in _open_connection self._do_auth(self._user, self._password, File "C:\Users\Arthur Thomas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\mysql\connector\connection.py", line 219, in _do_auth self._auth_switch_request(username, password) File "C:\Users\Arthur Thomas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\mysql\connector\connection.py", line 318, in _auth_switch_request raise errors.get_exception(packet) mysql.connector.errors.ProgrammingError: 1049 (42000): Unknown database 'glocultur' Process finished with exit code 1 Python is 3.9 MsSQL is 8.0.25 Windows 10 I built the GloCultur inside MySQL. It has 8 tables and each has 24 records. It is in its own directory. Any ideas about what might be wrong or how to fix it? Thanks, Arthur -----Original Message----- From: Alan Gauld Sent: Monday, July 5, 2021 04:05 To: tutor at python.org Subject: Re: [Tutor] Connecting to a new mysql database On 04/07/2021 21:42, arthomas152 at comcast.net wrote: > This works: > > import mysql.connector > > mydb = mysql.connector.connect( > host='localhost', > user='root', > password='xxxxxxxxxxxxxxx', > database='sakila') > mycursor = mydb.cursor() > mycursor.execute("Show tables;") > myresult = mycursor.fetchall() > for x in myresult: > print(x) > mydb.close() > > > But, when I try to open a new mysql database, I receive the message > "unknown database." Its always best to show us the code that does NOT work. And to include a full cut n paste of the error message. For example, I'm not sure what you mean by "open a new database" Do you mean you are trying to create a database from Python? Or do you mean accessing a newly created database with no data in it yet? How is the database being created? What are you trying to do with it? Real code that breaks will hopefully answer those questions. > Mysql reads the database without a problem. Do I need to register my > database somehow? This suggests that the database is already created? So what exactly are you doing with MySQL to "read" it? Again the actual SQL commands you are using (and output?) would help. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From robertvstepp at gmail.com Mon Jul 5 20:20:49 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Mon, 5 Jul 2021 19:20:49 -0500 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: I haven't had a chance to dive into your responses yet, but it looks like I have not clearly communicated how I am using the tables and what I mean by "game". On Mon, Jul 5, 2021 at 6:27 PM Cameron Simpson wrote: > > On 05Jul2021 12:17, Alan Gauld wrote: > >On 05/07/2021 02:33, boB Stepp wrote: > >> 4) I haven't made it this far yet, but part of each game will be a > >> record of each hand played. The current table "games" will refer to > >> these tables, one per game. > > > >Can you elaborate by what you mean by games? > >Do you mean each type of game(clock, spyder, canfield, etc) > >will have its own table or do you mean each game session? > >(game1,game2 etc.) > > > >If the latter I'd expect it all to be a single table. > >But if the former you may have different attributes > >per game type and multiple tables makes sense. > > We may have a nomenclature problem here. By "game" I mean a particular kind of solitaire game with its own scoring parameters. Further if I decide to try out different strategies of playing that particular type of solitaire game then that would count as a separate entry in the games table. For instance if there were such a thing as "boBSolitaire" then there might be in the games table entries for "boBSolitaireMinimizeReservePile", "boBSolitaireMaximizeFoundationPiles", etc. Dennis asked what I meant by "hand". I defined what I meant in the module docstring, but it is simply "I just now played a hand of boBSolitaire." There would be a score for that hand which would be added to the running total score. So the games table would look like (I bet this will get mangled in the archive): game_id | game_name | strategy | max_num_reserve_cards | max_num_foundation_cards | reserve_card_value | foundation_card_value | total_score ================================================================================================================== 1 | boBSolitair1 | myidea1 | 13 | 52 | -1 | 1 | - 548 2 | boBSolitair2 | myidea2 | 13 | 52 | -1 | 1 | 0 3 | AlanSolitair1 | hisidea1 | 28 | 104 | -2 | 1 | 291 4 ... Each one of these solitaire games would have its own collection of hands played over time and look like: boBSolitair ========= hand_num | date_played | time_recorded | hand_score ========================================== 1 | 2021-07-04 | 1342 | -5 2 | 2021-07-04 | 1356 | 43 3 | 2021-07-05 | 0704 | -26 4 ... Likewise "boBSolitair2" and "AlanSolitair1" would have their own hands played tables. So each "game" is a combination of a particular type of solitaire played with a certain strategy applied. Each such game will have a record of ALL of the hands played using that type of solitaire with a particular strategy used in the game play. Later I will compare all solitaire games played of the same kind of solitaire to determine the effectiveness of the different strategies applied. Does this help make things clearer? boB Stepp From alan.gauld at yahoo.co.uk Mon Jul 5 20:25:09 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 6 Jul 2021 01:25:09 +0100 Subject: [Tutor] Fwd: Connecting to a new mysql database In-Reply-To: <08be6a69-0ffd-8e73-243c-810480b620d5@yahoo.co.uk> References: <000901d771de$5caeb1b0$160c1510$@comcast.net> <08be6a69-0ffd-8e73-243c-810480b620d5@yahoo.co.uk> Message-ID: On 06/07/2021 01:05, Alan Gauld wrote: >> mydb = mysql.connector.connect( >> host='localhost', >> user='root', >> password='xxxxxxxxxxxxxxx', >> database='sakila') > When I run this code, I receive error messages. >> import mysql.connector >> >> mydb = mysql.connector.connect( >> host='localhost', >> user='root', >> password='xxxxxxxxxxxxxxx', >> database='GloCultur') > Here are the results? > > C:\Users\Arthur > Thomas\AppData\Local\Microsoft\WindowsApps\python3.9.exe" > D:/GloCulture/Glo_opensesame.py > Traceback (most recent call last): > File "D:\GloCulture\Glo_opensesame.py", line 3, in > mydb = mysql.connector.connect( .... > line 219, in _do_auth > self._auth_switch_request(username, password) > File "C:\Users\Arthur > Thomas\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\mysql\connector\connection.py", > line 318, in _auth_switch_request > raise errors.get_exception(packet) > mysql.connector.errors.ProgrammingError: 1049 (42000): Unknown database > 'glocultur' > > Process finished with exit code 1 > I built the GloCultur inside MySQL. It has 8 tables and each has 24 > records. It is in its own directory. If it says no such database that implies a path issue. I don't know how MySql derives the location of its databases but I'd start by looking at that. You say it works from MySql? How are you running MySql? Is it from the same folder as you are running python (which defines the CWD for the process)? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From phillor9 at gmail.com Mon Jul 5 20:45:09 2021 From: phillor9 at gmail.com (Phil) Date: Tue, 6 Jul 2021 10:45:09 +1000 Subject: [Tutor] Really basic won't run error In-Reply-To: References: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> <9a8cba7b-8fee-fe8d-f90c-15b115fda493@DancesWithMice.info> <8feb4e2a-01a2-a687-2c14-0f4ab02668bc@gmail.com> <57c8f99b-0598-e321-d8cf-5c3017e41f7f@gmail.com> <56126e9a-55b4-eb79-1674-ab901ab198f4@gmail.com> Message-ID: On 6/7/21 10:26 am, Dennis Lee Bieber wrote: > Frames are top-level "windows", Thank you for the explanation and the links. It looks like I have more reading ahead of me. -- Regards, Phil From robertvstepp at gmail.com Mon Jul 5 22:27:31 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Mon, 5 Jul 2021 21:27:31 -0500 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On Mon, Jul 5, 2021 at 3:28 AM Cameron Simpson wrote: > > On 04Jul2021 20:33, boB Stepp wrote: > >3) Am I handling the database stuff properly so far? > > Just to this, there's some stuff to criticise here. > > > ( > > self.game_id, > > _, # Already have game_name value. > > self.strategy, > > self.num_reserve_cards, > > self.num_foundation_cards, > > self.reserve_card_value, > > self.foundation_card_value, > > self.total_score, > > ) = self._get_game_data() > > This is quite fragile... I definitely concur as this bothered me before I even composed my post. But I wanted to get out a question as to my overall approach before I wasted too much time going down the wrong rabbit holes. However, I had hopes that someone would have a better way than the above... > ... If the columns in your game table change, even in > their order, this will assign the wrong things to the various > attributes. You can have sqlite3 return a Row object instead of a bare > tuple of values for each row. There's an example in the module docs in > the "Row Objects" section. Adapting it, after you connect to the > database, add: > > self.con.row_factory = sqlite3.Row > > Then the row fetch methods return Row insteance, which still act like > tuples but also have attributes named after the columns. So the > assignment above would become: This definitely is useful and the way to go. I had seen this in the docs and meant to read about it, but I was/am currently worrying about SQL usage. > Also, very importantly, this is the wrong way to insert values into an > SQL query. The format field {self.game_name} will just get the bare > game name inserted raw into the SQL, which will likely be a syntax error > or be misread as a column name. See this document: > > https://xkcd.com/327/ I am aware of this, but game_name is supposed to be fully vetted by this point and would map str in Python to TEXT in that column. Can there still be issues with this getting inserted into the table? But I suppose many bad things can happen during code development which might alter the safeness of that value, so I take your point and will adopt this as a generally used technique. > The execute method accepts a dict for things to fill in in the SQL. > There's an example in the docs: > > cur.execute("select * from people where name_last=:who and age=:age", {"who": who, "age": age}) > > That will place syntacticly correct values at the ":who" and ":age" > placeholders in the SQL string, using the values from the dict. Do that > instead. > > > game_data = cur.fetchall()[0] > > You ccould just fetchone(). In retrospect I don't know why I didn't use fetchone() as there will always only be one row returned as game_name must be unique in that table. > > if not game_data: > > This test is too late. game_data won't be empty or false, you'll get an > exception during the fetch. You could go: I beg to differ here. This test is meant to cover the "new game" case where there is not an entry yet for that game. I tested this in the interpreter and as I originally wrote it returns an empty list. Thanks, Cameron! boB Stepp From robertvstepp at gmail.com Mon Jul 5 22:59:22 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Mon, 5 Jul 2021 21:59:22 -0500 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On Mon, Jul 5, 2021 at 9:22 PM Dennis Lee Bieber wrote: > > On Mon, 5 Jul 2021 19:20:49 -0500, boB Stepp > declaimed the following: > > > game_id | game_name | strategy | max_num_reserve_cards | > max_num_foundation_cards | reserve_card_value | foundation_card_value > | total_score > ================================================================================================================== > 1 | boBSolitair1 | myidea1 | 13 > | 52 | -1 > | 1 | - 548 > 2 | boBSolitair2 | myidea2 | 13 > | 52 | -1 > | 1 | 0 > 3 | AlanSolitair1 | hisidea1 | 28 > | 104 | -2 > | 1 | 291 > > Note: if "game name" and "strategy" are always linked one-for-one, you > should get rid of one of the columns -- or create a table of What I wrote in an effort to create a non-mangled table (not much luck there) was to have short names. In actual usage "game_name" is meant to be a user-created identifier that ultimately will show up in the UI menu to select that game. The "strategy" field will be for an expanded description of what strategy the user wishes to employ for that particular game. It might just be a word or two or be a lengthy paragraph. > strategy(*ID*, _game_name_, _strategy_) > > and only have one column which is a foreign key (that is, it stores the ID > from another table, for use in joining tables). Call it "strategy" and drop > game_name from your main table. > > That allows you to have just "boBSolitair" for "game_name", and records > with different "strategies" > > strategy > ID game_name strategy > 1 boBSolitair myidea1 > 2 boBSolitair myidea2 > 3 AlanSolitair hisidea1 > 4 boBSolitar myidea3 > > That then makes your first table > variants > ID strategyID ... > 1 1 ... > 2 2 ... > 3 3 ... (and yes, the example data is rather redundant looking > 4 4 ... > > >Each one of these solitaire games would have its own collection of > >hands played over time and look like: > > > >boBSolitair > >========= > >hand_num | date_played | time_recorded | hand_score > >========================================== > >1 | 2021-07-04 | 1342 | -5 > >2 | 2021-07-04 | 1356 | 43 > >3 | 2021-07-05 | 0704 | -26 > >4 ... > > > >Likewise "boBSolitair2" and "AlanSolitair1" would have their own hands > >played tables. > > > And that is where the problem comes in. > > You should NOT have a table for each variant. Instead you should have > one table, which has a field pointing to the variant to which it applies... > > Calling it "hand_number" is meaningless -- it is just a primary key ID > field I disagree. It has the meaning of the 1st hand played, the second hand, etc. This also makes it a good primary key. > plays > ID variantID date_played time_recorded score > 1 1 ... > 2 2 ... > 3 3 ... > 4 1 ... > 5 1 ... > 6 3 ... > 7 4 ... > > > SELECT s.game_name, s.strategy, v.max_num_reserved_cards, v. ..., > p.date_played, p.time_recorded, p.score from strategy as s inner join > variant as v on v.strategyID = s.ID inner join plays as p on p.variantID = > v.ID where s.game_name = "AlanSolitair" and s.strategy = "hisidea1"; > > (that should return the plays with ID 3 and 6). > > Have you read any tutorials on relational database design (these should > be engine agnostic -- and cover things like at least the first three Codd > Normal Forms). Yes and no. Yes, too many years ago and at the time I never did more than play around a little bit with databases. No, in that I have obviously forgotten what I had all too briefly known. Nonetheless I suspected I was making things much harder than they should be, which is why I ask these questions... > https://www.codementor.io/@nigelbpeck/design-principles-for-relational-data-tzzujzkiq > > https://medium.com/@expertwithsagarjaybhay/principles-relational-database-5fad42e888af > (one disagreement: In relational theory, the non-key data is related to the > primary key WITHIN a single table; that first bullet is talking about > joining data from multiple tables... > Theory Practice > relation table > tuple row > ) > https://medium.com/@kimtnguyen/relational-database-schema-design-overview-70e447ff66f9 > (also uses the non-theory definition of "relation" -- but does mention 1NF, > 2NF, 3NF) > > Wikipedia appears to be the best reference (and uses relation correctly > as representing A-TABLE). > https://en.wikipedia.org/wiki/Relational_model > https://en.wikipedia.org/wiki/Database_normalization > > For most/casual uses, 3NF is sufficient. Thanks for these references. I also have plenty of books on these topics however dusty they be.... boB Stepp From cs at cskk.id.au Mon Jul 5 23:37:42 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 6 Jul 2021 13:37:42 +1000 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 05Jul2021 21:27, boB Stepp wrote: >> Also, very importantly, this is the wrong way to insert values into >> an >> SQL query. The format field {self.game_name} will just get the bare >> game name inserted raw into the SQL, which will likely be a syntax error >> or be misread as a column name. See this document: >> >> https://xkcd.com/327/ > >I am aware of this, but game_name is supposed to be fully vetted by >this point and would map str in Python to TEXT in that column. Can >there still be issues with this getting inserted into the table? Well, you're not even quoting it. So you'd be making a piece of SQL like: SELECT * FROM games WHERE game_name = my_game_name That tries to compare two columns, "game_name" and "my_game_name", and there is no my_game_name column. You presumably want: SELECT * FROM games WHERE game_name = 'my_game_name' But that takes you down the rabbit hole of correct SQL quoting and always remembering to do it. Got a quote inside you string? Etc. Never trust you string to be "vetted". This is a syntax issue. You _should_ be able to support a game name like "; drop table ..." without issue or risk. (You might forbid it, but it shouldn't represent an SQL injection risk if you don't.) The placeholder approach in fact often separates the SQL from the parameters in the network protocol, so it isn't even trying to insert quoted values into the SQL - it will be sending some SQL-with-placeholders and sepearate protocol fields for the values to go in the placeholders. What happens in SQLite itself I'm not sure. But the placeholder approach means this is not your problem! The db driver / python module handles that. _Always_ use placeholders! >> > game_data = cur.fetchall()[0] >> >> You ccould just fetchone(). > >In retrospect I don't know why I didn't use fetchone() as there will >always only be one row returned as game_name must be unique in that >table. > >> > if not game_data: >> >> This test is too late. game_data won't be empty or false, you'll get an >> exception during the fetch. You could go: > >I beg to differ here. This test is meant to cover the "new game" case >where there is not an entry yet for that game. I tested this in the >interpreter and as I originally wrote it returns an empty list. I suggest that this line: game_data = cur.fetchall()[0] will raise an index error when fetchall() returns an empty list. That is why the test is too late in the code you had. Cheers, Cameron Simpson From __peter__ at web.de Tue Jul 6 02:33:45 2021 From: __peter__ at web.de (Peter Otten) Date: Tue, 6 Jul 2021 08:33:45 +0200 Subject: [Tutor] Python Question In-Reply-To: References: <2097543254.877902.1625442956875.ref@mail.yahoo.com> <2097543254.877902.1625442956875@mail.yahoo.com> Message-ID: On 05/07/2021 10:11, Alan Gauld via Tutor wrote: > On 05/07/2021 00:55, riskxexbiz--- via Tutor wrote: > Picking out one line: > >> if investor != ["conservative" or "moderate" or "aggressive"]: > > This does not do what you want. Python evaluates the bit > inside the list as [(True or True or True)] so your code > is read by Python as: > > if investor != [True] Let's see: >>> ["conservative" or "moderate" or "aggressive"] ['conservative'] So no, the expression a or b or c picks the first value that evaluates to True in a boolean context, i. e. with bool(value) == True. If there is no such -- sometimes called "truthy" -- value the expression evaluates to the last tested value, e. g.: >>> 0 or "" or 0.0 0.0 >>> 0 or 0.0 or "" '' From gisselle312 at hotmail.com Mon Jul 5 20:32:11 2021 From: gisselle312 at hotmail.com (edna rey) Date: Tue, 6 Jul 2021 00:32:11 +0000 Subject: [Tutor] I need help In-Reply-To: <56126e9a-55b4-eb79-1674-ab901ab198f4@gmail.com> References: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> <9a8cba7b-8fee-fe8d-f90c-15b115fda493@DancesWithMice.info> <8feb4e2a-01a2-a687-2c14-0f4ab02668bc@gmail.com> <57c8f99b-0598-e321-d8cf-5c3017e41f7f@gmail.com> , <56126e9a-55b4-eb79-1674-ab901ab198f4@gmail.com> Message-ID: Nicol?s left and now Paola has been left alone watching the performance star award show. She has decided to start recording the first letter of the names of the actors and actresses who appear in the show and to count the number of times in a row a letter appears. Entry A list with each of the initial letters of the actor's or actress's name, separated by a space, in the same order of appearance in the show. Exit A row with the initial letters of the names of the actors or actresses that appear in the show in their order of arrival, separated by a space, and another row with the number of actors or actresses of the same font that arrived consecutively Example Entry Exit Y Y Y X X P P P L Q V V E E E E V V R K K K Y X P L Q V E V R K 3 2 3 1 1 2 4 2 1 3 G G U U U U A A A Z Z Z T T T V V Q Q A A A G U A Z T V Q A 2 4 3 3 3 2 2 3 E O O E E O O O R R R V J Z Z Z B E O E O R V J Z B 1 2 2 3 3 1 1 3 1 letras= input(str ("IngreselasInicialesDelosActores")).upper() frecuencia = [] for n in (len(letras)): if n in frecuencia: frecuencia[n] +=1 else: frecuencia[n] = 1 print(frecuencia) IngreselasInicialesDelosActoresG G U U U U A A A Z Z Z T T T V V Q Q A A A G G U U U U A A A Z Z Z T T T V V Q Q A A A 43 {'G': 2, ' ': 21, 'U': 4, 'A': 6, 'Z': 3, 'T': 3, 'V': 2, 'Q': 2} From learn2program at gmail.com Tue Jul 6 04:14:02 2021 From: learn2program at gmail.com (Alan Gauld) Date: Tue, 6 Jul 2021 09:14:02 +0100 Subject: [Tutor] Python Question In-Reply-To: References: <2097543254.877902.1625442956875.ref@mail.yahoo.com> <2097543254.877902.1625442956875@mail.yahoo.com> Message-ID: <5d8c82c0-4ecd-3fc8-8408-81e56ea46122@yahoo.co.uk> On 06/07/2021 07:33, Peter Otten wrote: > On 05/07/2021 10:11, Alan Gauld via Tutor wrote: > if investor != ["conservative" or "moderate" or "aggressive"]: >> This does not do what you want. Let's see: > >>> ["conservative" or "moderate" or "aggressive"] > ['conservative'] > > So no, the expression > > a or b or c > > picks the first value that evaluates to True in a boolean context, Oops! Good catch, I should have got that. But it still doesn't do what the OP intended. :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Tue Jul 6 04:53:31 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 6 Jul 2021 09:53:31 +0100 Subject: [Tutor] I need help In-Reply-To: References: <053949f2-1ca5-885d-705b-f5d13c728967@gmail.com> <9a8cba7b-8fee-fe8d-f90c-15b115fda493@DancesWithMice.info> <8feb4e2a-01a2-a687-2c14-0f4ab02668bc@gmail.com> <57c8f99b-0598-e321-d8cf-5c3017e41f7f@gmail.com> <56126e9a-55b4-eb79-1674-ab901ab198f4@gmail.com> Message-ID: On 06/07/2021 01:32, edna rey wrote: > E O O E E O O O R R R V J Z Z Z B > > E O E O R V J Z B > > 1 2 2 3 3 1 1 3 1 Notice that you need two result lists not one. The first stores the unique letters The second stores the frequency counts. Python provides a solution for that in the form of a dictionary, but I'm going to assume that your course has not covered them yet and work with simple lists. (There is an even more specific tool for this kind of scenario in the Collections module, but I'll ignore that too!) > letras= input(str ("IngreselasInicialesDelosActores")).upper() You don't need the str() call since it is already a string. > frecuencia = [] Here you create an empty list. If it was a dictionary - do you know about them yet? The code below would work. But with lists it won't. You would need to size the list to store all of the possible letters. You an do that like: frecuencia = 'abcdefghijklmnopqrstuvwxyz'.upper() You also need a third list to store the counts: counts = [0] * len(frecuencia) > for n in (len(letras)): This gets the index of a letter, you probably just want the letter for c in letras: > if n in frecuencia: > frecuencia[n] +=1 This then looks to see if the index is in the second list. You probably want to see where the letter is in the second list: index = frecuencia.index(c) Now you can update the counts list: counts[index] += 1 > else: > frecuencia[n] = 1 Because we initialized the counts to zero you don't need the else clause. > print(frecuencia) You need to print only the letters used and their counts so: for n in range(len(frecuencia)): if counts[n] > 0: print(frecuencia[n],':',counts[n]) > IngreselasInicialesDelosActoresG G U U U U A A A Z Z Z T T T V V Q Q A A A > G G U U U U A A A Z Z Z T T T V V Q Q A A A > 43 > {'G': 2, ' ': 21, 'U': 4, 'A': 6, 'Z': 3, 'T': 3, 'V': 2, 'Q': 2} This last line suggests you are familiar with dictionaries? If so there is a simpler solution using a dictionary. Note that the solution above assumes minimal knowledge of Python. It does not show the best way to do it in Python, just a very basic technique that hopefully you can understand. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Tue Jul 6 05:06:37 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 6 Jul 2021 10:06:37 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 06/07/2021 01:20, boB Stepp wrote: > Each one of these solitaire games would have its own collection of > hands played over time and look like: > > boBSolitair > ========= > hand_num | date_played | time_recorded | hand_score > ========================================== > 1 | 2021-07-04 | 1342 | -5 > 2 | 2021-07-04 | 1356 | 43 > 3 | 2021-07-05 | 0704 | -26 > 4 ... > > Likewise "boBSolitair2" and "AlanSolitair1" would have their own hands > played tables. This is the bit swe are saying is suspect. Rather than a hands table per game type just make game type a field of a single hands table. > So each "game" is a combination of a particular type of solitaire > played with a certain strategy applied. I'm not sure how you plan on creating/recording "strategies". That sounds like a set of rules or heuristics that are typically very difficult to codify as data. I'd expect those to be stored as Python code within your game class(es?) So you'd have a top level abstract Game class, then subclasses for each variant (boBSolitair1, AlanSolitair1, etc) that overrides the strategy containing method(s) as appropriate. > record of ALL of the hands played using that type of solitaire with a > particular strategy used in the game play. The bit I'm not sure of is does this mean that each game type can have multiple strategies? And each hand may have a different strategy? So the object model becomes (I need really UML for this!): Warning ASCII art coming up! Monospace font needed... +++++++++++++++++++ Hand | Game<-boBSolitair1 | Strategy +++++++++++++++++++= > Later I will compare all > solitaire games played of the same kind of solitaire to determine the > effectiveness of the different strategies applied. > > Does this help make things clearer? Yes, but introduces the whole new issue of what exactly a Strategy looks like and how it might be stored? Is it just a function/method/algorithm? Or is it a set of data/parameters? Or a mixture of both? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Tue Jul 6 05:11:05 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 6 Jul 2021 10:11:05 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 06/07/2021 03:59, boB Stepp wrote: >> You should NOT have a table for each variant. Instead you should have >> one table, which has a field pointing to the variant to which it applies... >> >> Calling it "hand_number" is meaningless -- it is just a primary key ID >> field > > I disagree. It has the meaning of the 1st hand played, the second > hand, etc. This also makes it a good primary key. Only if you have multiple tables, so that the tablename effectively becomes part of the key. But if you only use a single hand table then you need a composite key of (gameType,handNo) Note: SQLite does support composite keys, see my tutorial for an example in the address book. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From cs at cskk.id.au Tue Jul 6 19:40:39 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Wed, 7 Jul 2021 09:40:39 +1000 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 06Jul2021 10:06, Alan Gauld wrote: >On 06/07/2021 01:20, boB Stepp wrote: >> Each one of these solitaire games would have its own collection of >> hands played over time and look like: >> >> boBSolitair >> ========= >> hand_num | date_played | time_recorded | hand_score >> ========================================== >> 1 | 2021-07-04 | 1342 | -5 >> 2 | 2021-07-04 | 1356 | 43 >> 3 | 2021-07-05 | 0704 | -26 >> 4 ... >> >> Likewise "boBSolitair2" and "AlanSolitair1" would have their own hands >> played tables. >This is the bit swe are saying is suspect. >Rather than a hands table per game type just make game >type a field of a single hands table. Just to this, a normal approach would be a single table with an additional game_id column. A specific hand is then identified by the (game_id,hand_num) tuple. A whole game would be all hands with a particular game_id value. This is what the "relational" in RDBMS is for - SELECT lets you pull out all sorts of things based on these relationships. So a "table" is normally one-for-one with a record type - you'd have one "hands" table for all "hand records" above. A particular game is just an abstraction on top of that. Like any other subset eg high scoring hands, or hands played on a particular date, etc. Cheers, Cameron Simpson From robertvstepp at gmail.com Tue Jul 6 20:16:11 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Tue, 6 Jul 2021 19:16:11 -0500 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On Tue, Jul 6, 2021 at 7:03 PM Cameron Simpson wrote: > > On 06Jul2021 10:06, Alan Gauld wrote: > >On 06/07/2021 01:20, boB Stepp wrote: > >> Each one of these solitaire games would have its own collection of > >> hands played over time and look like: > >> > >> boBSolitair > >> ========= > >> hand_num | date_played | time_recorded | hand_score > >> ========================================== > >> 1 | 2021-07-04 | 1342 | -5 > >> 2 | 2021-07-04 | 1356 | 43 > >> 3 | 2021-07-05 | 0704 | -26 > >> 4 ... > >> > >> Likewise "boBSolitair2" and "AlanSolitair1" would have their own hands > >> played tables. > >This is the bit swe are saying is suspect. > >Rather than a hands table per game type just make game > >type a field of a single hands table. > > Just to this, a normal approach would be a single table with an > additional game_id column. A specific hand is then identified by the > (game_id,hand_num) tuple. A whole game would be all hands with a > particular game_id value. > > This is what the "relational" in RDBMS is for - SELECT lets you pull out > all sorts of things based on these relationships. > > So a "table" is normally one-for-one with a record type - you'd have one > "hands" table for all "hand records" above. A particular game is just an > abstraction on top of that. Like any other subset eg high scoring hands, > or hands played on a particular date, etc. Yeah, after Dennis' posts I started to (vaguely) recall this stuff. I had forgotten that I need a different mindset about data and their interrelationships in RDBMSs that SQL then exploits. I just broke out one of my newer SQL books and will probably be reading through that for a while as I do not recall this stuff well enough obviously. This will be overkill for the current project, but beneficial for later, more substantial ones. Cheers! boB Stepp From roel at roelschroeven.net Wed Jul 7 05:51:37 2021 From: roel at roelschroeven.net (Roel Schroeven) Date: Wed, 7 Jul 2021 11:51:37 +0200 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: <740df502-e1ea-d8e4-e2b8-82d383971cc6@roelschroeven.net> Op 7/07/2021 om 1:40 schreef Cameron Simpson: > Just to this, a normal approach would be a single table with an > additional game_id column. A specific hand is then identified by the > (game_id,hand_num) tuple. A whole game would be all hands with a > particular game_id value. > > This is what the "relational" in RDBMS is for - SELECT lets you pull out > all sorts of things based on these relationships. > This is all very helpful; just one small nitpick concerning terminology. Contrary to what many people believe, and what I used to think before I took a course on RDBMSs, a relation in the relational model is *not* the link between tables. A relation in the relational model is what SQL calls a table: each row is a predicate, and a relation is a set of predicates belonging to the same predicate variable. As the Wikipedia article on Database design (https://en.wikipedia.org/wiki/Database_design#Determining_data_relationships) says: "(NOTE: A common misconception is that the relational model is so called because of the stating of relationships between data elements therein. This is not true. The relational model is so named because it is based upon the mathematical structures known as relations .) " See also e.g. the Wikipedia article on the relational model: https://en.wikipedia.org/wiki/Relational_model) I know, that all sounds overly formal and mathematical and not terribly useful, and I don't know the correct name for the link between tables. Relationship, perhaps, as you said? Still I think it's important, to avoid needless confusion, to at least be aware of the official terminology. -- "Met een spitsvondig citaat bewijs je niets." -- Voltaire From vinub at calday.co.uk Wed Jul 7 06:54:52 2021 From: vinub at calday.co.uk (Bithov Vinu Student) Date: Wed, 7 Jul 2021 11:54:52 +0100 Subject: [Tutor] Fixing a python curses bug in my program Message-ID: Hi, I've been trying to write an intelligent flashcard application (similar to Anki) that is for the command line. The program would, more specifically, take a CSV file (called a review file) that contained flashcards (field 1 is front, field 2 is the back, 3-5 are metadata filled in by the program), filter for all cards due today, never-before-seen, or due today and never-before seen (these "filters" are based on command line arguments). This filtered deck of cards is prompted to the user by the program using curses, the user presses any key to reveal the answer, and grades the card based on its easiness. The program then uses the SuperMemo 2 algorithm (cf. SuperMemo, Anki, Mnemosyne) and the aformentioned metadata to calculate the date at which the flashcard ought to be next reviewed, for optimum recall of the fact. [Here](https://bit.ly/3hMEFbr) is a heavily commented version of the code; my problem is not the overall program but rather the init_curses() function, the prompt_review_card() function , and the quiz function, which are the only functions that use curses directly. When I run the program with the correct command line arguments (exemplar usage is shown in the comments and a test file is attached to this email) it abruptly exits and my shell has its formatting changed so that every Enter results in it tabbing slightly away from me. How would I fix the curses bug? I'm well aware that at the moment the overall code is a mess and could do with some serious refactoring (for which tips and advice is appreciated) but at the moment I'm just looking to get a functioning prototype working before I make it completely robust. Thanks, Bithov Vinu= === EXEMPLAR FILE === bioch: Why do open chain saccharides have reducing properties? because of the carbonyl group (which can accept protons) bioch: Which component of starch is branched? amylopectin 0 2.5 0 0001-01-01 bioch: What stages of the pentose phosphate pathway form the non-oxidative phase? ribulose 5-phosphate -> fructose 6-phosphate 1 2.6 1 07/07/2021 === EXEMPLAR FILE === -- Student Account -- Calday Grange Grammar School is a charitable company limited by guarantee and registered in England and Wales with company number 8332696. The Registered Office is at Grammar School Lane, West Kirby, Wirral, CH48 8GG From alan.gauld at yahoo.co.uk Wed Jul 7 07:09:14 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 7 Jul 2021 12:09:14 +0100 Subject: [Tutor] Fixing a python curses bug in my program In-Reply-To: References: Message-ID: On 07/07/2021 11:54, Bithov Vinu Student wrote: > version of the code; my problem is not the overall program but rather the > init_curses() function, the prompt_review_card() function , and the quiz > function, which are the only functions that use curses directly. When I run > the program with the correct command line arguments (exemplar usage is > shown in the comments and a test file is attached to this email) it > abruptly exits and my shell has its formatting changed so that every Enter > results in it tabbing slightly away from me. Ideally you should use the curses.wrapper() function. But if you need to use some unusual initialization 9although it doesn't look like you do!) wrap the curses stuff in a try/finally: try: screen = init_curses() # your for loop here finally: curses.endwin() That should at least get you back to a sane screen. As to what's causing it to crash, that's harder to tell without an error trace. Hopefully the above will aid you in getting there. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From bouncingcats at gmail.com Wed Jul 7 09:04:59 2021 From: bouncingcats at gmail.com (David) Date: Wed, 7 Jul 2021 23:04:59 +1000 Subject: [Tutor] Fixing a python curses bug in my program In-Reply-To: References: Message-ID: On Wed, 7 Jul 2021 at 21:00, Bithov Vinu Student wrote: > [Here](https://bit.ly/3hMEFbr) is a heavily commented version of the code For anyone wondering, that goes to a pastebin: https://pastebin.com/e4UgGcqs which seems to function ok without agreeing to cookies or javascript. From alan.gauld at yahoo.co.uk Wed Jul 7 19:43:01 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 8 Jul 2021 00:43:01 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On 06/07/2021 10:06, Alan Gauld via Tutor wrote: > So the object model becomes (I need really UML for this!): > Warning ASCII art coming up! Monospace font needed... > > +++++++++++++++++++ > Hand > | > Game<-boBSolitair1 > | > Strategy > +++++++++++++++++++= I've been thinking about this and am unclear about the strategy business. Can a Player change strategy for every hand? Or is it fixed during a game? In other words if I start a Game of AlanSolitair1 can I play one hand with Strategy1 then play the next with Strategy2 and then switch back to Strategy1? And how would the scoring and reporting work in that case? I'm suspecting that the strategy should be the same for every set of hands - let's call it a Session? So does that mean we have another conceptual class - a Session - to consider. Or is a Session the same as a Game? What exactly is a Game? - is it a set of rules/plays or is it just a set of hands? I'm assuming it's the former, in which case a session is a contiguous set of hands using the same Game (but possibly different Strategies?) Or is a Game a set of hands with each hand associated with a GameType(which holds the rules/plays?) and Strategy? (Or does a Game/Session only have one GameType?) There are many ambiguities in the specification. But those need to be resolved before the database schema design can be completed. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From robertvstepp at gmail.com Wed Jul 7 19:58:36 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Wed, 7 Jul 2021 18:58:36 -0500 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2021 at 6:44 PM Alan Gauld via Tutor wrote: > > On 06/07/2021 10:06, Alan Gauld via Tutor wrote: > > > So the object model becomes (I need really UML for this!): > > Warning ASCII art coming up! Monospace font needed... > > > > +++++++++++++++++++ > > Hand > > | > > Game<-boBSolitair1 > > | > > Strategy > > +++++++++++++++++++= > > I've been thinking about this and am unclear about the strategy business. I thought I addressed the definitions of strategy, hands and games yesterday, but perhaps you missed that post -- it was the one with the mangled tables. I said: ------------------------------------------------------------------------------------------- By "game" I mean a particular kind of solitaire game with its own scoring parameters. Further if I decide to try out different strategies of playing that particular type of solitaire game then that would count as a separate entry in the games table. For instance if there were such a thing as "boBSolitaire" then there might be in the games table entries for "boBSolitaireMinimizeReservePile", "boBSolitaireMaximizeFoundationPiles", etc. Dennis asked what I meant by "hand". I defined what I meant in the module docstring, but it is simply "I just now played a hand of boBSolitaire." There would be a score for that hand which would be added to the running total score. So the games table would look like (I bet this will get mangled in the archive): ------------------------------------------------------------------------------------------- And from a slightly later post: ------------------------------------------------------------------------------------------- ...The "strategy" field will be for an expanded description of what strategy the user wishes to employ for that particular game. It might just be a word or two or be a lengthy paragraph. ------------------------------------------------------------------------------------------- > Can a Player change strategy for every hand? Or is it fixed > during a game? In other words if I start a Game of AlanSolitair1 > can I play one hand with Strategy1 then play the next with > Strategy2 and then switch back to Strategy1? And how would > the scoring and reporting work in that case? So a particular "game" is the collection of hands played, all with the same rules, using the same strategy throughout. > I'm suspecting that the strategy should be the same for > every set of hands - let's call it a Session? So does that > mean we have another conceptual class - a Session - to > consider. Or is a Session the same as a Game? Yes, as you state Session and Game would be synonymous. > What exactly is a Game? - is it a set of rules/plays or is > it just a set of hands? I'm assuming it's the former, in > which case a session is a contiguous set of hands using > the same Game (but possibly different Strategies?) See above. > Or is a Game a set of hands with each hand associated > with a GameType(which holds the rules/plays?) and Strategy? > (Or does a Game/Session only have one GameType?) Same set of rules, same strategy. > There are many ambiguities in the specification. But > those need to be resolved before the database schema > design can be completed. I pray that all ambiguities have been removed?!? boB Stepp From robertvstepp at gmail.com Wed Jul 7 23:57:33 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Wed, 7 Jul 2021 22:57:33 -0500 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: Message-ID: On Wed, Jul 7, 2021 at 8:52 PM Dennis Lee Bieber wrote: > > On Thu, 8 Jul 2021 00:43:01 +0100, Alan Gauld via Tutor > declaimed the following: > > >On 06/07/2021 10:06, Alan Gauld via Tutor wrote: [...] > >I've been thinking about this and am unclear about the strategy business. [...] > >There are many ambiguities in the specification. But > >those need to be resolved before the database schema > >design can be completed. > > Concur: we have just words to build off now (hand, game, strategy), > with no proper description of just what each entails. As I recall, "hand" > has a datetime for completion, with some score. [...] Guys, I *do* know you are trying to be helpful, but I am beginning to feel frustrated with the direction this thread has taken. I am sure I bear the brunt of this, as something I thought was rather simple in concept has become something else. I thought my communications skills were better than this, but apparently I have really fumbled the ball on this thread. I will endeavor to try again and hopefully do a better job of communication. First, I have "meta"-program goals: 1) Always to learn useful things. 2) More specifically to take a relatively simple concept -- but not too simple! -- and use it as a learning tool to better understand best software construction practices. 3) Amongst the things I wish to better understand are database usage and design, OOP, applying MVC to a program's structure effectively, and, once I get there, learn tkinter and UI design better, and so on. It has often been said on this list that a good way to learn Python and other programming concepts, it is a good idea to pick a project that interests one and try to implement it. While reading technical stuff, I like to break up the study by playing various forms of solitaire. As Dennis notes there are many, ... , many forms of solitaire. The ones I enjoy playing are open-ended and have scoring systems. Also there is flexibility in the rules, so that, for instance, I don't have to play cards to the foundations area unless I feel it is beneficial. Or I can instead focus on building long sequences of cards in the build area, or, whatever. The games I play do not have a single pass through the deck, but allow one to turn the cards over and go through them again and again until there are no legal plays left. But regardless of what the rules are -- for the program the rules don't matter -- each play of a hand results in a score. The reserve cards left subtract from the hand score by a certain number of points per card left and the foundation cards add points positively to the hand score by a certain number of points per card there. The name of the program captures its purpose: Solitaire Scorekeeper. The summary line of the module as originally posted read: "Program to keep track of solitaire scores." It is that simple in concept. The program does not care what the rules are, what the actual cards are, or how they are played. The particular solitaire game and its rules being played only matter in that it might help in giving a sensible name to the game to differentiate it from other games I might be playing and scoring. To make it more interesting as a programming project I wish to collect other data, including a desire to see how effective different approaches to playing a particular solitaire game are to getting the best possible cumulative score over time. So, to keep things simple, say I am playing a solitaire called boB_Solitaire. It doesn't matter what its rules are. It has a scoring system. I want to try two naive strategies and see how the cumulative scores tend to play out: (1) Do everything possible to play cards to the foundations area in preference to any other possible play available. This will increase the positive score for the hand. Say each foundation card is worth +1 point for a total maximum of 52 points if I am fortunate and get the entire deck of cards to that area (no jokers please). Or naive strategy (2) do everything possible to reduce the number of cards in the reserve pile. Each card left in the reserve pile counts -2 points per card. The pile starts out with 13 cards in it, so the max possible negative score would be -26. Therefore, for this particular solitaire game and scoring system each hand at the end will have a possible score ranging from -26 to +52, inclusive. In this scenario I would have two solitaire games I'm tracking over time, boB_Solitaire1 and boB_Solitaire2. They both use the same exact set of rules of play, but the first uses naive strategy (1) and the second naive strategy (2). So for the purposes of the data about each game, *not* the record of hands played I would want to have in a "games" table: In this example there would be two entries in the games table, one row for boB_Solitaire1, one for boB_Solitaire2. The columns would be: A primary key field. Name of the game -- here boB_Solitaire1 or boB_Solitaire2. A text field for "strategy": For boB_Solitaire1 it would be something like: "Play to maximize number of cards to the foundations area." For boB_Solitaire2 it would be something like: "Play to minimize the number of cards left in the reserve pile." An integer field for max_num_of_reserve_cards: Some kinds of solitaire have differing numbers of starting cards in the reserve pile than others. This is needed to compute scores for each hand. An integer field for max_num_of_foundation_cards: Often this will be 52 for a normal pack of cards without jokers being used, but some games use multiple packs of cards or even limit to less than a pack of cards to the foundation areas. Again, this is needed to compute a hand score. An integer field for reserve_card_value: Usually a negative integer. Again, the exact value for a given game's rules may vary game to game. Needed to compute a hand score. An integer field for foundation_card_value: Usually a positive integer. Exact value per card depends on a game's rules. Needed to compute a hand score. A computed integer field for the cumulative score for a game. Yes, it does not have to be in the table, but could be computed on demand, but I am stating things as originally presented in the code. A date field for the date the game was originally started/created. I did not mention this previously, but intended all along to have it recorded. As you (Dennis in particular) demonstrated, I only need one more table to record the scores for all hands of all games played. It would have these columns: A primary key field. A foreign key field referencing the games table and to which row/game the hand score belongs to. A date field to record the date the hand score was recorded. A time field to record the time that the hand was recorded. The actual hand score for the hand played. That's it. Two tables. Later, once the code to implement all of the above is done, I could explore analyzing the collected data. I might look at the average score per hand. I might look at the distribution of possible scores for a hand and perhaps generate a bar graph for the results. Use such data to make calls on which strategies appear to do best for a given kind of solitaire game. Whatever. But the program under discussion is as simple as I outlined above. There is no recording of specific cards or card layouts or .... Just scores, the needed information to compute scores, some dates and time for when played, and simple text descriptions of the strategic approach used. That's it! Remember this is simply a vehicle for me to learn with your ever kindly assistance. And to better automate something I do anyway either by hand or in a plain text file. I hope this clears things up?!? boB Stepp From z at bonjourpy.org Thu Jul 8 02:51:43 2021 From: z at bonjourpy.org (LI Bonjour) Date: Thu, 8 Jul 2021 06:51:43 +0000 Subject: [Tutor] there might be something error in dataclasses.py Message-ID: >>> from dataclasses import dataclass >>> @dataclass ... class A: ... a:list[str] = None ... ... >>> a = A() >>> a.a = [] >>> b [A(a=['1']), A(a=['1']), A(a=['1'])] >>> for x in b: ... x.a.append('1') ... >>> b [A(a=['1', '1', '1', '1']), A(a=['1', '1', '1', '1']), A(a=['1', '1', '1', '1'])] >>> I think it should be this below. [A(a=['1','1']), A(a=['1','1']), A(a=['1','1'])] is there anything wrong here . please mail me ASAP,thanks From roel at roelschroeven.net Thu Jul 8 07:24:04 2021 From: roel at roelschroeven.net (Roel Schroeven) Date: Thu, 8 Jul 2021 13:24:04 +0200 Subject: [Tutor] there might be something error in dataclasses.py In-Reply-To: References: Message-ID: <2d44e159-5cd4-5bab-51ff-8ae230cfd9b9@roelschroeven.net> Op 8/07/2021 om 8:51 schreef LI Bonjour: >>>> from dataclasses import dataclass >>>> @dataclass > ... class A: > ... a:list[str] = None > ... > ... >>>> a = A() >>>> a.a = [] >>>> b Where does this 'b' come from? What is its value? > [A(a=['1']), A(a=['1']), A(a=['1'])] Oh, 'b' seems to be a list of instances of class A. But how is it constructed? That can make all the difference. >>>> for x in b: > ... x.a.append('1') > ... >>>> b > [A(a=['1', '1', '1', '1']), A(a=['1', '1', '1', '1']), A(a=['1', '1', '1', '1'])] > I think it should be this below. > [A(a=['1','1']), A(a=['1','1']), A(a=['1','1'])] > is there anything wrong here . > What probably happened is that all the elements of b refer to one single instance of A. If you then change that instance, that change becomes visible via all references to that instance. If you don't want it, initialize b with something like ??? b = [A() for _ in range(3)] instead of e.g. ??? b = [A()] * 3 -- "A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools." -- Douglas Adams From PyTutor at DancesWithMice.info Thu Jul 8 13:09:45 2021 From: PyTutor at DancesWithMice.info (dn) Date: Fri, 9 Jul 2021 05:09:45 +1200 Subject: [Tutor] there might be something error in dataclasses.py In-Reply-To: References: Message-ID: <9088bf04-18f4-9140-de30-083412c734ef@DancesWithMice.info> On 08/07/2021 18.51, LI Bonjour wrote: >>>> from dataclasses import dataclass >>>> @dataclass > ... class A: > ... a:list[str] = None > ... > ... >>>> a = A() ... > is there anything wrong here . > please mail me ASAP,thanks Further to earlier response:- Whilst there is no published-error, there is a lack of consistency/logic: Python 3.9.5 (default, May 14 2021, 00:00:00) [GCC 10.3.1 20210422 (Red Hat 10.3.1-1)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from dataclasses import dataclass >>> @dataclass ... class A(): ... a:list[ str ] = None ... >>> type( A.a ) >>> a = A() >>> type( a.a ) (will leave you to compare running the original and this code through mypy) Whilst A.a is described to be a list of strings, it is immediately given the value of None - which is neither a string nor a list. Which should it be? a:list[ str ] = [] Which also obviates the initialisation of a.a! -- Regards, =dn From manpritsinghece at gmail.com Sat Jul 10 11:38:53 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Sat, 10 Jul 2021 21:08:53 +0530 Subject: [Tutor] Correct style of line break when chaining methods Message-ID: Dear sir, consider a line of code : csknum = crick.iloc[:, [3, 4, 5]].isin(["CSK"]).all(axis="columns").sum() This line contains chained methods . What would be an appropriate way to break this line ? The way i have done it is given below : csknum = (crick.iloc[:, [3, 4, 5]] .isin(["CSK"]) .all(axis="columns") .sum()) Need your comments Regards Manprit Singh From mats at wichmann.us Sat Jul 10 11:49:59 2021 From: mats at wichmann.us (Mats Wichmann) Date: Sat, 10 Jul 2021 09:49:59 -0600 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: Message-ID: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> On 7/10/21 9:38 AM, Manprit Singh wrote: > Dear sir, > > consider a line of code : > > csknum = crick.iloc[:, [3, 4, 5]].isin(["CSK"]).all(axis="columns").sum() > > This line contains chained methods . What would be an appropriate way to > break this line ? > The way i have done it is given below : > > csknum = (crick.iloc[:, [3, 4, 5]] > .isin(["CSK"]) > .all(axis="columns") > .sum()) > > Need your comments 1. that line doesn't need breaking, it's short enough 2. try not to - it might be a place where you're okay to go a bit over 80 columns 3. if necessary, the above is okay, but if you're going to use parens as a way to be able to break the line, you might break right after the opening paren so you're not tryint to artificially line up chunks underneath something that's a ways off to the right. By which I mean: csknum = ( crick.iloc[:, [3, 4, 5]] .isin(["CSK"]) .all(axis="columns") .sum() ) these are but opinions! From manpritsinghece at gmail.com Sat Jul 10 12:03:06 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Sat, 10 Jul 2021 21:33:06 +0530 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> Message-ID: Dear sir, See in this particular example, the length of the line is 75 characters , and according to PEP 8, in Python a line must not exceed 69 characters. So I divided this line . And Secondarily when doing work with Numpy arrays and Pandas we are mostly working with chained methods, and at that time, what i feel is a line break can increase readability also . Need further comments and guidance. Regards Manprit Singh Regards On Sat, Jul 10, 2021 at 9:20 PM Mats Wichmann wrote: > On 7/10/21 9:38 AM, Manprit Singh wrote: > > Dear sir, > > > > consider a line of code : > > > > csknum = crick.iloc[:, [3, 4, 5]].isin(["CSK"]).all(axis="columns").sum() > > > > This line contains chained methods . What would be an appropriate way to > > break this line ? > > The way i have done it is given below : > > > > csknum = (crick.iloc[:, [3, 4, 5]] > > .isin(["CSK"]) > > .all(axis="columns") > > .sum()) > > > > Need your comments > > 1. that line doesn't need breaking, it's short enough > 2. try not to - it might be a place where you're okay to go a bit over > 80 columns > 3. if necessary, the above is okay, but if you're going to use parens as > a way to be able to break the line, you might break right after the > opening paren so you're not tryint to artificially line up chunks > underneath something that's a ways off to the right. By which I mean: > > csknum = ( > crick.iloc[:, [3, 4, 5]] > .isin(["CSK"]) > .all(axis="columns") > .sum() > ) > > > these are but opinions! > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From robertvstepp at gmail.com Sat Jul 10 12:20:26 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 10 Jul 2021 11:20:26 -0500 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> Message-ID: On Sat, Jul 10, 2021 at 11:03 AM Manprit Singh wrote: > > Dear sir, > > See in this particular example, the length of the line is 75 characters , > and according to PEP 8, in Python a line must not exceed 69 characters. Are you sure you did not misread the max recommended line length? https://www.python.org/dev/peps/pep-0008/#id19 says: Maximum Line Length Limit all lines to a maximum of 79 characters. There are some qualifications later for comments and docstrings, but these don't apply to your example. boB Stepp From manpritsinghece at gmail.com Sat Jul 10 12:32:57 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Sat, 10 Jul 2021 22:02:57 +0530 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> Message-ID: Dear boB stepp, Mats Wichmann & Mark Lawrence Thanks for your comments and help ..... today i really appreciate the members like you in this group . Thanks and regards Manprit Singh On Sat, Jul 10, 2021 at 9:50 PM boB Stepp wrote: > On Sat, Jul 10, 2021 at 11:03 AM Manprit Singh > wrote: > > > > Dear sir, > > > > See in this particular example, the length of the line is 75 characters , > > and according to PEP 8, in Python a line must not exceed 69 characters. > > Are you sure you did not misread the max recommended line length? > > https://www.python.org/dev/peps/pep-0008/#id19 says: > > > Maximum Line Length > > Limit all lines to a maximum of 79 characters. > > > There are some qualifications later for comments and docstrings, but > these don't apply to your example. > > boB Stepp > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From robertvstepp at gmail.com Sat Jul 10 12:33:46 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 10 Jul 2021 11:33:46 -0500 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> Message-ID: On Sat, Jul 10, 2021 at 11:20 AM boB Stepp wrote: > > On Sat, Jul 10, 2021 at 11:03 AM Manprit Singh > wrote: > > > > Dear sir, > > > > See in this particular example, the length of the line is 75 characters , > > and according to PEP 8, in Python a line must not exceed 69 characters. > > Are you sure you did not misread the max recommended line length? > > https://www.python.org/dev/peps/pep-0008/#id19 says: > > > Maximum Line Length > > Limit all lines to a maximum of 79 characters. > > > There are some qualifications later for comments and docstrings, but > these don't apply to your example. Some further thoughts about this reflecting guidance I've read on Tutor and elsewhere. The concept of 80 characters line length is a historical artifact from the early punch card and terminal days. However, 80 characters is often cited as "about" the right line length for humans to process chunks of information. But modern software and monitors can substantially stretch this "about" line length to just about any length you find comprehensible and convenient. And remember PEP 8 is a guideline only. I am currently using an automatic Python code formatter called "black". It determines its default line length a bit longer than 79/80 characters (from https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length): Line length You probably noticed the peculiar default line length. Black defaults to 88 characters per line, which happens to be 10% over 80. This number was found to produce significantly shorter files than sticking with 80 (the most popular), or even 79 (used by the standard library). In general, 90-ish seems like the wise choice. And Mats suggested the need for judgement when you are "close to" whatever your predetermined line length is. Splitting lines unnecessarily can make the code harder to read than fudging your line length limit sometimes. boB Stepp From alan.gauld at yahoo.co.uk Sat Jul 10 14:58:03 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 10 Jul 2021 19:58:03 +0100 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: Message-ID: On 10/07/2021 16:38, Manprit Singh wrote: > The way i have done it is given below : > > csknum = (crick.iloc[:, [3, 4, 5]] > .isin(["CSK"]) > .all(axis="columns") > .sum()) > I don't believe Python has any standard idiom for breaking chained lines but Smalltalk does so you might copy that. Smalltalk aligns the dot operators: csknum = (crick.iloc[:, [3, 4, 5]] .isin(["CSK"]) .all(axis="columns") .sum()) It makes it easier to identify the object to which the chain belongs. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From breamoreboy at gmail.com Sat Jul 10 12:18:09 2021 From: breamoreboy at gmail.com (Mark Lawrence) Date: Sat, 10 Jul 2021 17:18:09 +0100 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> Message-ID: On 10/07/2021 17:03, Manprit Singh wrote: > Dear sir, > > See in this particular example, the length of the line is 75 characters , > and according to PEP 8, in Python a line must not exceed 69 characters. > So I divided this line . > And Secondarily when doing work with Numpy arrays and Pandas we are mostly > working with chained methods, and at that time, what i feel is a line break > can increase readability also . > Need further comments and guidance. > > Regards > Manprit Singh > PEP 8 quite clearly states "This document gives coding conventions for the Python code comprising the standard library in the main Python distribution". It also has a section titled "A Foolish Consistency is the Hobgoblin of Little Minds". I'll leave you to draw your own conclusions :) -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From robertvstepp at gmail.com Sun Jul 11 16:02:00 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Sun, 11 Jul 2021 15:02:00 -0500 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: <1cuceg50jl535plon1ni4utfk6p52ch1k4@4ax.com> Message-ID: On Sun, Jul 11, 2021 at 2:52 PM Dennis Lee Bieber wrote: > > > Probably of no interest at this point... Untrue. Don't take my lack of response as having no interest. Instead, you have convinced me to go off on a SQL/database design study tangent. Even though with what you have given me I could go ahead and finish the program, your information has strongly motivated me to study databases in more depth, especially as I have forgotten so much. For future programs I envision doing I will have great need for database use, so I might as well dive in now. I will return later to the program and this thread, hopefully soon as I am almost finished with the current text I am studying. Anyway, many thanks! boB Stepp From learn2program at gmail.com Sun Jul 11 16:59:52 2021 From: learn2program at gmail.com (Alan Gauld) Date: Sun, 11 Jul 2021 21:59:52 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: <1cuceg50jl535plon1ni4utfk6p52ch1k4@4ax.com> Message-ID: On 11/07/2021 21:02, boB Stepp wrote: > ahead and finish the program, your information has strongly motivated > me to study databases in more depth, especially as I have forgotten so > much. For future programs I envision doing I will have great need for > database use, so I might as well dive in now. Almost every real-world programmer needs a good knowledge of SQL and database design. When dealing with large volumes of data a RDBMS can take a lot of the strain leaving the programmer much less to hand-craft in Python (or whatever language they choose). Studying data design is never time wasted! I spent the last 10 years of my career with the label "Enterprise Architect". That meant I was in charge of the design of many large projects. But, in each case I had a "Data Architect" working with me, and frequently a "Network architect" too. These are both specialist skills that no matter how good you get at system level design, you always want an expert in your corner. In fact, on large projects, if there is a performance problem it will 90% of the time be either database or network related. - Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From learn2program at gmail.com Sun Jul 11 17:02:05 2021 From: learn2program at gmail.com (Alan Gauld) Date: Sun, 11 Jul 2021 22:02:05 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: <1cuceg50jl535plon1ni4utfk6p52ch1k4@4ax.com> Message-ID: <35d86b50-8ea7-1692-7351-81f55a6acc49@yahoo.co.uk> On 11/07/2021 20:52, Dennis Lee Bieber wrote: > I couldn't get the syntax for SQLite3 triggers to take, Can you elaborate? Apart from the requirement to only use pure SQL I've not had any issues. (Although there may be a need to switch on some PRAGMAs...) What wasn't working in SQLite? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From roel at roelschroeven.net Sun Jul 11 15:55:40 2021 From: roel at roelschroeven.net (Roel Schroeven) Date: Sun, 11 Jul 2021 21:55:40 +0200 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> Message-ID: Manprit Singh schreef op 10/07/2021 om 18:03: > And Secondarily when doing work with Numpy arrays and Pandas we are mostly > working with chained methods, and at that time, what i feel is a line break > can increase readability also . I think so too. As a general rule, or rather guideline, I think a line should as much as reasonably possible express one thought. The exact meaning of "one thought" is quite vague, of course; it's more the idea that counts than a hard rule. In this case, I think each of the chained operations constitutes a separate thought and therefore deservers to be on its own line. It can work in the other direction too: sometimes a line gets quite long just because there are e.g. some long identifiers in it, and it can be a good idea to keep it on one line even if it's a bit over 79 characters. 90-ish is the approximate cut-off I use mostly. I know, that strictly doesn't comply with PEP 8. I've always been a fan of "A Foolish Consistency is the Hobgoblin of Little Minds" in PEP 8, and "Readability counts" from PEP 20 (The Zen of Python). When increasing line length within reason aids readability/clarity, by all means go for it, I'd say. -- "Honest criticism is hard to take, particularly from a relative, a friend, an acquaintance, or a stranger." -- Franklin P. Jones Roel Schroeven From mats at wichmann.us Sun Jul 11 19:42:05 2021 From: mats at wichmann.us (Mats Wichmann) Date: Sun, 11 Jul 2021 17:42:05 -0600 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> Message-ID: <731f101f-41fe-3c9f-b939-8aae559b5349@wichmann.us> On 7/11/21 1:55 PM, Roel Schroeven wrote: > Manprit Singh schreef op 10/07/2021 om 18:03: > >> And Secondarily when doing work with Numpy arrays and Pandas we are >> mostly >> working with chained methods, and at that time, what i feel is a line >> break >> can increase readability also . > > I think so too. As a general rule, or rather guideline, I think a line > should as much as reasonably possible express one thought. The exact > meaning of "one thought" is quite vague, of course; it's more the idea > that counts than a hard rule. > > In this case, I think each of the chained operations constitutes a > separate thought and therefore deservers to be on its own line. and this gets back to the original thought... the tendency in the Numpy world seems to me (just from brief glances, mind) to indeed be to use these strings of chained methods, and it doesn't have to be that way, you can save the results and not chain them... :) From PyTutor at DancesWithMice.info Sun Jul 11 23:52:46 2021 From: PyTutor at DancesWithMice.info (dn) Date: Mon, 12 Jul 2021 15:52:46 +1200 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> Message-ID: On 11/07/2021 04.33, boB Stepp wrote: > On Sat, Jul 10, 2021 at 11:20 AM boB Stepp wrote: >> On Sat, Jul 10, 2021 at 11:03 AM Manprit Singh >> wrote: >>> >>> Dear sir, >>> >>> See in this particular example, the length of the line is 75 characters , >>> and according to PEP 8, in Python a line must not exceed 69 characters. >> >> Are you sure you did not misread the max recommended line length? ... >> Limit all lines to a maximum of 79 characters. ... > The concept of 80 characters line length is a historical artifact from > the early punch card and terminal days. However, 80 characters is > often cited as "about" the right line length for humans to process > chunks of information. But modern software and monitors can > substantially stretch this "about" line length to just about any > length you find comprehensible and convenient. And remember PEP 8 is > a guideline only. 0 bring back punch-cards! 1 I can't remember why "79", instead of the 80-column preceding 'standard' (which also followed punched cards, even when terminals/screens were capable of more than 24x80 - but back-then some languages mandated the use of the first columns and perhaps also the last). Perhaps the 80th column would be where the "\" line-continuation character fits? 2 as someone with aged and ageing eyes, I find a narrower line easier to read (within reason). Thus, the longer the line, the harder it is to read. Even more-so if the screen necessitates horizontal-scrolling! NB if I simplify the code I (could) write, in order to be considerate of others who might follow, is it unreasonable to request 'narrow lines' for the benefit of old fogies like me? 3 when including code within a lesson or article (conference presentation or other slide), the margins are often considerably narrower than (even) 79-columns. (I struck this issue whilst remotely attending an on-line Code Review, just last week) 4 I find myself torn between coding on a wide-screen in portrait or landscape mode (horizontal or vertical orientation). The portrait display gives a view of more lines of code - usually am able to scan between a function call and its definition (there's another 'rule' about such proximity, somewhere). Contrarily, the landscape mode enables multiple 'panes' so that I am able to read a library function/method call on one pane and its definition in another. Thus, code-width should be no wider than the width of my vertical screen's 'width'; and no wider than will fit in the two or three panes that may span the width of my horizontal screen! (and we're back to that 'magic number' of columns, or even slightly less) 5 I notice that my (simple-text) email-client also limits line-length, even though it is displaying the text using a variable-width font... > I am currently using an automatic Python code formatter called > "black". It determines its default line length a bit longer than > 79/80 characters (from > https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length): > > > > Line length > > You probably noticed the peculiar default line length. Black defaults > to 88 characters per line, which happens to be 10% over 80. This > number was found to produce significantly shorter files than sticking > with 80 (the most popular), or even 79 (used by the standard library). > In general, 90-ish seems like the wise choice. > Black is described as "opinionated", by which they mean that their opinion counts for more than any and all of the several and varying opinions already stated 'here'! Yes, that's an irony! See also XCKD's "standards" cartoon (https://xkcd.com/927/) > And Mats suggested the need for judgement when you are "close to" > whatever your predetermined line length is. Splitting lines > unnecessarily can make the code harder to read than fudging your line > length limit sometimes. ...and equally, splitting the line (with appropriate columnar formatting) may make each 'unit', of a long and complex LoC, easier to comprehend! +1 *Human* readability trumps all! Hobgoblins be gone... -- Regards, =dn From Richard at Damon-Family.org Mon Jul 12 08:44:52 2021 From: Richard at Damon-Family.org (Richard Damon) Date: Mon, 12 Jul 2021 06:44:52 -0600 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> Message-ID: <48459ad3-f4f7-c326-1ed6-f8767af01931@Damon-Family.org> On 7/11/21 10:52 PM, dn via Tutor wrote: > > 0 bring back punch-cards! > > 1 I can't remember why "79", instead of the 80-column preceding > 'standard' (which also followed punched cards, even when > terminals/screens were capable of more than 24x80 - but back-then some > languages mandated the use of the first columns and perhaps also the > last). Perhaps the 80th column would be where the "\" line-continuation > character fits? The problem with 80 column lines was that some terminals on getting the 80th character would automatically do a new line, and then the following new line from the code would give you a blank line. -- Richard Damon From learn2program at gmail.com Mon Jul 12 14:14:31 2021 From: learn2program at gmail.com (Alan Gauld) Date: Mon, 12 Jul 2021 19:14:31 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: References: <1cuceg50jl535plon1ni4utfk6p52ch1k4@4ax.com> <35d86b50-8ea7-1692-7351-81f55a6acc49@yahoo.co.uk> Message-ID: <7a3c65dd-805c-1cd5-6dab-ef36c782b336@yahoo.co.uk> On 12/07/2021 16:47, Dennis Lee Bieber wrote: > Pity the current SQLite documentation is only HTML -- I'd prefer a PDF. > I have both editions of "The Definitive Guide" (though even the 2nd ed > is getting old). Personally, the 1st edition is the volume to have I have the O'Reilly guide "Using SQLite" which is pretty good IMHO. Its from 2010 and covers v3.6 and touches on the "new" 3.7 features. The example it gives for a trigger also uses another table but should work for the local table: CREATE TRIGGER access_audit BEFORE UPDATE ON access FOR EACH ROW BEGIN ?? INSERT INTO audit VALUES (OLD.level, NEW.level, CURRENT_TIMESTAMP); END; I see no reason why you couldn't just change the table from 'audit' to 'access' on the INSERT statement. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From learn2program at gmail.com Mon Jul 12 14:16:23 2021 From: learn2program at gmail.com (Alan Gauld) Date: Mon, 12 Jul 2021 19:16:23 +0100 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: <11poeg5jma2v2kj1dhhbqfr7c4546e3fj5@4ax.com> References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> <11poeg5jma2v2kj1dhhbqfr7c4546e3fj5@4ax.com> Message-ID: <37cbe384-9f92-02e1-c1a4-6eae4b88de5f@yahoo.co.uk> On 12/07/2021 16:53, Dennis Lee Bieber wrote: > For punched cards, one way of documenting removed code was to punch > whatever the system/language used to indicate "comment" in column 80, then > flip the card around. Listings would then include the "old" code (in > reverse -- right to left). Similarly for paper tape, if you punched all 5 holes it signified a comment (or null statement as our docs called it). -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PyTutor at DancesWithMice.info Mon Jul 12 21:21:29 2021 From: PyTutor at DancesWithMice.info (dn) Date: Tue, 13 Jul 2021 13:21:29 +1200 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: <48459ad3-f4f7-c326-1ed6-f8767af01931@Damon-Family.org> References: <2b9a9faa-12a1-8b68-7af3-4baf4fad94af@wichmann.us> <48459ad3-f4f7-c326-1ed6-f8767af01931@Damon-Family.org> Message-ID: <09eaa81b-4efd-5f79-a7d6-7407fcbc55b5@DancesWithMice.info> On 13/07/2021 00.44, Richard Damon wrote: > On 7/11/21 10:52 PM, dn via Tutor wrote: >> >> 0 bring back punch-cards! >> >> 1 I can't remember why "79", instead of the 80-column preceding >> 'standard' (which also followed punched cards, even when >> terminals/screens were capable of more than 24x80 - but back-then some >> languages mandated the use of the first columns and perhaps also the >> last). Perhaps the 80th column would be where the "\" line-continuation >> character fits? Thanks @Richard and @Dennis for the trips down 'memory lane'. Yes, there were several different applications for the last (and first) columns on punched-cards. What I really meant was: <<>>, as opposed to the full, 'round', 80 (or some other number of columns) - hence the "\" comment... -- Regards, =dn From adityakarthik9 at gmail.com Tue Jul 13 00:26:39 2021 From: adityakarthik9 at gmail.com (adityakarthik9 at gmail.com) Date: Tue, 13 Jul 2021 09:56:39 +0530 Subject: [Tutor] ImportError: attempted relative import with no known parent package Message-ID: <2B51AA95-76DC-45AD-AC1D-D8A16A298278@hxcore.ol> Hi. I am trying to execute a dump file using a .py file. I am using ubuntu for my projects. The command I give in the ubuntu terminal is : python3 main.py file.dump When I type this command, I get an error stating that: File "main.py", line 3, in from .Histogram import Histogram ImportError: attempted relative import with no known parent package I am trying to plot histogram for my dump file. So the output of this command should give me the frequency with which an assembly instruction is repeated in the dump file. I am using the program from the following link to get the results. https://github.com/riscv-newop/riscv-newop/tree/master/rvnewop From my understanding, the error is occurring due to some directory mis-match but I made sure that all the files are in the same directory. I am not sure where I am going wrong. I request your help. Kindly revert back at your nearest convenience. Thank You, Yours Sincerely, Aditya Karthik Sent from [1]Mail for Windows 10 References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 From __peter__ at web.de Tue Jul 13 04:43:13 2021 From: __peter__ at web.de (Peter Otten) Date: Tue, 13 Jul 2021 10:43:13 +0200 Subject: [Tutor] ImportError: attempted relative import with no known parent package In-Reply-To: <2B51AA95-76DC-45AD-AC1D-D8A16A298278@hxcore.ol> References: <2B51AA95-76DC-45AD-AC1D-D8A16A298278@hxcore.ol> Message-ID: On 13/07/2021 06:26, adityakarthik9 at gmail.com wrote: > Hi. > > > > I am trying to execute a dump file using a .py file. > > > > I am using ubuntu for my projects. > > > > The command I give in the ubuntu terminal is : > > > > python3 main.py file.dump > > > > When I type this command, I get an error stating that: > > > > File "main.py", line 3, in > > from .Histogram import Histogram > > > > ImportError: attempted relative import with no known parent package > > > > I am trying to plot histogram for my dump file. So the output of this > command should give me the frequency with which an assembly instruction is > repeated in the dump file. > > I am using the program from the following link to get the results. > > https://github.com/riscv-newop/riscv-newop/tree/master/rvnewop > > From my understanding, the error is occurring due to some directory > mis-match but I made sure that all the files are in the same directory. > > I am not sure where I am going wrong. If you installed the program with pip as suggested here https://github.com/riscv-newop/riscv-newop python3 -m rvnewop path/to/file.dump should work. rvnewop is a package, and you should never cd into a package directory or include it into Python's search path in another way. The parent directory of the package directory should of course be in the search path, but you can trust pip to take care of that. From chris.barnes.clb at gmail.com Tue Jul 13 11:17:45 2021 From: chris.barnes.clb at gmail.com (Chris Barnes) Date: Tue, 13 Jul 2021 10:17:45 -0500 Subject: [Tutor] Invalid syntax (pyflakes E) Message-ID: Below is my code. I'm a student in Programming for Engineers and I'm having trouble with some syntax errors. I've highlighted in yellow where I'm having the issue. Any help would be appreciated. import pint; def GetInputFromArray(promt, choices): print(prompt) for i in range(0, len(choices)): print("Enter {} for {}".format(i, choices[i])) selection = int(input("Enter Value: ")) return choices[selection] a = GetInput("Select Pressure", ["atm", "in_Hg", "psi"]) print("You selected: {}".format(a)) v = 1.0 from pint import UnitRegsitry ureg = UnitRegistry() v = v * ureg.parse_expression(a) print(" Here is the value with units {}".format(v)) print(v.to(ureg.atm)) print(v.dimensionality) ureg = UnitRegistry(autoconvert_offset_to_baseunit = True) def getPressureInAtm(): print("You will input the pressure value and then select the units of the input") p = float(input("Enter Pressure:")) choices = ["atm"", "in_Hg", "psi"] print("Youl will now select the units of the value entered") for i in range(0, len(choices)): print("Enter {} for {}".format(i, choices[i])) units = int(input("Enter Value: ")) p1 = p * u.parse_expression(choices[units]) print(getPressureinAtm()) Thanks, Chris Barnes From alan.gauld at yahoo.co.uk Tue Jul 13 13:26:36 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 13 Jul 2021 18:26:36 +0100 Subject: [Tutor] Invalid syntax (pyflakes E) In-Reply-To: References: Message-ID: On 13/07/2021 16:17, Chris Barnes wrote: > Below is my code. I'm a student in Programming for Engineers and I'm > having trouble with some syntax errors. I've highlighted in yellow where > I'm having the issue. This is a text based list and rich text files often get mangled. So many will not see the yellow. Its far better to simply copy the full error text into your mail. Python error messages contain a lot of useful information so it helps if we can see them in their entirety. > import pint; > > def GetInputFromArray(promt, choices): > print(prompt) > for i in range(0, len(choices)): > print("Enter {} for {}".format(i, choices[i])) > selection = int(input("Enter Value: ")) > return choices[selection] > > a = GetInput("Select Pressure", ["atm", "in_Hg", "psi"]) > print("You selected: {}".format(a)) > > v = 1.0 > > from pint import UnitRegsitry > ureg = UnitRegistry() > v = v * ureg.parse_expression(a) I don;t know your lkibrary so will have to assume that multiplying a float by whatever parse_expression returns yields a valid object. But so far that is the most suspicious line... > print(" Here is the value with units {}".format(v)) > > print(v.to(ureg.atm)) > print(v.dimensionality) > > ureg = UnitRegistry(autoconvert_offset_to_baseunit = True) > > def getPressureInAtm(): > print("You will input the pressure value and then select the units of > the input") > p = float(input("Enter Pressure:")) > choices = ["atm"", "in_Hg", "psi"] > print("Youl will now select the units of the value entered") > for i in range(0, len(choices)): > print("Enter {} for {}".format(i, choices[i])) > units = int(input("Enter Value: ")) Why not use your previous function? units = GetInputFromArray(prompt,choices) The main reason we create functions is to use them multiple times. > p1 = p * u.parse_expression(choices[units]) Where is u defined? Should it be ureg? > > print(getPressureinAtm()) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PyTutor at DancesWithMice.info Tue Jul 13 14:33:21 2021 From: PyTutor at DancesWithMice.info (dn) Date: Wed, 14 Jul 2021 06:33:21 +1200 Subject: [Tutor] Invalid syntax (pyflakes E) In-Reply-To: References: Message-ID: <435baaea-f35f-0381-1960-33263bc8d7dd@DancesWithMice.info> On 14/07/2021 03.17, Chris Barnes wrote: > Below is my code. I'm a student in Programming for Engineers and I'm > having trouble with some syntax errors. I've highlighted in yellow where > I'm having the issue. Any help would be appreciated. Adding to @Alan's response. Lack of visibility (for us) implies too much guessing... > import pint; No need for terminating semi-colon > def GetInputFromArray(promt, choices): > print(prompt) promt or prompt? > for i in range(0, len(choices)): > print("Enter {} for {}".format(i, choices[i])) Are you new to Python after learning some other language? Beyond language syntax, Python has its own idiom(s) (which require 'learning' in order to release their full power/benefit) - am also throwing-in f-strings, because they (seem to me) easier to read: for counter, choice in choices: print( f"Enter { counter } for { choice }" ) > selection = int(input("Enter Value: ")) The (non-syntax) problems with this are that you have assumed the user will enter the numeric value, rather than the "choice"; and that if the data-received is anything other than an integer, 'crash'! > return choices[selection] > > a = GetInput("Select Pressure", ["atm", "in_Hg", "psi"]) > print("You selected: {}".format(a)) GetInputFromArray or GetInput? Also, pythonista generally prefer "get_input" as a format for function names. > v = 1.0 > > from pint import UnitRegsitry > ureg = UnitRegistry() UnitRegsitry or UnitRegistry? Again, pythonista prefer all import-s to be together at the top of the code. There are times when it makes sense to import the same module, or components thereof, in multiple ways. However, the code 'here' does not actually use "pint" after importing it (first import statement). If that observation is correct, why bother? The two lines can be condensed: from pint import UnitRegistry as ureg but... > v = v * ureg.parse_expression(a) > > print(" Here is the value with units {}".format(v)) > > print(v.to(ureg.atm)) > > print(v.dimensionality) > > ureg = UnitRegistry(autoconvert_offset_to_baseunit = True) What's the intention here? UnitRegistry was imported and then re-labelled and employed as ureg. Why now (quite legally) refer to it as UnitRegistry? Is a lack of consistency, part of the problem? Worse: the name "ureg" is being re-used, and for something quite different. Once it has been calculated what is its purpose? > def getPressureInAtm(): > print("You will input the pressure value and then select the units of > the input") > p = float(input("Enter Pressure:")) > choices = ["atm"", "in_Hg", "psi"] > print("Youl will now select the units of the value entered") > for i in range(0, len(choices)): > print("Enter {} for {}".format(i, choices[i])) > units = int(input("Enter Value: ")) > p1 = p * u.parse_expression(choices[units]) > > print(getPressureinAtm()) This seems to be largely a repeat of code 'higher up'. What is the intention? Proof-reading after typing will help. Read each err.msg carefully. Usually the answer is 'right there' (although such is far too easy for an experienced coder to say!). Be aware that sometimes the line-number mentioned is where the error became obvious, but the actual error may have occurred 'higher-up', eg unbalanced quotes, brackets, etc. -- Regards, =dn From mats at wichmann.us Tue Jul 13 15:26:50 2021 From: mats at wichmann.us (Mats Wichmann) Date: Tue, 13 Jul 2021 13:26:50 -0600 Subject: [Tutor] Invalid syntax (pyflakes E) In-Reply-To: <80preghhhbh9v4mpd2ft5652bg2qln13f5@4ax.com> References: <435baaea-f35f-0381-1960-33263bc8d7dd@DancesWithMice.info> <80preghhhbh9v4mpd2ft5652bg2qln13f5@4ax.com> Message-ID: <78fa74d0-e3a5-b4b3-5672-b0cb3cc8a19c@wichmann.us> On 7/13/21 1:07 PM, Dennis Lee Bieber wrote: > On Wed, 14 Jul 2021 06:33:21 +1200, dn via Tutor > declaimed the following: > > >>> from pint import UnitRegsitry >>> ureg = UnitRegistry() >> >> UnitRegsitry or UnitRegistry? >> > Hmmm, I missed that typo... using a somewhat intelligent editor (IDE or almost-IDE) can eliminate these types of errors - accept the offered autocomplete and you won't have mismatched param/usage, function names, etc. I'm just sayin' - there's enough to learn, why not look into something that helps with the "niggly stuff"? From cs at cskk.id.au Tue Jul 13 19:38:34 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Wed, 14 Jul 2021 09:38:34 +1000 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: <9mlregp12f2hkdq7j8mg9ao2fqcc7s6vu7@4ax.com> References: <9mlregp12f2hkdq7j8mg9ao2fqcc7s6vu7@4ax.com> Message-ID: On 13Jul2021 15:01, Dennis Lee Bieber wrote: >On Mon, 12 Jul 2021 15:53:48 -0400, Dennis Lee Bieber > declaimed the following: > For the OP... DateTime handling is going to be tricky between > SQLite3 >and Python. > > SQLite3 documentation states that something declared "date" or >"datetime" as data type is mapped to SQLite3 NUMERIC >https://www.sqlite.org/datatype3.html section 2.2 and 3.1.1. > > However... CURRENT_TIMESTAMP returns a formatted ISO date/time string, >which SQLite3 will happily store in a nominally numeric field (SQLite3 will >store anything in any field, possibly doing some conversions... string >"123" in a numeric/integer field becomes actual integer 123). > > The only way to get a UNIX type value is to use > strftime('%s', date/time value) >and that is not allowed in DEFAULT or GENERATED AS clause in a table >definition (something about dynamic clauses ) Personally I like to use UNIX timestamps. They're integers (or floats for higher precision) and are unambiguous. I can always render a date or date time string for them for presentation purposes. Cheers, Cameron Simpson From adityakarthik9 at gmail.com Wed Jul 14 00:06:23 2021 From: adityakarthik9 at gmail.com (adityakarthik9 at gmail.com) Date: Wed, 14 Jul 2021 09:36:23 +0530 Subject: [Tutor] ImportError: attempted relative import with no known parent package Message-ID: > Hi. > > > > I am trying to execute a dump file using a .py file. > > > > I am using ubuntu for my projects. > > > > The command I give in the ubuntu terminal is : > > > > python3 main.py file.dump > I am pointing to the path: ".local/lib/python3/packages/notebook/riscv-newop" Inside "riscv-newop" directory, there is a file "rvnewop" which consists of main.py which I am trying to execute on my dump file which is also inside the same folder rvnewop. > > > When I type this command, I get an error stating that: > > > > File "main.py", line 3, in > > from .Histogram import Histogram > > > > ImportError: attempted relative import with no known parent package > > > > I am trying to plot histogram for my dump file. So the output of this > command should give me the frequency with which an assembly instruction is > repeated in the dump file. > > I am using the program from the following link to get the results. > > https://github.com/riscv-newop/riscv-newop/tree/master/rvnewop > > From my understanding, the error is occurring due to some directory > mis-match but I made sure that all the files are in the same directory. If I use the below command: python3 -m rvnewop path/to/file.dump I think it should work but the rvnewop folder consists of many .py files. I do not want the output for all of the files. I am only trying to execute the main.py file which imports the Histogram.py file both of which are inside the rvnewop folder. My only doubt is : What directory should I be pointing to when I execute the command: python3 -m rvnewop path/to/file.dump And how to get the results only with main.py for my dump file without getting the error which is mentioned above . Thank You > Sent from [1]Mail for Windows 10 References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 From adityakarthik9 at gmail.com Wed Jul 14 03:20:56 2021 From: adityakarthik9 at gmail.com (adityakarthik9 at gmail.com) Date: Wed, 14 Jul 2021 12:50:56 +0530 Subject: [Tutor] Error while importing a .py file from another .py file. Message-ID: <9CB04FD3-9E99-41D3-97CF-EE39380E6CBD@hxcore.ol> Hi. I am trying to execute a dump file using a .py file. I am using ubuntu for my projects. The command I give in the ubuntu terminal is : python3 main.py file.dump I am pointing to the path: ".local/lib/python3/packages/notebook/riscv-newop" Inside "riscv-newop" directory, there is a file "rvnewop" which consists of main.py which I am trying to execute on my dump file which is also inside the same folder rvnewop. When I type this command, I get an error stating that: File "main.py", line 3, in from .Histogram import Histogram ImportError: attempted relative import with no known parent package I am trying to plot histogram for my dump file. So the output of this command should give me the frequency with which an assembly instruction is repeated in the dump file. I am using the program from the following link to get the results. https://github.com/riscv-newop/riscv-newop/tree/master/rvnewop From my understanding, the error is occurring due to some directory mis-match but I made sure that all the files are in the same directory. If I use the below command: python3 -m rvnewop path/to/file.dump I think it should work but the rvnewop folder consists of many .py files. I do not want the output for all of the files. I am only trying to execute the main.py file which imports the Histogram.py file both of which are inside the rvnewop folder. My only doubt is : What directory should I be pointing to when I execute the command: python3 -m rvnewop path/to/file.dump And how to get the results only with main.py for my dump file without getting the error which is mentioned above . Thank You > Sent from [1]Mail for Windows 10 References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 From adityakarthik9 at gmail.com Wed Jul 14 09:11:59 2021 From: adityakarthik9 at gmail.com (Aditya Karthik) Date: Wed, 14 Jul 2021 18:41:59 +0530 Subject: [Tutor] TypeError: argument of type 'NoneType' is not iterable Message-ID: I want to run the contents of a folder named rvnewop for my dump file file.dump So i type this command in the terminal: python3 -m rvnewop path/to/file.dump But then I get an error : Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/aditya/.local/lib/python3.8/site-packages/riscv-newop/rvnewop/__main__.py", line 3, in main() File "/home/aditya/.local/lib/python3.8/site-packages/riscv-newop/rvnewop/main.py", line 22, in main program = Histogram.parse(args.filename, args.isa) File "/home/aditya/.local/lib/python3.8/site-packages/riscv-newop/rvnewop/Histogram.py", line 10, in parse program = Program(name=filename, isa=isa) File "/home/aditya/.local/lib/python3.8/site-packages/riscv-newop/rvnewop/Program.py", line 13, in __init__ self.rv = RV32(isa=isa) File "/home/aditya/.local/lib/python3.8/site-packages/riscv-newop/rvnewop/RV32.py", line 34, in __init__ if "32I" in isa: TypeError: argument of type 'NoneType' is not iterable The rvnewop file that i want to use is predefined and has been developed by a developer and it should work fine for my dump file. I don't think there should be any problems with the code as it is predefined and proven to work fine. I think I am committing an error somewhere while calling the files/directories in the command line when i try to execute using my command, i point to the path: ./.local/lib/python3.8/site-packages/riscv-newop whereas the files i am trying to access are from the path:/.local/lib/python3.8/site-packages/riscv-newop/rvnewop But, i don't think you are supposed to cd into the package directory or include it in the python's search path in another way. Not sure if that is the error I am committing. Please Help! For the codes, kindly refer to the link, https://github.com/riscv-newop/riscv-newop/tree/master/rvnewop Is it possible to run only a particular .py file for example main.py from the rvnewop folder instead of running the entire folder ? I am kinda lost here due of these issues. Can anyone please help me ? From alan.gauld at yahoo.co.uk Wed Jul 14 09:51:55 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 14 Jul 2021 14:51:55 +0100 Subject: [Tutor] TypeError: argument of type 'NoneType' is not iterable In-Reply-To: References: Message-ID: On 14/07/2021 14:11, Aditya Karthik wrote: > I want to run the contents of a folder named rvnewop for my dump file > file.dump > > So i type this command in the terminal: > > python3 -m rvnewop path/to/file.dump > > But then I get an error : caveat: I haven't looked at the code, just the error message... > program = Histogram.parse(args.filename, args.isa) Here it refers to args.isa > File > "/home/aditya/.local/lib/python3.8/site-packages/riscv-newop/rvnewop/Histogram.py", > line 10, in parse > program = Program(name=filename, isa=isa) This passes that value on... > self.rv = RV32(isa=isa) and on... > line 34, in __init__ > if "32I" in isa: > TypeError: argument of type 'NoneType' is not iterable And then it gets tested and is None. So it looks like the problem is the args.isa. Where does that come from? args suggests the command line argument... > when i try to execute using my command, i point to the path: > ./.local/lib/python3.8/site-packages/riscv-newop > > whereas the files i am trying to access are from the > path:/.local/lib/python3.8/site-packages/riscv-newop/rvnewop > > But, i don't think you are supposed to cd into the package directory or > include it in the python's search path in another way. If it has been set up correctly then the path should be OK. > Is it possible to run only a particular .py file for example main.py from > the rvnewop folder instead of running the entire folder ? It is, you just have to specify the full path, but that doesn't look like how this package should be used. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From __peter__ at web.de Wed Jul 14 09:59:45 2021 From: __peter__ at web.de (Peter Otten) Date: Wed, 14 Jul 2021 15:59:45 +0200 Subject: [Tutor] TypeError: argument of type 'NoneType' is not iterable In-Reply-To: References: Message-ID: On 14/07/2021 15:11, Aditya Karthik wrote: > I want to run the contents of a folder named rvnewop for my dump file > file.dump > > So i type this command in the terminal: > > python3 -m rvnewop path/to/file.dump > > But then I get an error : It looks like you need to provide a value for the --isa option, e. g. python3 -m rvnewop --isa I32 path/to/file.dump I'd say that this is a bug in the user interface -- the developer should make the --isa option required or provide a default. PS: I see you have tried to contact me via private mail. Please don't; if you send your messages to the newsgroup/mailing list others have a chance to read them and you are more likely to get a useful answer. From cs at cskk.id.au Thu Jul 15 20:12:43 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 16 Jul 2021 10:12:43 +1000 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: <09eaa81b-4efd-5f79-a7d6-7407fcbc55b5@DancesWithMice.info> References: <09eaa81b-4efd-5f79-a7d6-7407fcbc55b5@DancesWithMice.info> Message-ID: On 13Jul2021 13:21, DL Neil wrote: >On 13/07/2021 00.44, Richard Damon wrote: >> On 7/11/21 10:52 PM, dn via Tutor wrote: >>> 1 I can't remember why "79", instead of the 80-column preceding >>> 'standard' (which also followed punched cards, even when >>> terminals/screens were capable of more than 24x80 - but back-then some >>> languages mandated the use of the first columns and perhaps also the >>> last). Perhaps the 80th column would be where the "\" line-continuation >>> character fits? > >Thanks @Richard and @Dennis for the trips down 'memory lane'. Yes, there >were several different applications for the last (and first) columns on >punched-cards. > >What I really meant was: <<on the* "79">>>, as opposed to the full, 'round', 80 (or some other >number of columns) - hence the "\" comment... Another consideration is that various physical terminals had differing behaviour of the cursor at the very end of the line. (And many terminals were 80 characters.) Anyway, if you wrote a character to column 80 the cursor might be on that line or the next line depending on the terminal's behaviour. Particularly, if you did that on the bottom line of the screen the terminal might scroll. Or not. This makes things hard for full screen programmes such as visual editors etc. The curses library etc avoided the rightmost column for this kind of reason IIRC. Anyway, the upshot of this is that on an n-column display, it is prudent to use n-1 characters of text. Cheers, Cameron Simpson From cs at cskk.id.au Thu Jul 15 20:05:09 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 16 Jul 2021 10:05:09 +1000 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: <6b3uegp3ihia5ne6n1tlkka3laravdided@4ax.com> References: <6b3uegp3ihia5ne6n1tlkka3laravdided@4ax.com> Message-ID: On 14Jul2021 12:28, Dennis Lee Bieber wrote: >On Wed, 14 Jul 2021 09:38:34 +1000, Cameron Simpson >declaimed the following: >>Personally I like to use UNIX timestamps. They're integers (or floats >>for higher precision) and are unambiguous. I can always render a date or >>date time string for them for presentation purposes. >> > That's what I was expecting, especially as SQLite associates "datetime" >as a NUMERIC type... > > But to get a UNIX timestamp in SQLite requires something like > > strftime("%s", 'now', 'unixepoch') > >(which, technically, is still returning a string, but as "seconds" is all >numeric, should convert it to integer storage). > > That can't be specified in "default" or "generated as" clauses, meaning >having to create triggers to set a value. Can it not? From https://sqlite.org/lang_createtable.html under "The DEFAULT clause": If the default value of a column is an expression in parentheses, then the expression is evaluated once for each row inserted and the results used in the new row. You may be better off than you thought. Cheers, Cameron Simpson From cs at cskk.id.au Thu Jul 15 22:37:14 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 16 Jul 2021 12:37:14 +1000 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: <3mo1fgd0m5gvpe3npb7oqv303mn9jeed30@4ax.com> References: <3mo1fgd0m5gvpe3npb7oqv303mn9jeed30@4ax.com> Message-ID: On 15Jul2021 21:49, Dennis Lee Bieber wrote: >On Fri, 16 Jul 2021 10:05:09 +1000, Cameron Simpson >declaimed the following: >>Can it not? From https://sqlite.org/lang_createtable.html under "The >>DEFAULT clause": >> >> If the default value of a column is an expression in parentheses, >> then the expression is evaluated once for each row inserted and the >> results used in the new row. >> >>You may be better off than you thought. > > When I tried I got some error message about not being able to use a >"dynamic" expression. > > Didn't try further. Could have just been a syntax error on my part but >there is this: https://www.sqlite.org/lang_createtable.html >""" >3.2. The DEFAULT clause > >... An explicit DEFAULT clause may specify that the default value is NULL, >a string constant, a blob constant, a signed-number, or any constant >expression enclosed in parentheses. ... For the purposes of the DEFAULT >clause, an expression is considered constant if it contains no sub-queries, >column or table references, bound parameters, or string literals enclosed >in double-quotes instead of single-quotes. >""" > > The message I'd received implies the clause I supplied was not >considered "constant". That seemed in conflict with the line I quoted, so I dug a bit more. This works: create table z ( col1 INTEGER DEFAULT (strftime('%s', 'now', 'unixepoch'))); This is not constant: sqlite> create table z ( col1 INTEGER DEFAULT (strftime("%s", 'now', 'unixepoch')+"")); Error: default value of column [col1] is not constant Earlier in the DEFAULT specification it says: For the purposes of the DEFAULT clause, an expression is considered constant if it contains no sub-queries, column or table references, bound parameters, or string literals enclosed in double-quotes instead of single-quotes. This implies that double quotes have a different meaning. I think "constant' means "not requiring a subselect of some kind". Elsewhere in the syntax documentation was: A string constant is formed by enclosing the string in single quotes ('). Double quotes enclose an identifier such as a column name, so the double quoted expression is a column access, not a constant. Cheers, Cameron Simpson From mats at wichmann.us Fri Jul 16 17:43:11 2021 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 16 Jul 2021 15:43:11 -0600 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <09eaa81b-4efd-5f79-a7d6-7407fcbc55b5@DancesWithMice.info> Message-ID: On 7/15/21 6:12 PM, Cameron Simpson wrote: > Another consideration is that various physical terminals had differing > behaviour of the cursor at the very end of the line. (And many terminals > were 80 characters.) Anyway, if you wrote a character to column 80 the > cursor might be on that line or the next line depending on the > terminal's behaviour. Particularly, if you did that on the bottom line > of the screen the terminal might scroll. Or not. This makes things hard > for full screen programmes such as visual editors etc. > > The curses library etc avoided the rightmost column for this kind of > reason IIRC. sheesh, some threads just won't come to a conclusion and die :) this is all a VERY long time ago. the development of a screen-addressable editor (vi, which started life as an extension mode to ex) at UC Berkeley became possible basically because of the ADM 3A (yeah, it's got its own wikipedia page: https://en.wikipedia.org/wiki/ADM-3A)... Slightly more advanced terminals like the VT-100 and an HP model (26something, I don't remember the code number) and some Beehive model turned up and suggested generalizing the cursor-addressing functions, which is where curses came in. I was a student there at the time. The 3A was funky, as I recall the Beehive was even more funky... and there's really no reason to hang on to quirks from those early days, since all "terminal" behavior is emulated now.... From alan.gauld at yahoo.co.uk Fri Jul 16 17:55:55 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 16 Jul 2021 22:55:55 +0100 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <09eaa81b-4efd-5f79-a7d6-7407fcbc55b5@DancesWithMice.info> Message-ID: On 16/07/2021 22:43, Mats Wichmann wrote: > On 7/15/21 6:12 PM, Cameron Simpson wrote: >> The curses library etc avoided the rightmost column for this kind of >> reason IIRC. > > sheesh, some threads just won't come to a conclusion and die :) > > this is all a VERY long time ago. the development of a > screen-addressable editor (vi, which started life as an extension mode > to ex) And contrary to popular opinion vi does not really use curses, rather it uses termcap (and now terminfo), the raw terminal control codes. There is, so far as I can tell, no actual curses specific code in vi. It does include curses but never enters curses mode, it just uses the termcap/info control definitions. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mats at wichmann.us Fri Jul 16 18:01:12 2021 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 16 Jul 2021 16:01:12 -0600 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: References: <09eaa81b-4efd-5f79-a7d6-7407fcbc55b5@DancesWithMice.info> Message-ID: <0440f35c-027e-44ee-9faa-fb979057956d@wichmann.us> On 7/16/21 3:55 PM, Alan Gauld via Tutor wrote: > On 16/07/2021 22:43, Mats Wichmann wrote: >> this is all a VERY long time ago. the development of a >> screen-addressable editor (vi, which started life as an extension mode >> to ex) > > And contrary to popular opinion vi does not really use curses, > rather it uses termcap (and now terminfo), the raw terminal > control codes. There is, so far as I can tell, no actual > curses specific code in vi. It does include curses but never > enters curses mode, it just uses the termcap/info control > definitions. I don't know about vim, which is a rewrite I've never poked at the code for, but for the original vi, definitely true - curses grew out of the learnings from building vi, but it wasn't pulled back into vi. From cs at cskk.id.au Fri Jul 16 19:00:48 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Sat, 17 Jul 2021 09:00:48 +1000 Subject: [Tutor] Correct style of line break when chaining methods In-Reply-To: <0440f35c-027e-44ee-9faa-fb979057956d@wichmann.us> References: <0440f35c-027e-44ee-9faa-fb979057956d@wichmann.us> Message-ID: On 16Jul2021 16:01, Mats Wichmann wrote: >On 7/16/21 3:55 PM, Alan Gauld via Tutor wrote: >>And contrary to popular opinion vi does not really use curses, >>rather it uses termcap (and now terminfo), the raw terminal >>control codes. There is, so far as I can tell, no actual >>curses specific code in vi. It does include curses but never >>enters curses mode, it just uses the termcap/info control >>definitions. > >I don't know about vim, which is a rewrite I've never poked at the >code for, but for the original vi, definitely true - curses grew out of >the learnings from building vi, but it wasn't pulled back into vi. Aye. Back in the day I sometimes dialed in to the university on a TRS-80 and a 300baud modem. The TRS-80's cursor codes were quite limited. Vi had a distinct mode for terminals with no "delete line" function - if you deleted a line of text it would just leave a marker there (a line with a single "@" at the left) rather than close up the display. You could do this with curses I suppose, but you wouldn't. At some point I wrote some assembler to provide a few capabilities line delete-line for the TRS-80 and vi behaved a lot more like a modern one, but on a slow serial line with no delete/scroll-up the placeholder approach was very effective. I've still got the terminfo entries for each :-) -rw-r--r-- 1 cameron cameron 998 29 May 1999 /Users/cameron/rc/term/info/t/trs80m1 -rw-r--r-- 1 cameron cameron 1119 29 May 1999 /Users/cameron/rc/term/info/t/trs80m1cs Cheers, Cameron Simpson From shashank.edu at gmail.com Sun Jul 18 01:44:07 2021 From: shashank.edu at gmail.com (Shashank Jain) Date: Sun, 18 Jul 2021 11:14:07 +0530 Subject: [Tutor] zoneinfo not working properly Message-ID: I was using python 3.9.3 then zoneinfo library was working fine without any issues. but now after installing 3.9.6 there are problems. Ex. Using below code there no outputs. and further there are issues while working with timezones with zoninfo where I tried same things with pytz that was giving correct output. for i in zoneinfo.available_timezones(): print(i) From mats at wichmann.us Sun Jul 18 16:47:56 2021 From: mats at wichmann.us (Mats Wichmann) Date: Sun, 18 Jul 2021 14:47:56 -0600 Subject: [Tutor] zoneinfo not working properly In-Reply-To: References: Message-ID: On 7/17/21 11:44 PM, Shashank Jain wrote: > I was using python 3.9.3 then zoneinfo library was working fine without any > issues. but now after installing 3.9.6 there are problems. Ex. > > Using below code there no outputs. and further there are issues while > working with timezones with zoninfo where I tried same things with pytz > that was giving correct output. > > for i in zoneinfo.available_timezones(): > print(i) I'm sure this is no comfort to you, but I tried this with 3.9.6 and there were no problems. It suggests something may have gone haywire with your installation, rather than that zoneinfo itself is broken in 3.9.6 (which should not happen anyway - it's got a decent test suite which should catch things) From robertvstepp at gmail.com Sun Jul 18 20:22:15 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Sun, 18 Jul 2021 19:22:15 -0500 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: <990A5EAF-0613-4687-99C0-94044D99BA87@wichmann.us> References: <5n8udg1hcimme45un3chg850msa4ficqi0@4ax.com> <990A5EAF-0613-4687-99C0-94044D99BA87@wichmann.us> Message-ID: On Fri, Jul 2, 2021 at 12:55 PM Mats Wichmann wrote: > > On July 2, 2021 10:59:20 AM MDT, Mats Wichmann wrote: > >On July 2, 2021 10:06:02 AM MDT, Dennis Lee Bieber > > wrote: > >>On Thu, 1 Jul 2021 19:02:14 -0500, boB Stepp > >>declaimed the following: > >> > >>>One thing that is mildly troubling is that if I wish to write > >>>cross-platform applications then I must go to the bother of detecting > >>>the user's OS and stashing "data" appropriately for that OS. But > >that > >>>should not normally be too troublesome. > > > >Take a look at the applies package on pypi for some help with this. > > Appdirs. Sorry, autocorrect got it. My son ran across platformdirs, a fork of appdirs, at https://github.com/platformdirs/platformdirs boB Stepp From phillor9 at gmail.com Sun Jul 18 22:54:07 2021 From: phillor9 at gmail.com (Phil) Date: Mon, 19 Jul 2021 12:54:07 +1000 Subject: [Tutor] object does not support item assignment Message-ID: I'm in the process of converting a C++ programme that I wrote some time ago to Python but I've run into a problem at an early stage. The following is a snippet of the code. One Led works but how do I work with 8 Leds? led1 =, led2 =, etc and then add them to a list? That might be OK for 8 Leds but I think there must be a better way. TypeError: 'Led' object does not support item assignment class Led: ??? def __init__(self, pos): ??????? self.pos = pos ??????? self.led = Led((50, 50)) ??????? for i in range(8): ??????????? self.led[i] = Led((50, 50)) -- Regards, Phil From mats at wichmann.us Sun Jul 18 23:11:56 2021 From: mats at wichmann.us (Mats Wichmann) Date: Sun, 18 Jul 2021 21:11:56 -0600 Subject: [Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved? In-Reply-To: References: <5n8udg1hcimme45un3chg850msa4ficqi0@4ax.com> <990A5EAF-0613-4687-99C0-94044D99BA87@wichmann.us> Message-ID: <86efbdc9-88d3-04b5-3128-c59a1df0670a@wichmann.us> On 7/18/21 6:22 PM, boB Stepp wrote: > On Fri, Jul 2, 2021 at 12:55 PM Mats Wichmann wrote: >> Appdirs. Sorry, autocorrect got it. > > My son ran across platformdirs, a fork of appdirs, at > https://github.com/platformdirs/platformdirs Thanks for the note - for me personally, at least, that's useful - Black already shifted over to using the fork in the last days so it looks like this one will have some legs. From robertvstepp at gmail.com Sun Jul 18 23:19:53 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Sun, 18 Jul 2021 22:19:53 -0500 Subject: [Tutor] object does not support item assignment In-Reply-To: References: Message-ID: On Sun, Jul 18, 2021 at 9:55 PM Phil wrote: > One Led works but how do I work with 8 Leds? led1 =, led2 =, etc and > then add them to a list? That might be OK for 8 Leds but I think there > must be a better way. > > TypeError: 'Led' object does not support item assignment > > class Led: > def __init__(self, pos): > self.pos = pos > > self.led = Led((50, 50)) > > for i in range(8): > self.led[i] = Led((50, 50)) Are you sure you don't want something more like this: Python 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> class Led: ... def __init__(self, pos): ... self.pos = pos ... >>> leds = {} >>> for i in range(8): ... leds[i] = Led((50, 50)) ... >>> leds {0: <__main__.Led object at 0x0000024347DCCE80>, 1: <__main__.Led object at 0x0000024347D794C0>, 2: <__main__.Led object at 0x000002434826F6A0>, 3: <__main__.Led object at 0x000002434826F730>, 4: <__main__.Led object at 0x000002434826F400>, 5: <__main__.Led object at 0x000002434826F490>, 6: <__main__.Led object at 0x000002434826F4F0>, 7: <__main__.Led object at 0x000002434826F550>} This gives you 8 Led objects to work with that you could address like: leds[0].turn_on() leds[1].turn_off() ... If you had a couple of methods turn_on() and turn_off() in your Led class. Not really sure what you are aiming for, but this is my best guess. HTH! boB Stepp From PyTutor at DancesWithMice.info Sun Jul 18 23:24:12 2021 From: PyTutor at DancesWithMice.info (dn) Date: Mon, 19 Jul 2021 15:24:12 +1200 Subject: [Tutor] object does not support item assignment In-Reply-To: References: Message-ID: <1c1b4f15-c819-4a9e-ed4e-fe08c05b674f@DancesWithMice.info> On 19/07/2021 14.54, Phil wrote: > I'm in the process of converting a C++ programme that I wrote some time > ago to Python but I've run into a problem at an early stage. > > The following is a snippet of the code. > > One Led works but how do I work with 8 Leds? led1 =, led2 =, etc and > then add them to a list? That might be OK for 8 Leds but I think there > must be a better way. > > TypeError: 'Led' object does not support item assignment > > class Led: > ??? def __init__(self, pos): > ??????? self.pos = pos > > ??????? self.led = Led((50, 50)) > > ??????? for i in range(8): > ??????????? self.led[i] = Led((50, 50)) There seems to be two entities here: - Led: a single light-emitting diode - Festoon: a series (collection) of Led objects (which likely are not in an electrical "series"!) It is worth having an Led class, if you are going to be doing things with each LED, eg switching it on, switching it off, recording its state, etc. It appears as if you would also like to do something with all the LEDs/Led objects at once, eg turn them all on. Depending upon this latter point, you may want a separate Festoon class, or it may be easier to load (however many) individual Led objects into a list or dict. class Led: def __init__(self, pos): self.pos = pos - and - my_led = Led((50, 50)) - or - leds = list() for i in range(8): leds.append( Led((50, 50)) ) Enough to take the next step? -- Regards =dn -- Regards, =dn From phillor9 at gmail.com Mon Jul 19 00:17:36 2021 From: phillor9 at gmail.com (Phil) Date: Mon, 19 Jul 2021 14:17:36 +1000 Subject: [Tutor] object does not support item assignment In-Reply-To: <1c1b4f15-c819-4a9e-ed4e-fe08c05b674f@DancesWithMice.info> References: <1c1b4f15-c819-4a9e-ed4e-fe08c05b674f@DancesWithMice.info> Message-ID: <5bf2b4a6-743c-0f7a-2dda-6e73f667f3ee@gmail.com> On 19/7/21 1:24 pm, dn via Tutor wrote: > On 19/07/2021 14.54, Phil wrote: >> I'm in the process of converting a C++ programme that I wrote some time >> ago to Python but I've run into a problem at an early stage. Thank you boB and dn. I do have several methods including on and off and I'm currently trying to produce a group of LEDs so that I can scan the on LED across the group; a Cylon effect. I did think of a dictionary but the list idea is very close to what I had in mind and I can now see where I made my mistake. -- Regards, Phil From phillor9 at gmail.com Mon Jul 19 01:48:42 2021 From: phillor9 at gmail.com (Phil) Date: Mon, 19 Jul 2021 15:48:42 +1000 Subject: [Tutor] Set and get class members Message-ID: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> I've read through several tutorial on this but I cannot relate what I'm reading to the problem. Say I wanted to set all the LED's status to false, or that of a particular LED, do I need a set function or can I access the attribute directly? In either case, what I have done below is not correct. class Led: ??? def __init__(self, pos): ??????? self.pos = pos ??????? self.state ??????? def setState(self, s): ??????????? self.state = s ??????? self.leds = list() ??????? for i in range(8): ??????????? self.leds[i].setState(False) ??????????? self.leds.append(Led((50 + (i * 30), 50))) -- Regards, Phil From DomainAdmin at DancesWithMice.info Sun Jul 18 23:08:22 2021 From: DomainAdmin at DancesWithMice.info (David L Neil) Date: Mon, 19 Jul 2021 15:08:22 +1200 Subject: [Tutor] object does not support item assignment In-Reply-To: References: Message-ID: On 19/07/2021 14.54, Phil wrote: > I'm in the process of converting a C++ programme that I wrote some time > ago to Python but I've run into a problem at an early stage. > > The following is a snippet of the code. > > One Led works but how do I work with 8 Leds? led1 =, led2 =, etc and > then add them to a list? That might be OK for 8 Leds but I think there > must be a better way. > > TypeError: 'Led' object does not support item assignment > > class Led: > ??? def __init__(self, pos): > ??????? self.pos = pos > > ??????? self.led = Led((50, 50)) > > ??????? for i in range(8): > ??????????? self.led[i] = Led((50, 50)) There seems to be two entities here: - Led: a single light-emitting diode - Festoon: a series (collection) of Led objects (which likely are not in an electrical "series"!) It is worth having an Led class, if you are going to be doing things with each LED, eg switching it on, switching it off, recording its state, etc. It appears as if you would also like to do something with all the LEDs/Led objects at once, eg turn them all on. Depending upon this latter point, you may want a separate Festoon class, or it may be easier to load (however many) individual Led objects into a list or dict. class Led: def __init__(self, pos): self.pos = pos - and - my_led = Led((50, 50)) - or - leds = list() for i in range(8): leds.append( Led((50, 50)) ) Enough to take the next step? -- Regards =dn From sagepav at gmail.com Sun Jul 18 22:12:19 2021 From: sagepav at gmail.com (sage pavlovich) Date: Sun, 18 Jul 2021 19:12:19 -0700 Subject: [Tutor] For loops with lists Message-ID: <1775967E-73A8-4B12-B957-5B35696AC498@hxcore.ol> Hey I'm trying to do calculate Tzone finding a final result for a list of values from 0-8760. It is saying my syntax is wrong but not sure why. Any tips? The last section of code is where the errors are occuring import pandas as pd # import pandas data manipulation library import numpy as np # import numpy numerical library import matplotlib.pyplot as plt # import matplotlib plotting library import seaborn # import seaborn for advanced plotting data = pd.read_csv('weatherdata.csv') # load the weather file into a dataframe called 'data' print(data) timedata = data timedata['Datetime'] = pd.to_datetime(data['Date'] + ' ' + data['HH:MM'], format='%d/%m/%y %H:%M') # convert date and time timedata = timedata.set_index('Datetime') # set as index del timedata['Date'] # remove date column del timedata['HH:MM'] # remove time column timedata.index = timedata.index.map(lambda t: t.replace(year=2020)) # overwrite year print(timedata) T_air = timedata['Dry Bulb Temp'] RH = timedata['Relative Humidity'] x = list(range(0,12)) monthly_Temp = timedata['Dry Bulb Temp'].resample('M').mean() monthly_RH = timedata['Relative Humidity'].resample('M').mean() fig, ax1 = plt.subplots() ax2 = ax1.twinx() ax1.plot(x, monthly_Temp, 'g-') ax2.plot(x, monthly_RH, 'b-') ax1.set_xlabel('Months of the Year') ax1.set_ylabel('Temperature', color='g') ax2.set_ylabel('Relative Humidity', color='b') pltojb = plt.title('Temperature vs Humidity') plt.grid(True) plt.show() Q_North = timedata['Total Solar North'] # access total solar north energy column and make list Q_South = timedata['Total Solar South'] # access the total solar south energy column and make list Q_North.plot() plt.ylabel('Solar Gains, W') # label the y axis plt.title('Northern Solar Gains 2020') # add a title plt.grid() Wind_Direction = timedata['Wind Direction'] South_or_North_Wind = timedata[((Wind_Direction >= 135) & (Wind_Direction <= 225)) | ((Wind_Direction >= 315) | (Wind_Direction <= 45))] Windspeed=South_or_North_Wind['Wind Speed'] print (Windspeed) Windspeed.plot() for i in range(0,8760) print (i) P1 = (0.8*1.2)*((Windspeed[i]^2)/2) P2 = (-0.5*1.2)*((Windspeed[i]^2)/2) Vvent[i] = (sqrt(15)/5)*(sqrt(P1[i]-P2[i])) print (Vvent) Qvent[i] = (1.2*Vvent[i])*(T_air[i]-Tzone[i]) Tzone[i] = (Tair[i]+(Q_South[i]+Q_North[i])+109.333)/(1.2*Vvent[i]) From sagepav at gmail.com Sun Jul 18 22:12:19 2021 From: sagepav at gmail.com (sage pavlovich) Date: Sun, 18 Jul 2021 19:12:19 -0700 Subject: [Tutor] For loops with lists Message-ID: <1775967E-73A8-4B12-B957-5B35696AC498@hxcore.ol> Hey I'm trying to do calculate Tzone finding a final result for a list of values from 0-8760. It is saying my syntax is wrong but not sure why. Any tips? The last section of code is where the errors are occuring import pandas as pd # import pandas data manipulation library import numpy as np # import numpy numerical library import matplotlib.pyplot as plt # import matplotlib plotting library import seaborn # import seaborn for advanced plotting data = pd.read_csv('weatherdata.csv') # load the weather file into a dataframe called 'data' print(data) timedata = data timedata['Datetime'] = pd.to_datetime(data['Date'] + ' ' + data['HH:MM'], format='%d/%m/%y %H:%M') # convert date and time timedata = timedata.set_index('Datetime') # set as index del timedata['Date'] # remove date column del timedata['HH:MM'] # remove time column timedata.index = timedata.index.map(lambda t: t.replace(year=2020)) # overwrite year print(timedata) T_air = timedata['Dry Bulb Temp'] RH = timedata['Relative Humidity'] x = list(range(0,12)) monthly_Temp = timedata['Dry Bulb Temp'].resample('M').mean() monthly_RH = timedata['Relative Humidity'].resample('M').mean() fig, ax1 = plt.subplots() ax2 = ax1.twinx() ax1.plot(x, monthly_Temp, 'g-') ax2.plot(x, monthly_RH, 'b-') ax1.set_xlabel('Months of the Year') ax1.set_ylabel('Temperature', color='g') ax2.set_ylabel('Relative Humidity', color='b') pltojb = plt.title('Temperature vs Humidity') plt.grid(True) plt.show() Q_North = timedata['Total Solar North'] # access total solar north energy column and make list Q_South = timedata['Total Solar South'] # access the total solar south energy column and make list Q_North.plot() plt.ylabel('Solar Gains, W') # label the y axis plt.title('Northern Solar Gains 2020') # add a title plt.grid() Wind_Direction = timedata['Wind Direction'] South_or_North_Wind = timedata[((Wind_Direction >= 135) & (Wind_Direction <= 225)) | ((Wind_Direction >= 315) | (Wind_Direction <= 45))] Windspeed=South_or_North_Wind['Wind Speed'] print (Windspeed) Windspeed.plot() for i in range(0,8760) print (i) P1 = (0.8*1.2)*((Windspeed[i]^2)/2) P2 = (-0.5*1.2)*((Windspeed[i]^2)/2) Vvent[i] = (sqrt(15)/5)*(sqrt(P1[i]-P2[i])) print (Vvent) Qvent[i] = (1.2*Vvent[i])*(T_air[i]-Tzone[i]) Tzone[i] = (Tair[i]+(Q_South[i]+Q_North[i])+109.333)/(1.2*Vvent[i]) From alan.gauld at yahoo.co.uk Mon Jul 19 06:20:55 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 19 Jul 2021 11:20:55 +0100 Subject: [Tutor] For loops with lists In-Reply-To: <1775967E-73A8-4B12-B957-5B35696AC498@hxcore.ol> References: <1775967E-73A8-4B12-B957-5B35696AC498@hxcore.ol> Message-ID: On 19/07/2021 03:12, sage pavlovich wrote: > Hey I'm trying to do calculate Tzone finding a final result for a list of > values from 0-8760. It is saying my syntax is wrong but not sure why. Any > tips? The last section of code is where the errors are occuring rather than you try to describe what/where you think the problem is just include the full error message - it contains all the information we need(in conjunction with the code of course!) That way we don't need to guess... > timedata = data I'm not sure why you are doing that? Why not just all it timeata to start with? > timedata['Datetime'] = pd.to_datetime(data['Date'] + ' ' + data['HH:MM'],> format='%d/%m/%y %H:%M') # convert date and time And this is now very confusing since you are referring to timedata and data in the same line but they are the same object! Use one name or the other but not both. That's a recipe for madness when it comes to debugging. > timedata = timedata.set_index('Datetime') # set as index > del timedata['Date'] # remove date column > del timedata['HH:MM'] # remove time column And now you overwrite the timedata so it points to something else. > timedata.index = timedata.index.map(lambda t: t.replace(year=2020)) # > overwrite year Are you sure you mean to replace the index with the result of the map? That sounds dangerous to me! > print(timedata) > T_air = timedata['Dry Bulb Temp'] > RH = timedata['Relative Humidity'] > x = list(range(0,12)) > > monthly_Temp = timedata['Dry Bulb Temp'].resample('M').mean() > monthly_RH = timedata['Relative Humidity'].resample('M').mean() > > fig, ax1 = plt.subplots() > ax2 = ax1.twinx() > ax1.plot(x, monthly_Temp, 'g-') > ax2.plot(x, monthly_RH, 'b-') > ax1.set_xlabel('Months of the Year') > ax1.set_ylabel('Temperature', color='g') > ax2.set_ylabel('Relative Humidity', color='b') > pltojb = plt.title('Temperature vs Humidity') > plt.grid(True) > plt.show() > > > > Q_North = timedata['Total Solar North'] # access total solar north energy > column and make list > Q_South = timedata['Total Solar South'] # access the total solar south > energy column and make list > Q_North.plot() > > plt.ylabel('Solar Gains, W') # label the y axis > plt.title('Northern Solar Gains 2020') # add a title > plt.grid() > > Wind_Direction = timedata['Wind Direction'] > South_or_North_Wind = timedata[((Wind_Direction >= 135) & (Wind_Direction > <= 225)) | ((Wind_Direction >= 315) | (Wind_Direction <= 45))] > Windspeed=South_or_North_Wind['Wind Speed'] > print (Windspeed) > Windspeed.plot() > > for i in range(0,8760) No colon after the loop, this is likely your syntax error > print (i) > P1 = (0.8*1.2)*((Windspeed[i]^2)/2) > P2 = (-0.5*1.2)*((Windspeed[i]^2)/2) > Vvent[i] = (sqrt(15)/5)*(sqrt(P1[i]-P2[i])) > print (Vvent) > Qvent[i] = (1.2*Vvent[i])*(T_air[i]-Tzone[i]) > Tzone[i] = (Tair[i]+(Q_South[i]+Q_North[i])+109.333)/(1.2*Vvent[i]) And no indentation of the block either... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Mon Jul 19 06:35:24 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 19 Jul 2021 11:35:24 +0100 Subject: [Tutor] Set and get class members In-Reply-To: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> References: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> Message-ID: On 19/07/2021 06:48, Phil wrote: > Say I wanted to set all the LED's status to false, or that of a > particular LED, do I need a set function or can I access the attribute > directly? Your class is defining a single Led. You can (in Python) set the state directly ) or you can create a set() method (not a function, they are not the same!) or you can create a state property. The simplest route is to set it directly. If using a method I'd suggest a switch() method instead that toggles state on/off. You can still force a state by direct access when necessary. In either case, what I have done below is not correct. > > class Led: > ??? def __init__(self, pos): > > ??????? self.pos = pos > ??????? self.state > > ??????? def setState(self, s): > ??????????? self.state = s By defining the method inside init() it will disappear after the init() method completes. You need to define it at the class level as a separate method: class Led: def __init__(self...):.... def set(...):..... > ??????? self.leds = list() > ??????? for i in range(8): > ??????????? self.leds[i].setState(False) > ??????????? self.leds.append(Led((50 + (i * 30), 50))) Should a single LED know about a lot of other LEDS? Maybe you need a separate variable that holds the list of LEDs? [A more exotic choice would be to make the list of LEDs be a class variable(not an instance one) and have all LEDs append themselves to that when created. You can then create a class method that will turn all LEDs on/off. class LED: allLEDS = [] #class variable def __init__(self,state): ... init LED attributes... LED.allLEDS.append(self) def switch(self): self.state = not self.state @classmethod def switch_all(cls): for led in LED.allLEDS: led.switch() The problem with this approach in Python is that Python does not have reliable destructors so you cannot reliably delete the LEDs when they are destroyed. ] My personal recommendation is to keep it as simple as possible. Access the state attribute directly and have the LEDs stored in a global list variable. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mats at wichmann.us Mon Jul 19 13:28:27 2021 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 19 Jul 2021 11:28:27 -0600 Subject: [Tutor] object does not support item assignment In-Reply-To: References: Message-ID: On 7/19/21 10:22 AM, Dennis Lee Bieber wrote: > On Mon, 19 Jul 2021 12:54:07 +1000, Phil declaimed the > following: > > >> class Led: >> ??????? self.led = Led((50, 50)) > > How many Led classes do you have defined. To me, that assignment is > just creating another instance of the very class you have defined, and the > __init__() of that instance will define another instance, ad infinitum. what's more, because of the way Python evaluates things, at the time that line runs Led won't be defined yet, so you'll get a NameError. The class definition has to be fully executed before Python stores the object reference as the name Led. From phillor9 at gmail.com Wed Jul 21 02:27:53 2021 From: phillor9 at gmail.com (Phil) Date: Wed, 21 Jul 2021 16:27:53 +1000 Subject: [Tutor] Set and get class members In-Reply-To: References: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> Message-ID: On 19/7/21 8:35 pm, Alan Gauld via Tutor wrote: > > My personal recommendation is to keep it as simple as possible. > Access the state attribute directly and have the LEDs stored > in a global list variable. This is what has been suggested; self.leds = list() for i in range(8): self.leds[i].self.state = False # this is not correct. If state is True then the LED is on. self.leds.append(Led((50 + (i * 30), 50))) # this results in 8 equally spaced LEDs Is this what you have in mind? No matter how I try to set the state of a LED the error message is: IndexError: list assignment index out of range This is how I came up with a working solution in C++ using set methods quite some time ago. ? l = new Led[num_leds]; ? for (int i = 0; i < num_leds; i++) ? { ??? l[i] = new Led(60 + (i * 40), 120); ??? l[i].setState(false); ??? l[i].setLedSize(30); ??? l[i].setOnColour(green); ??? l[i].setOffColour(black); ? } All I need to do is set the state of say LED[3]. I've been sitting in a broken down motorhome for most of the day and my head is in a bit of a spin which could be why I'm having trouble with this simple problem. -- Regards, Phil From alan.gauld at yahoo.co.uk Wed Jul 21 03:46:55 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 21 Jul 2021 08:46:55 +0100 Subject: [Tutor] Set and get class members In-Reply-To: References: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> Message-ID: On 21/07/2021 07:27, Phil wrote: > On 19/7/21 8:35 pm, Alan Gauld via Tutor wrote: >> >> My personal recommendation is to keep it as simple as possible. >> Access the state attribute directly and have the LEDs stored >> in a global list variable. > > This is what has been suggested; > > self.leds = list() This should NOT be inside the class, it should be external. So: leds = list() # create empty list > for i in range(8): > self.leds[i].self.state = False You don't need the self (self is like 'this' in C++) and you can't use indexing until you've filled the list. So append the LEDs or use a list comprehension: leds = [Led(...) for _ in range(8)] # create list of 8 LEDs Then you can iterate over them: for led in leds: led.state = False # turn it off. > self.leds.append(Led((50 + (i * 30), 50))) # this results in 8 equally spaced LEDs You could do it this way but this line needs to be first. You can't index the list before adding the item. for i in range(8): leds.append(Led(...)) leds[i].state = False But its less pythonic to use indexing like that. > No matter how I try to set the state of a LED the error message is: > > IndexError: list assignment index out of range You just need to create the Led and put it in the list before you try to access it > This is how I came up with a working solution in C++ using set methods The set methods are irrelevant you could use direct access in C++ too, just make the state public. > ? l = new Led[num_leds]; > > > ? for (int i = 0; i < num_leds; i++) > ? { > ??? l[i] = new Led(60 + (i * 40), 120); Here you create the led and add it to the array. > ??? l[i].setState(false); Now you access it. The opposite way round to what you did in python. And the correct way. > All I need to do is set the state of say LED[3]. Oncer you have created and added them to the list you can use indexing: led[3].state = True -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From __peter__ at web.de Wed Jul 21 04:02:02 2021 From: __peter__ at web.de (Peter Otten) Date: Wed, 21 Jul 2021 10:02:02 +0200 Subject: [Tutor] Set and get class members In-Reply-To: References: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> Message-ID: On 21/07/2021 08:27, Phil wrote: > On 19/7/21 8:35 pm, Alan Gauld via Tutor wrote: >> >> My personal recommendation is to keep it as simple as possible. >> Access the state attribute directly and have the LEDs stored >> in a global list variable. > > This is what has been suggested; > > ??????? self.leds = list() This is *not* Alan's suggestion. "self" shouts "part of an (LED) instance. > ??????? for i in range(8): > ??????????? self.leds[i].self.state = False # this is not correct. If > state is True then the LED is on. > ??????????? self.leds.append(Led((50 + (i * 30), 50))) # this results > in 8 equally spaced LEDs > > Is this what you have in mind? > > No matter how I try to set the state of a LED the error message is: > > IndexError: list assignment index out of range > > This is how I came up with a working solution in C++ using set methods > quite some time ago. > > ? l = new Led[num_leds]; > > > ? for (int i = 0; i < num_leds; i++) > ? { > ??? l[i] = new Led(60 + (i * 40), 120); > ??? l[i].setState(false); > ??? l[i].setLedSize(30); > ??? l[i].setOnColour(green); > ??? l[i].setOffColour(black); > ? } > > All I need to do is set the state of say LED[3]. > > I've been sitting in a broken down motorhome for most of the day and my > head is in a bit of a spin which could be why I'm having trouble with > this simple problem. The difference is that in Python usually you do not preallocate the list. Instead of leds = [None] * num_leds for i in range(num_leds): leds[i] = Led(...) you better write leds = [] for i in range(num_leds): leds.append(Led(...)) or with a list comprehension: leds = [Led(...) for i in range(num_leds)] In both C++ and Python it would be better to use the constructor/initializer to specify the attributes: red = "red" black = "black" green = "green" class Led: def __init__( self, pos, state=False, size=50, on_color=green, off_color=black ): self.pos = pos self.state = state self.size = size self.on_color = on_color self.off_color = off_color num_leds = 8 leds = [] for i in range(num_leds): leds.append( Led((60 + (i * 40), 120)) ) # modify a specific LED leds[3].on_color = red To set all states you can write a helper function def set_states(leds, state): for led in leds: led.state = state # switch on all LEDs set_states(leds, True) # switch off all green LEDs set_states((led for led in leds if led.on_color == green), False) Once you are a bit more fluent in Python and understand the difference between instance, class and global namespace you can wrap the leds list into a class as suggested by dn and turn set_states into a method. For now I suggest that you stick to the simple approach shown above. From PyTutor at DancesWithMice.info Wed Jul 21 04:28:16 2021 From: PyTutor at DancesWithMice.info (dn) Date: Wed, 21 Jul 2021 20:28:16 +1200 Subject: [Tutor] Set and get class members In-Reply-To: References: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> Message-ID: <05a83086-0008-7052-9255-0ebe82007e09@DancesWithMice.info> On 21/07/2021 18.27, Phil wrote: > On 19/7/21 8:35 pm, Alan Gauld via Tutor wrote: >> >> My personal recommendation is to keep it as simple as possible. >> Access the state attribute directly and have the LEDs stored >> in a global list variable. Didn't we cover this the other day in "[Tutor] object does not support item assignment"? Class 1: an LED - can be replicated as many times as there are LEDs to represent. Class 2 or more likely a list: representing the LED array > This is what has been suggested; > > ??????? self.leds = list() no need for "self." "self" is only used inside the LED class! > ??????? for i in range(8): what might happen inside this loop: - instantiate an LED - append it to the list (or collection class) > ??????????? self.leds[i].self.state = False # this is not correct. If > state is True then the LED is on. > ??????????? self.leds.append(Led((50 + (i * 30), 50))) # this results in > 8 equally spaced LEDs > > Is this what you have in mind? > > No matter how I try to set the state of a LED the error message is: > > IndexError: list assignment index out of range Once the list of LED objects has been established, access will be of the form: list[ index ].turn_on() where list[ 0 ] etc are LED objects. Thus by choosing the list-index the code will direct itself to a particular LED. If you want each LED to be addressable by some sort of name, then don't use a list, use a dict instead: dict[ name ].turn_on() > This is how I came up with a working solution in C++ using set methods > quite some time ago. > > ? l = new Led[num_leds]; > > > ? for (int i = 0; i < num_leds; i++) > ? { > ??? l[i] = new Led(60 + (i * 40), 120); > ??? l[i].setState(false); > ??? l[i].setLedSize(30); > ??? l[i].setOnColour(green); > ??? l[i].setOffColour(black); > ? } > > All I need to do is set the state of say LED[3]. In the __init__() for the LED class, I would set these as default values, rather than setting them dynamically. > I've been sitting in a broken down motorhome for most of the day and my > head is in a bit of a spin which could be why I'm having trouble with > this simple problem. Know the feeling, even without a motorhome... -- Regards, =dn From breamoreboy at gmail.com Wed Jul 21 03:41:00 2021 From: breamoreboy at gmail.com (Mark Lawrence) Date: Wed, 21 Jul 2021 08:41:00 +0100 Subject: [Tutor] Set and get class members In-Reply-To: References: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> Message-ID: <90527a4c-0863-23f1-c08d-2165f0eef442@gmail.com> On 21/07/2021 07:27, Phil wrote: > On 19/7/21 8:35 pm, Alan Gauld via Tutor wrote: >> >> My personal recommendation is to keep it as simple as possible. >> Access the state attribute directly and have the LEDs stored >> in a global list variable. > > This is what has been suggested; > > ??????? self.leds = list() > ??????? for i in range(8): > ??????????? self.leds[i].self.state = False # this is not correct. If > state is True then the LED is on. > ??????????? self.leds.append(Led((50 + (i * 30), 50))) # this results > in 8 equally spaced LEDs > > Is this what you have in mind? > > No matter how I try to set the state of a LED the error message is: > > IndexError: list assignment index out of range Of course, you're trying to assign to an empty list, only then do you try to append to the list. The cleanest way that I see to do what you want is to get your __init__ method for the Led class to set the state, don't do it directly in the for loop. Then if you want you can use a list comprehension:- self.leds = [Led((50 + (i * 30), 50)) for i in range(8)] -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From phillor9 at gmail.com Wed Jul 21 19:03:29 2021 From: phillor9 at gmail.com (Phil) Date: Thu, 22 Jul 2021 09:03:29 +1000 Subject: [Tutor] Set and get class members In-Reply-To: References: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> Message-ID: <02fbf7b0-10ae-4567-6c18-03f57b7eb939@gmail.com> On 21/7/21 5:46 pm, Alan Gauld via Tutor wrote: Thank you everyone for pointing me in the correct direction. I have now achieved my initial goal to scan a lit LED across a line of unlit LEDs, though quite likely not in a true Pythonic form. I have a splitting headache and in no mood to play amateur programmer at the moment, however, I will return to this project in a day or two. Thank you all again. -- Regards, Phil From PyTutor at DancesWithMice.info Wed Jul 21 19:58:52 2021 From: PyTutor at DancesWithMice.info (dn) Date: Thu, 22 Jul 2021 11:58:52 +1200 Subject: [Tutor] Set and get class members In-Reply-To: <02fbf7b0-10ae-4567-6c18-03f57b7eb939@gmail.com> References: <374a97c7-60ff-b700-1f68-fe2c62f3eb03@gmail.com> <02fbf7b0-10ae-4567-6c18-03f57b7eb939@gmail.com> Message-ID: <2e487ab5-49a2-c8b7-afbd-51ce483b56f4@DancesWithMice.info> On 22/07/2021 11.03, Phil wrote: > On 21/7/21 5:46 pm, Alan Gauld via Tutor wrote: > > Thank you everyone for pointing me in the correct direction. I have now > achieved my initial goal to scan a lit LED across a line of unlit LEDs, > though quite likely not in a true Pythonic form. I have a splitting > headache and in no mood to play amateur programmer at the moment, > however, I will return to this project in a day or two. > > Thank you all again. When complexity seems to be doing its best to cause migraines, the solution is to tackle one small part at a time, eg represent a single LED and its switching. (Most of the time) It should thereafter become a less-complex task to figure-out how to join them all together. Many professionals will dispute this - the more time spent in design up-front, the less expensive the overall project. However, your "scale" is different! I've been working (in my spare time - whatever that is. OK, I've been supposed to have been working...) on some basic "maker" projects for kids. Came across a library book: Cook, Craft, and Evans, "Raspberry Pi Projects for Dummies", Wiley, 2015. It includes two "Stomp" projects (a game unfamiliar to me: requiring one to stomp/stamp on insect-bugs) featuring a ring of LEDs, lit in-turn, and a foot-switch. The provided-code includes all the lighting components and deals with the co-incidence of foot-switch and LED state. Sadly it is not O-O code, but may be of-interest... I've managed to acquire a ring-light of LEDs, but the rest is all good-intentions! (and I assume there are many other such articles on-line or in books) Your question and a visit from an (very) old ex-colleague coincided. I was reminded of visiting a different university (his), many, many, years ago; and being amused by the ops-staff and their re-programming of the front panel of a Burroughs B6700 mainframe's front-panel. Actually such beasts had several panels of lights/toggle-switches. One of them would display the Burroughs "B" when the CPU was idle. They were very proud of re-working another panel to have 'blinkenlights' which followed one another, over and around the panel's matrix - these days I think it would be called a 'snake' pattern. Countless hours went into such endeavors - with which I can identify, having used my Computer Operator boredom-time (like 0400~0600 on the night-shift) to teach myself more advanced programming (and thus running jobs of my own - sneaking them into idle time from the bottom/low-pri settings of the submission queue - shhh don't tell!). In case you don't know about mainframes and front panels: https://en.wikipedia.org/wiki/Front_panel Some photos, including the "B": https://4310b1a9-a-b8516d33-s-sites.googlegroups.com/a/retrocomputingtasmania.com/home/home/projects/burroughs-b6700-mainframe/gallery/B6700%20dual%20CPU%20MCP%20idle%20light%20pattern.JPG More FYI about what they are doing with de-comm mainframes: http://www.retrocomputingtasmania.com/home/projects/burroughs-b6700-mainframe -- Regards, =dn From basictech21 at yahoo.com Thu Jul 22 23:19:22 2021 From: basictech21 at yahoo.com (Chukwuemeka Ohakwe) Date: Fri, 23 Jul 2021 03:19:22 +0000 (UTC) Subject: [Tutor] SYNTAX ERROR MESSAGE References: <1420817910.559824.1627010362763.ref@mail.yahoo.com> Message-ID: <1420817910.559824.1627010362763@mail.yahoo.com> From? random import randintdef pick(words):? ? num_words = len(words)? ? num_picked = randint(0, num_words - 1)? ? word_picked = words[num_picked]? ? return word_pickedprint(pick(name), pick(verb), 'a', pick(noun), end='.\n') Good day friends!please why does this program send error message when I run it? The error is highlighted at r in random. It reads: Syntax Error? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Invalid Syntax? From alexkleider at gmail.com Fri Jul 23 18:56:48 2021 From: alexkleider at gmail.com (Alex Kleider) Date: Fri, 23 Jul 2021 15:56:48 -0700 Subject: [Tutor] SYNTAX ERROR MESSAGE In-Reply-To: <1420817910.559824.1627010362763@mail.yahoo.com> References: <1420817910.559824.1627010362763.ref@mail.yahoo.com> <1420817910.559824.1627010362763@mail.yahoo.com> Message-ID: On Fri, Jul 23, 2021 at 2:01 PM Chukwuemeka Ohakwe via Tutor < tutor at python.org> wrote: > From random import randintdef pick(words): num_words = len(words) > num_picked = randint(0, num_words - 1) word_picked = words[num_picked] > return word_pickedprint(pick(name), pick(verb), 'a', pick(noun), > end='.\n') > > Good day friends!please why does this program send error message when I > run it? > The error is highlighted at r in random. It reads: Syntax Error > Invalid > Syntax > It seems you are not posting in text mode (a common problem, one of which I've been guilty myself in the past) so your code is all mushed together and is no longer Python. I've tried to unravel it and get the following: from random import randint def pick(words): num_words = len(words) num_picked = randint(0, num_words - 1) word_picked = words[num_picked] return word_picked print(pick(name), pick(verb), 'a', pick(noun), end='.\n') The error I get is: Traceback (most recent call last): File "tutor.py", line 9, in print(pick(name), pick(verb), 'a', pick(noun), end='.\n') NameError: name 'name' is not defined Also: 'verb' and ''noun" are not defined. If you define them as follows: name = "Chuck" verb = "code" noun = "Python" ..then the code executes although it's not clear to me what you are trying to achieve. _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From cs at cskk.id.au Fri Jul 23 19:10:15 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Sat, 24 Jul 2021 09:10:15 +1000 Subject: [Tutor] SYNTAX ERROR MESSAGE In-Reply-To: <1420817910.559824.1627010362763@mail.yahoo.com> References: <1420817910.559824.1627010362763@mail.yahoo.com> Message-ID: On 23Jul2021 03:19, Chukwuemeka Ohakwe wrote: >From? random import randintdef pick(words):? ? num_words = len(words)? ? num_picked = randint(0, num_words - 1)? ? word_picked = words[num_picked]? ? return word_pickedprint(pick(name), pick(verb), 'a', pick(noun), end='.\n') > >Good day friends!please why does this program send error message when I run it? >The error is highlighted at r in random. It reads: Syntax Error? ? ? ? >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? >?Invalid Syntax? That is hard to say because your code is all formatted on one line and you have not recited the complete error message. Please include your code between triple backticks (the ` character), like this: ``` code here etc etc ``` but hard against the left hand edge. Do the same with the error message, including all of it. If what you included above is accurate, it may be because you have "From". Python is case sensitive, and "from" needs to be all lower case. Cheers, Cameron Simpson From PyTutor at DancesWithMice.info Fri Jul 23 19:15:05 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sat, 24 Jul 2021 11:15:05 +1200 Subject: [Tutor] SYNTAX ERROR MESSAGE In-Reply-To: <1420817910.559824.1627010362763@mail.yahoo.com> References: <1420817910.559824.1627010362763.ref@mail.yahoo.com> <1420817910.559824.1627010362763@mail.yahoo.com> Message-ID: <6e87082a-5cc8-b4bf-b895-9bfe99757592@DancesWithMice.info> On 23/07/2021 15.19, Chukwuemeka Ohakwe via Tutor wrote: > From? random import randintdef pick(words):? ? num_words = len(words)? ? num_picked = randint(0, num_words - 1)? ? word_picked = words[num_picked]? ? return word_pickedprint(pick(name), pick(verb), 'a', pick(noun), end='.\n') > > Good day friends!please why does this program send error message when I run it? > The error is highlighted at r in random. It reads: Syntax Error? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Invalid Syntax? >>> From random import randint File "", line 1 From random import randint ^ SyntaxError: invalid syntax >>> from random import randint Yes, it can be confusing when Python notices the error at one point (the 'arrow'/highlight), but the actual fault was caused earlier in the code. The trick is to look a short distance 'backwards'! -- Regards, =dn From mmssdd1920 at gmail.com Sat Jul 24 02:31:52 2021 From: mmssdd1920 at gmail.com (Msd De) Date: Sat, 24 Jul 2021 12:01:52 +0530 Subject: [Tutor] fixed quad Message-ID: I have nested integrals and need to use a fixed quad to carry out the integration Below is my code. Not sure of whether use of lambda function is appropriate need help with regards to use of lambda function. Thank you LB=0.0 UB=etatip K001_Intg=[None]*64;K0d1_Intg=[None]*64;Kd01_Intg=[None]*64;Kdd1_Intg=[None]*64 def kfac_11(eta,Ex): distGen=array_defl[i] a=2*FCON*abs(EL-Ex) ............... ............. ............ K001_Intg=F0_xp*Fd_xp*f_xp*etaTerm/W K0d1_Intg=Kdd_0*F0_xp**2*f_xp*etaTerm/W Kd01_Intg=K00_0*Fd_xp**2*f_xp*etaTerm/W Kdd1_Intg=F0_xp*Fd_xp*f_xp*etaTerm/W return [K001_Intg,K0d1_Intg,Kd01_Intg,Kdd1_Intg]; DFF_11=[] *def DEx11(Ex): * distGen=array_defl[i] global xp xp=(array_xefl[i,]) arr_KFact=[] *K_Fact=integrate.fixed_quad(kfac_11, LB, UB, args=(Ex, ), n=64) * ......... ......... ....... DEx_11=(ak2/ak1)*TP_1*DFF_11*CONST return DEx_11 JRuss11={} array_JRuss11=np.zeros(64) *for i in range(64): * *JRuss11[i]=integrate.fixed_quad(lambda Ex: DEx11(Ex),ELOW, EHIGH, n=64)* array_JRuss11[int(i)]=(JRuss11[i][0]) IntgRusscurr11=[] Russcurrent11=[] arr_Russcurrent11=np.zeros(64) *def currRuss11(xiRuss11): * IntgRusscurr11=array_JRuss11*np.sqrt(xiRuss11**2-etatip**2)*CCONST * return *IntgRusscurr11 From __peter__ at web.de Sat Jul 24 03:52:12 2021 From: __peter__ at web.de (Peter Otten) Date: Sat, 24 Jul 2021 09:52:12 +0200 Subject: [Tutor] fixed quad In-Reply-To: References: Message-ID: On 24/07/2021 08:31, Msd De wrote: > I have nested integrals and need to use a fixed quad to carry out the > integration > Below is my code. Not sure of whether use of lambda function is appropriate > need help with regards to use of lambda function. > Thank you > *for i in range(64): * > *JRuss11[i]=integrate.fixed_quad(lambda Ex: DEx11(Ex),ELOW, EHIGH, > n=64)* > array_JRuss11[int(i)]=(JRuss11[i][0]) I'm sorry. I can't help you with the actual task. However, the lambda above is superfluous. Consider the simple analog do_stuff(lambda x: f(x)) If you rewrite it with a "normal" function this becomes def g(x): return f(x) do_stuff(g) and it should be easy to see that g() wraps f() without doing any extra work; so you better write do_stuff(f) directly, or in your case fixed_quad(DEx11, ...) From bhaskarjha.dev.001 at gmail.com Sat Jul 24 10:43:32 2021 From: bhaskarjha.dev.001 at gmail.com (Bhaskar Jha) Date: Sat, 24 Jul 2021 20:13:32 +0530 Subject: [Tutor] How to better understand args and kwargs Message-ID: Hello, I am a newbie with Python. And, I want to understand what's the use of args and kwargs.. i understand that these are used when we do not know the number of arguments that will be passed.. but how is args different from kwargs.. and, why do we allow for a situation when someone can pass unlimited arguments to a function.. other languages such as C++ do not make provisions for this. So, why does Python do it? Kind Regards Bhaskar From alan.gauld at yahoo.co.uk Sat Jul 24 14:34:58 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 24 Jul 2021 19:34:58 +0100 Subject: [Tutor] How to better understand args and kwargs In-Reply-To: References: Message-ID: On 24/07/2021 15:43, Bhaskar Jha wrote: > I am a newbie with Python. And, I want to understand what's the use of args > and kwargs.. i understand that these are used when we do not know the > number of arguments that will be passed.. but how is args different from > kwargs.. args is for fixed arguments, kwargs is for arguments specified using keywords. def a(*args): for arg in args: print(arg) def b(**kwargs): for nm,val in **kwargs.items(): print(nm,':',val) a(1,2,3) # use positional args b(a=4,b=5,c=6) # use keyword args def f(*args,**kwargs): for a in args: print(a,) for nm,val in kwargs.items(): print(nm,':',val) f(1,2,x=22,c="foo") # use combination of positional and keyword > and, why do we allow for a situation when someone can pass > unlimited arguments to a function.. As for practical uses, one common example is a function like print() which can take any number of arguments. Another common use-case is where you are writing a function that wraps another (possibly more complex) function and simply passes through the values given along with some hard coded choices. The user can pass in whatever values they want to pass to the underlying function without the wrapper having to understand all of them itself. It isn't a common scenario but it does crop up quite often in real world projects. > other languages such as C++ do not make > provisions for this. So, why does Python do it? Many languages do, including C, C++ and C# Here is the link for C++: https://en.cppreference.com/w/cpp/language/variadic_arguments But it tends to be a rare scenario so you don't her it discussed very often. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From robertvstepp at gmail.com Sun Jul 25 13:14:21 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Sun, 25 Jul 2021 12:14:21 -0500 Subject: [Tutor] OT: "Your tests are only as good as your mocks." Comments? Message-ID: From https://swizec.com/blog/what-i-learned-from-software-engineering-at-google/#stubs-and-mocks-make-bad-tests The author of this article notes an example from his practice where his mock database that he used in his tests passed his tests when the actual code in production no longer had a database column that was in his mock database. As I have begun to play around with databases recently and how to test code relying on them, this really caught my attention. The overall article itself is a recap of what he read in a book about how Google does things ("Software Engineering at Google"). In this situation Google advocates for using "fakes" in place of mocks, where these fakes are simplified implementations of the real thing maintained by the same team to ensure API parity. How would the development and maintaining of these fakes be done so that the fakes don't drift from coding reality like the mocks might? It is not clear to me exactly what is going on here. And a more Python-specific question: Does the Python ecosystem provide tools for creating and managing fakes? -- Wishing you only the best, boB Stepp From mats at wichmann.us Sun Jul 25 13:48:23 2021 From: mats at wichmann.us (Mats Wichmann) Date: Sun, 25 Jul 2021 11:48:23 -0600 Subject: [Tutor] OT: "Your tests are only as good as your mocks." Comments? In-Reply-To: References: Message-ID: On 7/25/21 11:14 AM, boB Stepp wrote: > Python-specific question:? Does the Python ecosystem provide tools for > creating and managing fakes? > Well, there's this: https://pypi.org/project/Faker/ there are a large bunch of wrappers that adapt this for particular environments. I see there's a package called sqlfaker which may be unrelated code-wise, and looks like it might be dead. and for more sophisticated generation of test data, look at Hypothesis. From __peter__ at web.de Sun Jul 25 18:17:18 2021 From: __peter__ at web.de (Peter Otten) Date: Mon, 26 Jul 2021 00:17:18 +0200 Subject: [Tutor] OT: "Your tests are only as good as your mocks." Comments? In-Reply-To: References: Message-ID: On 25/07/2021 19:14, boB Stepp wrote: > From > https://swizec.com/blog/what-i-learned-from-software-engineering-at-google/#stubs-and-mocks-make-bad-tests > > > The author of this article notes an example from his practice where his > mock > database that he used in his tests passed his tests when the actual code in > production no longer had a database column that was in his mock > database.? As > I have begun to play around with databases recently and how to test code > relying on them, this really caught my attention. > > The overall article itself is a recap of what he read in a book about how > Google does things ("Software Engineering at Google").? In this situation > Google advocates for using "fakes" in place of mocks, where these fakes are > simplified implementations of the real thing maintained by the same team to > ensure API parity.? How would the development and maintaining of these > fakes > be done so that the fakes don't drift from coding reality like the mocks > might?? It is not clear to me exactly what is going on here.? And a more > Python-specific question:? Does the Python ecosystem provide tools for > creating and managing fakes? Bob, my observation is that you have a penchant to attack simple problems in the most complex way that exists. You are not Google! All your teams are likely just you;) You have a sqlite database of a few Megabytes -- or is it Kilobytes? You can easily copy the "real thing" into your test environment and run a few tests with it, and also compare the database layout with that of a smaller test db that contains hand-crafted data for your other tests. To increase the off-topicity a bit, here's a link which IIRC I first saw on comp.lang.python a few -- tempus fugit -- years ago. https://www.theonion.com/beaver-overthinking-dam-1819568416 Have fun! From alan.gauld at yahoo.co.uk Sun Jul 25 19:08:34 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 26 Jul 2021 00:08:34 +0100 Subject: [Tutor] OT: "Your tests are only as good as your mocks." Comments? In-Reply-To: References: Message-ID: On 25/07/2021 18:14, boB Stepp wrote: > The author of this article notes an example from his practice where his mock > database that he used in his tests passed his tests when the actual code in > production no longer had a database column that was in his mock database. That's always an issue with test environments. On big projects a large part of the budget is spent on testing and maintaining sync between environments. That usually means separate test and development teams and sophisticated automated tools to check for any changes to checked-in schemas etc. Detecting unpublished changes to the schema is an important part of the testing since such changes tend to impact many components of the system. > The overall article itself is a recap of what he read in a book about how > Google does things ("Software Engineering at Google"). In this situation > Google advocates for using "fakes" in place of mocks, where these fakes are > simplified implementations of the real thing maintained by the same team to > ensure API parity. I've never heard them called fakes before but having test databases based on the real thing is a common test strategy. Usually there is an automated tool that takes a copy of the dev database and then mangles it in ways that appease the data protection Gods... > How would the development and maintaining of these fakes > be done so that the fakes don't drift from coding reality like the mocks > might? - Tools. - Regular sync of baselines (via more tools). - Making the test schema the definitive one, so any failures are the fault of the devs. If they haven't published their changes in advance they are to blame for failures in test! > It is not clear to me exactly what is going on here. And a more > Python-specific question: Does the Python ecosystem provide tools for > creating and managing fakes? No idea, I've never found much tool support for this kind of thing, certainly on bigger projects there is a whole team responsible for building test tools that includes syncing and creating test data. And most of it is bespoke. As an example we had one project with around 500 developers, of which about 100+ were involved in testing and 30+ in creating the various builds and version control. In other words about 30% of the total technical staff were doing nothing but test related tasks. But that project had around 66 million lines of code (COBOL, SQL and C++) and a (DB2) database with over 10,000 tables and took up over 600GB of space. At that scale these things matter. For most Python type projects things are very much smaller and the scale of testing can be similarly scaled down. By the time you get to a small team of 6 or less (the happy spot for agile techniques) testing is usually a case of some fairly limited and informal system tests (possibly done by representative users) and a lot of unit testing done by the devs themselves. Unit testing rarely needs more than mocks. It's in system testing that "fakes" become critical. All IMHO of course. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PyTutor at DancesWithMice.info Sun Jul 25 22:55:41 2021 From: PyTutor at DancesWithMice.info (dn) Date: Mon, 26 Jul 2021 14:55:41 +1200 Subject: [Tutor] OT: "Your tests are only as good as your mocks." Comments? In-Reply-To: References: Message-ID: <28e5ad90-ad6d-697c-f70e-528491ebd4fd@DancesWithMice.info> On 26/07/2021 05.14, boB Stepp wrote: > From > https://swizec.com/blog/what-i-learned-from-software-engineering-at-google/#stubs-and-mocks-make-bad-tests > > > The author of this article notes an example from his practice where his > mock > database that he used in his tests passed his tests when the actual code in > production no longer had a database column that was in his mock > database.? As > I have begun to play around with databases recently and how to test code > relying on them, this really caught my attention. > > The overall article itself is a recap of what he read in a book about how > Google does things ("Software Engineering at Google").? In this situation > Google advocates for using "fakes" in place of mocks, where these fakes are > simplified implementations of the real thing maintained by the same team to > ensure API parity.? How would the development and maintaining of these > fakes > be done so that the fakes don't drift from coding reality like the mocks > might?? It is not clear to me exactly what is going on here.? And a more > Python-specific question:? Does the Python ecosystem provide tools for > creating and managing fakes? It's an amusing story, and one which the author identified as particularly relevant in larger organisations - but (almost) irrelevant in a one-man band. Thus, there seem to be two components to the question(s) and the thinking behind them. Firstly, the way teams and corporations operate, and secondly Python tools which support something recommended by an organisation (which may/not be used by their Python teams). Faking is newer than the more established techniques of stubs and mocks. Accordingly, it is generating a lot of light, but we have yet to see if there will be much heat! We'll get to that, but start with your interest in moving beyond the sole-coder into a professional dev.team environment:- Issue 1: Pride (as in, "...goeth before...") There is a mystique to working for a "FAANG company" (per article), that somehow translates into a Queen sound-track ("We are the champions"). In the ?good, old, days we referred to "Blue Chip companies" and thought them good, eye-catching content for one's resume (been there, done that, t-shirt too ragged to wear). However, the reality is, they (and their work-methods) are indeed unlike most others'. Whether they are better, or not, is up for debate... (further assumption: that all components of 'the organisation' are equal - and magnificent. The reality is that departments/projects differ widely from each-other - ranging from those which do shine-brightly, to those which challenge the proverbial pig-sty for churned-up mud and olfactory discomfort) Just because their employees think they're 'great' doesn't mean that their approach will suit any/all of the rest of us. Issue 2: Arrogance Within an organisation certain team-leaders attempt to build 'unity' through a them-and-us strategy. Which like the above, tends to engender a 'we are better than them' attitude. This in-turn amplifies any point of difference, often to the point of interfering with or preventing inter-communication. These days I'd probably be pilloried (are HR allowed to use 'cruel and unusual punishment'?) for it, but (with my Project Rescue PM-hat on (Project Manager)) have walked into situations like this; and all other efforts failing, mandated that teams to get themselves into the same room and 'hash things out' (professionally), or ... I would start "banging heads together" (unprofessionally) - or worse... - and yes, I've suffered through scenarios involving the DB-team not speaking with the dev.teams attempting to 'read' or 'write'. Sigh! (in fact thinking of the wasted time/money: BIG SIGH!) See also the author's comment about "Hyrum's Law" - which can only be said to be magnified when team inter-communication dwindles. Issue 3: Metrics There is a rule of human nature, that if some measurement is being used, work-practice will adapt to maximise on that point. Some of us will remember the idea that 'good programmers' wrote more LoC ("Lines of Code") per day, than others. If you were being measured on that, would it be better to write a three~five-line for-loop block or a single-line list-comprehension? How many of us really think that shipping our working "minutely" (per the article) is even remotely a good-idea? Perhaps we value our reputations? Tell me again: who claims to "do no harm"? Is there an emphasis on ensuring and assuring tests (at all levels) if there is a rush to 'production'? (this attitude to testing has been a problem, in many and varied forms, for as long as there has been programming) Issue 4: the Future is a rush! Whilst it is undeniably exciting, the problem with racing-forwards is that it will be difficult to anticipate where (future) problems lie. It is fair to say: no-one can predict the future (least-wise not with '20/20 vision'). However, Santayana's aphorism also applies: "Those who cannot remember the past are condemned to repeat it". In this case, "The Mythical Man-Month" (Brooks). That lesson recounts how adding more personnel to a 'late' project actually had the opposite effect to that intended. Which applies to the author's descriptions of adding too many new staff (to anything) in an uncontrolled, indeed uncontrollable, fashion. People don't know each other, responsibilities keep shifting, communication fractures, and becoming difficult/impossible, dries-up! Is this a technical problem or a management failing? Issue 5: Solving social problems with 'technical solutions' Which neatly ties-together much of the above: yes, we've probably all experienced the 'please upgrade' issue, and the laggards' 'can I upgrade from so-many-versions-ago to the-new-version all at-once?' plea. However, just as the author comments (earlier in the article) about 'engineers losing control', so too will users! People's feelings represents an huge proportion of their willingness/decision to install, use, and continue to use, an application. There are ways to talk to people - nay, to make the point: "ways to talk WITH people"! Accordingly, 'here' at Python, we have a current version of 3.9... yet cheerfully engage with folk using earlier releases - even (albeit with some alarm) those who are somehow compelled to stay with Python 2! With such critique in-mind, let's look at practicalities:- You (and the author) are quite right, such faults will not be discovered in what is often 'the normal course' of a team's/an individual's work-flow! However, remember that one should not (unit) test to stubs/mocks/interfaces; but test to values! If your code is to divide two numbers, it had better test for 'that zero problem'; but who-cares from where the data has been drawn? If I happen to be doing the DB work, using 'my' (project's) Git repo; and you are writing app.code within 'your' Git repo, we can both unit test "until the cows come home" - and never, ever find such an 'error' as the author described. Unit tests, must by-definition come up short. Such investigation is (a part of) the province of "Integration Testing". Who manages that part of the author's CI/CD process? Answer: not the DB-team, not the app.devs, ... Who then? Oops! FYI At one time I considered a technical answer to this issue and thought that MySQL's in-memory DB-engine might serve, ie by putting tables/table-stubs into memory, by which the speed-increase might counter the disadvantage of using a 'real' DB. It wasn't suitable, largely because of the limitations on which data-types which can be handled - and thus it failed to enable a realistic replacement of the 'real data'. (https://dev.mysql.com/doc/refman/8.0/en/memory-storage-engine.html) There is always going to be a problem with modelling - you'd think that as we do this all-day, every-day, we-computer-people would consider this. Do we? Adequately? A mock is a mimic - similar but not the same, and quite possibly even over-emphasising certain aspects (even at the risk of minimising others). A stub is an abbreviated form - 'stuff' has, by definition, been left-out. These are 'judgement calls'. Do we sometimes get these 'wrong'? Remember that should you stub your toe there may be others prepared to mock your clumsiness. (Yuk!) Any tool can be assumed to be offering us 'more' than it really is - it is easy to assume that we have 'everything covered' when we don't - after all, isn't that our fate: that no matter how much testing we perform, there will always be one user who can find some combination of circumstances we did not foresee... Finally, in this cynical observation of 'real life', there was talk of "fakes are simplified implementations of the real thing maintained by the same team to ensure API parity". Which is "the same team"? The DB-guys who are only interested in their work, and who work to no metric which involves helping 'you'? Your team - the ones who have no idea that the DB-team have 'shifted the goal-posts'? Whither "API parity"? It may be that instead of discovering that the information used to build the mock is now out-of-date, all that happens is that you (belatedly) discover that the "fake" is too-fake... The deck-chairs have been rearranged and given new labels ("mock", "fake"), but they are still on SS Titanic! So, then, what is the answer? (I'm not sure about the "the"!) Once again, back in the ?good, old, days (oh, no, here he goes again...) - which included the "waterfall approach" to systems development, we called it "Change Control". No-one was allowed to make 'changes' to a system without appropriate documentation to provide (all-concerned) notice! Today, imagine a stand-up SCRUM/meeting, and I (ever so grandly) announce that today I shall be coding a change to the database, adding a new field, and it will all be so-wonderful. You prick-up your ears and ask for more info - will it affect your application-code? We agree to 'meet afterwards', and the change is reviewed, and either reversed or accommodated. No nasty surprise AFTER we both thought 'job done'! How well we work together! Some people think of (unit- and integration-) testing as 'extra work'. After all, it is the application-code which 'gets the job done'! One of the contributions of TDD is that testing is integral to development, to proving, and to maintenance/refactoring. Accordingly, as much care should be invested in the testing routines as is into the application's! Whether we use mocks, stubs, fakes, or you-name-it, there are no guarantees. Each must be used with care. Nothing can be taken for-granted! A tool being used at one 'layer' of the testing process cannot really be expected to cross-layers - even if some 'higher' layer of testing uses the same tool. How one layer is tested is quite different to the objectives of testing an higher/lower layer! NB I don't see one of these as 'the tool to rule them all'. Each has its place - use 'the best tool for the job'. A number of references/allusions have been included here, because I know the OP likes to 'read around' and consider issues wider than mere syntax. Having survived this far, if you enjoy our favorite language's occasional allusion to the Monty Python series, you will likely also enjoy Terry Pratchett's (fictional) writings. Herewith a couple of pertinent quotes, the first about 'planning' (planning testing?), and the second referencing our industry's habit of making great promises (and assumptions) but being a little short on delivery against others' (users') expectations (as per the article?):- "Plan A hadn't worked. Plan B had failed. Everything depended on Plan C, and there was one drawback to this: he had only ever planned as far as B." "Crowley had been extremely impressed with the warranties offered by the computer industry, and had in fact sent a bundle Below to the department that drew up the Immortal Soul agreements, with a yellow memo form attached just saying:?Learn, guys.'" (both from "Good Omens") -- Regards, =dn From rakesh7biswas at gmail.com Sun Jul 25 23:14:09 2021 From: rakesh7biswas at gmail.com (Rakesh Biswas) Date: Mon, 26 Jul 2021 08:44:09 +0530 Subject: [Tutor] OT: "Your tests are only as good as your mocks." Comments? In-Reply-To: <28e5ad90-ad6d-697c-f70e-528491ebd4fd@DancesWithMice.info> References: <28e5ad90-ad6d-697c-f70e-528491ebd4fd@DancesWithMice.info> Message-ID: Fantastic write up on the philosophy of work where plan C remains unplanned. ? On Mon, Jul 26, 2021, 8:33 AM dn via Tutor wrote: > On 26/07/2021 05.14, boB Stepp wrote: > > From > > > https://swizec.com/blog/what-i-learned-from-software-engineering-at-google/#stubs-and-mocks-make-bad-tests > > > > > > The author of this article notes an example from his practice where his > > mock > > database that he used in his tests passed his tests when the actual code > in > > production no longer had a database column that was in his mock > > database. As > > I have begun to play around with databases recently and how to test code > > relying on them, this really caught my attention. > > > > The overall article itself is a recap of what he read in a book about how > > Google does things ("Software Engineering at Google"). In this situation > > Google advocates for using "fakes" in place of mocks, where these fakes > are > > simplified implementations of the real thing maintained by the same team > to > > ensure API parity. How would the development and maintaining of these > > fakes > > be done so that the fakes don't drift from coding reality like the mocks > > might? It is not clear to me exactly what is going on here. And a more > > Python-specific question: Does the Python ecosystem provide tools for > > creating and managing fakes? > > > It's an amusing story, and one which the author identified as > particularly relevant in larger organisations - but (almost) irrelevant > in a one-man band. > > Thus, there seem to be two components to the question(s) and the > thinking behind them. Firstly, the way teams and corporations operate, > and secondly Python tools which support something recommended by an > organisation (which may/not be used by their Python teams). Faking is > newer than the more established techniques of stubs and mocks. > Accordingly, it is generating a lot of light, but we have yet to see if > there will be much heat! We'll get to that, but start with your interest > in moving beyond the sole-coder into a professional dev.team environment:- > > > Issue 1: Pride (as in, "...goeth before...") > There is a mystique to working for a "FAANG company" (per article), that > somehow translates into a Queen sound-track ("We are the champions"). In > the ?good, old, days we referred to "Blue Chip companies" and thought > them good, eye-catching content for one's resume (been there, done that, > t-shirt too ragged to wear). However, the reality is, they (and their > work-methods) are indeed unlike most others'. Whether they are better, > or not, is up for debate... (further assumption: that all components of > 'the organisation' are equal - and magnificent. The reality is that > departments/projects differ widely from each-other - ranging from those > which do shine-brightly, to those which challenge the proverbial pig-sty > for churned-up mud and olfactory discomfort) Just because their > employees think they're 'great' doesn't mean that their approach will > suit any/all of the rest of us. > > Issue 2: Arrogance > Within an organisation certain team-leaders attempt to build 'unity' > through a them-and-us strategy. Which like the above, tends to engender > a 'we are better than them' attitude. This in-turn amplifies any point > of difference, often to the point of interfering with or preventing > inter-communication. These days I'd probably be pilloried (are HR > allowed to use 'cruel and unusual punishment'?) for it, but (with my > Project Rescue PM-hat on (Project Manager)) have walked into situations > like this; and all other efforts failing, mandated that teams to get > themselves into the same room and 'hash things out' (professionally), or > ... I would start "banging heads together" (unprofessionally) - or worse... > - and yes, I've suffered through scenarios involving the DB-team not > speaking with the dev.teams attempting to 'read' or 'write'. Sigh! (in > fact thinking of the wasted time/money: BIG SIGH!) See also the author's > comment about "Hyrum's Law" - which can only be said to be magnified > when team inter-communication dwindles. > > Issue 3: Metrics > There is a rule of human nature, that if some measurement is being used, > work-practice will adapt to maximise on that point. Some of us will > remember the idea that 'good programmers' wrote more LoC ("Lines of > Code") per day, than others. If you were being measured on that, would > it be better to write a three~five-line for-loop block or a single-line > list-comprehension? How many of us really think that shipping our > working "minutely" (per the article) is even remotely a good-idea? > Perhaps we value our reputations? Tell me again: who claims to "do no > harm"? Is there an emphasis on ensuring and assuring tests (at all > levels) if there is a rush to 'production'? (this attitude to testing > has been a problem, in many and varied forms, for as long as there has > been programming) > > Issue 4: the Future is a rush! > Whilst it is undeniably exciting, the problem with racing-forwards is > that it will be difficult to anticipate where (future) problems lie. It > is fair to say: no-one can predict the future (least-wise not with > '20/20 vision'). However, Santayana's aphorism also applies: "Those who > cannot remember the past are condemned to repeat it". In this case, "The > Mythical Man-Month" (Brooks). That lesson recounts how adding more > personnel to a 'late' project actually had the opposite effect to that > intended. Which applies to the author's descriptions of adding too many > new staff (to anything) in an uncontrolled, indeed uncontrollable, > fashion. People don't know each other, responsibilities keep shifting, > communication fractures, and becoming difficult/impossible, dries-up! Is > this a technical problem or a management failing? > > Issue 5: Solving social problems with 'technical solutions' > Which neatly ties-together much of the above: yes, we've probably all > experienced the 'please upgrade' issue, and the laggards' 'can I upgrade > from so-many-versions-ago to the-new-version all at-once?' plea. > However, just as the author comments (earlier in the article) about > 'engineers losing control', so too will users! People's feelings > represents an huge proportion of their willingness/decision to install, > use, and continue to use, an application. There are ways to talk to > people - nay, to make the point: "ways to talk WITH people"! > Accordingly, 'here' at Python, we have a current version of 3.9... yet > cheerfully engage with folk using earlier releases - even (albeit with > some alarm) those who are somehow compelled to stay with Python 2! > > > With such critique in-mind, let's look at practicalities:- > > You (and the author) are quite right, such faults will not be discovered > in what is often 'the normal course' of a team's/an individual's > work-flow! However, remember that one should not (unit) test to > stubs/mocks/interfaces; but test to values! If your code is to divide > two numbers, it had better test for 'that zero problem'; but who-cares > from where the data has been drawn? > > If I happen to be doing the DB work, using 'my' (project's) Git repo; > and you are writing app.code within 'your' Git repo, we can both unit > test "until the cows come home" - and never, ever find such an 'error' > as the author described. Unit tests, must by-definition come up short. > Such investigation is (a part of) the province of "Integration Testing". > Who manages that part of the author's CI/CD process? Answer: not the > DB-team, not the app.devs, ... Who then? Oops! > > > FYI At one time I considered a technical answer to this issue and > thought that MySQL's in-memory DB-engine might serve, ie by putting > tables/table-stubs into memory, by which the speed-increase might > counter the disadvantage of using a 'real' DB. It wasn't suitable, > largely because of the limitations on which data-types which can be > handled - and thus it failed to enable a realistic replacement of the > 'real data'. > (https://dev.mysql.com/doc/refman/8.0/en/memory-storage-engine.html) > > > There is always going to be a problem with modelling - you'd think that > as we do this all-day, every-day, we-computer-people would consider > this. Do we? Adequately? > > A mock is a mimic - similar but not the same, and quite possibly even > over-emphasising certain aspects (even at the risk of minimising others). > > A stub is an abbreviated form - 'stuff' has, by definition, been left-out. > > These are 'judgement calls'. Do we sometimes get these 'wrong'? > > Remember that should you stub your toe there may be others prepared to > mock your clumsiness. (Yuk!) > > Any tool can be assumed to be offering us 'more' than it really is - it > is easy to assume that we have 'everything covered' when we don't - > after all, isn't that our fate: that no matter how much testing we > perform, there will always be one user who can find some combination of > circumstances we did not foresee... > > Finally, in this cynical observation of 'real life', there was talk of > "fakes are simplified implementations of the real thing maintained by > the same team to ensure API parity". Which is "the same team"? The > DB-guys who are only interested in their work, and who work to no metric > which involves helping 'you'? Your team - the ones who have no idea that > the DB-team have 'shifted the goal-posts'? Whither "API parity"? It may > be that instead of discovering that the information used to build the > mock is now out-of-date, all that happens is that you (belatedly) > discover that the "fake" is too-fake... The deck-chairs have been > rearranged and given new labels ("mock", "fake"), but they are still on > SS Titanic! > > > So, then, what is the answer? (I'm not sure about the "the"!) > > Once again, back in the ?good, old, days (oh, no, here he goes again...) > - which included the "waterfall approach" to systems development, we > called it "Change Control". No-one was allowed to make 'changes' to a > system without appropriate documentation to provide (all-concerned) notice! > > Today, imagine a stand-up SCRUM/meeting, and I (ever so grandly) > announce that today I shall be coding a change to the database, adding a > new field, and it will all be so-wonderful. You prick-up your ears and > ask for more info - will it affect your application-code? We agree to > 'meet afterwards', and the change is reviewed, and either reversed or > accommodated. > > No nasty surprise AFTER we both thought 'job done'! How well we work > together! > > > Some people think of (unit- and integration-) testing as 'extra work'. > After all, it is the application-code which 'gets the job done'! One of > the contributions of TDD is that testing is integral to development, to > proving, and to maintenance/refactoring. Accordingly, as much care > should be invested in the testing routines as is into the application's! > > Whether we use mocks, stubs, fakes, or you-name-it, there are no > guarantees. Each must be used with care. Nothing can be taken for-granted! > > A tool being used at one 'layer' of the testing process cannot really be > expected to cross-layers - even if some 'higher' layer of testing uses > the same tool. How one layer is tested is quite different to the > objectives of testing an higher/lower layer! > > NB I don't see one of these as 'the tool to rule them all'. Each has its > place - use 'the best tool for the job'. > > > A number of references/allusions have been included here, because I know > the OP likes to 'read around' and consider issues wider than mere syntax. > > Having survived this far, if you enjoy our favorite language's > occasional allusion to the Monty Python series, you will likely also > enjoy Terry Pratchett's (fictional) writings. Herewith a couple of > pertinent quotes, the first about 'planning' (planning testing?), and > the second referencing our industry's habit of making great promises > (and assumptions) but being a little short on delivery against others' > (users') expectations (as per the article?):- > > "Plan A hadn't worked. Plan B had failed. Everything depended on Plan > C, and there was one drawback to this: he had only ever planned as far > as B." > > "Crowley had been extremely impressed with the warranties offered by the > computer industry, and had in fact sent a bundle Below to the department > that drew up the Immortal Soul agreements, with a yellow memo form > attached just saying:?Learn, guys.'" > (both from "Good Omens") > -- > Regards, > =dn > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From alan.gauld at yahoo.co.uk Thu Jul 29 03:53:26 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 29 Jul 2021 08:53:26 +0100 Subject: [Tutor] Proper SQLite cursor handling? In-Reply-To: <4203gghhrp2l43d6q9f6eb2c8i5d3jeq3s@4ax.com> References: <1cuceg50jl535plon1ni4utfk6p52ch1k4@4ax.com> <35d86b50-8ea7-1692-7351-81f55a6acc49@yahoo.co.uk> <4203gghhrp2l43d6q9f6eb2c8i5d3jeq3s@4ax.com> Message-ID: On 28/07/2021 17:19, Dennis Lee Bieber wrote: > On Sun, 11 Jul 2021 18:34:01 -0400, Dennis Lee Bieber > Codd's opinion, NO field should allow NULL values. That implies that the > relevant fields are to be pulled out into a subordinate table, with a > unique foreign key to ensure that there are only 0 (the former NULL value) > or 1 record linked back to the parent record. That may be the purist view (or maybe just Codd!) but from a pragmatic view introducing new tables with a single value linked to from just one other table is a maintenance and performance nightmare. Plus it still begs the question about how you represent NULL values - thee are real world scenarios where things are optional and so can be NULL. You have to either invent spurious default values or some other way to fake it if using NOT NULL columns. > As a result, I've hacked my SQLite3 test scripts. One result is that > the BEFORE and AFTER UPDATE triggers are now gone -- the subordinate table > can have a default timestamp for datetime column, and NOT NULL will trap a > missing score. There may be a few cases where you are dealing with dynamic updates, but apart from simplistic databases like SQLite I still prefer writing a stored procedure or trigger to deal with those. > -- Doing this split does complicate the JOINs needed to retrieve data, > -- but also removed the need for UPDATE triggers to ensure end time > -- and score are filled in, as a default time can be specified, > -- and NOT NULL traps missing score. The avoidance of update triggers may cancel the performance penalty of the join. But it doesn't cancel the potential added complexity of all the selects, deletes, updates etc. which now have to manage two tables rather than one making them more fault prone. Its conceptually similar to the questions you must ask in OOP when determining when to create a new class for an attribute of an object. Does the added abstraction provide more benefit than the added complexity of maintaining two classes? Its the same debate with tables. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From jankenin at gmx.de Thu Jul 29 14:02:22 2021 From: jankenin at gmx.de (Jan Kenin) Date: Thu, 29 Jul 2021 20:02:22 +0200 Subject: [Tutor] Acceleration of long lists Message-ID: Hello, Often I append instances of objects in list-objects. Then I realize that the progress of my program slows down, when the list becomes larger and larger, e.g. larger than 3000. How can I accerlerate this? Perhaps it is a question of memory. Can I initialize the list in order to do the memory allocation only once? Thanks for all ideas! jankenin From alan.gauld at yahoo.co.uk Thu Jul 29 19:06:56 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 30 Jul 2021 00:06:56 +0100 Subject: [Tutor] Acceleration of long lists In-Reply-To: References: Message-ID: On 29/07/2021 19:02, Jan Kenin wrote: > Often I append instances of objects in list-objects. Then I realize that > the progress of my program slows down, when the list becomes larger and > larger, e.g. larger than 3000. 3000 is nothing to a modern computer. Remember the list is not holding your objects, it is simply holding a reference to each object. When you append an object you are only adding a new reference - a few bytes. Also, I believe that in the implementation details Python allocates memory for lists in chunks so you only actually create new memory when an existing chunk runs out, not for every append operation. (That's certainly how I would build it!) > How can I accerlerate this? Perhaps it is > a question of memory. Can I initialize the list in order to do the > memory allocation only once? No, that's a memory optimisation that's in the hands of the implementers and indeed may well work differently in different versions of Python (CPython, Jython, IronPython etc) The solution with any kind of performance problem is to measure to find out what is slowing things down. Have you tried using the profiler to see which functions are taking the time? Also, what kind of slow-down do you see? fractions of a second? seconds?, minutes? And is it only certain operations? If an operation is slow you probably won't notice it for a single object, but once you get into thousands repeating it then the delay becomes obvious. Again profiling is your friend. But without much more details (and ideally code) we can't be more specific. Also, if your code is doing embedded loops on these lists then that can slow things dramatically. And in Python it's easy to introduce loops without realizing - for example using 'in' tests causes a loop. If you repeat the same 'in' test multiple times you will loop over the entire list each time. Also searching for items might well be faster if you use a dictionary rather than a loop. But without sight of your code we can only make generic suggestions. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mats at wichmann.us Thu Jul 29 23:23:48 2021 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 29 Jul 2021 21:23:48 -0600 Subject: [Tutor] Acceleration of long lists In-Reply-To: References: Message-ID: On July 29, 2021 5:06:56 PM MDT, Alan Gauld via Tutor wrote: >On 29/07/2021 19:02, Jan Kenin wrote: > >> Often I append instances of objects in list-objects. Then I realize that >> the progress of my program slows down, when the list becomes larger and >> larger, e.g. larger than 3000. > >3000 is nothing to a modern computer. >Remember the list is not holding your objects, it is simply holding a >reference to each object. When you append an object you are only adding >a new reference - a few bytes. Once upon a time, if you were adding complex objects, things did slow down because the garbage collector scanned the list during append, and that got slower as the list grew (not for simple objects like ints, strings, etc. though), and you could usually improve things by turning it off before your append loop and on again afterwards. I dont think that's been an issue for something like 15 years now. >Also, I believe that in the implementation details Python >allocates memory for lists in chunks so you only actually >create new memory when an existing chunk runs out, not >for every append operation. (That's certainly how I would >build it!) in fact cpython doubles the size each time is has to allocate more so the frequency of getting a new chunk goes down. >> How can I accerlerate this? Perhaps it is >> a question of memory. Can I initialize the list in order to do the >> memory allocation only once? You dont mention numpy so I assume it's not involved - I have seen mentions of numpy arrays having this problem and indeed working better if you preallocate (numpy.empty(length)). That might be hooey, I have not experimented with this. >No, that's a memory optimisation that's in the hands of the >implementers and indeed may well work differently >in different versions of Python (CPython, Jython, IronPython etc) > >The solution with any kind of performance problem is to measure >to find out what is slowing things down. Have you tried using >the profiler to see which functions are taking the time? > >Also, what kind of slow-down do you see? fractions of a second? >seconds?, minutes? And is it only certain operations? If an operation >is slow you probably won't notice it for a single object, but >once you get into thousands repeating it then the delay >becomes obvious. Again profiling is your friend. But without >much more details (and ideally code) we can't be more specific. > >Also, if your code is doing embedded loops on these lists then >that can slow things dramatically. And in Python it's easy >to introduce loops without realizing - for example using >'in' tests causes a loop. If you repeat the same 'in' test >multiple times you will loop over the entire list each time. >Also searching for items might well be faster if you use >a dictionary rather than a loop. But without sight of your >code we can only make generic suggestions. > Here's a trick to try for fun, no idea if it will show a difference in your case since there could be something else going on: assuming your objects are unique and hashable, add them to a set instead, and then at the end, if you need a list, convert to a list. -- Sent from a mobile device with K-9 Mail. Please excuse my brevity. From __peter__ at web.de Fri Jul 30 03:44:10 2021 From: __peter__ at web.de (Peter Otten) Date: Fri, 30 Jul 2021 09:44:10 +0200 Subject: [Tutor] Acceleration of long lists In-Reply-To: References: Message-ID: On 29/07/2021 20:02, Jan Kenin wrote: > Often I append instances of objects in list-objects. Then I realize that > the progress of my program slows down, when the list becomes larger and > larger, e.g. larger than 3000. How can I accerlerate this? Perhaps it is > a question of memory. Can I initialize the list in order to do the > memory allocation only once? You can initialize the list at once, but not the objects in the list. However, the effect of that optimization is negligible, even for lists that are much longer than 3000 items: PS C:\Users\Peter> py -m timeit "a = [] >> for i in range(10**6): a.append(i)" 1 loop, best of 5: 430 msec per loop PS C:\Users\Peter> py -m timeit "a = [None] * 10**6 >> for i, k in enumerate(range(10**6)): a[i] = k" 1 loop, best of 5: 442 msec per loop > Thanks for all ideas! If you are seeing a user-perceivable slowdown for a small list of only(!) 3000 items there must be another problem than the one you suspected. Can you provide an example with more details? If it's not too long some actual code would be nice. From s.molnar at sbcglobal.net Fri Jul 30 10:27:26 2021 From: s.molnar at sbcglobal.net (Stephen P. Molnar) Date: Fri, 30 Jul 2021 10:27:26 -0400 Subject: [Tutor] Labeling and Sorting a Test File References: <61040C4E.4020001.ref@sbcglobal.net> Message-ID: <61040C4E.4020001@sbcglobal.net> First of all, let me say this is not a school exercise. I have a project that I am working on that is going to generate a large number of text files, perhaps in the thousands. The format of these files is: Detected 8 CPUs Reading input ... done. Setting up the scoring function ... done. Analyzing the binding site ... done. Using random seed: -596016 Performing search ... done. Refining results ... done. mode | affinity | dist from best mode | (kcal/mol) | rmsd l.b.| rmsd u.b. -----+------------+----------+---------- 1 -4.780168282 0.000 0.000 2 -4.767296818 8.730 11.993 3 -4.709057289 13.401 15.939 4 -4.677956834 8.271 11.344 5 -4.633563903 8.581 11.967 6 -4.569815226 4.730 8.233 7 -4.540149947 8.578 11.959 8 -4.515237403 8.096 10.215 9 -4.514233086 5.064 7.689 Writing output ... done. I have managed, through a combination of fruitlessly searching Google and making errors, this Python code (attached): #!/usr/bin/env python3 # -*- coding: utf-8 -*- import numpy as np with open("Ligand.list") as ligands_f: for line in ligands_f: ligand = line.strip() for num in range(1,11): #print(ligand) name_in = "{}.{}.log".format(ligand,num) data = np.genfromtxt(name_in,skip_header=28, skip_footer=1) name_s = ligand+'-BE' f = open(name_s, 'a') f.write(str(data[1,1])+'\n') f.close() The result of applying the code to ten files is: -4.709057289 -4.66850894 -4.747875776 -4.631865671 -4.661709186 -4.686041874 -4.632855261 -4.617575733 -4.734570162 -4.727217506 This is almost the way I want the file, with two problems: 1. I want to the first line in the textile to be the name of the file, in the example 'Test'. 2. I want the list sorted in the order of decreasing negative order: -4.747875776 -4.734570162 -4.727217506 -4.709057289 -4.686041874 -4.66850894 -4.661709186 -4.632855261 -4.631865671 -4.617575733 As my Python programming skills are limited, at best, I would appreciate some pointers in the right direction to implement the answers to my questions. Thanks in advance. -- Stephen P. Molnar, Ph.D. 614.312.7528 (c) Skype: smolnar1 -------------- next part -------------- -4.709057289 -4.66850894 -4.747875776 -4.631865671 -4.661709186 -4.686041874 -4.632855261 -4.617575733 -4.734570162 -4.727217506 From wlfraed at ix.netcom.com Fri Jul 30 12:20:19 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Fri, 30 Jul 2021 12:20:19 -0400 Subject: [Tutor] Labeling and Sorting a Test File References: <61040C4E.4020001.ref@sbcglobal.net> <61040C4E.4020001@sbcglobal.net> Message-ID: <9e68ggpbumbsjjv7ps5hp4d0uonbjefc1f@4ax.com> On Fri, 30 Jul 2021 10:27:26 -0400, "Stephen P. Molnar" declaimed the following: >First of all, let me say this is not a school exercise. > >I have a project that I am working on that is going to generate a large >number of text files, perhaps in the thousands. >The format of these files is: > >Detected 8 CPUs >Reading input ... done. >Setting up the scoring function ... done. >Analyzing the binding site ... done. >Using random seed: -596016 >Performing search ... done. > What application is generating files interspersing progress reports with... >Refining results ... done. > >mode | affinity | dist from best mode > | (kcal/mol) | rmsd l.b.| rmsd u.b. >-----+------------+----------+---------- > 1 -4.780168282 0.000 0.000 > 2 -4.767296818 8.730 11.993 > 3 -4.709057289 13.401 15.939 > 4 -4.677956834 8.271 11.344 > 5 -4.633563903 8.581 11.967 > 6 -4.569815226 4.730 8.233 > 7 -4.540149947 8.578 11.959 > 8 -4.515237403 8.096 10.215 > 9 -4.514233086 5.064 7.689 >Writing output ... done. ... a tabular text dump of said data? > >I have managed, through a combination of fruitlessly searching Google >and making errors, this Python code (attached): > >#!/usr/bin/env python3 ># -*- coding: utf-8 -*- > >import numpy as np > >with open("Ligand.list") as ligands_f: > for line in ligands_f: > ligand = line.strip() > for num in range(1,11): > #print(ligand) > name_in = "{}.{}.log".format(ligand,num) > data = np.genfromtxt(name_in,skip_header=28, skip_footer=1) > name_s = ligand+'-BE' > f = open(name_s, 'a') > f.write(str(data[1,1])+'\n') > f.close() > Why open and reclose the file for each write operation, since you are going to be writing 10 lines, one per file. Also, what do you expect if the output file already exists when you process the first file?. with open("Ligand.list") as ligands_f: for line in ligands_f: ligand = line.strip() with open(ligand+"-BE", "w") as outfil: for num in range(1,11): #print(ligand) name_in = "{}.{}.log".format(ligand,num) data = np.genfromtxt(name_in,skip_header=28, skip_footer=1) outfil.write(str(data[1,1])+'\n') >The result of applying the code to ten files is: > >-4.709057289 >-4.66850894 >-4.747875776 >-4.631865671 >-4.661709186 >-4.686041874 >-4.632855261 >-4.617575733 >-4.734570162 >-4.727217506 > Which is somewhat meaningless without being shown the ten input files... Are those the first data entry, the last data entry, something between. I'm guessing the first entry from each file. >This is almost the way I want the file, with two problems: > >1. I want to the first line in the textile to be the name of the file, >in the example 'Test'. Which file? Your code has an input file of, apparently, partial file names, then a series of files using the partial file name read from that first file, with a series of numerics appended, and an output file name? Where does "Test" come from? As for writing it -- given my scratched rewrite of your main code -- you would write that between the "with open..." and "for num..." statements. > >2. I want the list sorted in the order of decreasing negative order: > Normal ascending numeric sort... You can't do that unless you create the list in memory. Since you are processing one item from each of 10 files, and immediately writing it out, you are stuck with the order found in the files. This means moving the output to OUTSIDE of the main code, and accumulating the values.. >-4.747875776 >-4.734570162 >-4.727217506 >-4.709057289 >-4.686041874 >-4.66850894 >-4.661709186 >-4.632855261 >-4.631865671 >-4.617575733 > >As my Python programming skills are limited, at best, I would appreciate Spend some time with the Python Tutorial, and the library reference manual. Given you seem to be interested in only the first data line reported in each file, I'd have skipped the entire numpy overhead (you are reading in a whole file, just to throw all but one line away). >>> _, affinity, _, _ = " 1 -4.780168282 0.000 0.000".split() >>> affinity '-4.780168282' >>> float(affinity) -4.780168282 >>> With no massive import, just basic Python... -=-=- with open("Ligand.list") as ligands_f: for line in ligands_f: ligand = line.strip() outfn = ligand+"-BE" results = [] for num in range(1,11): with open("%s.%s.log" % (ligand, num), "r") as infil: while True: ln = infil.readln() if ln.startswith("---"): ln = infil.readln() _, affinity, _, _ = ln.split() #if possibility of more than 4 "words" on line #words = ln.split() #affinity = words[1] results.append(float(affinity)) break with open(outfn, "w") as outfil: results.sort() outfil.write("%s\n" % outfn) #or whatever you really desire outfil.write("".join(["%s\n" % itm for itm in results])) -=-=- -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/