From jamestztsai at gmail.com Thu Sep 1 06:31:25 2022 From: jamestztsai at gmail.com (James Tsai) Date: Thu, 1 Sep 2022 03:31:25 -0700 (PDT) Subject: Local variable definition in Python list comprehension Message-ID: Hello, I find it very useful if I am allowed to define new local variables in a list comprehension. For example, I wish to have something like [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. For now this functionality can be achieved by writing [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. Is it worthwhile to add a new feature like this in Python? If so, how can I propose this to PEP? From ben.usenet at bsb.me.uk Thu Sep 1 10:14:55 2022 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 01 Sep 2022 15:14:55 +0100 Subject: Local variable definition in Python list comprehension References: Message-ID: <877d2nmba8.fsf@bsb.me.uk> James Tsai writes: > I find it very useful if I am allowed to define new local variables in > a list comprehension. For example, I wish to have something like > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > For now this functionality can be achieved by writing > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. x and y are, to a first approximation, new local variables defined in a list comprehension. I think you need to restate what it is you want. > Is it worthwhile to add a new feature like this in Python? If so, how > can I propose this to PEP? To make any sort of case you'd need to give an example that does not have a clearer way to write it already. Your working version is, to me, clearer that the ones you want to be able to write. -- Ben. From rosuav at gmail.com Thu Sep 1 12:15:28 2022 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 2 Sep 2022 02:15:28 +1000 Subject: Local variable definition in Python list comprehension In-Reply-To: References: Message-ID: On Fri, 2 Sept 2022 at 02:10, James Tsai wrote: > > Hello, > > I find it very useful if I am allowed to define new local variables in a list comprehension. For example, I wish to have something like > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > For now this functionality can be achieved by writing > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. > > Is it worthwhile to add a new feature like this in Python? If so, how can I propose this to PEP? Not everything has to be a one-liner. ChrisA From eryksun at gmail.com Thu Sep 1 12:34:07 2022 From: eryksun at gmail.com (Eryk Sun) Date: Thu, 1 Sep 2022 11:34:07 -0500 Subject: Local variable definition in Python list comprehension In-Reply-To: References: Message-ID: On 9/1/22, James Tsai wrote: > > I find it very useful if I am allowed to define new local variables in a > list comprehension. For example, I wish to have something like > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > For now this functionality can be achieved by writing > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. You can assign a local variable in the `if` expression. For example: >>> [(x, y) for x in range(10) if x + (y := x**2) < 30] [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)] From eryksun at gmail.com Thu Sep 1 12:41:21 2022 From: eryksun at gmail.com (Eryk Sun) Date: Thu, 1 Sep 2022 11:41:21 -0500 Subject: How to make a variable's late binding crosses the module boundary? In-Reply-To: References: <06fa821a-2c33-49a2-81e3-6f76dd0a7317n@googlegroups.com> <1e81aff1-d1e4-48e0-9b48-bc2fcebe38dfn@googlegroups.com> <6db8e7db-b3b9-46b9-a9c7-4cc608bc56een@googlegroups.com> Message-ID: On 8/31/22, Jach Feng wrote: > > I found that using "from test import *" in test2.py makes test2's relation > to "test" almost like the function's relation to the module where it is > defined. You can refer to all the variables of the module Note that in general, if __all__ doesn't exist to explicitly define the contents of a star import, then it imports everything except names that begin with an underscore. From jamestztsai at gmail.com Thu Sep 1 16:18:53 2022 From: jamestztsai at gmail.com (James Tsai) Date: Thu, 1 Sep 2022 13:18:53 -0700 (PDT) Subject: Local variable definition in Python list comprehension In-Reply-To: <877d2nmba8.fsf@bsb.me.uk> References: <877d2nmba8.fsf@bsb.me.uk> Message-ID: <41b28b33-e36f-4ff9-9d1f-b8b667af38c4n@googlegroups.com> ? 2022?9?1???? UTC+2 16:15:17? ??? > James Tsai writes: > > > I find it very useful if I am allowed to define new local variables in > > a list comprehension. For example, I wish to have something like > > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > > > For now this functionality can be achieved by writing > > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. > x and y are, to a first approximation, new local variables defined in a > list comprehension. I think you need to restate what it is you want. > > Is it worthwhile to add a new feature like this in Python? If so, how > > can I propose this to PEP? > To make any sort of case you'd need to give an example that does not > have a clearer way to write it already. Your working version is, to me, > clearer that the ones you want to be able to write. > > -- > Ben. By local variable definition I mean binding a variable to a single value, so it doesn't include giving an iterable that a variable can take values iteratively, e.g. 'for x in range(10)'. Does it not worth introducing a specific syntax to do this, instead of creating a new list ad hoc to define the variable like 'for y in [1]'? From jamestztsai at gmail.com Thu Sep 1 16:26:28 2022 From: jamestztsai at gmail.com (James Tsai) Date: Thu, 1 Sep 2022 13:26:28 -0700 (PDT) Subject: Local variable definition in Python list comprehension In-Reply-To: References: Message-ID: <30203f65-3dde-420a-b419-9581d070ea5dn@googlegroups.com> ? 2022?9?1???? UTC+2 18:16:03? ??? > On Fri, 2 Sept 2022 at 02:10, James Tsai wrote: > > > > Hello, > > > > I find it very useful if I am allowed to define new local variables in a list comprehension. For example, I wish to have something like > > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > > > For now this functionality can be achieved by writing > > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. > > > > Is it worthwhile to add a new feature like this in Python? If so, how can I propose this to PEP? > Not everything has to be a one-liner. > > ChrisA No but very often when I have written a neat list/dict/set comprehension, I find it very necessary to define local variable(s) to make it more clear and concise. Otherwise I have to break it down to several incrementally indented lines of for loops, if statements, and variable assignments, which I think look less nice. From jamestztsai at gmail.com Thu Sep 1 16:33:16 2022 From: jamestztsai at gmail.com (James Tsai) Date: Thu, 1 Sep 2022 13:33:16 -0700 (PDT) Subject: Local variable definition in Python list comprehension In-Reply-To: References: Message-ID: <83c577b5-aaa3-4335-9857-5e88e6be4015n@googlegroups.com> ? 2022?9?1???? UTC+2 18:34:36? ??? > On 9/1/22, James Tsai wrote: > > > > I find it very useful if I am allowed to define new local variables in a > > list comprehension. For example, I wish to have something like > > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > > > For now this functionality can be achieved by writing > > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. > You can assign a local variable in the `if` expression. For example: > > >>> [(x, y) for x in range(10) if x + (y := x**2) < 30] > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)] Yeah this works great but like [(x, y) for x in range(10) for y in [x**2]] I written before, is kind of a hack. And if initially I do not need an "if" condition in the list comprehension, this becomes less convenient. I still can write >>> [(x, y) for x in range(10) if (y := x**2) or True] But I wonder if Python could have a specific syntax to support this. From rosuav at gmail.com Thu Sep 1 17:01:31 2022 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 2 Sep 2022 07:01:31 +1000 Subject: Local variable definition in Python list comprehension In-Reply-To: <83c577b5-aaa3-4335-9857-5e88e6be4015n@googlegroups.com> References: <83c577b5-aaa3-4335-9857-5e88e6be4015n@googlegroups.com> Message-ID: On Fri, 2 Sept 2022 at 06:55, James Tsai wrote: > > ? 2022?9?1???? UTC+2 18:34:36? ??? > > On 9/1/22, James Tsai wrote: > > > > > > I find it very useful if I am allowed to define new local variables in a > > > list comprehension. For example, I wish to have something like > > > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > > > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > > > > > For now this functionality can be achieved by writing > > > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. > > You can assign a local variable in the `if` expression. For example: > > > > >>> [(x, y) for x in range(10) if x + (y := x**2) < 30] > > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)] > > Yeah this works great but like [(x, y) for x in range(10) for y in [x**2]] I written before, is kind of a hack. And if initially I do not need an "if" condition in the list comprehension, this becomes less convenient. I still can write > >>> [(x, y) for x in range(10) if (y := x**2) or True] > > But I wonder if Python could have a specific syntax to support this. > But why would you need to assign to y in that example? If you're using it more than once, you can use :=, and if you aren't, you don't need to. But do be aware that := does not create a comprehension-local name binding, but a nonlocal instead. > No but very often when I have written a neat list/dict/set comprehension, I find it very necessary > to define local variable(s) to make it more clear and concise. Otherwise I have to break it down > to several incrementally indented lines of for loops, if statements, and variable assignments, > which I think look less nice. Well, if it's outgrown a list comp, write it on multiple lines. Like I said, not everything has to be a one-liner. ChrisA From hjp-python at hjp.at Thu Sep 1 18:11:56 2022 From: hjp-python at hjp.at (Peter J. Holzer) Date: Fri, 2 Sep 2022 00:11:56 +0200 Subject: Local variable definition in Python list comprehension In-Reply-To: <83c577b5-aaa3-4335-9857-5e88e6be4015n@googlegroups.com> References: <83c577b5-aaa3-4335-9857-5e88e6be4015n@googlegroups.com> Message-ID: <20220901221156.lpnvwmkwmtorydz5@hjp.at> On 2022-09-01 13:33:16 -0700, James Tsai wrote: > ? 2022?9?1???? UTC+2 18:34:36? ??? > > On 9/1/22, James Tsai wrote: > > > > > > I find it very useful if I am allowed to define new local variables in a > > > list comprehension. For example, I wish to have something like > > > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > > > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > > > > > For now this functionality can be achieved by writing > > > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. > > You can assign a local variable in the `if` expression. For example: > > > > >>> [(x, y) for x in range(10) if x + (y := x**2) < 30] > > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)] > > Yeah this works great but like [(x, y) for x in range(10) for y in > [x**2]] I written before, is kind of a hack. And if initially I do not > need an "if" condition in the list comprehension, this becomes less > convenient. I still can write > >>> [(x, y) for x in range(10) if (y := x**2) or True] In that case [(x, x**2) for x in range(10)] seems to be somewhat more readable. I would also say that [(x, x**2) for x in range(10) if x + x**2 < 80] doesn't really seem worse than any of the variants with y in it. (Yes, I get that your real duplicated expression is probably a bit more complex than `x**2`, but by the time a temporary variable really improves readability it's probably a good time to split that across multiple lines, too.) hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From cs at cskk.id.au Thu Sep 1 18:11:06 2022 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 2 Sep 2022 08:11:06 +1000 Subject: Local variable definition in Python list comprehension In-Reply-To: References: Message-ID: On 02Sep2022 07:01, Chris Angelico wrote: >On Fri, 2 Sept 2022 at 06:55, James Tsai wrote: >> No but very often when I have written a neat list/dict/set >> comprehension, I find it very necessary >> to define local variable(s) to make it more clear and concise. Otherwise I have to break it down >> to several incrementally indented lines of for loops, if statements, and variable assignments, >> which I think look less nice. > >Well, if it's outgrown a list comp, write it on multiple lines. Like I >said, not everything has to be a one-liner. True, but a comprehension can be more expressive than a less "functional" expression (series of statements). James, can you provide (a) a real world example where you needed to write a series of statements or loops and (b) a corresponding example of how you would have preferred to have written that code, possibly inventing some syntax or misusing ":=" as if it workeed they way you'd like it to work? Cheers, Cameron Simpson From avi.e.gross at gmail.com Thu Sep 1 18:52:49 2022 From: avi.e.gross at gmail.com (Avi Gross) Date: Thu, 1 Sep 2022 18:52:49 -0400 Subject: Local variable definition in Python list comprehension In-Reply-To: References: <83c577b5-aaa3-4335-9857-5e88e6be4015n@googlegroups.com> Message-ID: Dumb question. Your y is purely a function of x. So create an f(x) where you want your y. It probably can even be anonymous inline. I mean your return values of (x, y) would be (x, f(x)) ... On Thu, Sep 1, 2022, 5:04 PM Chris Angelico wrote: > On Fri, 2 Sept 2022 at 06:55, James Tsai wrote: > > > > ? 2022?9?1???? UTC+2 18:34:36? ??? > > > On 9/1/22, James Tsai wrote: > > > > > > > > I find it very useful if I am allowed to define new local variables > in a > > > > list comprehension. For example, I wish to have something like > > > > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > > > > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > > > > > > > For now this functionality can be achieved by writing > > > > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. > > > You can assign a local variable in the `if` expression. For example: > > > > > > >>> [(x, y) for x in range(10) if x + (y := x**2) < 30] > > > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16)] > > > > Yeah this works great but like [(x, y) for x in range(10) for y in > [x**2]] I written before, is kind of a hack. And if initially I do not need > an "if" condition in the list comprehension, this becomes less convenient. > I still can write > > >>> [(x, y) for x in range(10) if (y := x**2) or True] > > > > But I wonder if Python could have a specific syntax to support this. > > > > But why would you need to assign to y in that example? If you're using > it more than once, you can use :=, and if you aren't, you don't need > to. But do be aware that := does not create a comprehension-local name > binding, but a nonlocal instead. > > > No but very often when I have written a neat list/dict/set > comprehension, I find it very necessary > > to define local variable(s) to make it more clear and concise. Otherwise > I have to break it down > > to several incrementally indented lines of for loops, if statements, and > variable assignments, > > which I think look less nice. > > Well, if it's outgrown a list comp, write it on multiple lines. Like I > said, not everything has to be a one-liner. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From jfong at ms4.hinet.net Thu Sep 1 20:44:46 2022 From: jfong at ms4.hinet.net (Jach Feng) Date: Thu, 1 Sep 2022 17:44:46 -0700 (PDT) Subject: How to make a variable's late binding crosses the module boundary? In-Reply-To: References: <06fa821a-2c33-49a2-81e3-6f76dd0a7317n@googlegroups.com> <1e81aff1-d1e4-48e0-9b48-bc2fcebe38dfn@googlegroups.com> <6db8e7db-b3b9-46b9-a9c7-4cc608bc56een@googlegroups.com> Message-ID: ery... at gmail.com ? 2022?9?2? ?????12:41:46 [UTC+8] ?????? > On 8/31/22, Jach Feng wrote: > > > > I found that using "from test import *" in test2.py makes test2's relation > > to "test" almost like the function's relation to the module where it is > > defined. You can refer to all the variables of the module > Note that in general, if __all__ doesn't exist to explicitly define > the contents of a star import, then it imports everything except names > that begin with an underscore. Thank you for pointing out these important difference:-) --Jach From dandyrandy223 at gmail.com Thu Sep 1 22:02:39 2022 From: dandyrandy223 at gmail.com (Randy Johnson) Date: Thu, 1 Sep 2022 19:02:39 -0700 (PDT) Subject: When should I use "parent=None" in __ini__ and "parent" in super() In-Reply-To: References: Message-ID: <0ee9047c-6104-4b75-9ba4-c552eaca4a43n@googlegroups.com> Those are contradictory for what you are trying to accomplish unless it is a Parent - Child relationship (MainWindow - Window): When you super() an object, it enherits all the properties from its parent object. Source: https://www.w3schools.com/python/python_inheritance.asp If what you want is 2 windows, you can define 2 instances of both Classes and run both at the end. ``` class User1(object): def __init__(self, fname, age, parent=None): self.fname = fname self.age = int(age) # Calling the return function that prints the details self.return_user() def return_user(self): print(self.fname) print(self.age) class User2(object): def __init__(self, lname, height, parent=None): self.lname = lname self.height = float(height) self.return_user() def return_user(self): print(self.lname) print(self.height, "CM") print((self.height / 2.54), "IN") User1("Hugh", 21) User2("Janus", 175.26) ``` From cs at cskk.id.au Fri Sep 2 19:08:57 2022 From: cs at cskk.id.au (Cameron Simpson) Date: Sat, 3 Sep 2022 09:08:57 +1000 Subject: When should I use "parent=None" in __ini__ and "parent" in super() In-Reply-To: References: Message-ID: On 23Sep2021 20:38, Mohsen Owzar wrote: >I'm writing since almost one-year codes in Python, using TKinter and >PyQt5. >I'm somehow able to writes GUIs in both of them. >But since I'm using more Pyqt5 and using classes with initialization and super() constructs, and also I saw lots of videos and examples of coding them, I still don?t know exactly how and when should I use the parent in __init()__ and super() construct. Basicly, q Qt widget can always have a parent. You need to accomodate that parameter. Let's look at your first example, which is more correct: class MyClass1(QWidget): def __init__(self, name, parent=None): super(MyClass1, self).__init__(parent) print(self.parent().test) Here's you're defining the initialiser for your MyClass1 subclass. You do not specific handle parent, that is handled by the superclass. So: - accept an optional parent parameter as parameter=None - pass that parameter to the superclass initialiser for it to handle - proceed with the rest of your initialiser Note: when you've got just only superclass, it is simpler to call it's initialiser like this: super().__init__(parent) which avoids having to hardwire knowledge of the superclass in this line of code. Your second example is less correct: class MyClass2(QWidget): def __init__(self, name): super(MyClass2, self).__init__() print(self.parent().test) This subclass _does not_ accept an optional parent parameter. The parent class accepts an optional parent parameter, as before, but you haven't got one to supply and because it is optional in QWidget, things still work. Usually if you're subclassing a single other class, you might try to have even less knowledge of the subperclass. Example: class MyClass1(QWidget): def __init__(self, name, **kw): super().__init__(**kw) ... do something with name ... Here you have a parameter you yourself expect to specially handle (name) and also accept arbitrary other keyword parameters, which you pass straight off to the superclass initialiser. You neither know nor care if a parent was supplied, but by passing all the remaining keywork parameters you support it anyway. If you have some special parameters of your own the pattern looks like this: class MyClass1(QWidget): def __init__(self, name, myparam1=None, **kw): super().__init__(**kw) ... do something with name ... ... do something with myparam1 ... Here your subclass handles name and myparam1, and still leaves the rest for the superclass. Cheers, Cameron Simpson From jamestztsai at gmail.com Fri Sep 2 18:28:39 2022 From: jamestztsai at gmail.com (James Tsai) Date: Fri, 2 Sep 2022 15:28:39 -0700 (PDT) Subject: Local variable definition in Python list comprehension In-Reply-To: References: Message-ID: <2e1a758c-b694-4076-ba97-0b105ffe4e7dn@googlegroups.com> ? 2022?9?2???? UTC+2 00:17:23? ??? > On 02Sep2022 07:01, Chris Angelico wrote: > >On Fri, 2 Sept 2022 at 06:55, James Tsai wrote: > >> No but very often when I have written a neat list/dict/set > >> comprehension, I find it very necessary > >> to define local variable(s) to make it more clear and concise. Otherwise I have to break it down > >> to several incrementally indented lines of for loops, if statements, and variable assignments, > >> which I think look less nice. > > > >Well, if it's outgrown a list comp, write it on multiple lines. Like I > >said, not everything has to be a one-liner. > True, but a comprehension can be more expressive than a less > "functional" expression (series of statements). > > James, can you provide (a) a real world example where you needed to > write a series of statements or loops and (b) a corresponding example of > how you would have preferred to have written that code, possibly > inventing some syntax or misusing ":=" as if it workeed they way you'd > like it to work? > > Cheers, > Cameron Simpson Yeah, I think list comprehension is particularly useful to construct a deeply nested list/dict. For example, I am now using Plotly to visualize a cellular network including several base stations and users. Here is the function I have written: def plot_network(area, base_stations, users): bs_poses = np.array([bs.pos for bs in base_stations]) ue_poses = np.array([ue.pos for ue in users]) fig = px.scatter(x=bs_poses[:, 0], y=bs_poses[:, 1]) fig.add_scatter(x=ue_poses[:, 0], y=ue_poses[:, 1]) fig.update_layout( xaxis=dict(range=[0, area[0]], nticks=5), yaxis=dict(range=[0, area[1]], nticks=5), shapes=[dict( type="circle", fillcolor="PaleTurquoise", x0=x-r, y0=y-r, x1=x+r, y1=y+r, hovertext=f"({x:.2f}, {y:.2f})", opacity=0.3 ) for bs in base_stations for x, y in [bs.pos] for r in [bs.cell_radius]], ) return fig Simply put, I want to scatter the BSs and users, and additionally I want to draw a big circle around each BS to represent its cell coverage. I can choose to write 'x0=bs.pos[0]-bs.cell_radius, y0=...' instead, but it becomes less concise, and if x, y, or r is the return value of a function instead of a property, it becomes more computationally expensive to repeat calling the function as well. I also can create the list of 'shapes' by appending to a list, like shapes = [] for bs in base_stations: x, y = bs.pos r = bs.cell_radius shapes.append(dict(...)) fig.update_layout( xaxis=dict(range=[0, area[0]], nticks=5), yaxis=dict(range=[0, area[1]], nticks=5), shapes=shapes ) But in my opinion this is much less concise. I think it looks better to create the list within the nested structure. So I totally agree that list comprehension adds much expressiveness in Python. I only wonder whether it is a good idea to introduce a specific syntax for local variable assignment in list comprehensions, instead of using "for r in [bs.cell_radius]". I am also surprised to know that the assignment operator ":=" in a list comprehension will assign a variable outside of the scope of the comprehension. I think it does not make sense since a list comprehension without a ":=" will never change name bindings outside itself. From drsalists at gmail.com Fri Sep 2 19:43:03 2022 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 2 Sep 2022 16:43:03 -0700 Subject: Local variable definition in Python list comprehension In-Reply-To: References: Message-ID: On Thu, Sep 1, 2022 at 9:16 AM Chris Angelico wrote: > On Fri, 2 Sept 2022 at 02:10, James Tsai wrote: > > > > Hello, > > > > I find it very useful if I am allowed to define new local variables in a > list comprehension. For example, I wish to have something like > > [(x, y) for x in range(10) for y := x ** 2 if x + y < 80], or > > [(x, y) for x in range(10) with y := x ** 2 if x + y < 80]. > > > > For now this functionality can be achieved by writing > > [(x, y) for x in range(10) for y in [x ** 2] if x + y < 80]. > > > > Is it worthwhile to add a new feature like this in Python? If so, how > can I propose this to PEP? > > Not everything has to be a one-liner. > So true! I like list comprehensions and generator expressions, but sometimes I end up regretting their use when there's a bug, and I have to convert one to a for loop + list.append in order to debug. From c.buhtz at posteo.jp Mon Sep 5 03:33:13 2022 From: c.buhtz at posteo.jp (c.buhtz at posteo.jp) Date: Mon, 05 Sep 2022 07:33:13 +0000 Subject: venv and packages with entry points Message-ID: <4cfe0453fba8f99f07ecdec72d591de0@posteo.de> Hello, I try to get it onto my head how virtual environments (via venv) works when I have packages with "entry points". I can define such entry points in the setup.cfg like this (full example [1]): [options.entry_points] console_scripts = hyperorg = hyperorg.__main__:main When I install such a package via pip the result is a shell script in /usr/bin/... What happens when I install such a package with an entry point via "pip install ." when an virtual environment is activated? Kind Christian [1] -- From alister.ware at ntlworld.com Sun Sep 4 15:43:15 2022 From: alister.ware at ntlworld.com (alister) Date: Sun, 4 Sep 2022 19:43:15 -0000 (UTC) Subject: Humour References: Message-ID: On Sun, 4 Sep 2022 02:08:20 -0700 (PDT), Ali Muhammad wrote: > Hi python devs it seems you do not have a sense of humour and I am here > to change that please I request to make it so on April 1st you change > the print function to a capital P this will be funny and people will use > language therefore you get more people more money more devs and happier > people in the world take action to solve depression. You may want to check the language History (unless you were deliberately taking the 'P' & I missed it ) :-) -- Hildebrant's Principle: If you don't know where you are going, any road will get you there. From mmontgomery at levado.to Sun Sep 4 15:47:07 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Sun, 04 Sep 2022 16:47:07 -0300 Subject: on GNU EMACS's python-mode, loading entire buffer References: <86y1v6zzf3.fsf@levado.to> <868rn4zg3m.fsf@levado.to> Message-ID: <86ilm2ncqs.fsf@levado.to> Meredith Montgomery writes: > Meredith Montgomery writes: > > [...] > >> I would also be interested in a command that restarts the REPL afresh >> and reloads my buffer --- sort of like keyboard's [F5] of the IDLE. > > A partial solution for this is the following procedure. > > (defun python-revert-and-send-buffer-to-repl () > "Revert current buffer and sends it to the Python REPL." > (interactive) > (revert-buffer "ignore-auto-no" "no-confirm") > (python-shell-send-buffer)) > > We can map this to the F5-key and that improves things. But a restart > of the REPL would be the ideal. (Sometimes we really want to start > afresh. Sometimes. Most often we don't want that.) It's not easy to restart the REPL. You can send "quit()" to it and invoke run-python again interactively by typing out one command after another, but if you write a procedure such as this one below, it doesn't work: it gives me the impression that there's a timing issue, that is, perhaps the procedure is too fast and something happens before it should. (defun python-save-send-buffer-to-repl () (interactive) (save-buffer) (python-shell-send-string "quit()") (run-python) (python-shell-send-buffer) (python-shell-switch-to-shell)) From mmontgomery at levado.to Sun Sep 4 17:05:31 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Sun, 04 Sep 2022 18:05:31 -0300 Subject: on implementing a toy oop-system Message-ID: <86r10qg89w.fsf@levado.to> Just for investigation sake, I'm trying to simulate OO-inheritance. (*) What did I do? I decided that objects would be dictionaries and methods would be procedures stored in the object. A class is just a procedure that creates such object-dictionary. So far so good. Trouble arrived when I decided to have inheritance. The problem might related to the lexical-nature of closures, but I'm not sure. (*) A problem Say I have a Counter-creating object procedure and a Car-creating object procedure. Cars want to track how many times they park, so they inherit a counter. --8<---------------cut here---------------start------------->8--- def Car(maker = None, color = None): o = inherit(Object(), Counter()) o["maker"] = maker o["color"] = color o["speed"] = 0 --8<---------------cut here---------------end--------------->8--- What Object() does is just to return a dictionary. What inherit() does is just to merge the two dictionaries into a single one (to be used by the Car procedure). This didn't come out as I expected, though. Let me show you step-by-step where the problem is. First, here's a counter in action. >>> c1 = Counter() >>> c1["inc"]()["inc"]() {'id': .id at 0x0000016547C648B0>, 'n': 2, [...]} >>> c1["n"] 2 That's good and expected: I incremented it twice. But look what happens with a Car's counter. >>> c = Car() >>> c = Car() >>> c {'id': .id at 0x0000016547C64B80>, 'n': 0, 'inc': .inc at 0x0000016547C64C10>, 'dec': .dec at 0x0000016547C64CA0>, 'maker': None, 'color': None, 'speed': 0, 'accelerate': .accelerate at 0x0000016547C64AF0>, 'park': .park at 0x0000016547C64D30>} >>> c["inc"]() {'id': .id at 0x0000016547C64B80>, 'n': 1, 'inc': .inc at 0x0000016547C64C10>, 'dec': .dec at 0x0000016547C64CA0>} We can see something got incremented! But... >>> c["n"] 0 Indeed, what got incremented is the dictionary attached to the /inc/ procedure of the Counter closure, so it's that dictionary that's being mutated. My /inherit/ procedure is not able to bring that procedure into the Car dictionary. Is that at all possible somehow? Alternatively, how would you do your toy oop-system? (*) Full code below from random import random def Object(): myid = random() def id(): return myid return { "id": id } def inherit(o, parentObject): o.update(parentObject) return o def Counter(begin_at = 0): o = Object() o["n"] = begin_at def inc(): nonlocal o o["n"] += 1 return o o["inc"] = inc def dec(): nonlocal o o["n"] -= 1 return o o["dec"] = dec return o def Car(maker = None, color = None): o = inherit(Object(), Counter()) o["maker"] = maker o["color"] = color o["speed"] = 0 def accelerate(speed): nonlocal o print(f"Car-{o['id']()}: accelerating to {speed}...") o["speed"] = speed return o o["accelerate"] = accelerate def park(): nonlocal o o["speed"] = 0 o["parked"] = True o["inc"]() print(f"Car-{o['id']()}: parked! ({o['n']} times)") return o o["park"] = park return o def tests(): c1 = Counter() c2 = Counter(100) c1["inc"]()["inc"]() c2["dec"]()["dec"]()["dec"]() print("c1 is 2:", c1["n"]) print("c2 is 97:", c2["n"]) car1 = Car("VW", "Red") car1["accelerate"](100) print("speed is 100:", car1["speed"]) car2 = Car("Ford", "Black") car2["accelerate"](120)["park"]() car2["accelerate"](50)["park"]() print("has parked 2 times:", car2["n"]) From mmontgomery at levado.to Mon Sep 5 13:07:12 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Mon, 05 Sep 2022 14:07:12 -0300 Subject: on implementing a toy oop-system References: <86r10qg89w.fsf@levado.to> Message-ID: <86leqxlphb.fsf@levado.to> ram at zedat.fu-berlin.de (Stefan Ram) writes: > Meredith Montgomery writes: >>Is that at all possible somehow? Alternatively, how would you do your >>toy oop-system? > > Maybe something along those lines: > > from functools import partial > > def counter_create( object ): > object[ "n" ]= 0 > def counter_increment( object ): > object[ "n" ]+= 1 > def counter_value( object ): > return object[ "n" ] > > counter_class =( counter_create, counter_increment, counter_value ) > > def inherit_from( class_, target ): > class_[ 0 ]( target ) > for method in class_[ 1: ]: > target[ method.__name__ ]= partial( method, target ) > > car = dict() > > inherit_from( counter_class, car ) > > print( car[ "counter_value" ]() ) > car[ "counter_increment" ]() > print( car[ "counter_value" ]() ) > > . The "create" part is simplified. I just wanted to show how > to make methods like "counter_increment" act on the object > that inherited them using "partial". That's simple and interesting. I'll study your more elaborate approach next, but I like what I see already. Thank you so much. From mmontgomery at levado.to Mon Sep 5 15:07:57 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Mon, 05 Sep 2022 16:07:57 -0300 Subject: on str.format and f-strings Message-ID: <868rmxljw2.fsf@levado.to> It seems to me that str.format is not completely made obsolete by the f-strings that appeared in Python 3.6. But I'm not thinking that this was the objective of the introduction of f-strings: the PEP at https://peps.python.org/pep-0498/#id11 says so explicitly. My question is whether f-strings can do the following nice thing with dictionaries that str.format can do: --8<---------------cut here---------------start------------->8--- def f(): d = { "name": "Meredith", "email": "mmontgomery at levado.to" } return "The name is {name} and the email is {email}".format(**d) --8<---------------cut here---------------end--------------->8--- Is there a way to do this with f-strings? From mmontgomery at levado.to Mon Sep 5 16:18:31 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Mon, 05 Sep 2022 17:18:31 -0300 Subject: on str.format and f-strings References: <868rmxljw2.fsf@levado.to> Message-ID: <8635d5lgmg.fsf@levado.to> ram at zedat.fu-berlin.de (Stefan Ram) writes: > Meredith Montgomery writes: > ... >> d = { "name": "Meredith", "email": "mmontgomery at levado.to" } >> return "The name is {name} and the email is {email}".format(**d) >>--8<---------------cut here---------------end--------------->8--- >>Is there a way to do this with f-strings? > > I cannot think of anything shorter now than: > > eval( 'f"The name is {name} and the email is {email}"', d ) > > , but with the spaces removed, it's even one character > shorter than the format expression: > > eval('f"The name is {name} and the email is {email}"',d) > "The name is {name} and the email is {email}".format(**d) > > . Lol. That's brilliant! Thanks very much! From mmontgomery at levado.to Mon Sep 5 18:59:28 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Mon, 05 Sep 2022 19:59:28 -0300 Subject: on the importance of exceptions Message-ID: <86o7vtjulr.fsf@levado.to> I'm trying to show people that exceptions are a very nice thing to have when it comes to detecting when something went awry somewhere. I'd like a real-world case, though. Here's what I'm sort of coming up with --- given my limited experience and imagination. Below, you have f calling g caling h calling j which returns True or False based on a random thing. (This simulates a task that sometimes succeeds and sometimes fails.) If while at f() I want to know what happened at j(), I'm going to have to propagate that information upwards through the execution stack. I can't do that right now: I'd have to change f, g and h. --8<---------------cut here---------------start------------->8--- from random import randint def f(): g() def g(): h() def h(): if j(): print("I got a 2.") else: print("I got a 1.") def j(): return randint(1,2) == 2 --8<---------------cut here---------------end--------------->8--- If instead of that, j() would be raising an exception when it fails, then all I only need to change f() to know what happens. I could replace j's test with opening a file, say. That would improve it a little. I'm sure you guys know excellent cases to show. Would you be so kind to share anything you might have in mind? From mmontgomery at levado.to Mon Sep 5 19:02:42 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Mon, 05 Sep 2022 20:02:42 -0300 Subject: on str.format and f-strings References: <868rmxljw2.fsf@levado.to> <8635d5lgmg.fsf@levado.to> <22d2ab37-8cfd-41d1-b7aa-7075661c8452n@googlegroups.com> Message-ID: <86czc9jugd.fsf@levado.to> Julio Di Egidio writes: > On Monday, 5 September 2022 at 22:18:58 UTC+2, Meredith Montgomery wrote: >> r... at zedat.fu-berlin.de (Stefan Ram) writes: > >> > , but with the spaces removed, it's even one character >> > shorter than the format expression: >> > >> > eval('f"The name is {name} and the email is {email}"',d) >> > "The name is {name} and the email is {email}".format(**d) >> >> Lol. That's brilliant! Thanks very much! > > Calling eval for that is like shooting a fly with a cannon. Indeed! But we're not looking for production-quality code. Just an extreme way to satisfy a silly requirement. > Besides, this one is even shorter: > > f"The name is {d['name']} and the email is {d['email']}" This doesn't quite satisfy the requeriments. We're trying to specify only the keys, not the dictionary. (But maybe the requirements did not say that explicitly. I'd have to look it up again --- it's been snipped. It's not important. Thanks much for your thoughts!) From mmontgomery at levado.to Mon Sep 5 21:07:25 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Mon, 05 Sep 2022 22:07:25 -0300 Subject: on str.format and f-strings References: <868rmxljw2.fsf@levado.to> <8635d5lgmg.fsf@levado.to> <22d2ab37-8cfd-41d1-b7aa-7075661c8452n@googlegroups.com> <86czc9jugd.fsf@levado.to> <83b659dd-546b-449a-9545-eef432dbdec6n@googlegroups.com> Message-ID: <86v8q1ia42.fsf@levado.to> Julio Di Egidio writes: > On Tuesday, 6 September 2022 at 01:03:02 UTC+2, Meredith Montgomery wrote: >> Julio Di Egidio writes: >> > On Monday, 5 September 2022 at 22:18:58 UTC+2, Meredith Montgomery wrote: >> >> r... at zedat.fu-berlin.de (Stefan Ram) writes: >> > >> >> > , but with the spaces removed, it's even one character >> >> > shorter than the format expression: >> >> > >> >> > eval('f"The name is {name} and the email is {email}"',d) >> >> > "The name is {name} and the email is {email}".format(**d) >> >> >> >> Lol. That's brilliant! Thanks very much! >> > >> > Calling eval for that is like shooting a fly with a cannon. >> >> Indeed! But we're not looking for production-quality code. Just an >> extreme way to satisfy a silly requirement. > > Indeed, as far as programming goes, even the premise is > totally nonsensical. Maybe you too better go to the pub? It surely isn't precise, but Stefan Ram caught my meaning. It's hard to be precise. I wanted to avoid having to write things like d['key']. Stefam Ram provided a solution. I did not care whether it was something sensible to do in Python from a serious-programming perspective. Thank you for thoughts anyhow. I appreciate it. From mmontgomery at levado.to Mon Sep 5 22:15:15 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Mon, 05 Sep 2022 23:15:15 -0300 Subject: on the importance of exceptions References: <86o7vtjulr.fsf@levado.to> Message-ID: <8635d5i6z0.fsf@levado.to> Meredith Montgomery writes: > I'm trying to show people that exceptions are a very nice thing to have > when it comes to detecting when something went awry somewhere. I'd like > a real-world case, though. Here's my contribution. I want to handle all errors in main() and the real job is done in does_a_job(), which, in turns, needs to delegate tasks to those other procedures that fail sometimes. It's does_a_job() that /must/ distinguish the error codes because errors come in as False from both opens_file() and reads_file(). So the checks must be done both in does_a_job() and in main(). (We also notice that does_a_job() has a return-type that's a union (sometimes an integer, sometimes a string), which makes distinguishing error code and success a bit harder.) --8<---------------cut here---------------start------------->8--- from random import randint def main(): r, data = does_a_job() if r < 0: if r == -1: print("Fatal. Could not open file.") return None if r == -2: print("Fatal. Could not read file") return None print(f"Great! We got the data: ``{r}''.") def does_a_job(): okay = opens_file() if not okay: return -1 okay, data = reads_file() if not okay: return -2 closes_file() return data def open_file(): # Opens okay with probability 80% return randint(1,10) <= 8 def read_file(): # Reads okay with probability 50% return randint(1,10) <= 5, "data I am" def closes_file(): # Works with probability 1 return True --8<---------------cut here---------------end--------------->8--- If we could give the program a final destination at does_a_job(), the job would be easier. But all we want to do in does_a_job() is to propagate the error conditions upwards to main() to really decide what to do. Exceptions lets us do that with a cleaner version. --8<---------------cut here---------------start------------->8--- from random import randint def main(): try: data = does_a_job() except FileNotFoundError: print("Fatal. Could not open file.") except MemoryError: print("Fatal. Could not read file") else: print(f"Great! We got the data: ``{data}''.") def does_a_job(): open_file() data = reads_file() close_file() return data def open_file(): # Opens okay with probability 80% if randint(1,10) <= 8: return True raise FileNotFoundError("Sorry: the file system is /so/ busy right now.") def reads_file(): # Reads okay with probability 50% if randint(1,10) <= 5: return "data I am" raise MemoryError("Sorry: not enough memory for /all/ this data.") def close_file(): # Works with probability 1 return True --8<---------------cut here---------------end--------------->8--- From mmontgomery at levado.to Mon Sep 5 22:22:34 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Mon, 05 Sep 2022 23:22:34 -0300 Subject: any author you find very good has written a book on Python? Message-ID: <86ilm1gs2d.fsf@levado.to> I never read a book on Python. I'm looking for a good one now. I just searched the web for names such as Charles Petzold, but it looks like he never wrote a book on Python. I also searched for Peter Seibel, but he also never did. I also tried to search for Richard Heathfield. (I took a look at his ``C Unleashed'' once and I liked what I saw.) This is how I search for books --- I go through the authors first. Charles Petzold, for instance, anything he writes is worth reading it. (Have you given his Annotated Turing a shot? It's a very nice read.) So that's my request --- any author you find very good has written a book on Python? It could be for in a certain specific context. For instance, I also searched for Hadley Wickham in the hope that he could have written a data-science-type of book using Python. I like his writing a lot, but he also only seems to have written only for the R language. Thank you! From mmontgomery at levado.to Tue Sep 6 11:36:10 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Tue, 06 Sep 2022 12:36:10 -0300 Subject: any author you find very good has written a book on Python? References: <86ilm1gs2d.fsf@levado.to> <87v8q1w3rd.fsf@nightsong.com> Message-ID: <86v8q0frbp.fsf@levado.to> Paul Rubin writes: > Meredith Montgomery writes: >> So that's my request --- any author you find very good has written a >> book on Python? > > The ones by David Beazley are great. Same with his non-book writings > about Python. See: http://dabeaz.com/ Distilled Python is looking really nice, actually. It seems so concise, so it looks like a really nice first read. Thank you for the recommendation. From cspealma at redhat.com Tue Sep 6 13:49:16 2022 From: cspealma at redhat.com (Calvin Spealman) Date: Tue, 6 Sep 2022 13:49:16 -0400 Subject: venv and packages with entry points In-Reply-To: <4cfe0453fba8f99f07ecdec72d591de0@posteo.de> References: <4cfe0453fba8f99f07ecdec72d591de0@posteo.de> Message-ID: Each virtual environment has its own bin/ directory and when activated its bin/ is in your $PATH On Tue, Sep 6, 2022 at 1:45 PM wrote: > Hello, > > I try to get it onto my head how virtual environments (via venv) works > when I have packages with "entry points". > > I can define such entry points in the setup.cfg like this (full example > [1]): > > [options.entry_points] > console_scripts = > hyperorg = hyperorg.__main__:main > > When I install such a package via pip the result is a shell script in > /usr/bin/... > > What happens when I install such a package with an entry point via "pip > install ." when an virtual environment is activated? > > Kind > Christian > > [1] -- > > -- > https://mail.python.org/mailman/listinfo/python-list > > -- CALVIN SPEALMAN SENIOR QUALITY ENGINEER calvin.spealman at redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] TRIED. TESTED. TRUSTED. From rosuav at gmail.com Tue Sep 6 13:57:03 2022 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 7 Sep 2022 03:57:03 +1000 Subject: on str.format and f-strings In-Reply-To: <868rmxljw2.fsf@levado.to> References: <868rmxljw2.fsf@levado.to> Message-ID: On Wed, 7 Sept 2022 at 03:52, Meredith Montgomery wrote: > > It seems to me that str.format is not completely made obsolete by the > f-strings that appeared in Python 3.6. But I'm not thinking that this > was the objective of the introduction of f-strings: the PEP at > > https://peps.python.org/pep-0498/#id11 > > says so explicitly. Precisely. It was never meant to obsolete str.format, and it does not. > My question is whether f-strings can do the > following nice thing with dictionaries that str.format can do: > > --8<---------------cut here---------------start------------->8--- > def f(): > d = { "name": "Meredith", "email": "mmontgomery at levado.to" } > return "The name is {name} and the email is {email}".format(**d) > --8<---------------cut here---------------end--------------->8--- > > Is there a way to do this with f-strings? No. That's not their job. That's str.format's job. ChrisA From list1 at tompassin.net Tue Sep 6 09:29:49 2022 From: list1 at tompassin.net (Thomas Passin) Date: Tue, 6 Sep 2022 09:29:49 -0400 Subject: Python-announce] ANN: GF4 Now Has A Plugin Capability Message-ID: A new plugin capability for the GF4 Waveform Calculator lets a Python file in the new plugins directory add a new command and command button. This is helpful for adding or developing new functionality. Basic information for writing and using plugins is included in the README file in the plugins directory. As always,the devel branch contains the most recent changes. There is also a new recent_changes.md file in the project root. GF4 is a Python program to display two-dimensional data, such as time series data, and to perform mathematical operations on the data or between two related data sets. The program aims to make data exploration easy and enjoyable. The project is hosted on Github: https://github.com/tbpassin/gf4-project/ From mmontgomery at levado.to Tue Sep 6 15:07:08 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Tue, 06 Sep 2022 16:07:08 -0300 Subject: on str.format and f-strings References: <868rmxljw2.fsf@levado.to> Message-ID: <86bkrsmieb.fsf@levado.to> Chris Angelico writes: > On Wed, 7 Sept 2022 at 03:52, Meredith Montgomery wrote: >> >> It seems to me that str.format is not completely made obsolete by the >> f-strings that appeared in Python 3.6. But I'm not thinking that this >> was the objective of the introduction of f-strings: the PEP at >> >> https://peps.python.org/pep-0498/#id11 >> >> says so explicitly. > > Precisely. It was never meant to obsolete str.format, and it does not. > >> My question is whether f-strings can do the >> following nice thing with dictionaries that str.format can do: >> >> --8<---------------cut here---------------start------------->8--- >> def f(): >> d = { "name": "Meredith", "email": "mmontgomery at levado.to" } >> return "The name is {name} and the email is {email}".format(**d) >> --8<---------------cut here---------------end--------------->8--- >> >> Is there a way to do this with f-strings? > > No. That's not their job. That's str.format's job. Chris! So good to see you around here again. Thank you so much for your input on this. From lkrupp at invalid.pssw.com.invalid Tue Sep 6 15:17:32 2022 From: lkrupp at invalid.pssw.com.invalid (Louis Krupp) Date: Tue, 6 Sep 2022 13:17:32 -0600 Subject: any author you find very good has written a book on Python? In-Reply-To: <86ilm1gs2d.fsf@levado.to> References: <86ilm1gs2d.fsf@levado.to> Message-ID: On 9/5/2022 8:22 PM, Meredith Montgomery wrote: > I never read a book on Python. I'm looking for a good one now. I just > searched the web for names such as Charles Petzold, but it looks like he > never wrote a book on Python. I also searched for Peter Seibel, but he > also never did. I also tried to search for Richard Heathfield. (I took > a look at his ``C Unleashed'' once and I liked what I saw.) This is how > I search for books --- I go through the authors first. Charles Petzold, > for instance, anything he writes is worth reading it. (Have you given > his Annotated Turing a shot? It's a very nice read.) > > So that's my request --- any author you find very good has written a > book on Python? > > It could be for in a certain specific context. For instance, I also > searched for Hadley Wickham in the hope that he could have written a > data-science-type of book using Python. I like his writing a lot, but > he also only seems to have written only for the R language. > > Thank you! I liked _Introducing Python_ by Bill Lubanovic. Louis From list1 at tompassin.net Tue Sep 6 16:05:57 2022 From: list1 at tompassin.net (Thomas Passin) Date: Tue, 6 Sep 2022 16:05:57 -0400 Subject: any author you find very good has written a book on Python? In-Reply-To: <86v8q0frbp.fsf@levado.to> References: <86ilm1gs2d.fsf@levado.to> <87v8q1w3rd.fsf@nightsong.com> <86v8q0frbp.fsf@levado.to> Message-ID: Mark Pilgram's "Dive Into Python" was good. Now he's updated it for Python 3: https://diveintopython3.net On 9/6/2022 11:36 AM, Meredith Montgomery wrote: > Paul Rubin writes: > >> Meredith Montgomery writes: >>> So that's my request --- any author you find very good has written a >>> book on Python? >> >> The ones by David Beazley are great. Same with his non-book writings >> about Python. See: http://dabeaz.com/ > > Distilled Python is looking really nice, actually. It seems so concise, > so it looks like a really nice first read. Thank you for the > recommendation. From hjp-python at hjp.at Tue Sep 6 17:48:28 2022 From: hjp-python at hjp.at (Peter J. Holzer) Date: Tue, 6 Sep 2022 23:48:28 +0200 Subject: any author you find very good has written a book on Python? In-Reply-To: <86ilm1gs2d.fsf@levado.to> References: <86ilm1gs2d.fsf@levado.to> Message-ID: <20220906214828.dcphdmxabqfoqvf7@hjp.at> On 2022-09-05 23:22:34 -0300, Meredith Montgomery wrote: > I never read a book on Python. I'm looking for a good one now. I just > searched the web for names such as Charles Petzold, but it looks like he > never wrote a book on Python. I also searched for Peter Seibel, but he > also never did. I also tried to search for Richard Heathfield. (I took > a look at his ``C Unleashed'' once and I liked what I saw.) This is how > I search for books --- I go through the authors first. Unfortunately I can't help you (never read a book on Python myself), but that seems like a weird way to search for a book on a language. I woudn't expect a single author to be able to write good books on more than a handful[1] of languages. Learning a language well takes time. Learning it so well that you can teach it well takes even longer. So I'd be quite ware of authors who write books on lots of different languages. hp [1] Possibly the hand of a carpenter. -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From jkn_gg at nicorp.f9.co.uk Tue Sep 6 17:07:26 2022 From: jkn_gg at nicorp.f9.co.uk (jkn) Date: Tue, 6 Sep 2022 14:07:26 -0700 (PDT) Subject: any author you find very good has written a book on Python? In-Reply-To: <86v8q0frbp.fsf@levado.to> References: <86ilm1gs2d.fsf@levado.to> <87v8q1w3rd.fsf@nightsong.com> <86v8q0frbp.fsf@levado.to> Message-ID: On Tuesday, September 6, 2022 at 4:36:38 PM UTC+1, Meredith Montgomery wrote: > Paul Rubin writes: > > > Meredith Montgomery writes: > >> So that's my request --- any author you find very good has written a > >> book on Python? > > > > The ones by David Beazley are great. Same with his non-book writings > > about Python. See: http://dabeaz.com/ > Distilled Python is looking really nice, actually. It seems so concise, > so it looks like a really nice first read. Thank you for the > recommendation. I concur with Paul's general recommendation of David Beazley's work. I bought a copy of Python Distilled recently, having 'grown up' with editions of his earlier 'Python Essential Reference', going back to the first edition (Python 1.5?) I confess to being slightly disappointed with 'Python Distilled', but I was probably expecting something that I shouldn't have. It is basically a relatively fast-paced introduction to 'modern' python, stripping down some of the fine detail that the 'Essential Reference' books leave in. I am not 100% sure how useful it would be for relative beginners; it depends what you are looking for. As a reference to functions and library usage etc., the essential reference books are (still) great, and cheap via eBay. As a stepping stone from 'fluent beginner', it might well be perfect. As a hand-holding learning guide, maybe not so great. I'm by no means trying to diss Beazley's work, I think it is great; just trying to indicate what you get for your money, and maybe the target audience. J^n From jkn_gg at nicorp.f9.co.uk Tue Sep 6 17:10:18 2022 From: jkn_gg at nicorp.f9.co.uk (jkn) Date: Tue, 6 Sep 2022 14:10:18 -0700 (PDT) Subject: any author you find very good has written a book on Python? In-Reply-To: References: <86ilm1gs2d.fsf@levado.to> <87v8q1w3rd.fsf@nightsong.com> <86v8q0frbp.fsf@levado.to> Message-ID: On Tuesday, September 6, 2022 at 9:06:31 PM UTC+1, Thomas Passin wrote: > Mark Pilgram's "Dive Into Python" was good. Now he's updated it for > Python 3: like, about ten years ago? (I think Mark Pilgrim dropped off the 'net many years ago...) > https://diveintopython3.net > On 9/6/2022 11:36 AM, Meredith Montgomery wrote: > > Paul Rubin writes: > > > >> Meredith Montgomery writes: > >>> So that's my request --- any author you find very good has written a > >>> book on Python? > >> > >> The ones by David Beazley are great. Same with his non-book writings > >> about Python. See: http://dabeaz.com/ > > > > Distilled Python is looking really nice, actually. It seems so concise, > > so it looks like a really nice first read. Thank you for the > > recommendation. From avi.e.gross at gmail.com Tue Sep 6 19:41:58 2022 From: avi.e.gross at gmail.com (avi.e.gross at gmail.com) Date: Tue, 6 Sep 2022 19:41:58 -0400 Subject: any author you find very good has written a book on Python? In-Reply-To: <86ilm1gs2d.fsf@levado.to> References: <86ilm1gs2d.fsf@levado.to> Message-ID: <010d01d8c24a$41c752e0$c555f8a0$@gmail.com> Subject: searching for books by an author you like on rather unrelated topics. I am curious if you normally look or books by a writer of Mysteries you like to see if they also wrote Science Fiction or Cookbooks and so on? Having said that, there are plenty of people in the Computer Science field who are quite multi-lingual and may know one or a few "languages" intimately and quite a bit about others and at least have heard of many more. Most such people never write a single book of the kind you are looking for. However, once someone does write some introductory book on some language or other system, sometimes with co-authors, some do indeed proceed to write additional books albeit not always of high quality. I have seen people who say were quite familiar with a language like C then turn around and try to teach languages like Python or R with an emphasis on doing things using similar methods like using explicit loops where others might use vectorized operations or comprehensions. I think the method of selecting an author is a bit flawed as a concept but not irrelevant. An author for an introductory textbook for non-programmers might not do as well in another book about the same language made for programmers who already can program in one or more other languages and mainly want to know how this language differs from others. And certainly they may not do well writing about more detailed or sophisticated aspects of the language. But the opposite is true. Many "experts" are horrible at explaining simpler things at the right level for newbies. You, do sound like you know something about programming in one or more other languages and simply want to add-on Python. There are quite a few books available including some older and some recent. What you likely should prioritize is newer books that focus over post-version2 as that is supposed to no longer be used when possible. Then you need to figure out if you are just curious or want a job using it and so on. One possibility is to visit a library (or online e-books) and flip through pages. Often you may find an earlier edition and after some perusal, consider getting an updated recent version of that book by the same author. I have read dozens of books on Python but leave recommendations to others as each is different and I wanted to see many sides. -----Original Message----- From: Python-list On Behalf Of Meredith Montgomery Sent: Monday, September 5, 2022 10:23 PM To: python-list at python.org Subject: any author you find very good has written a book on Python? I never read a book on Python. I'm looking for a good one now. I just searched the web for names such as Charles Petzold, but it looks like he never wrote a book on Python. I also searched for Peter Seibel, but he also never did. I also tried to search for Richard Heathfield. (I took a look at his ``C Unleashed'' once and I liked what I saw.) This is how I search for books --- I go through the authors first. Charles Petzold, for instance, anything he writes is worth reading it. (Have you given his Annotated Turing a shot? It's a very nice read.) So that's my request --- any author you find very good has written a book on Python? It could be for in a certain specific context. For instance, I also searched for Hadley Wickham in the hope that he could have written a data-science-type of book using Python. I like his writing a lot, but he also only seems to have written only for the R language. Thank you! -- https://mail.python.org/mailman/listinfo/python-list From list1 at tompassin.net Tue Sep 6 18:23:19 2022 From: list1 at tompassin.net (Thomas Passin) Date: Tue, 6 Sep 2022 18:23:19 -0400 Subject: any author you find very good has written a book on Python? In-Reply-To: References: <86ilm1gs2d.fsf@levado.to> <87v8q1w3rd.fsf@nightsong.com> <86v8q0frbp.fsf@levado.to> Message-ID: On 9/6/2022 5:10 PM, jkn wrote: > On Tuesday, September 6, 2022 at 9:06:31 PM UTC+1, Thomas Passin wrote: >> Mark Pilgram's "Dive Into Python" was good. Now he's updated it for >> Python 3: > > like, about ten years ago? (I think Mark Pilgrim dropped off the 'net > many years ago...) Yes, I thought so too, but I just found the website and it's operating. Could be for all I know that the Python3 version is ten years old, but the book should still be worthwhile. > >> https://diveintopython3.net >> On 9/6/2022 11:36 AM, Meredith Montgomery wrote: >>> Paul Rubin writes: >>> >>>> Meredith Montgomery writes: >>>>> So that's my request --- any author you find very good has written a >>>>> book on Python? >>>> >>>> The ones by David Beazley are great. Same with his non-book writings >>>> about Python. See: http://dabeaz.com/ >>> >>> Distilled Python is looking really nice, actually. It seems so concise, >>> so it looks like a really nice first read. Thank you for the >>> recommendation. From wlfraed at ix.netcom.com Tue Sep 6 20:35:45 2022 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Tue, 06 Sep 2022 20:35:45 -0400 Subject: any author you find very good has written a book on Python? References: <86ilm1gs2d.fsf@levado.to> Message-ID: <4jpfhhhuhefvnan6oe656ife2okp34kom0@4ax.com> On Mon, 05 Sep 2022 23:22:34 -0300, Meredith Montgomery declaimed the following: >I never read a book on Python. I'm looking for a good one now. I just >searched the web for names such as Charles Petzold, but it looks like he So far as I know, Petzold is a Windows Internals type person. Python is not a M$ product (even if they stuff it into their Win10 "app store" and is an option in Visual Studio. Searching for Python books using authors that may or may not have ever seen Python seems futile... Many of my Python books are O'Reilly publications, with specialized books from Packt and APress. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From lukasz at langa.pl Wed Sep 7 09:12:53 2022 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Wed, 7 Sep 2022 15:12:53 +0200 Subject: [RELEASE] Python versions 3.10.7, 3.9.14, 3.8.14, 3.7.14 now available with security content Message-ID: <0C5EFF1A-4D43-4127-84C3-BFE292233616@langa.pl> We have some security content, and plenty of regular bug fixes for 3.10. Let?s dive right in. CVE-2020-10735 Converting between int and str in bases other than 2 (binary), 4, 8 (octal), 16 (hexadecimal), or 32 such as base 10 (decimal) now raises a ValueError if the number of digits in string form is above a limit to avoid potential denial of service attacks due to the algorithmic complexity. Security releases for 3.9.14, 3.8.14, and 3.7.14 are made available simultaneously to address this issue, along with some less urgent security content. Upgrading your installations is highly recommended. Python 3.10.7 Get it here: https://www.python.org/downloads/release/python-3107/ This bugfix version of Python was released out-of-schedule to address the CVE, and as such contains a smaller number of changes compared to 3.10.6 (200 commits), or in fact 3.9.7 (187 commits) at the same stage of the release cycle a year ago. But there?s still over a 100 commits in this latest Python version so it?s worth checking out the change log . And now for something completely different In quantum mechanics, the uncertainty principle (also known as Heisenberg?s uncertainty principle) is any of a variety of mathematical inequalities asserting a fundamental limit to the accuracy with which the values for certain pairs of physical quantities of a particle, such as position and momentum or the time and the energy can be predicted from initial conditions. Such variable pairs are known as complementary variables or canonically conjugate variables; and, depending on interpretation, the uncertainty principle limits to what extent such conjugate properties maintain their approximate meaning, as the mathematical framework of quantum physics does not support the notion of simultaneously well-defined conjugate properties expressed by a single value. The uncertainty principle implies that it is in general not possible to predict the value of a quantity with arbitrary certainty, even if all initial conditions are specified. We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. Your friendly release team, Ned Deily @nad Steve Dower @steve.dower Pablo Galindo Salgado @pablogsal ?ukasz Langa @ambv -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From mmontgomery at levado.to Tue Sep 6 21:36:32 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Tue, 06 Sep 2022 22:36:32 -0300 Subject: any author you find very good has written a book on Python? References: <86ilm1gs2d.fsf@levado.to> <87v8q1w3rd.fsf@nightsong.com> <86v8q0frbp.fsf@levado.to> Message-ID: <86fsh4j78f.fsf@levado.to> jkn writes: > On Tuesday, September 6, 2022 at 4:36:38 PM UTC+1, Meredith Montgomery wrote: >> Paul Rubin writes: >> >> > Meredith Montgomery writes: >> >> So that's my request --- any author you find very good has written a >> >> book on Python? >> > >> > The ones by David Beazley are great. Same with his non-book writings >> > about Python. See: http://dabeaz.com/ >> Distilled Python is looking really nice, actually. It seems so concise, >> so it looks like a really nice first read. Thank you for the >> recommendation. > > I concur with Paul's general recommendation of David Beazley's work. > I bought a copy of Python Distilled recently, having 'grown up' with > editions of his earlier 'Python Essential Reference', going back to > the first edition (Python 1.5?) > > I confess to being slightly disappointed with 'Python Distilled', but > I was probably expecting something that I shouldn't have. It is > basically a relatively fast-paced introduction to 'modern' python, > stripping down some of the fine detail that the 'Essential Reference' > books leave in. > > I am not 100% sure how useful it would be for relative beginners; it > depends what you are looking for. As a reference to functions and > library usage etc., the essential reference books are (still) great, > and cheap via eBay. As a stepping stone from 'fluent beginner', it > might well be perfect. As a hand-holding learning guide, maybe not so > great. > > I'm by no means trying to diss Beazley's work, I think it is great; > just trying to indicate what you get for your money, and maybe the > target audience. You got a point. It's so concise that maybe it's too concise for a beginner. But I will take my chances. I liked it. From * at eli.users.panix.com Wed Sep 7 00:39:43 2022 From: * at eli.users.panix.com (Eli the Bearded) Date: Wed, 7 Sep 2022 04:39:43 -0000 (UTC) Subject: -ffast-math Message-ID: https://twitter.com/moyix/status/1567167774039973888 Brendan Dolan-Gavitt @moyix New blog post is live! In which I download 4 TB of Python packages containing native x86-64 libraries and see how many of them use -ffast-math, potentially altering floating point behavior in any program unlucky enough to load them! https://moyix.blogspot.com/2022/09/someones-been-messing-with-my-subnormals.html 8:08 AM - Sep 6, 2022 It's quite an adventure, a longish read but fun. Elijah ------ TL;DR: two dependencies using same misguided makefile cause most issues From maruf.gibl at gmail.com Wed Sep 7 06:28:57 2022 From: maruf.gibl at gmail.com (Maruful Islam) Date: Wed, 7 Sep 2022 03:28:57 -0700 (PDT) Subject: Question about learning Python Message-ID: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> I want to start learning python. I have a question about learning python. Is learning C essential or not for learning python? From duane.kaufman at gmail.com Wed Sep 7 08:04:20 2022 From: duane.kaufman at gmail.com (TheSeeker) Date: Wed, 7 Sep 2022 05:04:20 -0700 (PDT) Subject: any author you find very good has written a book on Python? In-Reply-To: <86ilm1gs2d.fsf@levado.to> References: <86ilm1gs2d.fsf@levado.to> Message-ID: <24ec187b-05fd-45af-9968-8ee8c4feaf9cn@googlegroups.com> On Monday, September 5, 2022 at 9:23:04 PM UTC-5, Meredith Montgomery wrote: > I never read a book on Python. I'm looking for a good one now. I just > searched the web for names such as Charles Petzold, but it looks like he > never wrote a book on Python. I also searched for Peter Seibel, but he > also never did. I also tried to search for Richard Heathfield. (I took > a look at his ``C Unleashed'' once and I liked what I saw.) This is how > I search for books --- I go through the authors first. Charles Petzold, > for instance, anything he writes is worth reading it. (Have you given > his Annotated Turing a shot? It's a very nice read.) > > So that's my request --- any author you find very good has written a > book on Python? > > It could be for in a certain specific context. For instance, I also > searched for Hadley Wickham in the hope that he could have written a > data-science-type of book using Python. I like his writing a lot, but > he also only seems to have written only for the R language. > > Thank you! Mike Driscoll (https://www.blog.pythonlibrary.org/about/) as written a number of books for different Python experience levels, which I have found useful. -Duane Kaufman From sandro at volery.com Wed Sep 7 10:13:05 2022 From: sandro at volery.com (Sandro Volery) Date: Wed, 7 Sep 2022 07:13:05 -0700 (PDT) Subject: Question about learning Python In-Reply-To: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> Message-ID: Hey Maruf > I want to start learning python. Good for you! Fun times ahead. > Is learning C essential or not for learning python? No, I would not say that learning C is essential for learning Python. However, C can serve as a great set of fundamentials in programming and understanding machines on a low level, such as memory management, etc., which all had to be done manually at this time. It also seems important to note the differences between execution on runtime, and compiled languages, but there is many videos / articles that do a better job at explaining it than it do. Best, Sandro From lal at solute.de Wed Sep 7 11:58:54 2022 From: lal at solute.de (Lars Liedtke) Date: Wed, 7 Sep 2022 17:58:54 +0200 Subject: Question about learning Python In-Reply-To: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> Message-ID: <261635f9-4ff8-ae68-e524-93ad5e2fb9a4@solute.de> Hello and welcome, the answer is a definitive "it depends" ;-) Generally you do not need knowledge in C for learning Python. But I'd say that it will not hurt to have some knowledge. Especially some packages use C-code to extend Python. But it seems to me that you are completely starting to learn how to program. Please correct me if I am wrong. So you can definitely learn how to program with learning Python and you can learn C afterwards if you need to. Cheers Lars -- Lars Liedtke Software Entwickler Phone: Fax: +49 721 98993- E-mail: lal at solute.de solute GmbH Zeppelinstra?e 15 76185 Karlsruhe Germany Marken der solute GmbH | brands of solute GmbH billiger.de | Shopping.de Gesch?ftsf?hrer | Managing Director: Dr. Thilo Gans, Bernd Vermaaten Webseite | www.solute.de Sitz | Registered Office: Karlsruhe Registergericht | Register Court: Amtsgericht Mannheim Registernummer | Register No.: HRB 110579 USt-ID | VAT ID: DE234663798 Informationen zum Datenschutz | Information about privacy policy http://solute.de/ger/datenschutz/grundsaetze-der-datenverarbeitung.php Am 07.09.22 um 12:28 schrieb Maruful Islam: > I want to start learning python. I have a question about learning python. > > Is learning C essential or not for learning python? From list1 at tompassin.net Wed Sep 7 12:33:27 2022 From: list1 at tompassin.net (Thomas Passin) Date: Wed, 7 Sep 2022 12:33:27 -0400 Subject: Question about learning Python In-Reply-To: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> Message-ID: On 9/7/2022 6:28 AM, Maruful Islam wrote: > I want to start learning python. I have a question about learning python. > > Is learning C essential or not for learning python? Not at all. From info at tundraware.com Wed Sep 7 13:39:25 2022 From: info at tundraware.com (Tim Daneliuk) Date: Wed, 7 Sep 2022 12:39:25 -0500 Subject: any author you find very good has written a book on Python? In-Reply-To: <86ilm1gs2d.fsf@levado.to> References: <86ilm1gs2d.fsf@levado.to> Message-ID: On 9/5/22 21:22, Meredith Montgomery wrote: > I never read a book on Python. I'm looking for a good one now. I just > searched the web for names such as Charles Petzold, but it looks like he > never wrote a book on Python. I also searched for Peter Seibel, but he > also never did. I also tried to search for Richard Heathfield. (I took > a look at his ``C Unleashed'' once and I liked what I saw.) This is how > I search for books --- I go through the authors first. Charles Petzold, > for instance, anything he writes is worth reading it. (Have you given > his Annotated Turing a shot? It's a very nice read.) > > So that's my request --- any author you find very good has written a > book on Python? > > It could be for in a certain specific context. For instance, I also > searched for Hadley Wickham in the hope that he could have written a > data-science-type of book using Python. I like his writing a lot, but > he also only seems to have written only for the R language. > > Thank you! David Beazley From rosuav at gmail.com Wed Sep 7 13:57:22 2022 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 8 Sep 2022 03:57:22 +1000 Subject: Question about learning Python In-Reply-To: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> Message-ID: On Thu, 8 Sept 2022 at 01:50, Maruful Islam wrote: > > I want to start learning python. I have a question about learning python. > > Is learning C essential or not for learning python? Absolutely not essential. In fact, I would strongly recommend learning Python before ever picking up C, as it's much easier to mess around in Python. Learning C will definitely help you to become a better programmer, but you can leave it until later. ChrisA From grant.b.edwards at gmail.com Wed Sep 7 14:53:37 2022 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 07 Sep 2022 11:53:37 -0700 (PDT) Subject: Question about learning Python References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> Message-ID: <6318e8b1.920a0220.5aa5e.017a@mx.google.com> On 2022-09-07, Chris Angelico wrote: > On Thu, 8 Sept 2022 at 01:50, Maruful Islam wrote: >> >> I want to start learning python. I have a question about learning python. >> >> Is learning C essential or not for learning python? > > Absolutely not essential. In fact, I would strongly recommend learning > Python before ever picking up C, as it's much easier to mess around in > Python. If you're a beginning programmer, then IMO learning C first is probably detrimental. C has quite a few quirks and pitfalls that will a) frustrate you and waste time, and b) influence your way of thinking about programs in a way that will be unhelpful for higher level languages. > Learning C will definitely help you to become a better programmer, but > you can leave it until later. Unless it's to work on a real project that's written in C, I'd put C a fair ways down the list of languages to learn. I'd probably put Python and then Scheme at the top of the list. Then maybe Smalltalk and something a bit more functional. -- Grant From rosuav at gmail.com Wed Sep 7 14:57:38 2022 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 8 Sep 2022 04:57:38 +1000 Subject: Question about learning Python In-Reply-To: <6318e8b1.920a0220.5aa5e.017a@mx.google.com> References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> Message-ID: On Thu, 8 Sept 2022 at 04:54, Grant Edwards wrote: > > On 2022-09-07, Chris Angelico wrote: > > On Thu, 8 Sept 2022 at 01:50, Maruful Islam wrote: > >> > >> I want to start learning python. I have a question about learning python. > >> > >> Is learning C essential or not for learning python? > > > > Absolutely not essential. In fact, I would strongly recommend learning > > Python before ever picking up C, as it's much easier to mess around in > > Python. > > If you're a beginning programmer, then IMO learning C first is > probably detrimental. C has quite a few quirks and pitfalls that will > a) frustrate you and waste time, and b) influence your way of thinking > about programs in a way that will be unhelpful for higher level > languages. Not as detrimental as starting with BASIC, and then moving on to x86 assembly language, and trying to massage the two together using CALL ABSOLUTE in order to get mouse input in your GW-BASIC programs. Don't be me, folks. ChrisA From grant.b.edwards at gmail.com Wed Sep 7 15:07:47 2022 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Wed, 07 Sep 2022 12:07:47 -0700 (PDT) Subject: Question about learning Python References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> Message-ID: <6318ec03.050a0220.865c7.049d@mx.google.com> On 2022-09-07, Chris Angelico wrote: > On Thu, 8 Sept 2022 at 04:54, Grant Edwards wrote: > >> If you're a beginning programmer, then IMO learning C first is >> probably detrimental. [...] > > Not as detrimental as starting with BASIC, and then moving on to x86 > assembly language, and trying to massage the two together using CALL > ABSOLUTE in order to get mouse input in your GW-BASIC programs. Ah the "good old days". From rosuav at gmail.com Wed Sep 7 15:15:09 2022 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 8 Sep 2022 05:15:09 +1000 Subject: Question about learning Python In-Reply-To: <6318ec03.050a0220.865c7.049d@mx.google.com> References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> <6318ec03.050a0220.865c7.049d@mx.google.com> Message-ID: On Thu, 8 Sept 2022 at 05:09, Grant Edwards wrote: > > On 2022-09-07, Chris Angelico wrote: > > On Thu, 8 Sept 2022 at 04:54, Grant Edwards wrote: > > > >> If you're a beginning programmer, then IMO learning C first is > >> probably detrimental. [...] > > > > Not as detrimental as starting with BASIC, and then moving on to x86 > > assembly language, and trying to massage the two together using CALL > > ABSOLUTE in order to get mouse input in your GW-BASIC programs. > > Ah the "good old days". > Indeed. The 1990s gave me all manner of skills, including the aforementioned mouse control in a BASIC program, writing a Terminate-and-Stay-Resident program that hooks an interrupt, tricks for *not* writing a TSR and still acting like one, building GUIs using pixel precision, building GUIs using pixel precision but fully automatically, using HTML tables to create layouts.... oh, yes, so many skills... To anyone suffering from https://xkcd.com/1479/ right now, I can assure you, quite a lot of that knowledge DOES eventually become obsolete when better methods come along. It just sometimes takes a decade or more. (And then occasionally it still haunts you. I'm finding table-based layouts in a site that I now have to manage. Eventually I'll fix it all, eventually....) ChrisA From PythonList at DancesWithMice.info Wed Sep 7 15:42:19 2022 From: PythonList at DancesWithMice.info (dn) Date: Thu, 8 Sep 2022 07:42:19 +1200 Subject: Question about learning Python In-Reply-To: References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> <6318ec03.050a0220.865c7.049d@mx.google.com> Message-ID: <74172380-1c23-333b-51f4-6418dd6de2e1@DancesWithMice.info> On 08/09/2022 07.15, Chris Angelico wrote: > On Thu, 8 Sept 2022 at 05:09, Grant Edwards wrote: >> >> On 2022-09-07, Chris Angelico wrote: >>> On Thu, 8 Sept 2022 at 04:54, Grant Edwards wrote: >>> >>>> If you're a beginning programmer, then IMO learning C first is >>>> probably detrimental. [...] >>> >>> Not as detrimental as starting with BASIC, and then moving on to x86 >>> assembly language, and trying to massage the two together using CALL >>> ABSOLUTE in order to get mouse input in your GW-BASIC programs. >> >> Ah the "good old days". >> > > Indeed. The 1990s gave me all manner of skills, including the > aforementioned mouse control in a BASIC program, writing a > Terminate-and-Stay-Resident program that hooks an interrupt, tricks > for *not* writing a TSR and still acting like one, building GUIs using > pixel precision, building GUIs using pixel precision but fully > automatically, using HTML tables to create layouts.... oh, yes, so > many skills... To anyone suffering from https://xkcd.com/1479/ right > now, I can assure you, quite a lot of that knowledge DOES eventually > become obsolete when better methods come along. It just sometimes > takes a decade or more. > > (And then occasionally it still haunts you. I'm finding table-based > layouts in a site that I now have to manage. Eventually I'll fix it > all, eventually....) OP: Python! Python has become one of the most popular first-languages to use in universities (etc). On-the-ground this varies by country, even by province/state. However, starting at a higher-level is recommendable - and should the learner decide that 'this computer stuff is not for me' (XKCD not withstanding) then the cost of effort-expended will be less. Also, there are are plenty of coders 'out there' who don't seem to have learned, or even need, the detail one acquires using a lower-level language. (no further comment on that!) TSRs? Now that was an ugly period of history! (trying to make a single-process operating system do multi-processing - only to find that many program[me]s assumed they had full use and undisputed control of the computer. Happy days...) History has its value. Talking to a group the other day, showed how IT-skills from patterns (eg Factory, Strategy, Decorator) and paradigms (eg Modular Programming, Structured Programming) through to Architectural Principles (eg SOLID) and project management approaches (eg Waterfall, Agile, SCRUM) all descend from hard-won knowledge and sometimes bitter-experience. Chunks of which pre-date Dartmouth BASIC, PCs, mini-computers, and 'family'/standardised-hardware operating systems! On the other hand, one can start too 'high' or too 'modern'. Like the person enthusing about MSFT's and AWS' programming AIs, thinking that such tools will replace programmers (one of the aims of the COBOL language back in the 1960s). His short-form description spoke volumes: 'it saves anyone from having to look-up Stack Overflow any more' - a 'blind' cut-and-paste prospect that saves the 'author' from the difficulties of 'learning stuff'; until it is time to, um, learn-stuff - to know why one needs to learn-stuff BEFORE taking from SO/AI. -- Regards, =dn From mmontgomery at levado.to Wed Sep 7 15:35:26 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Wed, 07 Sep 2022 16:35:26 -0300 Subject: Question about learning Python References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> Message-ID: <861qsnhta9.fsf@levado.to> Maruful Islam writes: > I want to start learning python. I have a question about learning python. > > Is learning C essential or not for learning python? Surely not necessary. There's a generous recent thread about books on Python. Have a look at it. From avi.e.gross at gmail.com Wed Sep 7 20:15:13 2022 From: avi.e.gross at gmail.com (avi.e.gross at gmail.com) Date: Wed, 7 Sep 2022 20:15:13 -0400 Subject: Question about learning Python In-Reply-To: <861qsnhta9.fsf@levado.to> References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <861qsnhta9.fsf@levado.to> Message-ID: <009e01d8c318$1103e910$330bbb30$@gmail.com> Maybe we should ask WHY the person asking the question about how to learn a computer language called Python is pairing it with the idea of whether to also learn C. What are they preparing for? Most people using Python have absolutely no reason to learn C, or C++ or C# or JAVA or one of a bewildering number of other languages. They may end up using functions that unknown to them have been partially implemented using libraries created in C or other languages but so what? They never see that except if they want to look at source code. So are they looking for a job that will require not just Python but also C? If anything, it is an impediment for many people to learn two rather different languages at about the same time as many things are very different and lessons in one often do not carry over well to the other. As examples, C uses braces to group things and Python uses indentation and C tends to need things like data types spelled out and Python mostly does not care what type it is. Now if a question is asked about how to learn a relatively few languages with different paradigms that represent an assortment of ways to think about and solve problems, sure, C might take a slot. But arguably you may want something more modern that is somewhat descended from C as a compiled language. If your goal is to use Python in ways it is designed to be used, learning C first may spoil your thinking and you may implement algorithms in Python that loosely translate the way it is done in C but can be done much nicer using Python costructs. -----Original Message----- From: Python-list On Behalf Of Meredith Montgomery Sent: Wednesday, September 7, 2022 3:35 PM To: python-list at python.org Subject: Re: Question about learning Python Maruful Islam writes: > I want to start learning python. I have a question about learning python. > > Is learning C essential or not for learning python? Surely not necessary. There's a generous recent thread about books on Python. Have a look at it. -- https://mail.python.org/mailman/listinfo/python-list From avi.e.gross at gmail.com Wed Sep 7 20:32:11 2022 From: avi.e.gross at gmail.com (avi.e.gross at gmail.com) Date: Wed, 7 Sep 2022 20:32:11 -0400 Subject: Question about learning Python In-Reply-To: References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> Message-ID: <00a001d8c31a$6fca13f0$4f5e3bd0$@gmail.com> Chris, I started with BASIC in high school and kept shifting my focus from one computer language to another long before I even looked at Python. Arguably each language had a REASON for existing so it supported some ideas or paradigms or ways of looking at things. Many at first were rather focused on doing one or a few things well, and others not so well or not at all. No need to rehash it. In a sense, many modern languages can be viewed as containing multiple overlapping modules which collectively do all kinds of things in many ways and can do whatever many earlier initial releases of earlier languages did, sometimes better but also sometimes slower. If you look at Python as a base and added modules, it can do all the mathematical things of say FORTRAN, manipulate text and produce reports and on and on. But it also can NOT do things done in basic, or at least not easily, like subroutines and goto's that are associated with a more primitive way to solve problems. Is there any purpose in teaching new students how to structure your code so that you use such methods? Well, maybe. I mean we have all kinds of slightly hidden ways to do a GOTO in some languages such as breaking out of a nested loop. Underneath it all, an IF/THEN/ELSE type of statement has regions you jump between. But arguably it is helpful to see operations on a higher level and leave the details of how it is accomplished in well tested code within the interpreter or compiler. C was designed on purpose to do some things that a language like PASCAL was designed not to allow or make easy. Both started off though as fairly simple languages that did things somewhat linearly. You can obviously emulate many things using a simplified subset of Python that would allow programs in those languages to be done. If that is all you want to learn, fine. But if your goal is to make use of OO and functional programming and other paradigms supported, ... As I posted elsewhere, I ask why the questioner specifically mentioned C. I also use R and there too, C is only needed if you want to speed up some function by writing parts in their version of C or C++. The fact that many interpreters are written (or sometimes were written) in C is not really relevant. -----Original Message----- From: Python-list On Behalf Of Chris Angelico Sent: Wednesday, September 7, 2022 2:58 PM To: python-list at python.org Subject: Re: Question about learning Python On Thu, 8 Sept 2022 at 04:54, Grant Edwards wrote: > > On 2022-09-07, Chris Angelico wrote: > > On Thu, 8 Sept 2022 at 01:50, Maruful Islam wrote: > >> > >> I want to start learning python. I have a question about learning python. > >> > >> Is learning C essential or not for learning python? > > > > Absolutely not essential. In fact, I would strongly recommend > > learning Python before ever picking up C, as it's much easier to > > mess around in Python. > > If you're a beginning programmer, then IMO learning C first is > probably detrimental. C has quite a few quirks and pitfalls that will > a) frustrate you and waste time, and b) influence your way of thinking > about programs in a way that will be unhelpful for higher level > languages. Not as detrimental as starting with BASIC, and then moving on to x86 assembly language, and trying to massage the two together using CALL ABSOLUTE in order to get mouse input in your GW-BASIC programs. Don't be me, folks. ChrisA -- https://mail.python.org/mailman/listinfo/python-list From jaapoort at gmail.com Thu Sep 8 09:51:51 2022 From: jaapoort at gmail.com (Jan Poort) Date: Thu, 8 Sep 2022 15:51:51 +0200 Subject: Fwd: In-Reply-To: References: Message-ID: error by starting up windows 10 i get this message ---------- Forwarded message --------- Van: Jan Poort Date: do 8 sep. 2022 om 15:49 Subject: To: -- *Jan Poort* From estates at hestates.org Wed Sep 7 21:34:13 2022 From: estates at hestates.org (Fulian Wang) Date: Thu, 8 Sep 2022 01:34:13 +0000 Subject: Question about learning Python In-Reply-To: <00a001d8c31a$6fca13f0$4f5e3bd0$@gmail.com> References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> <00a001d8c31a$6fca13f0$4f5e3bd0$@gmail.com> Message-ID: Python is an appropriate computer language for kids and teenagers. A very good book for beginners: Invent Your Own computer games with Python Author: Al Sweigart It's free online, but it's worth to have one. Get Outlook for iOS ________________________________ From: Python-list on behalf of avi.e.gross at gmail.com Sent: Wednesday, September 7, 2022 7:32:11 PM To: python-list at python.org Subject: RE: Question about learning Python Chris, I started with BASIC in high school and kept shifting my focus from one computer language to another long before I even looked at Python. Arguably each language had a REASON for existing so it supported some ideas or paradigms or ways of looking at things. Many at first were rather focused on doing one or a few things well, and others not so well or not at all. No need to rehash it. In a sense, many modern languages can be viewed as containing multiple overlapping modules which collectively do all kinds of things in many ways and can do whatever many earlier initial releases of earlier languages did, sometimes better but also sometimes slower. If you look at Python as a base and added modules, it can do all the mathematical things of say FORTRAN, manipulate text and produce reports and on and on. But it also can NOT do things done in basic, or at least not easily, like subroutines and goto's that are associated with a more primitive way to solve problems. Is there any purpose in teaching new students how to structure your code so that you use such methods? Well, maybe. I mean we have all kinds of slightly hidden ways to do a GOTO in some languages such as breaking out of a nested loop. Underneath it all, an IF/THEN/ELSE type of statement has regions you jump between. But arguably it is helpful to see operations on a higher level and leave the details of how it is accomplished in well tested code within the interpreter or compiler. C was designed on purpose to do some things that a language like PASCAL was designed not to allow or make easy. Both started off though as fairly simple languages that did things somewhat linearly. You can obviously emulate many things using a simplified subset of Python that would allow programs in those languages to be done. If that is all you want to learn, fine. But if your goal is to make use of OO and functional programming and other paradigms supported, ... As I posted elsewhere, I ask why the questioner specifically mentioned C. I also use R and there too, C is only needed if you want to speed up some function by writing parts in their version of C or C++. The fact that many interpreters are written (or sometimes were written) in C is not really relevant. -----Original Message----- From: Python-list On Behalf Of Chris Angelico Sent: Wednesday, September 7, 2022 2:58 PM To: python-list at python.org Subject: Re: Question about learning Python On Thu, 8 Sept 2022 at 04:54, Grant Edwards wrote: > > On 2022-09-07, Chris Angelico wrote: > > On Thu, 8 Sept 2022 at 01:50, Maruful Islam wrote: > >> > >> I want to start learning python. I have a question about learning python. > >> > >> Is learning C essential or not for learning python? > > > > Absolutely not essential. In fact, I would strongly recommend > > learning Python before ever picking up C, as it's much easier to > > mess around in Python. > > If you're a beginning programmer, then IMO learning C first is > probably detrimental. C has quite a few quirks and pitfalls that will > a) frustrate you and waste time, and b) influence your way of thinking > about programs in a way that will be unhelpful for higher level > languages. Not as detrimental as starting with BASIC, and then moving on to x86 assembly language, and trying to massage the two together using CALL ABSOLUTE in order to get mouse input in your GW-BASIC programs. Don't be me, folks. ChrisA -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list From estates at hestates.org Wed Sep 7 22:21:53 2022 From: estates at hestates.org (Fulian Wang) Date: Thu, 8 Sep 2022 02:21:53 +0000 Subject: any author you find very good has written a book on Python? In-Reply-To: References: <86ilm1gs2d.fsf@levado.to> Message-ID: Python is an appropriate computer language for kids and teenagers. A very good book for beginners: Invent Your Own computer games with Python Author: Al Sweigart It's free online, but it's worth to have one. Get Outlook for iOS ________________________________ From: Python-list on behalf of Tim Daneliuk via Python-list Sent: Wednesday, September 7, 2022 12:39:25 PM To: python-list at python.org Subject: Re: any author you find very good has written a book on Python? On 9/5/22 21:22, Meredith Montgomery wrote: > I never read a book on Python. I'm looking for a good one now. I just > searched the web for names such as Charles Petzold, but it looks like he > never wrote a book on Python. I also searched for Peter Seibel, but he > also never did. I also tried to search for Richard Heathfield. (I took > a look at his ``C Unleashed'' once and I liked what I saw.) This is how > I search for books --- I go through the authors first. Charles Petzold, > for instance, anything he writes is worth reading it. (Have you given > his Annotated Turing a shot? It's a very nice read.) > > So that's my request --- any author you find very good has written a > book on Python? > > It could be for in a certain specific context. For instance, I also > searched for Hadley Wickham in the hope that he could have written a > data-science-type of book using Python. I like his writing a lot, but > he also only seems to have written only for the R language. > > Thank you! David Beazley -- https://mail.python.org/mailman/listinfo/python-list From greg.ewing at canterbury.ac.nz Wed Sep 7 19:55:21 2022 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 8 Sep 2022 11:55:21 +1200 Subject: Question about learning Python In-Reply-To: References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> Message-ID: On 8/09/22 6:57 am, Chris Angelico wrote: > Not as detrimental as starting with BASIC, and then moving on to x86 > assembly language, and trying to massage the two together using CALL > ABSOLUTE in order to get mouse input in your GW-BASIC programs. Or starting with hand-assembled SC/MP machine code and then moving on to Applesoft BASIC. -- Greg From mmontgomery at levado.to Wed Sep 7 21:26:29 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Wed, 07 Sep 2022 22:26:29 -0300 Subject: Question about learning Python References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <861qsnhta9.fsf@levado.to> <009e01d8c318$1103e910$330bbb30$@gmail.com> Message-ID: <867d2efygq.fsf@levado.to> writes: > Maybe we should ask WHY the person asking the question about how to learn a > computer language called Python is pairing it with the idea of whether to > also learn C. Excellent point! From nsh531 at gmail.com Thu Sep 8 08:34:08 2022 From: nsh531 at gmail.com (=?UTF-8?B?16DXqteZINep15jXqNef?=) Date: Thu, 8 Sep 2022 15:34:08 +0300 Subject: WHAT THIS EXCEPTION MEANT? I ATTCHED SCREENSOHT Message-ID: [image: image.png] -- From wlfraed at ix.netcom.com Thu Sep 8 23:19:14 2022 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 08 Sep 2022 23:19:14 -0400 Subject: Question about learning Python References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> <6318ec03.050a0220.865c7.049d@mx.google.com> <74172380-1c23-333b-51f4-6418dd6de2e1@DancesWithMice.info> Message-ID: On Thu, 8 Sep 2022 07:42:19 +1200, dn declaimed the following: >TSRs? Now that was an ugly period of history! (trying to make a >single-process operating system do multi-processing - only to find that >many program[me]s assumed they had full use and undisputed control of >the computer. Happy days...) > I laughed when M$ MSDOS (2?) introduced TSRs... My TRS-80 running L(S)DOS had similar things at least a year earlier. And these were /run-time/ loadable. They called them "filters" (and device drivers were also an option). Key-click was one such -- though it also showed some quirks (like... If the processor was really busy, the key-board driver would buffer key-strokes, but the filter activated when an application /read/ the key-board). Filter to control printer formatting, a JobLog filter, Key-Stroke Multiply filter (I never used it, but it apparently uses a table of special keys and expands them to longer strings). Commands to load device drivers (or remove them!). Could even change the number of cylinders for a floppy drive -- My drives were "loose" enough to allow my to add 2 cylinders. >On the other hand, one can start too 'high' or too 'modern'. Like the >person enthusing about MSFT's and AWS' programming AIs, thinking that >such tools will replace programmers (one of the aims of the COBOL >language back in the 1960s). His short-form description spoke volumes: >'it saves anyone from having to look-up Stack Overflow any more' - a >'blind' cut-and-paste prospect that saves the 'author' from the >difficulties of 'learning stuff'; until it is time to, um, learn-stuff - >to know why one needs to learn-stuff BEFORE taking from SO/AI. I once worked with someone whose idea of programming was to find examples of working code, and cut&paste snippets to make an application. We needed to control four devices via GPIB... She wrote four short programs and a DCL script to run them in sequence. Problem: each program had to issue a GPIB initialization command before it could continue to talk to any device. But each initialization command would UNDO any configuration the previous program had set up! We discovered THAT when TDY to the remote site (and she'd already returned home). I had to debug the situation, and rewrite the four programs into a single consolidate program over a weekend (and I'd never worked with GPIB before this -- my task was the post processing of the data that was collected after the GPIB chain had been set up for data collection). I think I'm not giving away any secrets these days, but this was a Quick Response Contract for a proof of concept -- that a geo-bird could catch over the horizon GPS signals, and thereby improve the satellite tracking data for ephemeris generation. Had to be over-the-horizon as geo-birds are above GPS, and GPS aims signals down to earth. We had to catch side-lobes as GPS birds were high enough to avoid ionosphere/troposphere effects. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From rosuav at gmail.com Fri Sep 9 16:41:33 2022 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 10 Sep 2022 06:41:33 +1000 Subject: Question about learning Python In-Reply-To: References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> Message-ID: On Sat, 10 Sept 2022 at 06:38, Greg Ewing wrote: > > On 8/09/22 6:57 am, Chris Angelico wrote: > > Not as detrimental as starting with BASIC, and then moving on to x86 > > assembly language, and trying to massage the two together using CALL > > ABSOLUTE in order to get mouse input in your GW-BASIC programs. > > Or starting with hand-assembled SC/MP machine code and then moving > on to Applesoft BASIC. > I have no idea how we survived. Though, "survived with our sanity intact" clearly didn't happen, so perhaps there's that. ChrisA From rosuav at gmail.com Fri Sep 9 16:57:44 2022 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 10 Sep 2022 06:57:44 +1000 Subject: Question about learning Python In-Reply-To: References: <0d8fe3dc-8ff8-4580-ae65-1b9bb8e3e79fn@googlegroups.com> <6318e8b1.920a0220.5aa5e.017a@mx.google.com> <6318ec03.050a0220.865c7.049d@mx.google.com> <74172380-1c23-333b-51f4-6418dd6de2e1@DancesWithMice.info> Message-ID: On Sat, 10 Sept 2022 at 06:45, Dennis Lee Bieber wrote: > > On Thu, 8 Sep 2022 07:42:19 +1200, dn > declaimed the following: > > >TSRs? Now that was an ugly period of history! (trying to make a > >single-process operating system do multi-processing - only to find that > >many program[me]s assumed they had full use and undisputed control of > >the computer. Happy days...) > > > > I laughed when M$ MSDOS (2?) introduced TSRs... My TRS-80 running > L(S)DOS had similar things at least a year earlier. And these were > /run-time/ loadable. They called them "filters" (and device drivers were > also an option). Key-click was one such -- though it also showed some > quirks (like... If the processor was really busy, the key-board driver > would buffer key-strokes, but the filter activated when an application > /read/ the key-board). Filter to control printer formatting, a JobLog > filter, Key-Stroke Multiply filter (I never used it, but it apparently uses > a table of special keys and expands them to longer strings). Commands to > load device drivers (or remove them!). Could even change the number of > cylinders for a floppy drive -- My drives were "loose" enough to allow my > to add 2 cylinders. > To be fair on MS-DOS, you didn't *have* to use a TSR to hook interrupts, and the same idea of those filters would work (just hook the keyboard interrupt in a cooperative way; last installed is first executed). But the OS offered only one option for a program to run and put itself somewhere: "terminate process and increase the base segment address for subsequent processes", which would allow you to leave any amount of memory (on a sixteen-byte boundary) active. There's no reason that filters couldn't have been written that blit themselves into some other part of memory, point some interrupt vectors there, and then fully terminate. Sure, the OS wouldn't have offered any protection, but the OS didn't offer any actual protection anyway. All we'd need is a spare slab of memory for things to put their code into, one which everyone could allocate and deallocate from..... oh. Yeah, like virtual memory. ChrisA From avi.e.gross at gmail.com Fri Sep 9 16:55:56 2022 From: avi.e.gross at gmail.com (avi.e.gross at gmail.com) Date: Fri, 9 Sep 2022 16:55:56 -0400 Subject: WHAT THIS EXCEPTION MEANT? I ATTCHED SCREENSOHT In-Reply-To: References: Message-ID: <004501d8c48e$8eb11640$ac1342c0$@gmail.com> Nati, If you thought a bit first, you might remember this mailing list does not forward attachments so we are not seeing the image you called image.png and you neglected to also give us some text telling us what exception you saw or lots of additional details that might help. And, just FYI. Using upper case subjects with multiple spelling and grammar errors is not an ideal way to get anyone to want to read or reply. Yes, I realize your main language is ??????? but you are writing to an international audience where everyone expects some English. Try this next time: Subject: What does this exception mean? I attached a screenshot ([{that nobody can see]}) All uppercase is considered shouting by some unless it merits it. It is possible that the screenshot might have told us what was needed but generally people need to know what kind of Python you are running on what kind of machine/OS and what modules you imported that may be relevant and see what code may have been called that passed back the exception without handling it. You have posted here before many times and probably also know some of us want to know a bit about the purpose of the code as the answer often is not to do it that way and do something a bit different but that is valid in the circumstances. Avi -----Original Message----- From: Python-list On Behalf Of ??? ???? Sent: Thursday, September 8, 2022 8:34 AM To: python-list at python.org Subject: WHAT THIS EXCEPTION MEANT? I ATTCHED SCREENSOHT [image: image.png] -- -- https://mail.python.org/mailman/listinfo/python-list From mats at wichmann.us Fri Sep 9 18:04:26 2022 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 9 Sep 2022 16:04:26 -0600 Subject: WHAT THIS EXCEPTION MEANT? I ATTCHED SCREENSOHT In-Reply-To: References: Message-ID: <4a12c654-37c1-b65c-1e4f-6cbe040f5b90@wichmann.us> On 9/8/22 06:34, ??? ???? wrote: > [image: image.png] > The mailing list strips attachments, replacing them by a placeholder as you see above. If you want to try again, please paste the text of the exception. From c.buhtz at posteo.jp Sat Sep 10 14:46:33 2022 From: c.buhtz at posteo.jp (c.buhtz at posteo.jp) Date: Sat, 10 Sep 2022 18:46:33 +0000 Subject: [py-usr] flake8 gives me a W605 but Python don't Message-ID: <4MQ1zZ59Tlz6tr9@submission01.posteo.de> Hello, My `flake8` gives me a "W605 invalid escape sequence" [1] warning for this piece of example code. import re def foobar(): rex = re.compile('Version: \d+.\d+.\d+.*', re.MULTILINE) for match in rex.findall(' Version: 1.2.3 '): print(match) if __name__ == '__main__': foobar() But running this with Python 3.9.2 makes no problem. Python doesn't give me a `SyntaxWarning` or anything else. Python doesn't give me an error or warning. Only `flymake8` gives me this error. I do understand what is wrong with the pattern string in `compile()`. There should be a `r''` or the regex-escape characters should be escaped them selfs (e.g. `'Version: \\d'`). But my question is about why Python doesn't give me an error about it and why does it work. The pattern matches. Shouldn't there be an error or something? Does Python identify this string as an r-String by itself? [1] -- From python at mrabarnett.plus.com Sat Sep 10 15:37:23 2022 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 10 Sep 2022 20:37:23 +0100 Subject: [py-usr] flake8 gives me a W605 but Python don't In-Reply-To: <4MQ1zZ59Tlz6tr9@submission01.posteo.de> References: <4MQ1zZ59Tlz6tr9@submission01.posteo.de> Message-ID: <13ec65e9-5a5c-8de0-b34e-7a12777fdeda@mrabarnett.plus.com> On 2022-09-10 19:46, c.buhtz at posteo.jp wrote: > Hello, > > My `flake8` gives me a "W605 invalid escape sequence" [1] warning for > this piece of example code. > > import re > > def foobar(): > rex = re.compile('Version: \d+.\d+.\d+.*', re.MULTILINE) > > for match in rex.findall(' Version: 1.2.3 '): > print(match) > > > if __name__ == '__main__': > foobar() > > But running this with Python 3.9.2 makes no problem. Python doesn't > give me a `SyntaxWarning` or anything else. Python doesn't give me an > error or warning. Only `flymake8` gives me this error. > > I do understand what is wrong with the pattern string in `compile()`. > There should be a `r''` or the regex-escape characters should be > escaped them selfs (e.g. `'Version: \\d'`). > > But my question is about why Python doesn't give me an error about it > and why does it work. The pattern matches. Shouldn't there be an error > or something? Does Python identify this string as an r-String by itself? > > [1] -- Historically, invalid escapes such as '\d' have always been treated as '\\d', but that is, and always, bad idea; it's better to treat them as an error (or reserved for future use). At some point, in some future version of Python, they should become an error, but there's always a chance that some code out there could break, so there's a long period of deprecation first. How long? Who knows! Just take note of the warning, and don't write bad escapes. From reto at labrat.space Sat Sep 10 15:41:04 2022 From: reto at labrat.space (Reto) Date: Sat, 10 Sep 2022 21:41:04 +0200 Subject: [py-usr] flake8 gives me a W605 but Python don't In-Reply-To: <4MQ1zZ59Tlz6tr9@submission01.posteo.de> References: <4MQ1zZ59Tlz6tr9@submission01.posteo.de> Message-ID: <20220910194104.fb2ixwer6ehzemz3@feather> On Sat, Sep 10, 2022 at 06:46:33PM +0000, c.buhtz at posteo.jp wrote: > > But running this with Python 3.9.2 makes no problem. Python doesn't > give me a `SyntaxWarning` or anything else. Python doesn't give me an > error or warning. Only `flymake8` gives me this error. Well, it's not a syntax error. Escape sequences with no meaning simply yield the escape sequence. As in, '\d' is simply '\d', in contrast with say '\u' which is invalid and if fact throws a SyntaxError. >Unlike Standard C, all unrecognized escape sequences are left in the string unchanged, i.e., the backslash is left in the result. (This behavior is useful when debugging: if an escape sequence is mistyped, the resulting output is more easily recognized as broken.) It is also important to note that the escape sequences only recognized in string literals fall into the category of unrecognized escapes for bytes literals. from https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals From list1 at tompassin.net Sat Sep 10 17:07:50 2022 From: list1 at tompassin.net (Thomas Passin) Date: Sat, 10 Sep 2022 17:07:50 -0400 Subject: [py-usr] flake8 gives me a W605 but Python don't In-Reply-To: <4MQ1zZ59Tlz6tr9@submission01.posteo.de> References: <4MQ1zZ59Tlz6tr9@submission01.posteo.de> Message-ID: <1b078682-7799-3d47-aae4-a0d879ce9496@tompassin.net> I don't know what flake8 is complaining about, but I think you want a literal "." character in front of the two "\d" escapes. So "." should be replaced by "\." in two places. In addition to using a raw string, that is. If you may need to detect versions larger than 9 in any position, you would need to generalize your pattern, since "\d" is only going to look at a single character, IIUC. If you could be sure that the version string would always look exactly like your example, you could avoid a regex by splitting: _, version = version_string.split('Version:') version_parts = version.split('.') That's probably hoping for too much, though. On 9/10/2022 2:46 PM, c.buhtz at posteo.jp wrote: > Hello, > > My `flake8` gives me a "W605 invalid escape sequence" [1] warning for > this piece of example code. > > import re > > def foobar(): > rex = re.compile('Version: \d+.\d+.\d+.*', re.MULTILINE) > > for match in rex.findall(' Version: 1.2.3 '): > print(match) > > > if __name__ == '__main__': > foobar() > > But running this with Python 3.9.2 makes no problem. Python doesn't > give me a `SyntaxWarning` or anything else. Python doesn't give me an > error or warning. Only `flymake8` gives me this error. > > I do understand what is wrong with the pattern string in `compile()`. > There should be a `r''` or the regex-escape characters should be > escaped them selfs (e.g. `'Version: \\d'`). > > But my question is about why Python doesn't give me an error about it > and why does it work. The pattern matches. Shouldn't there be an error > or something? Does Python identify this string as an r-String by itself? > > [1] -- From mats at wichmann.us Sat Sep 10 19:54:35 2022 From: mats at wichmann.us (Mats Wichmann) Date: Sat, 10 Sep 2022 17:54:35 -0600 Subject: [py-usr] flake8 gives me a W605 but Python don't In-Reply-To: References: Message-ID: <972866b7-0dce-b1cf-734c-8690d13afd50@wichmann.us> On 9/10/22 13:20, Stefan Ram wrote: > writes: >> 'Version: \d+.\d+.\d+.*' > > All unrecognized escape sequences are left in the string > unchanged, i.e., the backslash is left in the result. > > This behavior is useful when debugging: if an escape > sequence is mistyped, the resulting output is more easily > recognized as broken. > > Some Python versions may emit a DeprecationWarning when > an unrecognized escape sequence is encountered; future > Python versions might emit a SyntaxWarning or SyntaxError. Consider: $ /usr/bin/python3.11 -X dev stuff.py /tmp/stuff.py:4: DeprecationWarning: invalid escape sequence '\d' rex = re.compile('Version: \d+.\d+.\d+.*', re.MULTILINE) Version: 1.2.3 From pablogsal at gmail.com Mon Sep 12 07:23:52 2022 From: pablogsal at gmail.com (Pablo Galindo Salgado) Date: Mon, 12 Sep 2022 12:23:52 +0100 Subject: [RELEASE] Python 3.11 release candidate 2 (3.11.0rc2) is available Message-ID: Python 3.11 is one month away, can you believe it? This snake is still trying to bite as it has been an interesting day of fighting fires, release blockers, and a bunch of late bugs but your friendly release team always delivers :) You can get this new release while is still fresh here: https://www.python.org/downloads/release/python-3110rc2/ ## This is the second release candidate of Python 3.11 This release, **3.11.0rc2**, is the last preview before the final release of Python 3.11.0 on 2022-10-24. Entering the release candidate phase, only reviewed code changes which are clear bug fixes are allowed between this release candidate and the final release. The second candidate and the last planned release preview is currently planned for Monday, 2022-09-05 while the official release is planned for Monday, 2022-10-24. There will be no ABI changes from this point forward in the 3.11 series and the goal is that there will be as few code changes as possible. ## Modification of the final release Due to the fact that we needed to delay the last release candidate by a week and because of personal scheduling problems I am delaying the final release to 2022-10-24 (three weeks from the original date). ## Call to action ?????????????? The 3.11 branch is now accepting changes for 3.11.**1**. To maximize stability, the final release will be cut from the v3.11.0rc2 tag. If you need the release manager (me) to cherry-pick any critical fixes, mark issues as release blockers, and/or add me as a reviewer on a critical backport PR on GitHub. To see which changes are currently cherry-picked for inclusion in 3.11.0, look at the short-lived branch-v3.11.0 https://github.com/python/cpython/tree/branch-v3.11.0 on GitHub. ?????????????? --- #### Core developers: all eyes on the docs now * Are all your changes properly documented? * Did you notice other changes you know of to have insufficient documentation? #### Community members We strongly encourage maintainers of third-party Python projects to prepare their projects for 3.11 compatibilities during this phase. As always, report any issues to [the Python bug tracker ](https://github.com/issues). Please keep in mind that this is a preview release and its use is **not** recommended for production environments. # Major new features of the 3.11 series, compared to 3.10 Some of the new major new features and changes in Python 3.11 are: ## General changes * [PEP 657](https://www.python.org/dev/peps/pep-0657/) -- Include Fine-Grained Error Locations in Tracebacks * [PEP 654](https://www.python.org/dev/peps/pep-0654/) -- Exception Groups and `except*` * [PEP 680](https://www.python.org/dev/peps/pep-0680/) -- tomllib: Support for Parsing TOML in the Standard Library * [gh-90908](https://github.com/python/cpython/issues/90908) -- Introduce task groups to asyncio * [gh-34627](https://github.com/python/cpython/issues/34627/) -- Atomic grouping (`(?>...)`) and possessive quantifiers (`*+, ++, ?+, {m,n}+`) are now supported in regular expressions. * The [Faster CPython Project](https://github.com/faster-cpython/) is already yielding some exciting results. Python 3.11 is up to 10-60% faster than Python 3.10. On average, we measured a 1.22x speedup on the standard benchmark suite. See [Faster CPython]( https://docs.python.org/3.11/whatsnew/3.11.html#faster-cpython) for details. ## Typing and typing language changes * [PEP 673](https://www.python.org/dev/peps/pep-0673/) -- Self Type * [PEP 646](https://www.python.org/dev/peps/pep-0646/) -- Variadic Generics * [PEP 675](https://www.python.org/dev/peps/pep-0675/) -- Arbitrary Literal String Type * [PEP 655](https://www.python.org/dev/peps/pep-0655/) -- Marking individual TypedDict items as required or potentially-missing * [PEP 681](https://www.python.org/dev/peps/pep-0681/) -- Data Class Transforms (Hey, **fellow core developer,** if a feature you find important is missing from this list, [let Pablo know](mailto:pablogsal at python.org ).) The next release will be the final release of Python 3.11.0, which is currently scheduled for Monday, 2022-10-24. # More resources * [Online Documentation](https://docs.python.org/3.11/) * [PEP 664](https://www.python.org/dev/peps/pep-0664/), 3.11 Release Schedule * Report bugs at [ https://github.com/python/cpython/issues](https://github.com/python/cpython/issues) . * [Help fund Python and its community](/psf/donations/). # And now for something completely different In general relativity, a white hole is a theoretical region of spacetime and singularity that cannot be entered from the outside, although energy-matter, light and information can escape from it. In this sense, it is the reverse of a black hole, which can be entered only from the outside and from which energy-matter, light and information cannot escape. White holes appear in the theory of eternal black holes. In addition to a black hole region in the future, such a solution of the Einstein field equations has a white hole region in its past. This region does not exist for black holes that have formed through gravitational collapse, however, nor are there any observed physical processes through which a white hole could be formed. Supermassive black holes are theoretically predicted to be at the centre of every galaxy and that possibly, a galaxy cannot form without one. Stephen Hawking and others have proposed that these supermassive black holes spawn a supermassive white hole. # We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation. https://www.python.org/psf/ If you have any questions, please reach out to me or another member of the release team :) Your friendly release team, Ned Deily @nad https://discuss.python.org/u/nad Steve Dower @steve.dower https://discuss.python.org/u/steve.dower Pablo Galindo Salgado @pablogsal https://discuss.python.org/u/pablogsal From mailinglists at vanwingerde.nl Mon Sep 12 16:35:46 2022 From: mailinglists at vanwingerde.nl (mailinglists at vanwingerde.nl) Date: Mon, 12 Sep 2022 20:35:46 +0000 Subject: DoesNotExist at /admin/blog/blog/add/: blog matching query does not exist Message-ID: <20220912203546.16349b0a@jaap.artio.shrl.nl> Suddenly I can no longer add blogs to Django. Django says 'blog matching query does not exist'. That seems strange to me because I want to add something to the database and not request a blog. What could be going on here? admin.py: class blogadmin(admin.ModelAdmin): def save_model(self, request, obj, form, change): if not change: obj.added_by_user = request.user.username obj.ip_added = request.META['REMOTE_ADDR'] if obj.ip_added == '127.0.0.1': obj.ip_added = request.META['HTTP_X_FORWARDED_FOR'] obj.publish = False del obj.publish_date else: obj.changed_by_user = request.user.username obj.ip_changed = request.META['REMOTE_ADDR'] if obj.ip_changed == '127.0.0.1': obj.ip_changed = request.META['HTTP_X_FORWARDED_FOR'] if obj.publish == True: if not obj.publish_date: obj.publish_date = obj.datetime_changed super(blogadmin,self).save_model(request, obj, form, change) I can add text, photographs etc., but no blogs. Notifications: DoesNotExist at /admin/blog/blog/add/ blog matching query does not exist. Request Method: POST Request URL: http://127.0.0.1:8001/admin/blog/blog/add/ Django Version: 4.1.1 Exception Type: DoesNotExist Exception Value: blog matching query does not exist. Traceback Environment: Request Method: POST Request URL: http://127.0.0.1:8001/admin/blog/blog/add/ Django Version: 4.1.1 Python Version: 3.10.5 Installed Applications: ['grappelli', 'django.contrib.admin', 'django.contrib.admindocs', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'blog.apps.BlogConfig'] Installed Middleware: ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'django.contrib.admindocs.middleware.XViewMiddleware'] Traceback (most recent call last): File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner response = get_response(request) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", line 686, in wrapper return self.admin_site.admin_view(view)(*args, **kwargs) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py", line 133, in _wrapped_view response = view_func(request, *args, **kwargs) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/views/decorators/cache.py", line 62, in _wrapped_view_func response = view_func(request, *args, **kwargs) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/sites.py", line 242, in inner return view(request, *args, **kwargs) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", line 1890, in add_view return self.changeform_view(request, None, form_url, extra_context) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py", line 46, in _wrapper return bound_method(*args, **kwargs) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py", line 133, in _wrapped_view response = view_func(request, *args, **kwargs) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", line 1750, in changeform_view return self._changeform_view(request, object_id, form_url, extra_context) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", line 1802, in _changeform_view self.save_model(request, new_object, form, not add) File "/var/django/liakoster.nl/blog-1/blog/admin.py", line 26, in save_model super(blogadmin,self).save_model(request, obj, form, change) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", line 1225, in save_model obj.save() File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", line 831, in save self.save_base( File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", line 882, in save_base updated = self._save_table( File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", line 1025, in _save_table results = self._do_insert( File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", line 1066, in _do_insert return manager._insert( File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/query.py", line 1790, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1656, in execute_sql for sql, params in self.as_sql(): File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1580, in as_sql value_rows = [ File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1581, in [ File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1582, in self.prepare_value(field, self.pre_save_val(field, obj)) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1530, in pre_save_val return field.pre_save(obj, add=True) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 1548, in pre_save return super().pre_save(model_instance, add) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 1411, in pre_save return super().pre_save(model_instance, add) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/fields/__init__.py", line 905, in pre_save return getattr(model_instance, self.attname) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/query_utils.py", line 178, in __get__ instance.refresh_from_db(fields=[field_name]) File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", line 741, in refresh_from_db db_instance = db_instance_qs.get() File "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/query.py", line 650, in get raise self.model.DoesNotExist( Exception Type: DoesNotExist at /admin/blog/blog/add/ Exception Value: blog matching query does not exist. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digitale ondertekening URL: From nospam at dfs.com Mon Sep 12 16:19:51 2022 From: nospam at dfs.com (DFS) Date: Mon, 12 Sep 2022 16:19:51 -0400 Subject: Obtain the query interface url of BCS server. In-Reply-To: <273b9481-192b-48b1-b057-fcfc22c6cf21n@googlegroups.com> References: <273b9481-192b-48b1-b057-fcfc22c6cf21n@googlegroups.com> Message-ID: On 9/12/2022 5:00 AM, hongy... at gmail.com wrote: > I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script. > > However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim? > > [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10 > > Regards, > Zhao You didn't say what you want to query. Are you trying to download entire sections of the Bilbao Crystallographic Server? Maybe the admins will give you access to the data. * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen brings up the table of space group symbols. * choose say #7: Pc * now click ITA Settings, then choose the last entry "P c 1 1" and it loads: https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita You might be able to fool around with that URL and substitute values and get back the data you want (in HTML) via Python. Do you really want HTML results? Hit Ctrl+U to see the source HTML of a webpage Right-click or hit Ctrl + Shift + C to inspect the individual elements of the page From enerel.amgalan at yahoo.com Tue Sep 13 01:41:46 2022 From: enerel.amgalan at yahoo.com (Enerel Amgalan) Date: Mon, 12 Sep 2022 23:41:46 -0600 Subject: DoesNotExist at /admin/blog/blog/add/: blog matching query does not exist In-Reply-To: <20220912203546.16349b0a@jaap.artio.shrl.nl> References: <20220912203546.16349b0a@jaap.artio.shrl.nl> Message-ID: <5260C860-3CFF-43C5-96B9-4B3A6FA47C2B@yahoo.com> unsubscribe -Bat > On Sep 12, 2022, at 14:32, mailinglists at vanwingerde.nl wrote: > > ?Suddenly I can no longer add blogs to Django. Django says 'blog > matching query does not exist'. That seems strange to me because I want > to add something to the database and not request a blog. What could be > going on here? > > admin.py: > class blogadmin(admin.ModelAdmin): > def save_model(self, request, obj, form, change): > if not change: > obj.added_by_user = request.user.username > obj.ip_added = request.META['REMOTE_ADDR'] > if obj.ip_added == '127.0.0.1': > obj.ip_added = request.META['HTTP_X_FORWARDED_FOR'] > obj.publish = False > del obj.publish_date > else: > obj.changed_by_user = request.user.username > obj.ip_changed = request.META['REMOTE_ADDR'] > if obj.ip_changed == '127.0.0.1': > obj.ip_changed = request.META['HTTP_X_FORWARDED_FOR'] > if obj.publish == True: > if not obj.publish_date: > obj.publish_date = obj.datetime_changed > super(blogadmin,self).save_model(request, obj, form, change) > > I can add text, photographs etc., but no blogs. > > Notifications: > DoesNotExist at /admin/blog/blog/add/ > > blog matching query does not exist. > > Request Method: POST > Request URL: http://127.0.0.1:8001/admin/blog/blog/add/ > Django Version: 4.1.1 > Exception Type: DoesNotExist > Exception Value: > > blog matching query does not exist. > > Traceback > Environment: > Request Method: POST > Request URL: http://127.0.0.1:8001/admin/blog/blog/add/ > > Django Version: 4.1.1 > Python Version: 3.10.5 > Installed Applications: > ['grappelli', > 'django.contrib.admin', > 'django.contrib.admindocs', > 'django.contrib.auth', > 'django.contrib.contenttypes', > 'django.contrib.sessions', > 'django.contrib.messages', > 'django.contrib.staticfiles', > 'blog.apps.BlogConfig'] > Installed Middleware: > ['django.middleware.security.SecurityMiddleware', > 'django.contrib.sessions.middleware.SessionMiddleware', > 'django.middleware.common.CommonMiddleware', > 'django.middleware.csrf.CsrfViewMiddleware', > 'django.contrib.auth.middleware.AuthenticationMiddleware', > 'django.contrib.messages.middleware.MessageMiddleware', > 'django.middleware.clickjacking.XFrameOptionsMiddleware', > 'django.contrib.admindocs.middleware.XViewMiddleware'] > > Traceback (most recent call last): > File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/exception.py", > line 55, in inner response = get_response(request) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/base.py", > line 197, in _get_response response = wrapped_callback(request, > *callback_args, **callback_kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 686, in wrapper return self.admin_site.admin_view(view)(*args, > **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py", > line 133, in _wrapped_view response = view_func(request, *args, > **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/views/decorators/cache.py", > line 62, in _wrapped_view_func response = view_func(request, *args, > **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/sites.py", > line 242, in inner return view(request, *args, **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 1890, in add_view return self.changeform_view(request, None, > form_url, extra_context) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py", > line 46, in _wrapper return bound_method(*args, **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py", > line 133, in _wrapped_view response = view_func(request, *args, > **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 1750, in changeform_view return self._changeform_view(request, > object_id, form_url, extra_context) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 1802, in _changeform_view self.save_model(request, new_object, > form, not add) File "/var/django/liakoster.nl/blog-1/blog/admin.py", > line 26, in save_model super(blogadmin,self).save_model(request, obj, > form, change) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 1225, in save_model obj.save() File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 831, in save self.save_base( File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 882, in save_base updated = self._save_table( File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 1025, in _save_table results = self._do_insert( File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 1066, in _do_insert return manager._insert( File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/manager.py", > line 85, in manager_method return getattr(self.get_queryset(), > name)(*args, **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/query.py", > line 1790, in _insert return > query.get_compiler(using=using).execute_sql(returning_fields) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1656, in execute_sql for sql, params in self.as_sql(): File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1580, in as_sql value_rows = [ File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1581, in [ File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1582, in self.prepare_value(field, > self.pre_save_val(field, obj)) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1530, in pre_save_val return field.pre_save(obj, add=True) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/fields/__init__.py", > line 1548, in pre_save return super().pre_save(model_instance, add) > File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/fields/__init__.py", > line 1411, in pre_save return super().pre_save(model_instance, add) > File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/fields/__init__.py", > line 905, in pre_save return getattr(model_instance, self.attname) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/query_utils.py", > line 178, in __get__ instance.refresh_from_db(fields=[field_name]) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 741, in refresh_from_db db_instance = db_instance_qs.get() File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/query.py", > line 650, in get raise self.model.DoesNotExist( > > Exception Type: DoesNotExist at /admin/blog/blog/add/ > Exception Value: blog matching query does not exist. > -- > https://mail.python.org/mailman/listinfo/python-list From mailinglists at vanwingerde.nl Tue Sep 13 07:58:25 2022 From: mailinglists at vanwingerde.nl (mailinglists at vanwingerde.nl) Date: Tue, 13 Sep 2022 11:58:25 +0000 Subject: DoesNotExist at /admin/blog/blog/add/: blog matching query does not exist In-Reply-To: <20220912203546.16349b0a@jaap.artio.shrl.nl> References: <20220912203546.16349b0a@jaap.artio.shrl.nl> Message-ID: <20220913115825.66a4a2dd@jaap.artio.shrl.nl> Sorry, the wrong list. Op 2022-09-12T20:35:46+0000 schreef mailinglists at vanwingerde.nl in bericht , inzake: het volgende. > Suddenly I can no longer add blogs to Django. Django says 'blog > matching query does not exist'. That seems strange to me because I > want to add something to the database and not request a blog. What > could be going on here? > > admin.py: > class blogadmin(admin.ModelAdmin): > def save_model(self, request, obj, form, change): > if not change: > obj.added_by_user = request.user.username > obj.ip_added = request.META['REMOTE_ADDR'] > if obj.ip_added == '127.0.0.1': > obj.ip_added = request.META['HTTP_X_FORWARDED_FOR'] > obj.publish = False > del obj.publish_date > else: > obj.changed_by_user = request.user.username > obj.ip_changed = request.META['REMOTE_ADDR'] > if obj.ip_changed == '127.0.0.1': > obj.ip_changed = request.META['HTTP_X_FORWARDED_FOR'] > if obj.publish == True: > if not obj.publish_date: > obj.publish_date = obj.datetime_changed > super(blogadmin,self).save_model(request, obj, form, change) > > I can add text, photographs etc., but no blogs. > > Notifications: > DoesNotExist at /admin/blog/blog/add/ > > blog matching query does not exist. > > Request Method: POST > Request URL: http://127.0.0.1:8001/admin/blog/blog/add/ > Django Version: 4.1.1 > Exception Type: DoesNotExist > Exception Value: > > blog matching query does not exist. > > Traceback > Environment: > Request Method: POST > Request URL: http://127.0.0.1:8001/admin/blog/blog/add/ > > Django Version: 4.1.1 > Python Version: 3.10.5 > Installed Applications: > ['grappelli', > 'django.contrib.admin', > 'django.contrib.admindocs', > 'django.contrib.auth', > 'django.contrib.contenttypes', > 'django.contrib.sessions', > 'django.contrib.messages', > 'django.contrib.staticfiles', > 'blog.apps.BlogConfig'] > Installed Middleware: > ['django.middleware.security.SecurityMiddleware', > 'django.contrib.sessions.middleware.SessionMiddleware', > 'django.middleware.common.CommonMiddleware', > 'django.middleware.csrf.CsrfViewMiddleware', > 'django.contrib.auth.middleware.AuthenticationMiddleware', > 'django.contrib.messages.middleware.MessageMiddleware', > 'django.middleware.clickjacking.XFrameOptionsMiddleware', > 'django.contrib.admindocs.middleware.XViewMiddleware'] > > Traceback (most recent call last): > File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/exception.py", > line 55, in inner response = get_response(request) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/core/handlers/base.py", > line 197, in _get_response response = wrapped_callback(request, > *callback_args, **callback_kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 686, in wrapper return self.admin_site.admin_view(view)(*args, > **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py", > line 133, in _wrapped_view response = view_func(request, *args, > **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/views/decorators/cache.py", > line 62, in _wrapped_view_func response = view_func(request, *args, > **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/sites.py", > line 242, in inner return view(request, *args, **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 1890, in add_view return self.changeform_view(request, None, > form_url, extra_context) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py", > line 46, in _wrapper return bound_method(*args, **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/utils/decorators.py", > line 133, in _wrapped_view response = view_func(request, *args, > **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 1750, in changeform_view return self._changeform_view(request, > object_id, form_url, extra_context) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 1802, in _changeform_view self.save_model(request, new_object, > form, not add) File "/var/django/liakoster.nl/blog-1/blog/admin.py", > line 26, in save_model super(blogadmin,self).save_model(request, obj, > form, change) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/contrib/admin/options.py", > line 1225, in save_model obj.save() File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 831, in save self.save_base( File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 882, in save_base updated = self._save_table( File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 1025, in _save_table results = self._do_insert( File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 1066, in _do_insert return manager._insert( File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/manager.py", > line 85, in manager_method return getattr(self.get_queryset(), > name)(*args, **kwargs) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/query.py", > line 1790, in _insert return > query.get_compiler(using=using).execute_sql(returning_fields) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1656, in execute_sql for sql, params in self.as_sql(): File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1580, in as_sql value_rows = [ File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1581, in [ File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1582, in self.prepare_value(field, > self.pre_save_val(field, obj)) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/sql/compiler.py", > line 1530, in pre_save_val return field.pre_save(obj, add=True) File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/fields/__init__.py", > line 1548, in pre_save return super().pre_save(model_instance, add) > File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/fields/__init__.py", > line 1411, in pre_save return super().pre_save(model_instance, add) > File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/fields/__init__.py", > line 905, in pre_save return getattr(model_instance, self.attname) > File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/query_utils.py", > line 178, in __get__ instance.refresh_from_db(fields=[field_name]) > File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/base.py", > line 741, in refresh_from_db db_instance = db_instance_qs.get() File > "/var/django/liakoster.nl/blog-1/lib/python3.10/site-packages/django/db/models/query.py", > line 650, in get raise self.model.DoesNotExist( > > Exception Type: DoesNotExist at /admin/blog/blog/add/ > Exception Value: blog matching query does not exist. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digitale ondertekening URL: From hongyi.zhao at gmail.com Tue Sep 13 03:46:43 2022 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Tue, 13 Sep 2022 00:46:43 -0700 (PDT) Subject: Obtain the query interface url of BCS server. In-Reply-To: References: <273b9481-192b-48b1-b057-fcfc22c6cf21n@googlegroups.com> Message-ID: On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote: > On 9/12/2022 5:00 AM, hongy... at gmail.com wrote: > > I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script. > > > > However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim? > > > > [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10 > > > > Regards, > > Zhao > You didn't say what you want to query. Are you trying to download > entire sections of the Bilbao Crystallographic Server? I am engaged in some related research and need some specific data used by BCS server. > Maybe the admins will give you access to the data. I don't think they will provide such convenience to researchers who have no cooperative relationship with them. > * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen > brings up the table of space group symbols. > > * choose say #7: Pc > > * now click ITA Settings, then choose the last entry "P c 1 1" and it > loads: > > https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita Not only that, but I want to obtain all such URLs programmatically! > You might be able to fool around with that URL and substitute values and > get back the data you want (in HTML) via Python. Do you really want > HTML results? > > Hit Ctrl+U to see the source HTML of a webpage > > Right-click or hit Ctrl + Shift + C to inspect the individual elements > of the page For batch operations, all these manual methods are inefficient. Best Regards, Zhao From loris.bennett at fu-berlin.de Tue Sep 13 07:39:02 2022 From: loris.bennett at fu-berlin.de (Loris Bennett) Date: Tue, 13 Sep 2022 13:39:02 +0200 Subject: poetry: local dependencies in dev and prod environments Message-ID: <871qsfwlk9.fsf@hornfels.zedat.fu-berlin.de> Hi, Say I have two modules: main application and a utility. With poetry I can add the utility as local dependency to the main application thus: poetry add ../utilities/mailer/dist/my_mailer-0.1.0-py3-none-any.whl This then generates the following in the pyproj.toml of the main application: [tool.poetry.dependencies] python = "^3.6" my-mailer = {path = "../utilities/mailer/dist/my_mailer-0.1.0-py3-none-any.whl"} With this configuration I can run the application in the development environment. However, if I want to install both the main application and the utility to my 'site-package' directory, I need to change pyproj.toml to [tool.poetry.dependencies] python = "^3.6" my-mailer = ">0.1.0" This seems a bit clunky to me and I wonder whether there is a better way of handling the situation. The problem is that the venv used by poetry for the main application correctly ignores the 'site-packages' directory, which contains the utility when it is installed. However, it would be handy if I could allow just this single package from 'site-packages' to be used within the venv. Is this possible? Cheers, Loris -- This signature is currently under construction. From nospam at dfs.com Tue Sep 13 09:32:58 2022 From: nospam at dfs.com (DFS) Date: Tue, 13 Sep 2022 09:32:58 -0400 Subject: Obtain the query interface url of BCS server. In-Reply-To: References: <273b9481-192b-48b1-b057-fcfc22c6cf21n@googlegroups.com> Message-ID: On 9/13/2022 3:46 AM, hongy... at gmail.com wrote: > On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote: >> On 9/12/2022 5:00 AM, hongy... at gmail.com wrote: >>> I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script. >>> >>> However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim? >>> >>> [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10 >>> >>> Regards, >>> Zhao >> You didn't say what you want to query. Are you trying to download >> entire sections of the Bilbao Crystallographic Server? > > I am engaged in some related research and need some specific data used by BCS server. What specific data? Is it available elsewhere? >> Maybe the admins will give you access to the data. > > I don't think they will provide such convenience to researchers who have no cooperative relationship with them. You can try. Tell the admins what data you want, and ask them for the easiest way to get it. >> * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen >> brings up the table of space group symbols. >> >> * choose say #7: Pc >> >> * now click ITA Settings, then choose the last entry "P c 1 1" and it >> loads: >> >> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita > > Not only that, but I want to obtain all such URLs programmatically! > >> You might be able to fool around with that URL and substitute values and >> get back the data you want (in HTML) via Python. Do you really want >> HTML results? >> >> Hit Ctrl+U to see the source HTML of a webpage >> >> Right-click or hit Ctrl + Shift + C to inspect the individual elements >> of the page > > For batch operations, all these manual methods are inefficient. Yes, but I don't think you'll be able to retrieve the URLs programmatically. The JavaScript code doesn't put them in the HTML result, except for that one I showed you, which seems like a mistake on their part. So you'll have to figure out the search fields, and your python program will have to cycle through the search values: Sample from above gnum = 007 what = gp trmat = b,-a-c,c unconv = P c 1 1 from = ita wBase = "https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen" wGnum = "?gnum=" + findgnum wWhat = "&what=" + findWhat wTrmat = "&trmat=" + findTrmat wUnconv = "&unconv=" + findUnconv wFrom = "&from=" + findFrom webpage = wBase + wGnum + wWhat + wTrmat + wUnconv + wFrom Then if that returns a hit, you'll have to parse the resulting HTML and extract the exact data you want. I did something similar a while back using the requests and lxml libraries ---------------------------------------------------------------- #build url wBase = "http://www.usdirectory.com" wForm = "/ypr.aspx?fromform=qsearch" wKeyw = "&qhqn=" + keyw wCityZip = "&qc=" + cityzip wState = "&qs=" + state wDist = "&rg=" + str(miles) wSort = "&sb=a2z" #sort alpha wPage = "&ap=" #used with the results page number webpage = wBase + wForm + wKeyw + wCityZip + wState + wDist #open URL page = requests.get(webpage) tree = html.fromstring(page.content) #no matches matches = tree.xpath('//strong/text()') if passNbr == 1 and ("No results were found" in str(matches)): print "No results found for that search" exit(0) ---------------------------------------------------------------- 2.x code file: https://file.io/VdptORSKh5CN > Best Regards, > Zhao From PythonList at DancesWithMice.info Tue Sep 13 17:51:53 2022 From: PythonList at DancesWithMice.info (dn) Date: Wed, 14 Sep 2022 09:51:53 +1200 Subject: Virtual PUG-meeting: An applied introduction to Finite State Machines Message-ID: An applied introduction to Finite State Machines 0730 UTC, Wed 21 Sep 2022 The basics of Finite State Machines and what they are good for. How to use FSM for solving optimization problems. - A simple traffic jam - A bigger traffic jam - A sudoku After this lecture there won't be any discrete optimization problem you won't be able to solve. Dr. Bjorn Madsen solves optimization problems using Python, for DoD, LEGO, Coca Cola, Airbus and many other companies. These meetups have two sessions. At least one is aimed at a beginner level - in either Python or FSMs. Sign-up and more details from https://www.meetup.com/nzpug-auckland/ -- Regards, =dn From sam.z.ezeh at gmail.com Tue Sep 13 18:18:38 2022 From: sam.z.ezeh at gmail.com (Sam Ezeh) Date: Tue, 13 Sep 2022 23:18:38 +0100 Subject: Virtual PUG-meeting: An applied introduction to Finite State Machines In-Reply-To: References: Message-ID: That seems interesting. Is this hosted online? And are there any suggested reading materials for those who might not be able to attend? Kind regards, Sam Ezeh On Tue, 13 Sept 2022 at 22:53, dn wrote: > An applied introduction to Finite State Machines > 0730 UTC, Wed 21 Sep 2022 > > The basics of Finite State Machines and what they are good for. How to > use FSM for solving optimization problems. > > - A simple traffic jam > - A bigger traffic jam > - A sudoku > > After this lecture there won't be any discrete optimization problem you > won't be able to solve. > > Dr. Bjorn Madsen solves optimization problems using Python, for DoD, > LEGO, Coca Cola, Airbus and many other companies. > > > These meetups have two sessions. At least one is aimed at a beginner > level - in either Python or FSMs. > > Sign-up and more details from https://www.meetup.com/nzpug-auckland/ > -- > Regards, > =dn > -- > https://mail.python.org/mailman/listinfo/python-list > From salvatorebruzzese05 at gmail.com Tue Sep 13 15:54:38 2022 From: salvatorebruzzese05 at gmail.com (Salvatore Bruzzese) Date: Tue, 13 Sep 2022 21:54:38 +0200 Subject: Uninstall tool not working. Message-ID: Hi, I was trying to uninstall version 3.10.7 of python but I've encountered problems with the uninstall tool. I open the python setup program, click on the uninstall button but it doesn't even start deleting python even though it says that the process has finished. Feel free to ask for more details in case I didn't explain it correctly. Thanks in advance for your help. From hongyi.zhao at gmail.com Tue Sep 13 19:29:16 2022 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Tue, 13 Sep 2022 16:29:16 -0700 (PDT) Subject: Obtain the query interface url of BCS server. In-Reply-To: References: <273b9481-192b-48b1-b057-fcfc22c6cf21n@googlegroups.com> Message-ID: <4ea83cbd-b549-485b-b8b8-58b50133ab4cn@googlegroups.com> On Tuesday, September 13, 2022 at 9:33:20 PM UTC+8, DFS wrote: > On 9/13/2022 3:46 AM, hongy... at gmail.com wrote: > > On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote: > >> On 9/12/2022 5:00 AM, hongy... at gmail.com wrote: > >>> I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script. > >>> > >>> However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim? > >>> > >>> [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10 > >>> > >>> Regards, > >>> Zhao > >> You didn't say what you want to query. Are you trying to download > >> entire sections of the Bilbao Crystallographic Server? > > > > I am engaged in some related research and need some specific data used by BCS server. > What specific data? All the data corresponding to the total catalog here: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen > Is it available elsewhere? This is an internationally recognized authoritative data source in this field. Data from other places, even if there are readily available electronic versions, are basically taken from here and are not comprehensive. > >> Maybe the admins will give you access to the data. > > > > I don't think they will provide such convenience to researchers who have no cooperative relationship with them. > You can try. Tell the admins what data you want, and ask them for the > easiest way to get it. > >> * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen > >> brings up the table of space group symbols. > >> > >> * choose say #7: Pc > >> > >> * now click ITA Settings, then choose the last entry "P c 1 1" and it > >> loads: > >> > >> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita > > > > Not only that, but I want to obtain all such URLs programmatically! > > > >> You might be able to fool around with that URL and substitute values and > >> get back the data you want (in HTML) via Python. Do you really want > >> HTML results? > >> > >> Hit Ctrl+U to see the source HTML of a webpage > >> > >> Right-click or hit Ctrl + Shift + C to inspect the individual elements > >> of the page > > > > For batch operations, all these manual methods are inefficient. > Yes, but I don't think you'll be able to retrieve the URLs > programmatically. The JavaScript code doesn't put them in the HTML > result, except for that one I showed you, which seems like a mistake on > their part. > > So you'll have to figure out the search fields, and your python program > will have to cycle through the search values: > > Sample from above > gnum = 007 > what = gp > trmat = b,-a-c,c > unconv = P c 1 1 > from = ita The problem is that I must first get all possible combinations of these variables. > wBase = "https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen" > wGnum = "?gnum=" + findgnum > wWhat = "&what=" + findWhat > wTrmat = "&trmat=" + findTrmat > wUnconv = "&unconv=" + findUnconv > wFrom = "&from=" + findFrom > webpage = wBase + wGnum + wWhat + wTrmat + wUnconv + wFrom > > Then if that returns a hit, you'll have to parse the resulting HTML and > extract the exact data you want. > > > > I did something similar a while back using the requests and lxml libraries > ---------------------------------------------------------------- > #build url > wBase = "http://www.usdirectory.com" > wForm = "/ypr.aspx?fromform=qsearch" > wKeyw = "&qhqn=" + keyw > wCityZip = "&qc=" + cityzip > wState = "&qs=" + state > wDist = "&rg=" + str(miles) > wSort = "&sb=a2z" #sort alpha > wPage = "&ap=" #used with the results page number > webpage = wBase + wForm + wKeyw + wCityZip + wState + wDist > > #open URL > page = requests.get(webpage) > tree = html.fromstring(page.content) > > #no matches > matches = tree.xpath('//strong/text()') > if passNbr == 1 and ("No results were found" in str(matches)): > print "No results found for that search" > exit(0) > ---------------------------------------------------------------- > > > > 2.x code file: https://file.io/VdptORSKh5CN > > > > > Best Regards, > > Zhao From PythonList at DancesWithMice.info Wed Sep 14 02:30:30 2022 From: PythonList at DancesWithMice.info (dn) Date: Wed, 14 Sep 2022 18:30:30 +1200 Subject: Virtual PUG-meeting: An applied introduction to Finite State Machines In-Reply-To: References: Message-ID: On 14/09/2022 10.18, Sam Ezeh wrote: > That seems interesting. > > Is this hosted online? And are there any suggested reading materials for > those who might not be able to attend? Yes there are - if you'd like to follow the link (below), the published meeting-details include references. One of my colleagues on the PUG Leadership Team has organised the speaker, but I recall mention of "notebooks", etc. So, there may be further resources which will be made-available during/after the talk itself... If you are a 'late-owl' in the (US) Pacific Time-Zone, you may like to join us 'live'. Sadly, the timings rapidly become anti-social for more easterly time-zones in the Americas. If you are anywhere in the rest of the world, you will find that the meeting falls sometime during your working-day, through to Europeans at-breakfast! You'll be most welcome! > Kind regards, > Sam Ezeh > > On Tue, 13 Sept 2022 at 22:53, dn > wrote: > > An applied introduction to Finite State Machines > 0730 UTC, Wed 21 Sep 2022 > > The basics of Finite State Machines and what they are good for. How to > use FSM for solving optimization problems. > > - A simple traffic jam > - A bigger traffic jam > - A sudoku > > After this lecture there won't be any discrete optimization problem you > won't be able to solve. > > Dr. Bjorn Madsen solves optimization problems using Python, for DoD, > LEGO, Coca Cola, Airbus and many other companies. > > > These meetups have two sessions. At least one is aimed at a beginner > level - in either Python or FSMs. > > Sign-up and more details from https://www.meetup.com/nzpug-auckland/ > > -- > Regards, > =dn > -- > https://mail.python.org/mailman/listinfo/python-list > > -- Regards, =dn From uri at speedy.net Wed Sep 14 09:03:57 2022 From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=) Date: Wed, 14 Sep 2022 16:03:57 +0300 Subject: Python 3.9.14 Message-ID: Hi, Python 3.9.14 has been released on Sept. 6, 2022. As I can see written on https://www.python.org/downloads/release/python-3914/: According to the release calendar specified in PEP 596, Python 3.9 is now in the "security fixes only" stage of its life cycle: the 3.9 branch only accepts security fixes and releases of those are made irregularly in source-only form until October 2025. Python 3.9 isn't receiving regular bug fixes anymore, and binary installers are no longer provided for it. Python 3.9.13 was the last full bugfix release of Python 3.9 with binary installers. Is there a safe way to install a 64-bit version of Python 3.9.14 on Windows? Thanks, Uri. ???? uri at speedy.net From info at carlharrisondesign.com Wed Sep 14 05:42:39 2022 From: info at carlharrisondesign.com (carlharrison) Date: Wed, 14 Sep 2022 11:42:39 +0200 Subject: problem downloading python Message-ID: <000101d8c81e$54916b20$fdb44160$@carlharrisondesign.com> Dear Sir or Madam, I am working on a PC with windows 10 and use Eset internet security. I have tried downloading python 3.10.7 for windows. Using a tutorial I see that a checkbox should appear called "Add python 3/7 to path" but this does not appear whichever version I try to download. I wondered if Eset was stopping this somehow and tried it with Eset switched off but the result is the same. Can you help? Best regards, Carl Harrison. From nospam at dfs.com Tue Sep 13 22:41:09 2022 From: nospam at dfs.com (DFS) Date: Tue, 13 Sep 2022 22:41:09 -0400 Subject: Obtain the query interface url of BCS server. In-Reply-To: <4ea83cbd-b549-485b-b8b8-58b50133ab4cn@googlegroups.com> References: <273b9481-192b-48b1-b057-fcfc22c6cf21n@googlegroups.com> <4ea83cbd-b549-485b-b8b8-58b50133ab4cn@googlegroups.com> Message-ID: <8bbUK.173095$3AK7.82100@fx35.iad> On 9/13/2022 7:29 PM, hongy... at gmail.com wrote: > On Tuesday, September 13, 2022 at 9:33:20 PM UTC+8, DFS wrote: >> On 9/13/2022 3:46 AM, hongy... at gmail.com wrote: >>> On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote: >>>> On 9/12/2022 5:00 AM, hongy... at gmail.com wrote: >>>>> I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script. >>>>> >>>>> However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim? >>>>> >>>>> [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10 >>>>> >>>>> Regards, >>>>> Zhao >>>> You didn't say what you want to query. Are you trying to download >>>> entire sections of the Bilbao Crystallographic Server? >>> >>> I am engaged in some related research and need some specific data used by BCS server. >> What specific data? > > All the data corresponding to the total catalog here: > https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen > >> Is it available elsewhere? > > This is an internationally recognized authoritative data source in this field. Data from other places, even if there are readily available electronic versions, are basically taken from here and are not comprehensive. > >>>> Maybe the admins will give you access to the data. >>> >>> I don't think they will provide such convenience to researchers who have no cooperative relationship with them. >> You can try. Tell the admins what data you want, and ask them for the >> easiest way to get it. >>>> * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen >>>> brings up the table of space group symbols. >>>> >>>> * choose say #7: Pc >>>> >>>> * now click ITA Settings, then choose the last entry "P c 1 1" and it >>>> loads: >>>> >>>> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita >>> >>> Not only that, but I want to obtain all such URLs programmatically! >>> >>>> You might be able to fool around with that URL and substitute values and >>>> get back the data you want (in HTML) via Python. Do you really want >>>> HTML results? >>>> >>>> Hit Ctrl+U to see the source HTML of a webpage >>>> >>>> Right-click or hit Ctrl + Shift + C to inspect the individual elements >>>> of the page >>> >>> For batch operations, all these manual methods are inefficient. >> Yes, but I don't think you'll be able to retrieve the URLs >> programmatically. The JavaScript code doesn't put them in the HTML >> result, except for that one I showed you, which seems like a mistake on >> their part. >> >> So you'll have to figure out the search fields, and your python program >> will have to cycle through the search values: >> >> Sample from above >> gnum = 007 >> what = gp >> trmat = b,-a-c,c >> unconv = P c 1 1 >> from = ita > > The problem is that I must first get all possible combinations of these variables. Shouldn't be too hard, but I've never done some of these things and have no code for you: space group number = gnum = 1 to 230 * use python to put each of those values, one at a time, into the group number field on the webpage * use python to simulate a button click of the ITA Settings button * it should load the HTML of the list of ITA settings for that space group * use python to parse the HTML and extract each of the ITA settings. The line of HTML has 'ITA number' in it. Find each of the 'href' values in the line(s). Real HTML from ITA Settings for space group 10: ITA number Setting10 P 1 2/m 110 P 1 1 2/m10 P 2/m 1 1 If you parse it right you'll have these addresses: "/cgi-bin/cryst/programs//nph-getgen?gnum=010&what=gp" "/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=c,a,b&unconv=P 1 1 2/m&from=ita" "/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=b,c,a&unconv=P 2/m 1 1&from=ita" Then you can parse each of these addresses and build a master list of the valid combinations of: gnum, what, trmat, unconv, from Check into the lxml library, and the 'etree' class. https://lxml.de You can also search gen.lib.rus.ec for the crystallography volumes, and maybe cut and paste data from them. >> wBase = "https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen" >> wGnum = "?gnum=" + findgnum >> wWhat = "&what=" + findWhat >> wTrmat = "&trmat=" + findTrmat >> wUnconv = "&unconv=" + findUnconv >> wFrom = "&from=" + findFrom >> webpage = wBase + wGnum + wWhat + wTrmat + wUnconv + wFrom >> >> Then if that returns a hit, you'll have to parse the resulting HTML and >> extract the exact data you want. >> >> >> >> I did something similar a while back using the requests and lxml libraries >> ---------------------------------------------------------------- >> #build url >> wBase = "http://www.usdirectory.com" >> wForm = "/ypr.aspx?fromform=qsearch" >> wKeyw = "&qhqn=" + keyw >> wCityZip = "&qc=" + cityzip >> wState = "&qs=" + state >> wDist = "&rg=" + str(miles) >> wSort = "&sb=a2z" #sort alpha >> wPage = "&ap=" #used with the results page number >> webpage = wBase + wForm + wKeyw + wCityZip + wState + wDist >> >> #open URL >> page = requests.get(webpage) >> tree = html.fromstring(page.content) >> >> #no matches >> matches = tree.xpath('//strong/text()') >> if passNbr == 1 and ("No results were found" in str(matches)): >> print "No results found for that search" >> exit(0) >> ---------------------------------------------------------------- >> >> >> >> 2.x code file: https://file.io/VdptORSKh5CN >> >> >> >>> Best Regards, >>> Zhao You're welcome. From nospam at dfs.com Tue Sep 13 22:50:08 2022 From: nospam at dfs.com (DFS) Date: Tue, 13 Sep 2022 22:50:08 -0400 Subject: Uninstall tool not working. In-Reply-To: References: Message-ID: On 9/13/2022 3:54 PM, Salvatore Bruzzese wrote: > Hi, > I was trying to uninstall version 3.10.7 of python but I've > encountered problems with the uninstall tool. > I open the python setup program, click on the uninstall button but it > doesn't even start deleting python even though it says that the > process has finished. > Feel free to ask for more details in case I didn't explain it correctly. > > Thanks in advance for your help. https://stackoverflow.com/questions/3515673/how-to-completely-remove-python-from-a-windows-machine From hongyi.zhao at gmail.com Wed Sep 14 08:44:19 2022 From: hongyi.zhao at gmail.com (hongy...@gmail.com) Date: Wed, 14 Sep 2022 05:44:19 -0700 (PDT) Subject: Obtain the query interface url of BCS server. In-Reply-To: <8bbUK.173095$3AK7.82100@fx35.iad> References: <273b9481-192b-48b1-b057-fcfc22c6cf21n@googlegroups.com> <4ea83cbd-b549-485b-b8b8-58b50133ab4cn@googlegroups.com> <8bbUK.173095$3AK7.82100@fx35.iad> Message-ID: On Wednesday, September 14, 2022 at 10:41:32 AM UTC+8, DFS wrote: > On 9/13/2022 7:29 PM, hongy... at gmail.com wrote: > > On Tuesday, September 13, 2022 at 9:33:20 PM UTC+8, DFS wrote: > >> On 9/13/2022 3:46 AM, hongy... at gmail.com wrote: > >>> On Tuesday, September 13, 2022 at 4:20:12 AM UTC+8, DFS wrote: > >>>> On 9/12/2022 5:00 AM, hongy... at gmail.com wrote: > >>>>> I want to do the query from with in script based on the interface here [1]. For this purpose, the underlying posting URL must be obtained, say, the URL corresponding to "ITA Settings" button, so that I can make the corresponding query URL and issue the query from the script. > >>>>> > >>>>> However, I did not find the conversion rules from these buttons to the corresponding URL. Any hints for achieving this aim? > >>>>> > >>>>> [1] https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen?list=new&what=gen&gnum=10 > >>>>> > >>>>> Regards, > >>>>> Zhao > >>>> You didn't say what you want to query. Are you trying to download > >>>> entire sections of the Bilbao Crystallographic Server? > >>> > >>> I am engaged in some related research and need some specific data used by BCS server. > >> What specific data? > > > > All the data corresponding to the total catalog here: > > https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen > > > >> Is it available elsewhere? > > > > This is an internationally recognized authoritative data source in this field. Data from other places, even if there are readily available electronic versions, are basically taken from here and are not comprehensive. > > > >>>> Maybe the admins will give you access to the data. > >>> > >>> I don't think they will provide such convenience to researchers who have no cooperative relationship with them. > >> You can try. Tell the admins what data you want, and ask them for the > >> easiest way to get it. > >>>> * this link: https://www.cryst.ehu.es/cgi-bin/cryst/programs/nph-getgen > >>>> brings up the table of space group symbols. > >>>> > >>>> * choose say #7: Pc > >>>> > >>>> * now click ITA Settings, then choose the last entry "P c 1 1" and it > >>>> loads: > >>>> > >>>> https://www.cryst.ehu.es/cgi-bin/cryst/programs//nph-trgen?gnum=007&what=gp&trmat=b,-a-c,c&unconv=P%20c%201%201&from=ita > >>> > >>> Not only that, but I want to obtain all such URLs programmatically! > >>> > >>>> You might be able to fool around with that URL and substitute values and > >>>> get back the data you want (in HTML) via Python. Do you really want > >>>> HTML results? > >>>> > >>>> Hit Ctrl+U to see the source HTML of a webpage > >>>> > >>>> Right-click or hit Ctrl + Shift + C to inspect the individual elements > >>>> of the page > >>> > >>> For batch operations, all these manual methods are inefficient. > >> Yes, but I don't think you'll be able to retrieve the URLs > >> programmatically. The JavaScript code doesn't put them in the HTML > >> result, except for that one I showed you, which seems like a mistake on > >> their part. > >> > >> So you'll have to figure out the search fields, and your python program > >> will have to cycle through the search values: > >> > >> Sample from above > >> gnum = 007 > >> what = gp > >> trmat = b,-a-c,c > >> unconv = P c 1 1 > >> from = ita > > > > The problem is that I must first get all possible combinations of these variables. > Shouldn't be too hard, but I've never done some of these things and have > no code for you: > > space group number = gnum = 1 to 230 > > * use python to put each of those values, one at a time, into the group > number field on the webpage > > * use python to simulate a button click of the ITA Settings button > > * it should load the HTML of the list of ITA settings for that space group This is the trickiest part of the problem. For this purpose, Vladimir gave the following suggestion here [1]: This is trivial with Selenium. Highlight the element by XPATH and extract URL with .get_attribute("href") [1] https://discuss.python.org/t/obtain-the-query-interface-url-of-bcs-server/18996/2?u=hongyi-zhao > * use python to parse the HTML and extract each of the ITA settings. > The line of HTML has 'ITA number' in it. Find each of the 'href' values > in the line(s). > > Real HTML from ITA Settings for space group 10: > > ITA number bgcolor="#bbbbbb">Setting bgcolor="#f0f0f0">10 href="/cgi-bin/cryst/programs//nph-getgen?gnum=010&what=gp">P 1 > 2/m 1 bgcolor="#f0f0f0">10 href="/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=c,a,b&unconv=P > 1 1 2/m&from=ita">P 1 1 2/m align="center" bgcolor="#f0f0f0">10 href="/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=b,c,a&unconv=P > 2/m 1 1&from=ita">P 2/m 1 1 > > > If you parse it right you'll have these addresses: > > "/cgi-bin/cryst/programs//nph-getgen?gnum=010&what=gp" > > "/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=c,a,b&unconv=P 1 > 1 2/m&from=ita" > > "/cgi-bin/cryst/programs//nph-trgen?gnum=010&what=gp&trmat=b,c,a&unconv=P 2/m > 1 1&from=ita" > > > Then you can parse each of these addresses and build a master list of > the valid combinations of: > > gnum, what, trmat, unconv, from > > Check into the lxml library, and the 'etree' class. https://lxml.de > > You can also search gen.lib.rus.ec for the crystallography volumes, and > maybe cut and paste data from them. The relevant search results are as follows: http://libgen.rs/search.php?&req=International+Tables+for+Crystallography&phrase=1&view=simple&column=def&sort=year&sortmode=DESC Best, Zhao From lal at solute.de Wed Sep 14 11:31:45 2022 From: lal at solute.de (Lars Liedtke) Date: Wed, 14 Sep 2022 17:31:45 +0200 Subject: problem downloading python In-Reply-To: <000101d8c81e$54916b20$fdb44160$@carlharrisondesign.com> References: <000101d8c81e$54916b20$fdb44160$@carlharrisondesign.com> Message-ID: Hello and welcome, Sadly I don't know about Eset internet security, or why you do not get the choice of letting the installer do that, but you could try to add Python manually to your PATH like it is described in https://www.geeksforgeeks.org/how-to-add-python-to-windows-path/ . But maybe some more windows savvy people than me might chime in to offer better advice ;-) . Cheers Lars Lars Liedtke Software Entwickler [Tel.] [Fax] +49 721 98993- [E-Mail] lal at solute.de solute GmbH Zeppelinstra?e 15 76185 Karlsruhe Germany [Logo Solute] Marken der solute GmbH | brands of solute GmbH [Marken] [Advertising Partner] Gesch?ftsf?hrer | Managing Director: Dr. Thilo Gans, Bernd Vermaaten Webseite | www.solute.de Sitz | Registered Office: Karlsruhe Registergericht | Register Court: Amtsgericht Mannheim Registernummer | Register No.: HRB 110579 USt-ID | VAT ID: DE234663798 Informationen zum Datenschutz | Information about privacy policy https://www.solute.de/ger/datenschutz/grundsaetze-der-datenverarbeitung.php Am 14.09.22 um 11:42 schrieb carlharrison: Dear Sir or Madam, I am working on a PC with windows 10 and use Eset internet security. I have tried downloading python 3.10.7 for windows. Using a tutorial I see that a checkbox should appear called "Add python 3/7 to path" but this does not appear whichever version I try to download. I wondered if Eset was stopping this somehow and tried it with Eset switched off but the result is the same. Can you help? Best regards, Carl Harrison. From wlfraed at ix.netcom.com Wed Sep 14 12:34:33 2022 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 14 Sep 2022 12:34:33 -0400 Subject: problem downloading python References: <000101d8c81e$54916b20$fdb44160$@carlharrisondesign.com> Message-ID: On Wed, 14 Sep 2022 11:42:39 +0200, "carlharrison" declaimed the following: >I am working on a PC with windows 10 and use Eset internet security. I have >tried downloading python 3.10.7 for windows. Using a tutorial I see that a >checkbox should appear called "Add python 3/7 to path" but this does not >appear whichever version I try to download. I wondered if Eset was stopping >this somehow and tried it with Eset switched off but the result is the same. >Can you help? Download from where? (There are a number of distributions available, including ones in the M$ "app" store, Visual Studio, etc.) "Add Python..." is a step when RUNNING the installer. It should not appear when DOWNLOADING the installer. The general sequence is: Download the installer file for the distribution. RUN the installer program to install Python (and any other stuff the distribution includes) Hide the installer program/icons -- since they do NOT run Python itself. Pure Python is run from a command line/shell interface; it is NOT a graphical IDE. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From nsh531 at gmail.com Thu Sep 15 07:31:33 2022 From: nsh531 at gmail.com (=?UTF-8?B?16DXqteZINep15jXqNef?=) Date: Thu, 15 Sep 2022 14:31:33 +0300 Subject: why I cannot to install packages? Message-ID: [image: image.png] From barry at barrys-emacs.org Thu Sep 15 11:50:55 2022 From: barry at barrys-emacs.org (Barry Scott) Date: Thu, 15 Sep 2022 16:50:55 +0100 Subject: why I cannot to install packages? In-Reply-To: References: Message-ID: <10C56923-D14C-4B41-9D75-3C47B29FEA76@barrys-emacs.org> > On 15 Sep 2022, at 12:31, ???? ????? wrote: > > [image: image.png] Images are stripped; please copy the text of the error you are seeing and post that. Barry > -- > https://mail.python.org/mailman/listinfo/python-list > From barry at barrys-emacs.org Thu Sep 15 11:49:43 2022 From: barry at barrys-emacs.org (Barry Scott) Date: Thu, 15 Sep 2022 16:49:43 +0100 Subject: Python 3.9.14 In-Reply-To: References: Message-ID: <87E87F5A-633C-406A-8AF6-8F547AF93083@barrys-emacs.org> > On 14 Sep 2022, at 14:03, ?????? wrote: > > Hi, > > Python 3.9.14 has been released on Sept. 6, 2022. As I can see written on > https://www.python.org/downloads/release/python-3914/: > > According to the release calendar specified in PEP 596, Python 3.9 is now > in the "security fixes only" stage of its life cycle: the 3.9 branch only > accepts security fixes and releases of those are made irregularly in > source-only form until October 2025. Python 3.9 isn't receiving regular bug > fixes anymore, and binary installers are no longer provided for it. Python > 3.9.13 was the last full bugfix release of Python 3.9 with binary > installers. > > > Is there a safe way to install a 64-bit version of Python 3.9.14 on Windows? What is stopping you using 3.10? If 3.9 is important then I guess you will need to build it for yourself. Barry > > Thanks, > Uri. > ???? > uri at speedy.net > -- > https://mail.python.org/mailman/listinfo/python-list From grant.b.edwards at gmail.com Thu Sep 15 15:41:40 2022 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 15 Sep 2022 12:41:40 -0700 (PDT) Subject: Can you mock a C function using ctypes? Message-ID: <63237ff4.c80a0220.23ec1.ce9c@mx.google.com> I've done unit testing of C functions using ctypes, and that works nicely until you need to provide a stub/mock function to be called by the C code under test. Can that be done using ctypes? For example, I open a library that contains functon foo() where foo() calls external function bar() which is not contained in the library. Then, I provide a Python bar() function that gets called by foo() when foo() is called? I suspect the anser is "no". I did a bit of googling, and found this question asked on Stack Overflow a few years ago, but the answer made no sense. It appeared to be ansering a different question, but I couldn't quite figure out what that question was. -- Grant From eryksun at gmail.com Thu Sep 15 16:10:21 2022 From: eryksun at gmail.com (Eryk Sun) Date: Thu, 15 Sep 2022 15:10:21 -0500 Subject: Can you mock a C function using ctypes? In-Reply-To: <63237ff4.c80a0220.23ec1.ce9c@mx.google.com> References: <63237ff4.c80a0220.23ec1.ce9c@mx.google.com> Message-ID: On 9/15/22, Grant Edwards wrote: > > Can that be done using ctypes? > > For example, I open a library that contains functon foo() where foo() > calls external function bar() which is not contained in the library. > Then, I provide a Python bar() function that gets called by foo() when > foo() is called? That's straight forward if the library allows the application to pass a function pointer to bar(). ctypes function prototypes are defined by ctypes.CFUNCTYPE(restype, *argtypes, use_errno=False, use_last_error=False) for the cdecl calling convention, or similarly by ctypes.WINFUNCTYPE() to use the Windows stdcall calling convention. A prototype can be instantiated as a function pointer that calls a Python function, e.g. c_bar = prototype(bar). From sjeik_appie at hotmail.com Thu Sep 15 17:09:17 2022 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Thu, 15 Sep 2022 23:09:17 +0200 Subject: Book/resource recommendation about Celery? Message-ID: Hi, I'm using Flask + Celery + RabbitMQ. Can anyone recommend a good book or other resource about Celery?? Thanks! Albert-Jan From grant.b.edwards at gmail.com Thu Sep 15 21:57:47 2022 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Thu, 15 Sep 2022 18:57:47 -0700 (PDT) Subject: Can you mock a C function using ctypes? References: <63237ff4.c80a0220.23ec1.ce9c@mx.google.com> Message-ID: <6323d81b.050a0220.7552d.1ed1@mx.google.com> On 2022-09-15, Eryk Sun wrote: > On 9/15/22, Grant Edwards wrote: >> >> Can that be done using ctypes? >> >> For example, I open a library that contains functon foo() where foo() >> calls external function bar() which is not contained in the library. >> Then, I provide a Python bar() function that gets called by foo() when >> foo() is called? > > That's straight forward if the library allows the application to pass > a function pointer to bar(). That's typically not the case. Does the pointer have to be passed? Or can it be "poked" into a global variable? If so, I guess it would be fairly trivial to write a dummy version of bar() in C that calls a function via a global pointer that could be set via ctypes? > ctypes function prototypes are defined by ctypes.CFUNCTYPE(restype, > *argtypes, use_errno=False, use_last_error=False) for the cdecl > calling convention, or similarly by ctypes.WINFUNCTYPE() to use the > Windows stdcall calling convention. A prototype can be instantiated > as a function pointer that calls a Python function, e.g. c_bar = > prototype(bar). From eryksun at gmail.com Fri Sep 16 01:51:00 2022 From: eryksun at gmail.com (Eryk Sun) Date: Fri, 16 Sep 2022 00:51:00 -0500 Subject: Can you mock a C function using ctypes? In-Reply-To: <6323d81b.050a0220.7552d.1ed1@mx.google.com> References: <63237ff4.c80a0220.23ec1.ce9c@mx.google.com> <6323d81b.050a0220.7552d.1ed1@mx.google.com> Message-ID: On 9/15/22, Grant Edwards wrote: > > Does the pointer have to be passed? Or can it be "poked" into a global > variable? If the library exports the function pointer as a global variable, it can be set in ctypes using the in_dll() method. Unfortunately, as far as I know, a ctypes function prototype can't be used directly for this, at least not easily, since the function pointer it creates doesn't have a `value` or `contents` attribute to set the target address. Instead, the exported global can be accessed as a void pointer. It's clunky, but it works. For example, Python's REPL supports a callback for reading a line from the terminal. Normally it's either hooked by a C extension, such as the readline module, or set to the default function PyOS_StdioReadline(). We can use a ctypes callback to hook into this and chain to the previous function. import ctypes PyOS_RFP = ctypes.CFUNCTYPE( ctypes.c_void_p, ctypes.c_void_p, # stdin (FILE *) ctypes.c_void_p, # stdout (FILE *) ctypes.c_char_p, # prompt ) @PyOS_RFP def readline_hook(stdin, stdout, prompt): print('HOOKED: ', end='', flush=True) return prev_rfp(stdin, stdout, prompt) rfp_vp = ctypes.c_void_p.in_dll(ctypes.pythonapi, 'PyOS_ReadlineFunctionPointer') if rfp_vp: # there's a previous RFP that can be hooked prev_rfp = ctypes.cast(rfp_vp, PyOS_RFP) rfp_vp.value = ctypes.cast(readline_hook, ctypes.c_void_p).value In this trivial example, "HOOKED: " is printed to the terminal before each line is read: HOOKED: >>> def f(): HOOKED: ... pass HOOKED: ... HOOKED: >>> Note that I defined the return type of PyOS_RFP to be c_void_p. Using c_char_p as the return type would convert the result to a bytes object, which is wrong in this case. If the setfunc of the callback's return type has to keep a reference to a converted Python object, such as a bytes object, then the callback has no choice but to leak the reference in order to keep the object alive indefinitely, which usually causes a memory leak. From Ralf_M at t-online.de Fri Sep 16 16:55:52 2022 From: Ralf_M at t-online.de (Ralf M.) Date: Fri, 16 Sep 2022 22:55:52 +0200 Subject: How to replace an instance method? Message-ID: I would like to replace a method of an instance, but don't know how to do it properly. My first naive idea was inst = SomeClass() def new_method(self, param): # do something return whatever inst.method = new_method however that doesn't work: self isn't passed as first parameter to the new inst.method, instead inst.method behaves like a static method. I had a closer look at the decorators classmethod and staticmethod. Unfortunetely I couldn't find a decorator / function "instancemethod" that turns a normal function into an instancemethod. The classmethod documentation contains a reference to the standard type hierarchie, and there is an explanation that an instancemethod is sort of a dynamically created wrapper around a function, which is accessable as __func__. So I modified the last line of the example above to inst.method.__func__ = new_method but got told that __func__ is read only. I found some information about methods in the Descriptor HowTo Guide, but it's about how it works internally and doesn't tell how to solve my problem (at least it doesn't tell me). Now I'm running out of ideas what to try next or what sections of the documentation to read next. Any ideas / pointers? Ralf M. From rosuav at gmail.com Fri Sep 16 17:15:49 2022 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 17 Sep 2022 07:15:49 +1000 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On Sat, 17 Sept 2022 at 07:07, Ralf M. wrote: > > I would like to replace a method of an instance, but don't know how to > do it properly. > > My first naive idea was > > inst = SomeClass() > def new_method(self, param): > # do something > return whatever > inst.method = new_method > > however that doesn't work: self isn't passed as first parameter to > the new inst.method, instead inst.method behaves like a static method. > > I had a closer look at the decorators classmethod and staticmethod. > Unfortunetely I couldn't find a decorator / function "instancemethod" > that turns a normal function into an instancemethod. > > The classmethod documentation contains a reference to the standard > type hierarchie, and there is an explanation that an instancemethod > is sort of a dynamically created wrapper around a function, which > is accessable as __func__. > So I modified the last line of the example above to > > inst.method.__func__ = new_method > > but got told that __func__ is read only. > > I found some information about methods in the Descriptor HowTo Guide, > but it's about how it works internally and doesn't tell how to solve > my problem (at least it doesn't tell me). > > Now I'm running out of ideas what to try next or what sections of the > documentation to read next. > > Any ideas / pointers? > You don't actually want a descriptor, since the purpose of descriptor protocol is to give you information about the instance when the attribute (the method, in this case) was attached to the class. In this case, you can handle it with something far far simpler: inst.method = functools.partial(new_method, inst) ChrisA From eryksun at gmail.com Fri Sep 16 17:34:26 2022 From: eryksun at gmail.com (Eryk Sun) Date: Fri, 16 Sep 2022 16:34:26 -0500 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On 9/16/22, Ralf M. wrote: > I would like to replace a method of an instance, but don't know how to > do it properly. A function is a descriptor that binds to any object as a method. For example: >>> f = lambda self, x: self + x >>> o = 42 >>> m = f.__get__(o) >>> type(m) >>> m.__self__ is o True >>> m(10) 52 From drsalists at gmail.com Fri Sep 16 18:35:12 2022 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 16 Sep 2022 15:35:12 -0700 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On Fri, Sep 16, 2022 at 2:06 PM Ralf M. wrote: > I would like to replace a method of an instance, but don't know how to > do it properly. > You appear to have a good answer, but... are you sure this is a good idea? It'll probably be confusing to future maintainers of this code, and I doubt static analyzers will like it either. I'm not the biggest fan of inheritance you'll ever meet, but maybe this is a good place for it? From list1 at tompassin.net Fri Sep 16 19:45:42 2022 From: list1 at tompassin.net (Thomas Passin) Date: Fri, 16 Sep 2022 19:45:42 -0400 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: <6ea832f7-06ab-c320-8000-16ab163a592c@tompassin.net> Here is an example of probably the easiest way to add an instance method: class Demo: def sqr(self, x): return x*x # Function to turn into a instance method def cube(self, x): return x*x*x d = Demo() print(d.sqr(2)) d.cube = cube.__get__(d) print(d.cube(3)) As for when someone might want to do this kind of thing, one place is when you are running scripts in an existing framework, and you can't change the class definition but you can access an instance. Or you might want to add the method to an existing class to debug and tune it up before you go through the formalities of actually changing an existing project. I once added a class to an running text editor so that it would highlight the current line. That was adding a class instead of a method, but the need was basically the same. On 9/16/2022 6:35 PM, Dan Stromberg wrote: > On Fri, Sep 16, 2022 at 2:06 PM Ralf M. wrote: > >> I would like to replace a method of an instance, but don't know how to >> do it properly. >> > > You appear to have a good answer, but... are you sure this is a good idea? > > It'll probably be confusing to future maintainers of this code, and I doubt > static analyzers will like it either. > > I'm not the biggest fan of inheritance you'll ever meet, but maybe this is > a good place for it? From drsalists at gmail.com Fri Sep 16 23:18:54 2022 From: drsalists at gmail.com (Dan Stromberg) Date: Fri, 16 Sep 2022 20:18:54 -0700 Subject: Python 3.9.14 In-Reply-To: References: Message-ID: ?On Wed, Sep 14, 2022 at 6:05 AM ??????? wrote:? > Hi, > > Python 3.9.14 has been released on Sept. 6, 2022. As I can see written on > https://www.python.org/downloads/release/python-3914/: > > According to the release calendar specified in PEP 596, Python 3.9 is now > in the "security fixes only" stage of its life cycle: the 3.9 branch only > accepts security fixes and releases of those are made irregularly in > source-only form until October 2025. Python 3.9 isn't receiving regular bug > fixes anymore, and binary installers are no longer provided for it. Python > 3.9.13 was the last full bugfix release of Python 3.9 with binary > installers. > > > Is there a safe way to install a 64-bit version of Python 3.9.14 on > Windows? > I use Windows as little as I can, but you could check to see if Conda or the Microsoft Store will give you CPython 3.9 binaries. If that isn't happening, you could try compiling your own from the python.org sources. On Linux, that's pretty simple: extract the archive, ./configure && make && sudo make install I don't know if it'll be easier or harder on WIndows. I want to say you could get MSYS2 as a development environment, but ISTR hearing that Python wants to be compiled with Microsoft's compiler on Windows for ABI compatibility. From Ralf_M at t-online.de Sat Sep 17 17:11:28 2022 From: Ralf_M at t-online.de (Ralf M.) Date: Sat, 17 Sep 2022 23:11:28 +0200 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: Am 16.09.2022 um 23:34 schrieb Eryk Sun: > On 9/16/22, Ralf M. wrote: >> I would like to replace a method of an instance, but don't know how to >> do it properly. > > A function is a descriptor that binds to any object as a method. For example: > > >>> f = lambda self, x: self + x > >>> o = 42 > >>> m = f.__get__(o) > >>> type(m) > > >>> m.__self__ is o > True > >>> m(10) > 52 Thank you and Chris A. for the two suggestions how to replace a method. I tried both inst.method = functools.partial(new_method, inst) and inst.method = new_method.__get__(inst) and both work in my toy example. I will try it on the real code next week. Even though the functools.partial solution is easier to understand (at least for me), I will probably use the __get__ solution as it avoids the import of an extra library. Ralf M. From Ralf_M at t-online.de Sat Sep 17 17:21:08 2022 From: Ralf_M at t-online.de (Ralf M.) Date: Sat, 17 Sep 2022 23:21:08 +0200 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: <5f661dae-4741-7e67-4d18-f2671b9fd49d@t-online.de> Am 17.09.2022 um 00:35 schrieb Dan Stromberg: > > > On Fri, Sep 16, 2022 at 2:06 PM Ralf M. > wrote: > > I would like to replace a method of an instance, but don't know how to > do it properly. > > > You appear to have a good answer, but...? are you sure this is a good idea? It's definitely a dirty hack. > It'll probably be confusing to future maintainers of this code, and I > doubt static analyzers will like it either. I agree that I will have to add sufficient comments for the future maintainer, should there ever be one (and even for me to still understand it next year). I don't use static analyzers. > I'm not the biggest fan of inheritance you'll ever meet, but maybe this > is a good place for it? Using a derived version of the class in question to overwrite the method was my first idea, however I don't instantiate the class in question myself, it is instantiated during the initialisation of another class, so I would at least have to derive a modified version of that as well. And that code is rather complex, with metaclasses and custom decorators, and I feel uncomfortable messing with that, while the method I intend to change is quite simple and straightforward. In case anybody is interested what I'm trying to achieve: It's simple in pandas to read an excel file into a dataframe, but only the cell value is read. Sometimes I need more / other information, e.g. some formatting or the hyperlink in a cell. Reopening the file with openpyxl and getting the info is possible, but cumbersome. Looking into the pandas code for reading excel files (which uses openpyxl internally) I noticed a method (of an internal pandas class) that extracts the value from an openpyxl cell. This method is rather simple and seems the ideal spot to change to get what I want. My idea is to instantiate pandas.ExcelFile (official pandas API), get the reader instance (an attribute of the ExcelFile object) and modify the method of the reader instance. The fact that the method I change and the ExcelFile attribute containing the reader are both private (start with _) doesn't make it any better, but I'm desperate enough to be willing to adapt my code to every major pandas release, if necessary. Ralf M. From rosuav at gmail.com Sat Sep 17 17:27:19 2022 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Sep 2022 07:27:19 +1000 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On Sun, 18 Sept 2022 at 07:20, Ralf M. wrote: > > Am 16.09.2022 um 23:34 schrieb Eryk Sun: > > On 9/16/22, Ralf M. wrote: > >> I would like to replace a method of an instance, but don't know how to > >> do it properly. > > > > A function is a descriptor that binds to any object as a method. For example: > > > > >>> f = lambda self, x: self + x > > >>> o = 42 > > >>> m = f.__get__(o) > > >>> type(m) > > > > >>> m.__self__ is o > > True > > >>> m(10) > > 52 > > Thank you and Chris A. for the two suggestions how to replace a method. > > I tried both > inst.method = functools.partial(new_method, inst) > and > inst.method = new_method.__get__(inst) > and both work in my toy example. > I will try it on the real code next week. > > Even though the functools.partial solution is easier to understand (at > least for me), I will probably use the __get__ solution as it avoids > the import of an extra library. > The two are basically equivalent. Using functools.partial emphasizes the fact that all you're doing is "locking in" the first parameter; using the __get__ method emphasizes the fact that functions are, fundamentally, the same thing as methods. Choose whichever one makes sense to you! ChrisA From eryksun at gmail.com Sat Sep 17 19:37:57 2022 From: eryksun at gmail.com (Eryk Sun) Date: Sat, 17 Sep 2022 18:37:57 -0500 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On 9/17/22, Chris Angelico wrote: > > The two are basically equivalent. Using functools.partial emphasizes > the fact that all you're doing is "locking in" the first parameter; > using the __get__ method emphasizes the fact that functions are, > fundamentally, the same thing as methods. Choose whichever one makes > sense to you! Functions are really not "fundamentally, the same thing as methods". They're only the same in that they're both callable. Also, a method's __getattribute__() falls back on looking up attributes on the underlying function (i.e. the method's __func__), such as inspecting the __name__ and __code__. A fundamental difference is that, unlike a function, a method is not a descriptor. Thus if a method object is set as an attribute of a type, the method does not rebind as a new method when accessed as an attribute of an instance of the type. From rosuav at gmail.com Sat Sep 17 20:21:07 2022 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Sep 2022 10:21:07 +1000 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On Sun, 18 Sept 2022 at 09:37, Eryk Sun wrote: > > On 9/17/22, Chris Angelico wrote: > > > > The two are basically equivalent. Using functools.partial emphasizes > > the fact that all you're doing is "locking in" the first parameter; > > using the __get__ method emphasizes the fact that functions are, > > fundamentally, the same thing as methods. Choose whichever one makes > > sense to you! > > Functions are really not "fundamentally, the same thing as methods". > They're only the same in that they're both callable. Also, a method's > __getattribute__() falls back on looking up attributes on the > underlying function (i.e. the method's __func__), such as inspecting > the __name__ and __code__. A fundamental difference is that, unlike a > function, a method is not a descriptor. Thus if a method object is set > as an attribute of a type, the method does not rebind as a new method > when accessed as an attribute of an instance of the type. An unbound method in Python 2 was distinctly different from a function, but in Python 3, they really truly are the same thing. A bound method object is a small wrapper around a function which binds its 'self' parameter; that's a distinction, but not a fundamental one. Yes, a bound method isn't a descriptor; that's not really a huge difference either, though. A method IS a function. A bound method is a function with one argument locked in, but still a function. ChrisA From avi.e.gross at gmail.com Sat Sep 17 20:28:05 2022 From: avi.e.gross at gmail.com (avi.e.gross at gmail.com) Date: Sat, 17 Sep 2022 20:28:05 -0400 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: <004e01d8caf5$857fe7a0$907fb6e0$@gmail.com> >From your description, Chris, it sounds like the functional programming technique often called currying. A factory function is created where one (or more) parameters are sort of frozen in so the user never sees or cares about them, and a modified or wrapped function is returned. In this case, the function now knows what object it is attached to as this. -----Original Message----- From: Python-list On Behalf Of Chris Angelico Sent: Saturday, September 17, 2022 8:21 PM To: python-list at python.org Subject: Re: How to replace an instance method? On Sun, 18 Sept 2022 at 09:37, Eryk Sun wrote: > > On 9/17/22, Chris Angelico wrote: > > > > The two are basically equivalent. Using functools.partial emphasizes > > the fact that all you're doing is "locking in" the first parameter; > > using the __get__ method emphasizes the fact that functions are, > > fundamentally, the same thing as methods. Choose whichever one makes > > sense to you! > > Functions are really not "fundamentally, the same thing as methods". > They're only the same in that they're both callable. Also, a method's > __getattribute__() falls back on looking up attributes on the > underlying function (i.e. the method's __func__), such as inspecting > the __name__ and __code__. A fundamental difference is that, unlike a > function, a method is not a descriptor. Thus if a method object is set > as an attribute of a type, the method does not rebind as a new method > when accessed as an attribute of an instance of the type. An unbound method in Python 2 was distinctly different from a function, but in Python 3, they really truly are the same thing. A bound method object is a small wrapper around a function which binds its 'self' parameter; that's a distinction, but not a fundamental one. Yes, a bound method isn't a descriptor; that's not really a huge difference either, though. A method IS a function. A bound method is a function with one argument locked in, but still a function. ChrisA -- https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Sat Sep 17 20:30:13 2022 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 18 Sep 2022 10:30:13 +1000 Subject: How to replace an instance method? In-Reply-To: <004e01d8caf5$857fe7a0$907fb6e0$@gmail.com> References: <004e01d8caf5$857fe7a0$907fb6e0$@gmail.com> Message-ID: On Sun, 18 Sept 2022 at 10:29, wrote: > > > From your description, Chris, it sounds like the functional programming > technique often called currying. A factory function is created where one (or > more) parameters are sort of frozen in so the user never sees or cares about > them, and a modified or wrapped function is returned. In this case, the > function now knows what object it is attached to as this. > Correct. I avoided the technical term but that is exactly what's happening. ChrisA From eryksun at gmail.com Sat Sep 17 21:37:58 2022 From: eryksun at gmail.com (Eryk Sun) Date: Sat, 17 Sep 2022 20:37:58 -0500 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On 9/17/22, Chris Angelico wrote: > > A method IS a function. A bound method is a function with one argument > locked in, but still a function. We were talking past each other. A method object is not a function object. You're talking about a function defined in a class that's accessed as a method of an instance of the class. In the class, that's just a function object; it's exactly a function, not merely fundamentally the same thing as one. It's only when accessed as an attribute of an instance of the type that the function's descriptor __get__() method is called to bind it and the instance to a method object that sets the instance as its __self__ and the function as its __func__. From c.buhtz at posteo.jp Sun Sep 18 05:46:45 2022 From: c.buhtz at posteo.jp (c.buhtz at posteo.jp) Date: Sun, 18 Sep 2022 09:46:45 +0000 Subject: Which architectures to support in a CI like Travis? Message-ID: <4MVjd65zQ8z9rxL@submission02.posteo.de> Hello, I am using TravisCI for my project on GitHub. The project is packaged for Debian, Ubuntu, Arch and several other distros. All this distros support multiple architectures and they have their own test machines to take care that all packages working on all archs. On my side (upstream) I wonder which arch I should "support" in my TravisCI configuration. I wan't to speed up travis and I want to save energy and carbon. I suspect that my Python code should run on much every platform that offers a Python interpreter. Of course there are edge cases. But they would be captured by the distros own test environments. So is there a good and objective reason to support multiple (and maybe) exotic platforms in a CI pipeline on upstream? Kind Christian From lal at solute.de Sun Sep 18 14:16:35 2022 From: lal at solute.de (Lars Liedtke) Date: Sun, 18 Sep 2022 20:16:35 +0200 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: <35bed7ed-8870-9fe6-3553-d65810f6c08d@solute.de> Hey, What has not been mentioned yet is simple delegation. Often you want to rewrite a method, maybe have different (more or less) parameters and additionally keep the old methods for backwards compatibility. Or mark it as deprecated at a later point. So you could write the new method and change the old method to call the new method but with the parameters the new method expects. If you explain this in the docstrings as well. Then you do not need to actually replace the method. Or you had a completely different use-case in mind, that I missed. Cheers Lars Lars Liedtke Software Entwickler [Tel.] [Fax] +49 721 98993- [E-Mail] lal at solute.de solute GmbH Zeppelinstra?e 15 76185 Karlsruhe Germany [Logo Solute] Marken der solute GmbH | brands of solute GmbH [Marken] [Advertising Partner] Gesch?ftsf?hrer | Managing Director: Dr. Thilo Gans, Bernd Vermaaten Webseite | www.solute.de Sitz | Registered Office: Karlsruhe Registergericht | Register Court: Amtsgericht Mannheim Registernummer | Register No.: HRB 110579 USt-ID | VAT ID: DE234663798 Informationen zum Datenschutz | Information about privacy policy https://www.solute.de/ger/datenschutz/grundsaetze-der-datenverarbeitung.php Am 16.09.22 um 22:55 schrieb Ralf M.: I would like to replace a method of an instance, but don't know how to do it properly. My first naive idea was inst = SomeClass() def new_method(self, param): # do something return whatever inst.method = new_method however that doesn't work: self isn't passed as first parameter to the new inst.method, instead inst.method behaves like a static method. I had a closer look at the decorators classmethod and staticmethod. Unfortunetely I couldn't find a decorator / function "instancemethod" that turns a normal function into an instancemethod. The classmethod documentation contains a reference to the standard type hierarchie, and there is an explanation that an instancemethod is sort of a dynamically created wrapper around a function, which is accessable as __func__. So I modified the last line of the example above to inst.method.__func__ = new_method but got told that __func__ is read only. I found some information about methods in the Descriptor HowTo Guide, but it's about how it works internally and doesn't tell how to solve my problem (at least it doesn't tell me). Now I'm running out of ideas what to try next or what sections of the documentation to read next. Any ideas / pointers? Ralf M. From martinp.dipaola at gmail.com Mon Sep 19 08:32:38 2022 From: martinp.dipaola at gmail.com (Martin Di Paola) Date: Mon, 19 Sep 2022 09:32:38 -0300 Subject: Which architectures to support in a CI like Travis? In-Reply-To: <4MVjd65zQ8z9rxL@submission02.posteo.de> References: <4MVjd65zQ8z9rxL@submission02.posteo.de> Message-ID: <20220919123238.6eiholrdotex757h@gmail.com> I would depend on the project. In the crytoanalysis tool that I developing, "cryptonita", I just manipule bytes. Nothing that could depend on the distro so my CI picks one OS and run the tests there. Project: https://github.com/cryptonitas/cryptonita CI: https://github.com/cryptonitas/cryptonita/blob/master/.github/workflows/test.yml On the other extreme I have "byexample", a tool that takes the examples in your docs and run them as automated tests. It supports different languages (Python, Ruby, Java, ...) and it works using the interpreter of each languages. An there is the challenge for its CI. Because byexample highly depends on the version of the interpreter, the CI config tries a lot of different scenarios Project: https://byexamples.github.io/ CI: https://github.com/byexamples/byexample/blob/master/.github/workflows/test.yml I don't tests different distros but I should for some cases that I suspect that it could be differences in how some interpreters behave. An about OS, I'm planning to add MacOS to the CI because I know that some users had problems in the past in that platform because how byexample interacts with the terminal. So two projects, both in Python, but with two totally different dependencies on the environment where they run, so their CI are different. The two examples are using Gitlab actions but the same applies to TravisCI. Thanks, Martin. On Sun, Sep 18, 2022 at 09:46:45AM +0000, c.buhtz at posteo.jp wrote: >Hello, > >I am using TravisCI for my project on GitHub. The project is packaged >for Debian, Ubuntu, Arch and several other distros. > >All this distros support multiple architectures and they have their own >test machines to take care that all packages working on all archs. > >On my side (upstream) I wonder which arch I should "support" in my >TravisCI configuration. I wan't to speed up travis and I want to save >energy and carbon. > >I suspect that my Python code should run on much every platform that >offers a Python interpreter. Of course there are edge cases. But they >would be captured by the distros own test environments. > >So is there a good and objective reason to support multiple (and maybe) >exotic platforms in a CI pipeline on upstream? > >Kind >Christian >-- >https://mail.python.org/mailman/listinfo/python-list From gweatherby at uchc.edu Mon Sep 19 09:06:42 2022 From: gweatherby at uchc.edu (Weatherby,Gerard) Date: Mon, 19 Sep 2022 13:06:42 +0000 Subject: How to replace an instance method? In-Reply-To: <5f661dae-4741-7e67-4d18-f2671b9fd49d@t-online.de> References: <5f661dae-4741-7e67-4d18-f2671b9fd49d@t-online.de> Message-ID: Just subclass and override whatever method you wish to modify ?Private? is conceptual. Mostly it means when the next version of a module comes out, code that you wrote that accesses *._ parts of the module might break. ___ import pandas class MyClass(pandas.ExcelFile.OpenpyxlReader): def _convert_cell(self, cell, convert_float: bool) -> 'Scalar': """override""" # do whatever you want, or call the base class version return super()._convert_cell(cell, convert_float) ? Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular Biology and Biophysics UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu On Sep 17, 2022, 5:29 PM -0400, Ralf M. , wrote: *** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. *** Am 17.09.2022 um 00:35 schrieb Dan Stromberg: On Fri, Sep 16, 2022 at 2:06 PM Ralf M. > wrote: I would like to replace a method of an instance, but don't know how to do it properly. You appear to have a good answer, but... are you sure this is a good idea? It's definitely a dirty hack. It'll probably be confusing to future maintainers of this code, and I doubt static analyzers will like it either. I agree that I will have to add sufficient comments for the future maintainer, should there ever be one (and even for me to still understand it next year). I don't use static analyzers. I'm not the biggest fan of inheritance you'll ever meet, but maybe this is a good place for it? Using a derived version of the class in question to overwrite the method was my first idea, however I don't instantiate the class in question myself, it is instantiated during the initialisation of another class, so I would at least have to derive a modified version of that as well. And that code is rather complex, with metaclasses and custom decorators, and I feel uncomfortable messing with that, while the method I intend to change is quite simple and straightforward. In case anybody is interested what I'm trying to achieve: It's simple in pandas to read an excel file into a dataframe, but only the cell value is read. Sometimes I need more / other information, e.g. some formatting or the hyperlink in a cell. Reopening the file with openpyxl and getting the info is possible, but cumbersome. Looking into the pandas code for reading excel files (which uses openpyxl internally) I noticed a method (of an internal pandas class) that extracts the value from an openpyxl cell. This method is rather simple and seems the ideal spot to change to get what I want. My idea is to instantiate pandas.ExcelFile (official pandas API), get the reader instance (an attribute of the ExcelFile object) and modify the method of the reader instance. The fact that the method I change and the ExcelFile attribute containing the reader are both private (start with _) doesn't make it any better, but I'm desperate enough to be willing to adapt my code to every major pandas release, if necessary. Ralf M. -- https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mYWFkAugwhU4HgCv9nRg1vSJhyJCA8RApcnyGTRNGQYTTmvVigqANAagTbBwo96YFdHmzfCYU8gN3KpVmcrmOg$ From mats at wichmann.us Mon Sep 19 10:10:09 2022 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 19 Sep 2022 08:10:09 -0600 Subject: Which architectures to support in a CI like Travis? In-Reply-To: <4MVjd65zQ8z9rxL@submission02.posteo.de> References: <4MVjd65zQ8z9rxL@submission02.posteo.de> Message-ID: <5a32d7e9-8e27-b860-6efd-8bf5e2604b2b@wichmann.us> On 9/18/22 03:46, c.buhtz at posteo.jp wrote: > Hello, > > I am using TravisCI for my project on GitHub. The project is packaged > for Debian, Ubuntu, Arch and several other distros. > > All this distros support multiple architectures and they have their own > test machines to take care that all packages working on all archs. > > On my side (upstream) I wonder which arch I should "support" in my > TravisCI configuration. I wan't to speed up travis and I want to save > energy and carbon. > > I suspect that my Python code should run on much every platform that > offers a Python interpreter. Of course there are edge cases. But they > would be captured by the distros own test environments. > > So is there a good and objective reason to support multiple (and maybe) > exotic platforms in a CI pipeline on upstream? > > Kind > Christian Kind of unrelated to the actual question, but if you start doing anything serious under Travis you'll run out of free minutes rather quickly. My project had to just give up on it after they changed their licensing models.... From pathakaditi96 at gmail.com Mon Sep 19 07:43:31 2022 From: pathakaditi96 at gmail.com (python 3.0 is not working) Date: Mon, 19 Sep 2022 17:13:31 +0530 Subject: Python is not working on my desktop Message-ID: <39D61C3C-B85F-4F49-B531-F6825422CF57@hxcore.ol> ? ? Sent from [1]Mail for Windows ? References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 From henhanna at gmail.com Sun Sep 18 11:49:37 2022 From: henhanna at gmail.com (Hen Hanna) Date: Sun, 18 Sep 2022 08:49:37 -0700 (PDT) Subject: Count-Trailing-Zeros(x) --- Can I use Log(x) to see if X is equal to ( Integer * 10 ^ k) ??? Message-ID: Count how many zeros are at the end of your int: def end_zeros(num): s = str(num) return len(s) - len(s.rstrip("0")) print(end_zeros(10)) # == 1 print(end_zeros(101)) # == 0 print(end_zeros(245)) # == 0 print(end_zeros(100100)) # == 2 ---------------- Writing a loop (using % and // ) feels like [reinventing the wheel] and seems intrinsically wrong. i like this code (above) because it's not reinventing the wheel --- the Python implementor has already done the loop (using % and // ), so i'm just using that efficient tool (routine). From henhanna at gmail.com Sun Sep 18 12:19:01 2022 From: henhanna at gmail.com (Hen Hanna) Date: Sun, 18 Sep 2022 09:19:01 -0700 (PDT) Subject: Caching (memoization) in (Gauche / Lisp / Scheme) and Python Message-ID: <9431b335-f2dd-46ca-ab1d-f4653dd67d9dn@googlegroups.com> so... for a few days i've been revising this Code (in Gauche / Lisp / Scheme) to make it run faster.. and last night i could improve it enough to give me the result i wanted in 72 minutes or so (on my slow PC at home). ( Maybe... within a few months, i'll write the same program in Python .... to see if it runs 10 or 20 times faster. ) this was the first time i've used Caching (memoization). ----- instead of calculating (at run-time) Factorial(x) and Combination(x,y) millions of times, i made 2 tables in advance... A simple Table-lookup (Vector-ref in Scheme) seems 100 -- 1000 times faster. One thought i had was... Maybe Python's Factorial(x) and Combination(x,y) (in Numpy ?) are already so fast that... i don't have to do the Caching (memoization) ??? From sadidalam2012 at gmail.com Mon Sep 19 01:51:14 2022 From: sadidalam2012 at gmail.com (Shadid Alam) Date: Mon, 19 Sep 2022 11:21:14 +0530 Subject: Regarding python 3.10 installation Message-ID: Hello I?ve been trying to install updated python version i.e. python 10.7. Its showing installed but whenever I open python it comes up with the older version i.e. 10.9 . Please fix this issue is possible From c.buhtz at posteo.jp Mon Sep 19 10:27:47 2022 From: c.buhtz at posteo.jp (c.buhtz at posteo.jp) Date: Mon, 19 Sep 2022 14:27:47 +0000 Subject: Which architectures to support in a CI like =?UTF-8?Q?Travis=3F?= In-Reply-To: <5a32d7e9-8e27-b860-6efd-8bf5e2604b2b@wichmann.us> References: <4MVjd65zQ8z9rxL@submission02.posteo.de> <5a32d7e9-8e27-b860-6efd-8bf5e2604b2b@wichmann.us> Message-ID: Dear Mats, thanks for the reply. Am 19.09.2022 16:10 schrieb Mats Wichmann: > Kind of unrelated to the actual question, but if you start doing > anything serious under Travis you'll run out of free minutes rather > quickly. My project had to just give up on it after they changed > their licensing models.... I'm new to Travis and didn't noticed that piece of information. ;) Now I am more scared that Travis asked me for my credit card data no matter that I choose a "free" plan. I'm a bit shocked and still don't understand the "credit" infos on my Travis account. There are "Credits" ("used 4470 of 10000 credits") and "OSS only credits" ("0 of your 0 monthly credits"). OSS == "open source software" ? I'm confused and thought that Travis is free for public github repos. Any suggestions. Kind Christian From dvl at psu.edu Mon Sep 19 13:31:31 2022 From: dvl at psu.edu (Christman, Roger Graydon) Date: Mon, 19 Sep 2022 17:31:31 +0000 Subject: memoization (original Subject lost because mailer lost the whole thread) Message-ID: "Hen Hanna" asked: > so... for a few days i've been revising this Code (in Gauche / Lisp / Scheme) to make it run faster.. and last night i could improve it enough to give me the result i wanted in 72 minutes or so (on my slow PC at home). > ( Maybe... within a few months, i'll write the same program in Python .... to see if it runs 10 or 20 times faster. ) > this was the first time i've used Caching (memoization). ----- instead of calculating (at run-time) Factorial(x) and Combination(x,y) millions of times, i made 2 tables in advance... A simple Table-lookup (Vector-ref in Scheme) seems 100 -- 1000 times faster. > One thought i had was... Maybe Python's Factorial(x) and Combination(x,y) (in Numpy ?) are already so fast that... i don't have to do the Caching (memoization) ??? Memoization will generally be very fast -- since it is essentially a table-lookup. If it uses a hash-table (which is common for dictionaries), it would be close to a constant-time access for any entry; otherwise, if it uses some tree structure, it might be logarithmic in the number of entries in the tree. But that fast access comes at a price of storage -- linear with respect to the number of items stored (versus no memory cost incurred when not memoizing). What effect this has when you call these functions "millions of times" depends very much one how many of those calls are on the same values. If all of the calls have different arguments, memoization would not find it in the table yet, and would have to recompute as normal -- and you would end up with no time savings, but a considerable memory allocation for all of those newly-cached values that you never retrieve. The big wins come from asking the same questions repeatedly. Now, as far as Python's functionality, I would not expect it to do any memoization for you. It certainly could not predict what arguments would provide, but it still could still have a reasonable implementation without memoization. Your Factorial(x) and Combination(x,y) would both require a time linear with respect to the value of x, with no memory cost incurred. But if you were to spend most of your application asking the same questions, I would not expect these functions to do any caching or memoization, since I would expect very few applications would have enough calls to these functions to make it worthwhile. So calling these functions, you can expect a linear time for each function call, and if you expect to frequently repeat arguments, then you should add your own memoization for an amortized linear time. And fortunately, Python makes memoization very easy, by using a dictionary as a default value. I've done that often for classroom purposes for cases where it makes a big difference (recursive Fibonacci accelerates from exponential time to linear time). And the process is straightforward enough that you could even define a decorator that could be applied to any function you choose. I don't have an example handy just because I never took the time to write one. Roger Christman Pennsylvania State University From eryksun at gmail.com Mon Sep 19 15:23:49 2022 From: eryksun at gmail.com (Eryk Sun) Date: Mon, 19 Sep 2022 14:23:49 -0500 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On 9/18/22, Stefan Ram wrote: > ram at zedat.fu-berlin.de (Stefan Ram) writes (abbreviated): >>types.MethodType( function, instance ) >>functools.partial( function, instance ) >>new_method.__get__( instance ) > > I wonder which of these three possibilities expresses > the idea of creating a new method from a function and > an instance most clearly. Using types.MethodType(function, instance) is the most clear and correct of the three. Using the function's descriptor __get__() method is equivalent in terms of the result. That said, the descriptor protocol is an intermediate-level concept, so manually calling __get__() isn't friendly to novices or casual users of the language. Using a functools.partial isn't the expected method type, with __func__ and __self__ attributes, and, unlike a method, it doesn't expose the wrapped function's __code__, __name__, __module__, __doc__, __annotations__, __defaults__, __kwdefaults__, __closure__, __globals__, or __builtins__. If dynamic inspection matters, using a functools.partial won't work directly with dis.dis(), inspect.getfile(), inspect.getsource(), inspect.getdoc(), inspect.get_annotations(), inspect.getcallargs(), or inspect.getclosurevars(). From 2QdxY4RzWzUUiLuE at potatochowder.com Mon Sep 19 15:19:01 2022 From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com) Date: Mon, 19 Sep 2022 15:19:01 -0400 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On 2022-09-18 at 09:11:28 +0000, Stefan Ram wrote: > ram at zedat.fu-berlin.de (Stefan Ram) writes (abbreviated): > >types.MethodType( function, instance ) > >functools.partial( function, instance ) > >new_method.__get__( instance ) > > I wonder which of these three possibilities expresses > the idea of creating a new method from a function and > an instance most clearly. The first one. And only the first one. The second one requires too much inside knowledge of Python to make the leap from currying to instance method. The third one doesn't even mention the function. Also, in Python, if I'm applying dunder methods directly (it's okay to write them), then I'm doing something wrong. From michael.stemper at gmail.com Mon Sep 19 14:32:30 2022 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Mon, 19 Sep 2022 13:32:30 -0500 Subject: Regarding python 3.10 installation In-Reply-To: References: Message-ID: In order to save time for the experts here: On 19/09/2022 00.51, Shadid Alam wrote: > Hello I?ve been trying to install updated python version i.e. python 10.7. There is no python 10.7. Are you possibly trying to install python 3.10.7? On what OS are you trying to install it? How are you trying to install it? Do you get any error messages when you try to install it? > Its showing installed but whenever I open python it comes up with the older How are you "opening" it? > version i.e. 10.9 . Please fix this issue is possible In order to do that, we'll need your root password and IP address. -- Michael F. Stemper Psalm 82:3-4 From PythonList at DancesWithMice.info Mon Sep 19 16:28:48 2022 From: PythonList at DancesWithMice.info (dn) Date: Tue, 20 Sep 2022 08:28:48 +1200 Subject: Python is not working on my desktop In-Reply-To: <39D61C3C-B85F-4F49-B531-F6825422CF57@hxcore.ol> References: <39D61C3C-B85F-4F49-B531-F6825422CF57@hxcore.ol> Message-ID: <73522323-7ab9-103d-1f17-a19761b7ca31@DancesWithMice.info> Regret to advise that many members will not click on links, and particularly not when the link is the only message-content. (flagged as likely spam-email!) Please describe the problem, including an explanation of from where the Python interpreter was downloaded, which OpSys is in-use, what commands were issued at the desktop, and copy-paste any results. Does the Python documentation (including the specific page covering installing Python on MS-Windows) help with the above? List-members are volunteers and not $paid. Please help us to help you! Also, please be advised that there is a Python-Tutor Discussion List which handles beginner and learner conversations. On 19/09/2022 23.43, python 3.0 is not working wrote: > ? > > ? > > Sent from [1]Mail for Windows -- Regards, =dn From eryksun at gmail.com Mon Sep 19 17:05:33 2022 From: eryksun at gmail.com (Eryk Sun) Date: Mon, 19 Sep 2022 16:05:33 -0500 Subject: How to replace an instance method? In-Reply-To: References: Message-ID: On 9/19/22, 2QdxY4RzWzUUiLuE at potatochowder.com <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: > On 2022-09-18 at 09:11:28 +0000, > Stefan Ram wrote: > >> ram at zedat.fu-berlin.de (Stefan Ram) writes (abbreviated): >> >types.MethodType( function, instance ) >> >functools.partial( function, instance ) >> >new_method.__get__( instance ) >> >> I wonder which of these three possibilities expresses >> the idea of creating a new method from a function and >> an instance most clearly. > > The first one. And only the first one. > > The second one requires too much inside knowledge of Python to make the > leap from currying to instance method. > > The third one doesn't even mention the function. The OP's example named the function "new_method". In general the third case would be func.__get__(instance). It's how the interpreter binds a new method when a function from the class hierarchy is accessed as an instance attribute. When a function from the class hierarchy is accessed as an attribute of the class, it's equivalent to calling func.__get__(None, cls), which just returns a reference to the function. To use the descriptor protocol to bind a function as a method of the class requires wrapping it with the classmethod descriptor type. For example, classmethod(func).__get__(None, cls) returns a method object with __self__ that references the cls type. Of course, calling types.MethodType(func, cls) is easier to understand and the preferred way. From hjp-python at hjp.at Tue Sep 20 03:11:24 2022 From: hjp-python at hjp.at (Peter J. Holzer) Date: Tue, 20 Sep 2022 09:11:24 +0200 Subject: memoization (original Subject lost because mailer lost the whole thread) In-Reply-To: References: Message-ID: <20220920071124.zzkh6rdcmbx5kigh@hjp.at> On 2022-09-19 17:31:31 +0000, Christman, Roger Graydon wrote: > And fortunately, Python makes memoization very easy, by using a > dictionary as a default value. I've done that often for classroom > purposes for cases where it makes a big difference (recursive > Fibonacci accelerates from exponential time to linear time). And the > process is straightforward enough that you could even define a > decorator that could be applied to any function you choose. Such a decorator is already part of the Python standard library: https://docs.python.org/3/library/functools.html#functools.lru_cache hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From 110119104012 at aalimec.ac.in Tue Sep 20 11:36:28 2022 From: 110119104012 at aalimec.ac.in (asika) Date: Tue, 20 Sep 2022 08:36:28 -0700 Subject: when I open a python idle it's constantly showing subprocess connection error Message-ID: <9351D2D8-F5F4-4347-86C3-9786B45602B1@hxcore.ol> ? ? Sent from [1]Mail for Windows ? References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 From 110119104012 at aalimec.ac.in Tue Sep 20 11:36:28 2022 From: 110119104012 at aalimec.ac.in (asika) Date: Tue, 20 Sep 2022 08:36:28 -0700 Subject: when I open a python idle it's constantly showing subprocess connection error Message-ID: <9351D2D8-F5F4-4347-86C3-9786B45602B1@hxcore.ol> ? ? Sent from [1]Mail for Windows ? References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 From mats at wichmann.us Tue Sep 20 13:26:34 2022 From: mats at wichmann.us (Mats Wichmann) Date: Tue, 20 Sep 2022 11:26:34 -0600 Subject: when I open a python idle it's constantly showing subprocess connection error In-Reply-To: <9351D2D8-F5F4-4347-86C3-9786B45602B1@hxcore.ol> References: <9351D2D8-F5F4-4347-86C3-9786B45602B1@hxcore.ol> Message-ID: On 9/20/22 09:36, asika wrote: > > > > > Sent from [1]Mail for Windows > > > > References > > Visible links > 1. https://go.microsoft.com/fwlink/?LinkId=550986 dunno if you were trying to send screenshots or something, that doesn't work here. Try: https://docs.python.org/3/library/idle.html#startup-failure Hint: you usually named a file you were working on the same as something IDLE uses, that's the usual cause these days. From phd at phdru.name Tue Sep 20 13:26:00 2022 From: phd at phdru.name (Oleg Broytman) Date: Tue, 20 Sep 2022 20:26:00 +0300 Subject: SQLObject 3.10.0 Message-ID: Hello! I'm pleased to announce version 3.10.0, the first release of branch 3.10 of SQLObject. What's new in SQLObject ======================= Contributors for this release are James Hudson, Juergen Gmach, Hugo van Kemenade. Many thanks! Features -------- * Allow connections in ``ConnectionHub`` to be strings. This allows to open a new connection in every thread. * Add compatibility with ``Pendulum``. Tests ----- * Run tests with Python 3.10. CI -- * GitHub Actions. * Stop testing at Travis CI. * Stop testing at AppVeyor. Documentation ------------- * DevGuide: source code must be pure ASCII. * DevGuide: ``reStructuredText`` format for docstrings is recommended. * DevGuide: de-facto good commit message format is required: subject/body/trailers. * DevGuide: ``conventional commit`` format for commit message subject lines is recommended. * DevGuide: ``Markdown`` format for commit message bodies is recommended. * DevGuide: commit messages must be pure ASCII. For a more complete list, please see the news: http://sqlobject.org/News.html What is SQLObject ================= SQLObject is an object-relational mapper. Your database tables are described as classes, and rows are instances of those classes. SQLObject is meant to be easy to use and quick to get started with. SQLObject supports a number of backends: MySQL, PostgreSQL, SQLite; connections to other backends - Firebird, Sybase, MSSQL and MaxDB (also known as SAPDB) - are lesser debugged). Python 2.7 or 3.4+ is required. Where is SQLObject ================== Site: http://sqlobject.org Development: http://sqlobject.org/devel/ Mailing list: https://lists.sourceforge.net/mailman/listinfo/sqlobject-discuss Download: https://pypi.org/project/SQLObject/3.10.0 News and changes: http://sqlobject.org/News.html StackOverflow: https://stackoverflow.com/questions/tagged/sqlobject Example ======= Create a simple class that wraps a table:: >>> from sqlobject import * >>> >>> sqlhub.processConnection = connectionForURI('sqlite:/:memory:') >>> >>> class Person(SQLObject): ... fname = StringCol() ... mi = StringCol(length=1, default=None) ... lname = StringCol() ... >>> Person.createTable() Use the object:: >>> p = Person(fname="John", lname="Doe") >>> p >>> p.fname 'John' >>> p.mi = 'Q' >>> p2 = Person.get(1) >>> p2 >>> p is p2 True Queries:: >>> p3 = Person.selectBy(lname="Doe")[0] >>> p3 >>> pc = Person.select(Person.q.lname=="Doe").count() >>> pc 1 Oleg. -- Oleg Broytman https://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From 110119104012 at aalimec.ac.in Tue Sep 20 11:36:28 2022 From: 110119104012 at aalimec.ac.in (asika) Date: Tue, 20 Sep 2022 08:36:28 -0700 Subject: when I open a python idle it's constantly showing subprocess connection error Message-ID: <9351D2D8-F5F4-4347-86C3-9786B45602B1@hxcore.ol> ? ? Sent from [1]Mail for Windows ? References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 From tripdarlinq at gmail.com Wed Sep 21 08:49:59 2022 From: tripdarlinq at gmail.com (tripd...@gmail.com) Date: Wed, 21 Sep 2022 05:49:59 -0700 (PDT) Subject: Did i debug this code correctly? Message-ID: <08edb534-487c-46e2-b014-79452faa100dn@googlegroups.com> I am trying to solve this job task given to my girlfriend and here's the task and what i did ~ The two Python functions liDDA and liB are supposed to solve the same task. Firstly, explain the task clearly and then fix any semantic errors in the two algorithms (so that they actually do solve the task correctly). #Suppose you have found the following (incomplete) Python code on a scrap #The two Python functions are not quite finished and contain some errors. -Firstly, give a detailed critique of the incomplete Python code given above. Secondly, fix any syntax errors so that the Python code runs, ensure the code follows good Python coding style and improve all the variable and function names. def liDDA(a,b,c,d) P = [] e = c-a f = d-b s = max(e,f) x,y = a,b for i in range(s): P.append((round(x), round(y))) x += e/s y += f/s print(P) liDDA(0,1,6,4)#when running this function i got this = [(0, 1), (1, 2), (2, 2), (3, 2), (4, 3), (5, 4)] Second function #function 2 to do same task def liB(a,b,c,d) P = [] e = c-a f = d-b g = 2*f - e y = b for x in range(a,c+1) P.append((x,y)) if (g>0) y += 1 g = g - 2*e g = g + 2*f print(P) liB(0,1,6,4) From mypenservices9 at gmail.com Wed Sep 21 09:49:46 2022 From: mypenservices9 at gmail.com (My Pen Services) Date: Wed, 21 Sep 2022 16:49:46 +0300 Subject: Issue with the Installation Message-ID: Hello, I am having an issue after installing the latest application. I do not seem to locate it on my PC nor am I able to get the shortcut for launching the application. When I try running the application again, I only get 3 options- 1. Customize 2. Repair 3. Uninstall What might be the issue? I have also attached a screenshot for the same. Regards George From diegofpsouza at gmail.com Wed Sep 21 12:41:24 2022 From: diegofpsouza at gmail.com (Diego Souza) Date: Wed, 21 Sep 2022 13:41:24 -0300 Subject: How to replace an instance method? Message-ID: Another possibility would be to use a lambda function or a callable object. This adds an overhead but would also allow you to inject new parameters that go into the function call. It also does not require any extra import. obj.old_method_name = lambda *a, **kw: new_method_name(obj, *a, **kw) A full example goes like this: class C: def __init__(self): self.value = 21 def get(self): return self.value def new_get(self): return self.value * 2 obj = C() print(obj.get()) obj.get = lambda *a, **kw: new_get(obj, *a, **kw) print(obj.get()) This would first output 21 and then 42. -- What you are trying to do requires more than just replacing the function _convert_cell. By default, OpenpyxlReader loads the workbook in read_only mode, discarding all links. This means that the cell object present in _convert_cell has no hyperlink attribute. There is no option to make it load the links. To force it to be loaded, we need to replace load_workbook as well. This method asks openpyxl to load the workbook, deciding whether it will discard the links or not. The second problem is that as soon as you instantiate an ExcelFile object it will instantiate an OpenpyxlReader and load the file. Leaving you with no time to replace the functions. Happily, ExcelFile gets the engine class from a static dictionary called _engines. This means that we can extend OpenpyxlReader, overwrite those two methods and replace the reference in ExcelFile._engines. The full source is: import pandas as pd class MyOpenpyxlReader(pd.ExcelFile.OpenpyxlReader): def load_workbook(self, filepath_or_buffer): from openpyxl import load_workbook return load_workbook( filepath_or_buffer, read_only=False, data_only=False, keep_links=True ) def _convert_cell(self, cell, convert_float: bool): value = super()._convert_cell(cell, convert_float) if cell.hyperlink is None: return value else: return (value, cell.hyperlink.target) pd.ExcelFile._engines["openpyxl"] = MyOpenpyxlReader df = pd.read_excel("links.xlsx") print(df) The source above worked on python 3.8.10, pandas 1.5.0, and openpyxl 3.0.10. The output for a sample xlsx file with the columns id, a page name (with links), and the last access is shown next. The first element in the second column's output tuple is the cell's text and the second element is the cell's link: id page last access 0 1 (google, https://www.google.com/) 2022-04-12 1 2 (gmail, https://gmail.com/) 2022-02-06 2 3 (maps, https://www.google.com/maps) 2022-02-17 3 4 (bbc, https://bbc.co.uk/) 2022-08-30 4 5 (reddit, https://www.reddit.com/) 2022-12-02 5 6 (stackoverflow, https://stackoverflow.com/) 2022-05-25 -- Should you do any of this? No. 1. What makes a good developer is his ability to create clear and maintainable code. Any of these options are clearly not clear, increase cognitive complexity, and reduce reliability. 2. We are manipulating internal class attributes and internal methods (those starting with _). Internal elements are not guaranteed to stay there over different versions, even minor updates. You should not manipulate them unless you are working on a fixed library version, like implementing tests and checking if the internal state has changed, hacking it, or debugging. Python assumes you will access these attributes wisely. 3. If you are working with other developers and you commit this code there is a huge chance another developer is using a slightly different pandas version that misses one of these elements. You will break the build, your team will complain and start thinking you are a naive developer. 4. Even if you adapt your code for multiple pandas versions you will end up with multiple ifs and different implementations. You don't want to maintain this over time. 5. It clearly takes more time to understand pandas' internals than writing your reader using openpyxl. It is not cumbersome, and if it changes the execution time from 20ms to 40ms but is much more reliable and maintainable we surely prefer the latter. The only scenario I see in which this would be acceptable is when you or your boss have an important presentation in the next hour, and you need a quick fix to make it work in order to demonstrate it. After the presentation is over and people have validated the functionality you should properly implement it. Keep It Simple and Stupid (KISS) -- Diego Souza Wespa Intelligent Systems Rio de Janeiro - Brasil On Mon, Sep 19, 2022 at 1:00 PM wrote: > > > From: "Weatherby,Gerard" > Date: Mon, 19 Sep 2022 13:06:42 +0000 > Subject: Re: How to replace an instance method? > Just subclass and override whatever method you wish to modify > ?Private? is conceptual. Mostly it means when the next version of a module comes out, code that you wrote that accesses *._ parts of the module might break. > ___ > > > import pandas > > > class MyClass(pandas.ExcelFile.OpenpyxlReader): > > def _convert_cell(self, cell, convert_float: bool) -> 'Scalar': > """override""" > # do whatever you want, or call the base class version > return super()._convert_cell(cell, convert_float) > > ? > Gerard Weatherby | Application Architect NMRbox | NAN | Department of Molecular Biology and Biophysics > UConn Health 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu > On Sep 17, 2022, 5:29 PM -0400, Ralf M. , wrote: > *** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. *** > > Am 17.09.2022 um 00:35 schrieb Dan Stromberg: > > > On Fri, Sep 16, 2022 at 2:06 PM Ralf M. > wrote: > > I would like to replace a method of an instance, but don't know how to > do it properly. > > > You appear to have a good answer, but... are you sure this is a good idea? > > It's definitely a dirty hack. > > It'll probably be confusing to future maintainers of this code, and I > doubt static analyzers will like it either. > > I agree that I will have to add sufficient comments for the future > maintainer, should there ever be one (and even for me to still > understand it next year). I don't use static analyzers. > > I'm not the biggest fan of inheritance you'll ever meet, but maybe this > is a good place for it? > > Using a derived version of the class in question to overwrite the > method was my first idea, however I don't instantiate the class in > question myself, it is instantiated during the initialisation of > another class, so I would at least have to derive a modified version of > that as well. And that code is rather complex, with metaclasses and > custom decorators, and I feel uncomfortable messing with that, while > the method I intend to change is quite simple and straightforward. > > In case anybody is interested what I'm trying to achieve: > > It's simple in pandas to read an excel file into a dataframe, but only > the cell value is read. Sometimes I need more / other information, e.g. > some formatting or the hyperlink in a cell. Reopening the file with > openpyxl and getting the info is possible, but cumbersome. > Looking into the pandas code for reading excel files (which uses > openpyxl internally) I noticed a method (of an internal pandas class) > that extracts the value from an openpyxl cell. This method is rather > simple and seems the ideal spot to change to get what I want. > > My idea is to instantiate pandas.ExcelFile (official pandas API), get > the reader instance (an attribute of the ExcelFile object) and modify > the method of the reader instance. > > The fact that the method I change and the ExcelFile attribute containing > the reader are both private (start with _) doesn't make it any better, > but I'm desperate enough to be willing to adapt my code to every major > pandas release, if necessary. > > Ralf M. > -- > https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mYWFkAugwhU4HgCv9nRg1vSJhyJCA8RApcnyGTRNGQYTTmvVigqANAagTbBwo96YFdHmzfCYU8gN3KpVmcrmOg$ From python at mrabarnett.plus.com Wed Sep 21 14:50:59 2022 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 21 Sep 2022 19:50:59 +0100 Subject: Issue with the Installation In-Reply-To: References: Message-ID: On 2022-09-21 14:49, My Pen Services wrote: > Hello, > > I am having an issue after installing the latest application. I do not seem > to locate it on my PC nor am I able to get the shortcut for launching the > application. On Windows, it should be in the main application menu. > When I try running the application again, I only get 3 options- > > 1. Customize > 2. Repair > 3. Uninstall > That's the installer. It's for installing or uninstalling Python, and nothing else. > What might be the issue? I have also attached a screenshot for the same. > This list automatically removes screenshots and other images. From cs at cskk.id.au Wed Sep 21 17:54:45 2022 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 22 Sep 2022 07:54:45 +1000 Subject: Did i debug this code correctly? In-Reply-To: <08edb534-487c-46e2-b014-79452faa100dn@googlegroups.com> References: <08edb534-487c-46e2-b014-79452faa100dn@googlegroups.com> Message-ID: Preamble: this list strips attachments. If you attached some screenshots orfile attachments, we do no receive them. We generally don't like screenshots unless they're vital to demonstrating some visual thing. We _much_ prefer text pasted inline in the message. So the below is written about a message with no attachments. On 21Sep2022 05:49, tripd... at gmail.com wrote: >I am trying to solve this job task given to my girlfriend and here's the task and what i did > >~ The two Python functions liDDA and liB are supposed to solve the same task. Firstly, explain >the task clearly and then fix any semantic errors in the two algorithms (so that they actually do >solve the task correctly). Well, as near as I can tell you've not explained the task. It isn't apparent to me what the purpose of either function is. I can see what they _do_, but I have no idea why. Without knowing the intended task, it is impossible to "fix" either of them because we don't know what the "correct" behaviour should be. >#Suppose you have found the following (incomplete) Python code on a scrap >#The two Python functions are not quite finished and contain some errors. >-Firstly, give a detailed critique of the incomplete Python code given above. Secondly, fix any syntax errors so that the Python code runs, ensure the code follows good Python coding style and improve all the variable and function names. I don't see any critique either. What I do see is no comments or docstrings. That would be the first thing I would want to change, probably followed by more expressive variable names. BTW, the second function contains a syntax error, and would not run. Cheers, Cameron Simpson From c.buhtz at posteo.jp Thu Sep 22 09:28:57 2022 From: c.buhtz at posteo.jp (c.buhtz at posteo.jp) Date: Thu, 22 Sep 2022 13:28:57 +0000 Subject: How to get the current set =?UTF-8?Q?LOG=5FMASK=20in=20Python=27s?= =?UTF-8?Q?=20syslog=20module=3F?= Message-ID: X-Post: https://stackoverflow.com/q/73814924/4865723 Hello, I'm aware that there is a `logging` package that is more _modern_ then [`syslog`](https://docs.python.org/3/library/syslog.html). But I have old code here to deal with that does use `syslog`. So that question is specific to `syslog` and not to `logging`. I would like to get the current `LOG_MASK`, which is kind of a logging level. According to the docu it seems that `syslog` doesn't have a mechanism for that. Does someone has an idea? The docu also tells me that `syslog` does let pass all messages by default. My point is I do manipulate via `syslog.setlogmask()` the current log leve. At the end I would like to set it back to its previous value. Kind From richardmoseley4 at gmail.com Thu Sep 22 09:44:38 2022 From: richardmoseley4 at gmail.com (Richard Moseley) Date: Thu, 22 Sep 2022 14:44:38 +0100 Subject: How to get the current set LOG_MASK in Python's syslog module? In-Reply-To: References: Message-ID: According to documentation syslog.setlogmask returns the current mask so save the value to reset later on. Oldval = syslog.setlogmask(newmask) This sets oldval to original mask. On Thu, 22 Sep 2022, 14:32 , wrote: > X-Post: https://stackoverflow.com/q/73814924/4865723 > > Hello, > > I'm aware that there is a `logging` package that is more _modern_ then > [`syslog`](https://docs.python.org/3/library/syslog.html). But I have > old code here to deal with that does use `syslog`. So that question is > specific to `syslog` and not to `logging`. > > I would like to get the current `LOG_MASK`, which is kind of a logging > level. According to the docu it seems that `syslog` doesn't have a > mechanism for that. > > Does someone has an idea? > > The docu also tells me that `syslog` does let pass all messages by > default. > My point is I do manipulate via `syslog.setlogmask()` the current log > leve. At the end I would like to set it back to its previous value. > > Kind > -- > https://mail.python.org/mailman/listinfo/python-list > From fabianjoseph063 at gmail.com Thu Sep 22 03:36:43 2022 From: fabianjoseph063 at gmail.com (Fabian Joseph) Date: Thu, 22 Sep 2022 00:36:43 -0700 (PDT) Subject: Problem when scraping the 100 Movie titles. In-Reply-To: References: <4af7159a-ceb6-4709-dff1-f9ee59398d29@wichmann.us> Message-ID: <1f038aa2-6a35-4fa8-9016-5d8368203f3dn@googlegroups.com> #Try using, it's save in json format of the website: import json import requests from bs4 import BeautifulSoup url = "https://www.empireonline.com/movies/features/best-movies-2/" soup = BeautifulSoup(requests.get(url).content, "html.parser") data = json.loads(soup.select_one("#__NEXT_DATA__").contents[0]) # uncomment this to print all data: #print(json.dumps(data, indent=4)) def find_articles(data): if isinstance(data, dict): for k, v in data.items(): if k.startswith("ImageMeta:"): yield v['image']['name'] else: yield from find_articles(v) elif isinstance(data, list): for i in data: yield from find_articles(i) for a in find_articles(data): print(a) From home.pjs at gmail.com Thu Sep 22 12:00:54 2022 From: home.pjs at gmail.com (Peter Smith) Date: Thu, 22 Sep 2022 09:00:54 -0700 (PDT) Subject: Python3.6 tkinter bug? In-Reply-To: References: <0780ab80-b8de-90cb-4f78-7e1ee5611af7@udel.edu> Message-ID: On Wednesday, February 1, 2017 at 11:55:35 AM UTC, Terry Reedy wrote: > On 2/1/2017 1:37 AM, Christian Gollwitzer wrote: > > Am 01.02.17 um 00:02 schrieb MRAB: > >> On 2017-01-31 22:34, Christian Gollwitzer wrote: > >>>> .!frame.!checkbutton > >>>> .!frame.!checkbutton2 > >>>> .!frame2.!checkbutton > >>>> .!frame2.!checkbutton2 > >>> > >>> > >> Perhaps someone who knows Tcl and tk can tell me, but I notice that in > >> the first example, the second part of the widget names are unique, > >> whereas in the second example, the second part of the widget names are > >> the reused (both "!checkbutton" and "!checkbutton2" occur twice). > > > > It is indeed the reason, but it has some strange legacy cause: the > > default name for the checkbutton-linked variable is the name of the > > button inside the parent. Therefore creating a checkbutton has the side > > effect of creating a variable with the button's name. > > > > In this case, the first buttons in the frames are linked to a variable > > called "!checkbutton" and the other two are linked to "!checkbutton2". > > (a variable name in Tcl can be anything apart from the empty string). > > This can also be demonstrated by this Tcl script: > > > > package require Tk > > > > pack [frame .f1] > > pack [frame .f2] > > > > pack [checkbutton .f1.c1 -text "A" ] > > pack [checkbutton .f1.c2 -text "B" ] > > > > pack [checkbutton .f2.c1 -text "C" ] > > pack [checkbutton .f2.c2 -text "D" ] > > > > which is equivalent to the Python code above. > > > > Note that this surprising behaviour was corrected for the (modern) ttk > > widgets, so if "checkbutton" is replaced by "ttk::checkbutton", they are > > not any longer linked. In Python, that would be > > > > from tkinter import ttk > > ... > > w = ttk.Checkbutton() > > > > (Personally, I'm not using the legacy widgets any longer) > Christian, could you repeat any relevant parts of your comments on the > tracker, especially any ideas on how we might fix tkinter? > https://bugs.python.org/issue29402 > >> Do the names need to be: > >> > >> .!frame.!checkbutton > >> .!frame.!checkbutton2 > >> .!frame2.!checkbutton3 > >> .!frame2.!checkbutton4 > Serhiy considered that but, not knowing that this would cause a > regression, we both liked numbering within parent better. > > There is a similar issue with radiobuttons on ttk.OptionMenus that > existed *before* the 3.6 name changes. > https://bugs.python.org/issue25684 > So there seems to be a systematic issue with tk or how we are (mis)using it. > > Good question. Maybe there should be unique variable names? I.e., if the > > script is changed into package require Tk > > > > pack [frame .f1] > > pack [frame .f2] > > > > pack [checkbutton .f1.c1 -text "A" -variable v1] > > pack [checkbutton .f1.c2 -text "B" -variable v2] > > > > pack [checkbutton .f2.c1 -text "C" -variable v3] > > pack [checkbutton .f2.c2 -text "D" -variable v4] > > > > then they are also not linked. > -- > Terry Jan Reedy From home.pjs at gmail.com Thu Sep 22 12:06:18 2022 From: home.pjs at gmail.com (Peter Smith) Date: Thu, 22 Sep 2022 09:06:18 -0700 (PDT) Subject: Python3.6 tkinter bug? In-Reply-To: References: <0780ab80-b8de-90cb-4f78-7e1ee5611af7@udel.edu> Message-ID: On Wednesday, February 1, 2017 at 11:55:35 AM UTC, Terry Reedy wrote: > On 2/1/2017 1:37 AM, Christian Gollwitzer wrote: > > Am 01.02.17 um 00:02 schrieb MRAB: > >> On 2017-01-31 22:34, Christian Gollwitzer wrote: > >>>> .!frame.!checkbutton > >>>> .!frame.!checkbutton2 > >>>> .!frame2.!checkbutton > >>>> .!frame2.!checkbutton2 > >>> > >>> > >> Perhaps someone who knows Tcl and tk can tell me, but I notice that in > >> the first example, the second part of the widget names are unique, > >> whereas in the second example, the second part of the widget names are > >> the reused (both "!checkbutton" and "!checkbutton2" occur twice). > > > > It is indeed the reason, but it has some strange legacy cause: the > > default name for the checkbutton-linked variable is the name of the > > button inside the parent. Therefore creating a checkbutton has the side > > effect of creating a variable with the button's name. > > > > In this case, the first buttons in the frames are linked to a variable > > called "!checkbutton" and the other two are linked to "!checkbutton2". > > (a variable name in Tcl can be anything apart from the empty string). > > This can also be demonstrated by this Tcl script: > > > > package require Tk > > > > pack [frame .f1] > > pack [frame .f2] > > > > pack [checkbutton .f1.c1 -text "A" ] > > pack [checkbutton .f1.c2 -text "B" ] > > > > pack [checkbutton .f2.c1 -text "C" ] > > pack [checkbutton .f2.c2 -text "D" ] > > > > which is equivalent to the Python code above. > > > > Note that this surprising behaviour was corrected for the (modern) ttk > > widgets, so if "checkbutton" is replaced by "ttk::checkbutton", they are > > not any longer linked. In Python, that would be > > > > from tkinter import ttk > > ... > > w = ttk.Checkbutton() > > > > (Personally, I'm not using the legacy widgets any longer) > Christian, could you repeat any relevant parts of your comments on the > tracker, especially any ideas on how we might fix tkinter? > https://bugs.python.org/issue29402 > >> Do the names need to be: > >> > >> .!frame.!checkbutton > >> .!frame.!checkbutton2 > >> .!frame2.!checkbutton3 > >> .!frame2.!checkbutton4 > Serhiy considered that but, not knowing that this would cause a > regression, we both liked numbering within parent better. > > There is a similar issue with radiobuttons on ttk.OptionMenus that > existed *before* the 3.6 name changes. > https://bugs.python.org/issue25684 > So there seems to be a systematic issue with tk or how we are (mis)using it. > > Good question. Maybe there should be unique variable names? I.e., if the > > script is changed into package require Tk > > > > pack [frame .f1] > > pack [frame .f2] > > > > pack [checkbutton .f1.c1 -text "A" -variable v1] > > pack [checkbutton .f1.c2 -text "B" -variable v2] > > > > pack [checkbutton .f2.c1 -text "C" -variable v3] > > pack [checkbutton .f2.c2 -text "D" -variable v4] > > > > then they are also not linked. > -- > Terry Jan Reedy It looks as if the issue is indeed that the expression to the right of CheckButton(... variable= must be an expression. This works for me global checkbix, checkbuttons checkbix += 1 b = tk.Checkbutton(frame, text=text, variable=checkbuttons[checkbix], ...) From wlfraed at ix.netcom.com Thu Sep 22 21:55:18 2022 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 22 Sep 2022 21:55:18 -0400 Subject: How to get the current set LOG_MASK in Python's syslog module? References: Message-ID: On Thu, 22 Sep 2022 13:28:57 +0000, c.buhtz at posteo.jp declaimed the following: >I would like to get the current `LOG_MASK`, which is kind of a logging >level. According to the docu it seems that `syslog` doesn't have a >mechanism for that. > There is a function .LOG_MASK() but it seems to just return 2^arg >>> import syslog >>> syslog.LOG_MASK(1) 2 >>> syslog.LOG_MASK(2) 4 >>> syslog.LOG_MASK(3) 8 >>> syslog.LOG_MASK(4) 16 >>> syslog.LOG_MASK(0) 1 >>> syslog.LOG_MASK(256) 1 >>> syslog.LOG_MASK(16) 65536 >>> syslog.__doc__ >>> syslog.LOG_MASK(17) 131072 >>> syslog.LOG_MASK(32) 1 >>> syslog.LOG_MASK(24) 16777216 >>> That is in the Windows Linux (Debian) subsystem. >Does someone has an idea? > >The docu also tells me that `syslog` does let pass all messages by >default. >My point is I do manipulate via `syslog.setlogmask()` the current log >leve. At the end I would like to set it back to its previous value. > >Kind -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From rosuav at gmail.com Thu Sep 22 23:32:51 2022 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 23 Sep 2022 13:32:51 +1000 Subject: How to get the current set LOG_MASK in Python's syslog module? In-Reply-To: References: Message-ID: On Thu, 22 Sept 2022 at 23:46, Richard Moseley wrote: > > According to documentation syslog.setlogmask returns the current mask so > save the value to reset later on. > > Oldval = syslog.setlogmask(newmask) > > This sets oldval to original mask. This on its own suggests an odd technique that should work but won't be great: oldval = syslog.setlogmask(1234) syslog.setlogmask(oldval) But the Python function just passes the value straight to the underlying system call, and thus treats zero specially: """ The setlogmask() function sets this logmask for the calling process, and returns the previous mask. If the mask argument is 0, the current logmask is not modified. """ So you should be able to do: current_mask = syslog.setlogmask(0) The Python docs do say to refer to the man pages, but IMO it would be worth mentioning this feature specifically in the docs. ChrisA From PythonList at DancesWithMice.info Fri Sep 23 01:22:33 2022 From: PythonList at DancesWithMice.info (dn) Date: Fri, 23 Sep 2022 17:22:33 +1200 Subject: NZPUG: Smart Iterator Challenge Message-ID: <8e873ba4-47d6-51b8-344b-20d6cf4e1746@DancesWithMice.info> A week-by-week Challenge series. A new venture by the Auckland Branch of the New Zealand Python Users' Group (AuckPUG) Challenge-week 1: Implementing a monolithic solution, starts today! All welcome! - will suit Python-Apprentices ready to move-on from 'the basics' and Python-Journeymen - will welcome participation and coaching-contributions from Python-Masters, with a special Challenge to be issued at that level Kiwi accent optional (most people are no-good at it anyway). With special thanks to our own Leam Hall, for feedback and advice - that said, all errors and omissions are mine. To avoid OT traffic on-list, please find all details, and file any follow-up questions through the Meetup-site: https://www.meetup.com/nzpug-auckland/ (or email myself directly). Are you up for a challenge? Regards =dn (for Pete and DJ) From mmontgomery at levado.to Fri Sep 23 16:59:12 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Fri, 23 Sep 2022 17:59:12 -0300 Subject: on implementing a toy oop-system References: <86r10qg89w.fsf@levado.to> Message-ID: <86o7v5kdsv.fsf@levado.to> ram at zedat.fu-berlin.de (Stefan Ram) writes: > Meredith Montgomery writes: >>Is that at all possible somehow? Alternatively, how would you do your >>toy oop-system? > > Maybe something along those lines: > > from functools import partial > > def counter_create( object ): > object[ "n" ]= 0 > def counter_increment( object ): > object[ "n" ]+= 1 > def counter_value( object ): > return object[ "n" ] > > counter_class =( counter_create, counter_increment, counter_value ) > > def inherit_from( class_, target ): > class_[ 0 ]( target ) > for method in class_[ 1: ]: > target[ method.__name__ ]= partial( method, target ) > > car = dict() > > inherit_from( counter_class, car ) > > print( car[ "counter_value" ]() ) > car[ "counter_increment" ]() > print( car[ "counter_value" ]() ) > > . The "create" part is simplified. I just wanted to show how > to make methods like "counter_increment" act on the object > that inherited them using "partial". I really liked this idea. I organized it my way. Have a look. (Thank you for the lecture!) --8<---------------cut here---------------start------------->8--- from functools import partial def Counter(name = None): o = {"name": name if name else "untitled", "n": 0} def inc(o): o["n"] += 1 return o o["inc"] = inc def get(o): return o["n"] o["get"] = get return o def Car(maker): o = {"maker": maker, "state": "off"} inherit_from(Counter, o) def on(o): if o["is_on"](): raise ValueError("oh, no: car is already on") o["inc"]() print(f"{o['maker']}: bruum!") o["state"] = "on" return o o["on"] = partial(on, o) def off(o): if o["is_off"](): raise ValueError("oh, no: car is already off") print(f"{o['maker']}: spat!") o["state"] = "off" return o o["off"] = partial(off, o) def is_on(o): return o["state"] == "on" o["is_on"] = partial(is_on, o) def is_off(o): return o["state"] == "off" o["is_off"] = partial(is_off, o) return o def main(): car1 = Car("Ford") car2 = Car("VW") for i in range(5): car1["on"](); car1["off"]() for i in range(3): car2["on"](); car2["off"]() print(f"car turned on = {car1['get']()} ({car1['maker']})") print(f"car turned on = {car2['get']()} ({car2['maker']})") ## (*) How to inherit the methods from a class ## def inherit_from(C, target): o = C() for k, v in o.items(): if callable(v): target[k] = partial(v, target) else: target[k] = v --8<---------------cut here---------------end--------------->8--- From rosuav at gmail.com Fri Sep 23 18:06:31 2022 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 24 Sep 2022 08:06:31 +1000 Subject: on implementing a toy oop-system In-Reply-To: <86o7v5kdsv.fsf@levado.to> References: <86r10qg89w.fsf@levado.to> <86o7v5kdsv.fsf@levado.to> Message-ID: On Sat, 24 Sept 2022 at 07:52, Meredith Montgomery wrote: > > def Counter(name = None): > o = {"name": name if name else "untitled", "n": 0} > def inc(o): > o["n"] += 1 > return o > o["inc"] = inc > def get(o): > return o["n"] > o["get"] = get > return o > Want a neat demo of how classes and closures are practically the same thing? def Counter(name=None): if not name: name = "untitled" n = 0 def inc(): nonlocal n; n += 1 def get(): return n return locals() Aside from using a nonlocal declaration rather than "self.n", this is extremely similar to classes, yet there are no classes involved. A class statement creates a namespace. The locals() function returns the function's namespace. Each call to Counter() creates a new closure context, just like each call to a constructor creates a new object. There's very little difference, at a fundamental level :) ChrisA From usenet at gkbrk.com Fri Sep 23 19:14:56 2022 From: usenet at gkbrk.com (Leo) Date: Fri, 23 Sep 2022 23:14:56 -0000 (UTC) Subject: How would one scrap property listing website like this? References: Message-ID: On Thu, 22 Sep 2022 09:36:47 -0700 (PDT), tripd... at gmail.com wrote: > https://nigeriapropertycentre.com/ > Has anyone scrap something like this before? > probably i should try power bi first to see if it can? You can try something like this. import urllib.request from bs4 import BeautifulSoup URL = "https://nigeriapropertycentre.com/for-rent/lagos" UA = "Mozilla/5.0 (X11; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/ 105.0" def fetch_url(url): headers = {"User-Agent": UA} req = urllib.request.Request(url, headers={"User-Agent": UA}) resp = urllib.request.urlopen(req) return resp.read().decode("utf-8") html = fetch_url(URL) soup = BeautifulSoup(html, "html.parser") for item in soup.find_all(itemtype="https://schema.org/ListItem"): row = {} row["name"] = item.find(itemprop="name").text row["url"] = item.find(itemprop="url").get("href", "") row["image"] = item.find(itemprop="image").get("src", "") row["content-title"] = item.find(class_="content-title").text row["address"] = item.find("address").text.strip() row["description"] = item.find(itemprop="description").text.strip() row["added-on"] = item.find("span", class_="added-on").text.strip() row["price"] = item.find("span", class_="price").parent.text.strip() row["aux"] = [] for li in item.find("ul", class_="aux-info").find_all("li"): row["aux"].append(li.text.strip()) print(row) From mmontgomery at levado.to Fri Sep 23 21:08:16 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Fri, 23 Sep 2022 22:08:16 -0300 Subject: on implementing a toy oop-system References: <86r10qg89w.fsf@levado.to> <86o7v5kdsv.fsf@levado.to> Message-ID: <868rm937gf.fsf@levado.to> Chris Angelico writes: > On Sat, 24 Sept 2022 at 07:52, Meredith Montgomery > wrote: >> >> def Counter(name = None): >> o = {"name": name if name else "untitled", "n": 0} >> def inc(o): >> o["n"] += 1 >> return o >> o["inc"] = inc >> def get(o): >> return o["n"] >> o["get"] = get >> return o >> > > Want a neat demo of how classes and closures are practically the same thing? > > def Counter(name=None): > if not name: name = "untitled" > n = 0 > def inc(): > nonlocal n; n += 1 > def get(): > return n > return locals() > > Aside from using a nonlocal declaration rather than "self.n", this is > extremely similar to classes, yet there are no classes involved. > > A class statement creates a namespace. The locals() function returns > the function's namespace. > > Each call to Counter() creates a new closure context, just like each > call to a constructor creates a new object. > > There's very little difference, at a fundamental level :) I started out this way, but I had to change direction to implement inheritance: the difficulty with closures seems to be lexical scoping, which makes it hard (or impossible) for me to move these closures to another ``object''. For instance, the nonlocal /n/ in /inc/ above is forever bound to that closure; there seems to be no way to make /inc/ update some /n/ in another ``object'', which is needed in my conception of inheritance. I think Python would have to let me duplicate closures. (Thanks for showing me /locals()/.) From axy at declassed.art Mon Sep 26 07:17:47 2022 From: axy at declassed.art (Axy) Date: Mon, 26 Sep 2022 12:17:47 +0100 Subject: Asynchronous execution of synchronous functions Message-ID: <4a6d6d26-318c-f596-5f92-89326085efff@declassed.art> Hi there, is there a library to call functions in context of a thread? For example, as in asyncsqlite which has a thread and a queue I mean has anyone generalized such an approach already? If not, I'll do it myself, no problem. It's a kind of tiny stuff, like atomicwrites, which is quite difficult to dig out with modern search engines that have already rolled down to hell a decade ago. Axy. From diegofpsouza at gmail.com Mon Sep 26 13:15:59 2022 From: diegofpsouza at gmail.com (Diego Souza) Date: Mon, 26 Sep 2022 14:15:59 -0300 Subject: Asynchronous execution of synchronous functions In-Reply-To: References: Message-ID: Did you check the ThreadPoolExecutor or the ProcessPoolExecutor? They won't give you atomic writes unless you add a Lock or a Condition, but they will execute your code in another thread or process. https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor Keep in mind that Python's threads have a global interpreter lock (GIL) that prevents full parallelism. Processes work as expected, but require IPC and pickable objects in and out. -- Diego Souza Wespa Intelligent Systems Rio de Janeiro - Brasil On Mon, Sep 26, 2022 at 1:01 PM wrote: > From: Axy > To: Python List > Date: Mon, 26 Sep 2022 12:17:47 +0100 > Subject: Asynchronous execution of synchronous functions > Hi there, > > is there a library to call functions in context of a thread? For > example, as in asyncsqlite which has a thread and a queue I mean has > anyone generalized such an approach already? > > If not, I'll do it myself, no problem. > > It's a kind of tiny stuff, like atomicwrites, which is quite difficult > to dig out with modern search engines that have already rolled down to > hell a decade ago. > > Axy. From axy at declassed.art Mon Sep 26 14:18:27 2022 From: axy at declassed.art (Axy) Date: Mon, 26 Sep 2022 19:18:27 +0100 Subject: Asynchronous execution of synchronous functions In-Reply-To: References: Message-ID: <0a52b519-e8e1-7b30-135e-f2ba2f122872@declassed.art> > Did you check the ThreadPoolExecutor or the ProcessPoolExecutor? They > won't give you atomic writes unless you add a Lock or a Condition, but > they will execute your code in another thread or process. Yes, I did, but they are too complicated to use. I'd like something for humans, such as asynchronizer = InThreadExecutor() result = await asynchronizer.run(myfunc, myargs, mykwargs) and I almost implemented that borrowing code from asyncsqlite (doves fly slowly, electric mail is even slower :)), but... > Keep in mind that Python's threads have a global interpreter lock > (GIL) that prevents full parallelism. Processes work as expected, but > require IPC and pickable objects in and out. yes, that became a problem. So, I revoke my question. Went out to redesign the whole approach. Thanks for reply! Axy. From jon+usenet at unequivocal.eu Mon Sep 26 14:17:01 2022 From: jon+usenet at unequivocal.eu (Jon Ribbens) Date: Mon, 26 Sep 2022 18:17:01 -0000 (UTC) Subject: Asynchronous execution of synchronous functions References: Message-ID: On 2022-09-26, Stefan Ram wrote: > So, I wanted to try to download all pages in parallel with > processes to avoid any GIL effect, while I don't understand > what the GIL actuall is. But processes didn't work here, so > I tried threads. This worked and now the total run time is > down to about 50 seconds. Downloading things from the network is *extremely* I/O-bound. So, as you have discovered, the GIL is going to make essentially no difference whatsoever. From songofacandy at gmail.com Mon Sep 26 22:22:43 2022 From: songofacandy at gmail.com (Inada Naoki) Date: Tue, 27 Sep 2022 11:22:43 +0900 Subject: Asynchronous execution of synchronous functions In-Reply-To: <0a52b519-e8e1-7b30-135e-f2ba2f122872@declassed.art> References: <0a52b519-e8e1-7b30-135e-f2ba2f122872@declassed.art> Message-ID: On Tue, Sep 27, 2022 at 3:19 AM Axy via Python-list wrote: > > > Did you check the ThreadPoolExecutor or the ProcessPoolExecutor? They > > won't give you atomic writes unless you add a Lock or a Condition, but > > they will execute your code in another thread or process. > > Yes, I did, but they are too complicated to use. I'd like something for > humans, such as > > asynchronizer = InThreadExecutor() > > result = await asynchronizer.run(myfunc, myargs, mykwargs) > There are two convenient APIs in asyncio. asyncio.to_thread and run_in_executor. to_thread creates and destroy a dedicated thread for the function. run_in_executor uses threadpool owned by loop. https://docs.python.org/3/library/asyncio-task.html?highlight=to_thread#asyncio.to_thread https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor > > and I almost implemented that borrowing code from asyncsqlite (doves fly > slowly, electric mail is even slower :)), but... > > > Keep in mind that Python's threads have a global interpreter lock > > (GIL) that prevents full parallelism. Processes work as expected, but > > require IPC and pickable objects in and out. > > yes, that became a problem. > > So, I revoke my question. Went out to redesign the whole approach. > > Thanks for reply! > > Axy. > -- > https://mail.python.org/mailman/listinfo/python-list -- Inada Naoki From boblatest at yahoo.com Wed Sep 28 14:22:15 2022 From: boblatest at yahoo.com (Robert Latest) Date: 28 Sep 2022 18:22:15 GMT Subject: Implementation of an lru_cache() decorator that ignores the first argument Message-ID: Hi all, in a (Flask) web application I often find that many equal (SQLAlchemy) queries are executed across subsequent requests. So I tried to cache the results of those queries on the module level like this: @lru_cache() def query_db(db, args): # do the "expensive" query return result This obviously doesn't work because each request uses a new database session, so the db argument always changes from one call to the next, triggering a new query against the database. But even if that weren't so, the function would keep returning the same value forever (unless it's kicked out of the cache) and not reflect the (infrequent) changes on the database. So what I need is some decorator that can be used like this: @lru_ignore_first(timeout=10) def query_db(db, args): # do the "expensive" query return result This is what I came up with. I'm quite happy with it so far. Question: Am I being too clever? is it too complicated? Am I overlooking something that will come back and bite me later? Thanks for any comments! from functools import wraps, lru_cache from time import time, sleep def lru_ignore_first(timeout=0, **lru_args): class TimeCloak(): '''All instances compare equal until timeout expires''' __slots__ = ('x', 't', 'timeout') def __init__(self, timeout): self.timeout = timeout self.t = 0 self.x = None def __hash__(self): return self.t def __eq__(self, other): return self.t == other.t def update(self, x): self.x = x if self.timeout: t = int(time()) if t >= self.t + self.timeout: self.t = t cloak = TimeCloak(timeout) def decorator(func): @lru_cache(**lru_args) def worker(cloak, *a, **b): return func(cloak.x, *a, **b) @wraps(func) def wrapped(first, *a, **kw): cloak.update(first) return worker(cloak, *a, **kw) return wrapped return decorator @lru_ignore_first(3) def expensive(first, par): '''This takes a long time''' print('Expensive:', first, par) return par * 2 for i in range(10): r = expensive(i, 100) sleep(1) print(r) From mmontgomery at levado.to Wed Sep 28 14:25:50 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Wed, 28 Sep 2022 15:25:50 -0300 Subject: on implementing a toy oop-system References: <86r10qg89w.fsf@levado.to> <86o7v5kdsv.fsf@levado.to> Message-ID: <86wn9npd8x.fsf@levado.to> Meredith Montgomery writes: > ram at zedat.fu-berlin.de (Stefan Ram) writes: > >> Meredith Montgomery writes: >>>Is that at all possible somehow? Alternatively, how would you do your >>>toy oop-system? >> >> Maybe something along those lines: >> >> from functools import partial >> >> def counter_create( object ): >> object[ "n" ]= 0 >> def counter_increment( object ): >> object[ "n" ]+= 1 >> def counter_value( object ): >> return object[ "n" ] >> >> counter_class =( counter_create, counter_increment, counter_value ) >> >> def inherit_from( class_, target ): >> class_[ 0 ]( target ) >> for method in class_[ 1: ]: >> target[ method.__name__ ]= partial( method, target ) >> >> car = dict() >> >> inherit_from( counter_class, car ) >> >> print( car[ "counter_value" ]() ) >> car[ "counter_increment" ]() >> print( car[ "counter_value" ]() ) >> >> . The "create" part is simplified. I just wanted to show how >> to make methods like "counter_increment" act on the object >> that inherited them using "partial". > > I really liked this idea. I organized it my way. Have a look. (Thank > you for the lecture!) But it lacks consistency. > from functools import partial > > def Counter(name = None): > o = {"name": name if name else "untitled", "n": 0} > def inc(o): > o["n"] += 1 > return o > o["inc"] = inc > def get(o): > return o["n"] > o["get"] = get > return o This parent class is not defined in the same way as the child class below. The class below uses partial to fix the object in the method, but the parent one does not. We need consistency. But if we curry the parent class's methods (that is, if we apply partial on it to fix the object in its first argument), we will curry them a second time in inherit_from. That won't work. I can't see an elegant solution there, so what I'm going to do is to keep a copy of the uncurried original method. The code below works, but you can see it's kinda ugly. I wish I could uncurry a procedure, but I don't think this is possible. (Is it?) # -*- mode: python; python-indent-offset: 2 -*- def Counter(name = None): self = {"name": name if name else "untitled", "n": 0} def inc(self): self["n"] += 1 return self self["inc_uncurried"] = inc self["inc"] = curry(inc, self) def get(self): return self["n"] self["get_uncurried"] = get self["get"] = curry(get, self) return self def Car(maker): self = {"maker": maker, "state": "off"} inherit_from(Counter, self) def on(self): if self["is_on"](): raise ValueError("oh, no: car is already on") self["inc"]() print(f"{self['maker']}: bruum!") self["state"] = "on" return self self["on_uncurried"] = on self["on"] = curry(on, self) def off(self): if self["is_off"](): raise ValueError("oh, no: car is already off") print(f"{self['maker']}: spat!") self["state"] = "off" return self self["off_uncurried"] = off self["off"] = curry(off, self) def is_on(self): return self["state"] == "on" self["is_on_uncurried"] = is_on self["is_on"] = curry(is_on, self) def is_off(self): return self["state"] == "off" self["is_off_uncurried"] = is_off self["is_off"] = curry(is_off, self) return self def main(): car1 = Car("Ford") car2 = Car("VW") for i in range(5): car1["on"](); car1["off"]() for i in range(3): car2["on"](); car2["off"]() print(f"car turned on = {car1['get']()} ({car1['maker']})") print(f"car turned on = {car2['get']()} ({car2['maker']})") >>> main() Ford: bruum! Ford: spat! Ford: bruum! Ford: spat! Ford: bruum! Ford: spat! Ford: bruum! Ford: spat! Ford: bruum! Ford: spat! VW: bruum! VW: spat! VW: bruum! VW: spat! VW: bruum! VW: spat! car turned on = 5 (Ford) car turned on = 3 (VW) From rosuav at gmail.com Wed Sep 28 17:11:55 2022 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 29 Sep 2022 07:11:55 +1000 Subject: Implementation of an lru_cache() decorator that ignores the first argument In-Reply-To: References: Message-ID: On Thu, 29 Sept 2022 at 05:36, Robert Latest via Python-list wrote: > in a (Flask) web application I often find that many equal (SQLAlchemy) queries > are executed across subsequent requests. So I tried to cache the results of > those queries on the module level like this: > > @lru_cache() > def query_db(db, args): > # do the "expensive" query > return result > > ... > This is what I came up with. I'm quite happy with it so far. Question: Am I > being too clever? is it too complicated? Am I overlooking something that will > come back and bite me later? Thanks for any comments! > > def lru_ignore_first(timeout=0, **lru_args): > ... I think this code is fairly specific to what you're doing, which means the decorator won't be as reusable (first hint of that is the entire "timeout" feature, which isn't mentioned at all in the function's name). So it's probably not worth trying to do this multi-layered approach, and it would be as effective, and a lot simpler, to just have code at the top of the query_db function to do the cache lookup. But you may find that your database is *itself* able to do this caching for you, and it will know when to evict from cache. If you really have to do it yourself, keep it really really simple, but have an easy way *in your own code* to do the cache purge; that way, you guarantee correctness, even at the expense of some performance. In terms of overall database performance, though: are you using transactions correctly? With PostgreSQL, especially, the cost of doing a series of queries in one transaction is barely higher than doing a single query in a transaction; or, putting it the other way around, doing several sequential transactions costs several times as much as doing one combined transaction. Check to see that you aren't accidentally running in autocommit mode or anything. It could save you a lot of hassle! ChrisA From PythonList at DancesWithMice.info Wed Sep 28 18:38:16 2022 From: PythonList at DancesWithMice.info (dn) Date: Thu, 29 Sep 2022 11:38:16 +1300 Subject: Implementation of an lru_cache() decorator that ignores the first argument In-Reply-To: References: Message-ID: On 29/09/2022 07.22, Robert Latest via Python-list wrote: ... > This is what I came up with. I'm quite happy with it so far. Question: Am I > being too clever? is it too complicated? Am I overlooking something that will > come back and bite me later? Thanks for any comments! Thank you for the chuckle: "Yes", you are clever; and "yes", this is likely a bit too clever (IMHO). The impression is that LRU will put something more-concrete, 'in front of' SQLAlchemy - which is more abstract, and which is in-turn 'in front of' the RDBMS (which is concrete...). Is this the code-smell making one's nose suspicious? The criticism is of SQLAlchemy. If the problem can't be solved with that tool, perhaps it is not the right-tool-for-the-job... Bias: With decades of SQL/RDBMS experience, it is easy to say, "drop the tool". +1 @Chris: depending upon how many transactions-between, it seems likely find that the RDBMS will cache sufficiently, as SOP. YMMV, ie there's only one way to find-out! -- Regards, =dn From stephen.berman at gmx.net Thu Sep 29 07:06:25 2022 From: stephen.berman at gmx.net (Stephen Berman) Date: Thu, 29 Sep 2022 13:06:25 +0200 Subject: on GNU EMACS's python-mode, loading entire buffer In-Reply-To: <86ilm2ncqs.fsf@levado.to> (Meredith Montgomery's message of "Sun, 04 Sep 2022 16:47:07 -0300") References: <86y1v6zzf3.fsf@levado.to> <868rn4zg3m.fsf@levado.to> <86ilm2ncqs.fsf@levado.to> Message-ID: <87wn9mwiby.fsf@gmx.net> On Sun, 04 Sep 2022 16:47:07 -0300 Meredith Montgomery wrote: > Meredith Montgomery writes: > >> Meredith Montgomery writes: >> >> [...] >> >>> I would also be interested in a command that restarts the REPL afresh >>> and reloads my buffer --- sort of like keyboard's [F5] of the IDLE. >> >> A partial solution for this is the following procedure. >> >> (defun python-revert-and-send-buffer-to-repl () >> "Revert current buffer and sends it to the Python REPL." >> (interactive) >> (revert-buffer "ignore-auto-no" "no-confirm") >> (python-shell-send-buffer)) >> >> We can map this to the F5-key and that improves things. But a restart >> of the REPL would be the ideal. (Sometimes we really want to start >> afresh. Sometimes. Most often we don't want that.) > > It's not easy to restart the REPL. You can send "quit()" to it and > invoke run-python again interactively by typing out one command after > another, but if you write a procedure such as this one below, it doesn't > work: it gives me the impression that there's a timing issue, that is, > perhaps the procedure is too fast and something happens before it > should. > > (defun python-save-send-buffer-to-repl () > (interactive) > (save-buffer) > (python-shell-send-string "quit()") > (run-python) > (python-shell-send-buffer) > (python-shell-switch-to-shell)) It does seem like a timing issue. This works for me: (defun python-save-send-buffer-to-repl () (interactive) (save-buffer) (python-shell-send-string "quit()") (sit-for 0.1) (run-python) (python-shell-send-buffer) (python-shell-switch-to-shell)) But if I decrease the wait to 0.05 it doesn't work. Steve Berman From jenkris at tutanota.com Thu Sep 29 11:54:48 2022 From: jenkris at tutanota.com (Jen Kris) Date: Thu, 29 Sep 2022 17:54:48 +0200 (CEST) Subject: PyObject_CallFunctionObjArgs segfaults Message-ID: Recently I completed a project where I used PyObject_CallFunctionObjArgs extensively with the NLTK library from a program written in NASM, with no problems.? Now I am on a new project where I call the Python random library.? I use the same setup as before, but I am getting a segfault with random.seed.? At the start of the NASM program I call a C API program that gets PyObject pointers to ?seed? and ?randrange? in the same way as I did before: int64_t Get_LibModules(int64_t * return_array) { PyObject * pName_random = PyUnicode_FromString("random"); PyObject * pMod_random = PyImport_Import(pName_random); if (pMod_random == 0x0){ PyErr_Print(); return 1;} PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange"); return_array[0] = (int64_t)pAttr_seed; return_array[1] = (int64_t)pAttr_randrange; return 0; } Later in the same program I call a C API program to call random.seed: int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) { PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_1); if (p_seed_calc == 0x0){ ??? PyErr_Print(); ??? return 1;} //Prepare return values long return_val = PyLong_AsLong(p_seed_calc); return return_val; } The first program correctly imports ?random? and gets pointers to ?seed? and ?randrange.?? I verified that the same pointer is correctly passed into C_API_2, and the seed value (1234) is passed as? Py_ssize_t value_1.? But I get this segfault: Program received signal SIGSEGV, Segmentation fault. 0x00007ffff64858d5 in _Py_INCREF (op=0x4d2) at ../Include/object.h:459 459???? ../Include/object.h: No such file or directory. So I tried Py_INCREF in the first program:? Py_INCREF(pMod_random); Py_INCREF(pAttr_seed); Then I moved Py_INCREF(pAttr_seed) to the second program.? Same segfault. Finally, I initialized ?random? and ?seed? in the second program, where they are used.? Same segfault.? The segfault refers to Py_INCREF, so this seems to do with reference counting, but Py_INCREF didn?t solve it.??? I?m using Python 3.8 on Ubuntu.? Thanks for any ideas on how to solve this.? Jen From python at mrabarnett.plus.com Thu Sep 29 13:06:49 2022 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 29 Sep 2022 18:06:49 +0100 Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: References: Message-ID: On 2022-09-29 16:54, Jen Kris via Python-list wrote: > Recently I completed a project where I used PyObject_CallFunctionObjArgs extensively with the NLTK library from a program written in NASM, with no problems.? Now I am on a new project where I call the Python random library.? I use the same setup as before, but I am getting a segfault with random.seed. > > At the start of the NASM program I call a C API program that gets PyObject pointers to ?seed? and ?randrange? in the same way as I did before: > > int64_t Get_LibModules(int64_t * return_array) > { > PyObject * pName_random = PyUnicode_FromString("random"); > PyObject * pMod_random = PyImport_Import(pName_random); > Both PyUnicode_FromString and PyImport_Import return new references or null pointers. > if (pMod_random == 0x0){ > PyErr_Print(); You're leaking a reference here (pName_random). > return 1;} > > PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); > PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange"); > > return_array[0] = (int64_t)pAttr_seed; > return_array[1] = (int64_t)pAttr_randrange; > You're leaking 2 references here (pName_random and pMod_random). > return 0; > } > > Later in the same program I call a C API program to call random.seed: > > int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) > { > PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_1); It's expecting all of the arguments to be PyObject*, but value_1 is Py_ssize_t instead of PyObject* (a pointer to a _Python_ int). The argument list must end with a null pointer. It returns a new reference or a null pointer. > > if (p_seed_calc == 0x0){ > ??? PyErr_Print(); > ??? return 1;} > > //Prepare return values > long return_val = PyLong_AsLong(p_seed_calc); > You're leaking a reference here (p_seed_calc). > return return_val; > } > > The first program correctly imports ?random? and gets pointers to ?seed? and ?randrange.?? I verified that the same pointer is correctly passed into C_API_2, and the seed value (1234) is passed as? Py_ssize_t value_1.? But I get this segfault: > > Program received signal SIGSEGV, Segmentation fault. > 0x00007ffff64858d5 in _Py_INCREF (op=0x4d2) at ../Include/object.h:459 > 459???? ../Include/object.h: No such file or directory. > > So I tried Py_INCREF in the first program: > > Py_INCREF(pMod_random); > Py_INCREF(pAttr_seed); > > Then I moved Py_INCREF(pAttr_seed) to the second program.? Same segfault. > > Finally, I initialized ?random? and ?seed? in the second program, where they are used.? Same segfault. > > The segfault refers to Py_INCREF, so this seems to do with reference counting, but Py_INCREF didn?t solve it. > > I?m using Python 3.8 on Ubuntu. > > Thanks for any ideas on how to solve this. > > Jen > From boblatest at yahoo.com Thu Sep 29 02:18:36 2022 From: boblatest at yahoo.com (Robert Latest) Date: 29 Sep 2022 06:18:36 GMT Subject: Implementation of an lru_cache() decorator that ignores the first argument References: Message-ID: Hi Chris and dh, thanks for your --as usually-- thoughtful and interesting answers. Indeed, when doing these web applications I find that there are several layers of useful, maybe less useful, and unknown caching. Many of my requests rely on a notoriously unreliable read-only database outside of my control, so I cache the required data into a local DB on my server, then I do some in-memory caching of expensive data plots because I haven't figured out how to reliably exploit the client-side caching ... then every middleware on that path may or may not implement its own version of clever or not-so-clever caching. Probably not a good idea to try and outsmart that by adding yet another thing that may break or not be up-to-date at the wrong moment. That said, the only caching that SQLAlchemy does (to my knowledge) is that it stores retrieved DB items by their primary keys in the session. Not worth much since the session gets created and dumped on each request by SQA's unit of work paradigm. But the DB backend itself may be caching repeated queries. Back to Python-theory: The "Cloak" object is the only way I could think of to sneak changing data past lru_cache's key lookup mechanism. Is there some other method? Just curious. From mmontgomery at levado.to Thu Sep 29 12:37:15 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Thu, 29 Sep 2022 13:37:15 -0300 Subject: on implementing a toy oop-system References: <86r10qg89w.fsf@levado.to> <86o7v5kdsv.fsf@levado.to> <86wn9npd8x.fsf@levado.to> Message-ID: <86zgeii1c4.fsf@levado.to> ram at zedat.fu-berlin.de (Stefan Ram) writes: > Meredith Montgomery writes: >>The code below works, but you can see it's kinda ugly. I wish I could >>uncurry a procedure, but I don't think this is possible. (Is it?) > > from functools import partial > from operator import add > add5 = partial( add, 5 ) > print( add5( 2 )) > # might be dependent on implementation details of "functools": > uncurried = add5.func > print( uncurried( 14, 7 )) It works on my system here. Thank you so much for your help. (I've been learning a lot with you!) > However, to evaluate a method call such as "o.m( a, a1, ... )", > currying does not necessarily have to be used. One can as well > determine the function to be used for "m" from the type of "o" > and then call that function with arguments "o", "a", "a1", ... Was that your last approach? I only glanced at it so far. But I remember seeing you describing an object and listing out which methods were defined for it and then there was one procedure that would take the role of invoking the methods (with the proper arguments). From mmontgomery at levado.to Thu Sep 29 14:00:00 2022 From: mmontgomery at levado.to (Meredith Montgomery) Date: Thu, 29 Sep 2022 15:00:00 -0300 Subject: on implementing a toy oop-system References: <86r10qg89w.fsf@levado.to> <86o7v5kdsv.fsf@levado.to> <86wn9npd8x.fsf@levado.to> <86zgeii1c4.fsf@levado.to> Message-ID: <86mtaihxi7.fsf@levado.to> ram at zedat.fu-berlin.de (Stefan Ram) writes: [...] >>>However, to evaluate a method call such as "o.m( a, a1, ... )", >>>currying does not necessarily have to be used. One can as well >>>determine the function to be used for "m" from the type of "o" >>>and then call that function with arguments "o", "a", "a1", ... >>Was that your last approach? > > Yes, I think so. > > (There are also languages with "multi-methods", where upon > a function call "m( o, o1, ... )" the decision which function > to call depends on all the types of all the arguments.) I think Clojure is one such. I've read Part 1 of ``Clojure in Action'' by Amit Rathore, 2012, Manning, ISBN 9781935182597. I liked it. From sandi.cimerman.umcg at gmail.com Thu Sep 29 14:00:55 2022 From: sandi.cimerman.umcg at gmail.com (Sandi Cimerman) Date: Thu, 29 Sep 2022 11:00:55 -0700 (PDT) Subject: non-standard glibc location In-Reply-To: References: <949d4c0c-37d1-c160-5a3c-e5d45632dbf5@tjol.eu> Message-ID: <99693c1f-dca4-4997-b70e-f90118434d0an@googlegroups.com> On Friday, September 8, 2017 at 3:16:12 AM UTC+2, Fetchinson . wrote: > On 9/7/17, Thomas Jollans wrote: > > On 2017-09-06 16:14, Fetchinson . via Python-list wrote: > >> Hi folks, > >> > >> I'm trying to install a binary package (tensorflow) which contains > >> some binary C extensions. Now my system glibc is 2.15 but the binaries > >> in the C extensions were created (apparently) with glibc 2.17. So I > >> thought no problemo I installed glibc 2.17 to a custom location, built > >> python2.7 from source (hopefully using my custom glibc) and installed > >> pip and everything else using this custom built python. But still when > >> I try to import tensorflow I get: > >> > >> ImportError: /lib64/libc.so.6: version `GLIBC_2.17' not found > >> (required by > >> /home/nogradi/fetch/custom/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so) > >> > >> So apparently it's trying to use my system glibc, not the custom one. > >> > >> How do I tell this extension to use the custom glibc? Is it even > >> possible? > > > > It's going to use the same libc as python, so first of all check which > > libc your python interpreter is actually linked to. Maybe your custom > > build didn't do quite what you wanted. > > > > ldd `which python` # or something like that > > > > Once you've convinced yourself that python has the correct libc, you > > could try building tensorflow from source rather than installing the > > binaries. > > > > Maybe something in here helps: > > https://github.com/tensorflow/tensorflow/issues/53 > Thanks a lot for all the comments, my problem was indeed that the > compiled python was still using the system glibc. The solution was to > set the environment variables > > p=/path/to/custom/glibc > export CFLAGS=-I${p}/include > export LDFLAGS="-Wl,--rpath=${p}/lib > -Wl,--dynamic-linker=${p}/lib/ld-linux-x86-64.so.2" > > And then the compiled python was using the new glibc. > > Side note 1 on tensorflow: the compiled tensorflow binary uses unicode > ucs4, so for python I had to ./configure --enable-unicode=ucs4 because > the default is ucs2 > > Side note 2 on tensorflow: it also depends on libstdc++ and my version > was too old for that as well. Instead of compiling gcc from source > (which includes libstdc++) I copied a binary libstdc++ from a newer > linux distro and it was working fine. > > And the reason, if anyone cares, I had to go through the above is that > I couldn't compile tensorflow from source. > > Thanks again, > Daniel > > > >> > >> But maybe I have an even more basic issue: how do I link python not > >> with the system glibc but with my custom glibc? > >> > >> Cheers, > >> Daniel > >> > >> > >> > > > > > > -- > > Thomas Jollans > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > > -- > Psss, psss, put it down! - http://www.cafepress.com/putitdown First of all, thank you a lot Fetchinson, it helped on my system (Centos 6) to today recompile the newest Python (3.10.7)! Of course I had foss2018 available on the system, so I "just" had to compile glibc, zlib, and openssl 1.1.1q ... but hey it was worth it! I have my environment up and running! FYI: 1. to me it helped that since I had nonstandard openssl location to edit Modules/Setup.dist as suggested here https://stackoverflow.com/questions/5937337/building-python-with-ssl-support-in-non-standard-location 2. to me it helped to configure with ./configure --with-libc=/tmp/glibc/lib/libc.so --prefix=/tmp/python --with-openssl=/tmp/openssl Hope this helps someone else who will try to do the same. Cheers From mwalmasri2003 at gmail.com Thu Sep 29 15:09:44 2022 From: mwalmasri2003 at gmail.com (Walid AlMasri) Date: Thu, 29 Sep 2022 22:09:44 +0300 Subject: python developer Message-ID: Hi all, I has been using python in scientific computing for many years I want to become a python developer so what is a good reference to follow ? Thanks in advance ! Walid From jenkris at tutanota.com Thu Sep 29 16:02:00 2022 From: jenkris at tutanota.com (Jen Kris) Date: Thu, 29 Sep 2022 22:02:00 +0200 (CEST) Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: References: Message-ID: Thanks very much to @MRAB for taking time to answer.? I changed my code to conform to your answer (as best I understand your comments on references), but I still get the same error.? My comments continue below the new code immediately below.? int64_t Get_LibModules(int64_t * return_array) { PyObject * pName_random = PyUnicode_FromString("random"); PyObject * pMod_random = PyImport_Import(pName_random); Py_INCREF(pName_random); Py_INCREF(pMod_random); if (pMod_random == 0x0){ PyErr_Print(); return 1;} PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange"); Py_INCREF(pAttr_seed); Py_INCREF(pAttr_randrange); return_array[0] = (int64_t)pAttr_seed; return_array[1] = (int64_t)pAttr_randrange; return 0; } int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) { PyObject * value_ptr = (PyObject * )value_1; PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_ptr, NULL); if (p_seed_calc == 0x0){ ??? PyErr_Print(); ??? return 1;} //Prepare return values long return_val = PyLong_AsLong(p_seed_calc); return return_val; } So I incremented the reference to all objects in Get_LibModules, but I still get the same segfault at PyObject_CallFunctionObjArgs.? Unfortunately, reference counting is not well documented so I?m not clear what?s wrong.? Sep 29, 2022, 10:06 by python at mrabarnett.plus.com: > On 2022-09-29 16:54, Jen Kris via Python-list wrote: > >> Recently I completed a project where I used PyObject_CallFunctionObjArgs extensively with the NLTK library from a program written in NASM, with no problems.? Now I am on a new project where I call the Python random library.? I use the same setup as before, but I am getting a segfault with random.seed. >> >> At the start of the NASM program I call a C API program that gets PyObject pointers to ?seed? and ?randrange? in the same way as I did before: >> >> int64_t Get_LibModules(int64_t * return_array) >> { >> PyObject * pName_random = PyUnicode_FromString("random"); >> PyObject * pMod_random = PyImport_Import(pName_random); >> > Both PyUnicode_FromString and PyImport_Import return new references or null pointers. > >> if (pMod_random == 0x0){ >> PyErr_Print(); >> > > You're leaking a reference here (pName_random). > >> return 1;} >> >> PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); >> PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange"); >> >> return_array[0] = (int64_t)pAttr_seed; >> return_array[1] = (int64_t)pAttr_randrange; >> > > You're leaking 2 references here (pName_random and pMod_random). > >> return 0; >> } >> >> Later in the same program I call a C API program to call random.seed: >> >> int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) >> { >> PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_1); >> > > It's expecting all of the arguments to be PyObject*, but value_1 is Py_ssize_t instead of PyObject* (a pointer to a _Python_ int). > > The argument list must end with a null pointer. > > It returns a new reference or a null pointer. > >> >> if (p_seed_calc == 0x0){ >> ??? PyErr_Print(); >> ??? return 1;} >> >> //Prepare return values >> long return_val = PyLong_AsLong(p_seed_calc); >> > You're leaking a reference here (p_seed_calc). > >> return return_val; >> } >> >> The first program correctly imports ?random? and gets pointers to ?seed? and ?randrange.?? I verified that the same pointer is correctly passed into C_API_2, and the seed value (1234) is passed as? Py_ssize_t value_1.? But I get this segfault: >> >> Program received signal SIGSEGV, Segmentation fault. >> 0x00007ffff64858d5 in _Py_INCREF (op=0x4d2) at ../Include/object.h:459 >> 459???? ../Include/object.h: No such file or directory. >> >> So I tried Py_INCREF in the first program: >> >> Py_INCREF(pMod_random); >> Py_INCREF(pAttr_seed); >> >> Then I moved Py_INCREF(pAttr_seed) to the second program.? Same segfault. >> >> Finally, I initialized ?random? and ?seed? in the second program, where they are used.? Same segfault. >> >> The segfault refers to Py_INCREF, so this seems to do with reference counting, but Py_INCREF didn?t solve it. >> >> I?m using Python 3.8 on Ubuntu. >> >> Thanks for any ideas on how to solve this. >> >> Jen >> > > -- > https://mail.python.org/mailman/listinfo/python-list > From jenkris at tutanota.com Thu Sep 29 16:47:14 2022 From: jenkris at tutanota.com (Jen Kris) Date: Thu, 29 Sep 2022 22:47:14 +0200 (CEST) Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: References: Message-ID: To update my previous email, I found the problem, but I have a new problem.? Previously I cast PyObject * value_ptr = (PyObject * )value_1 but that's not correct.? Instead I used PyObject * value_ptr = PyLong_FromLong(value_1) and that works.? HOWEVER, while PyObject_CallFunctionObjArgs does work now, it returns -1, which is not the right answer for random.seed.? I use "long return_val = PyLong_AsLong(p_seed_calc);" to convert it to a long.? So my question is why do I get -1 as return value?? When I query p_seed calc : get: (gdb) p p_seed_calc $2 = (PyObject *) 0x7ffff69be120 <_Py_NoneStruct> Thanks again. Jen Sep 29, 2022, 13:02 by python-list at python.org: > Thanks very much to @MRAB for taking time to answer.? I changed my code to conform to your answer (as best I understand your comments on references), but I still get the same error.? My comments continue below the new code immediately below.? > > int64_t Get_LibModules(int64_t * return_array) > { > PyObject * pName_random = PyUnicode_FromString("random"); > PyObject * pMod_random = PyImport_Import(pName_random); > > Py_INCREF(pName_random); > Py_INCREF(pMod_random); > > if (pMod_random == 0x0){ > PyErr_Print(); > return 1;} > > PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); > PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange"); > > Py_INCREF(pAttr_seed); > Py_INCREF(pAttr_randrange); > > return_array[0] = (int64_t)pAttr_seed; > return_array[1] = (int64_t)pAttr_randrange; > > return 0; > } > > int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) > { > PyObject * value_ptr = (PyObject * )value_1; > PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_ptr, NULL); > > if (p_seed_calc == 0x0){ > ??? PyErr_Print(); > ??? return 1;} > > //Prepare return values > long return_val = PyLong_AsLong(p_seed_calc); > > return return_val; > } > > So I incremented the reference to all objects in Get_LibModules, but I still get the same segfault at PyObject_CallFunctionObjArgs.? Unfortunately, reference counting is not well documented so I?m not clear what?s wrong.? > > > > > Sep 29, 2022, 10:06 by python at mrabarnett.plus.com: > >> On 2022-09-29 16:54, Jen Kris via Python-list wrote: >> >>> Recently I completed a project where I used PyObject_CallFunctionObjArgs extensively with the NLTK library from a program written in NASM, with no problems.? Now I am on a new project where I call the Python random library.? I use the same setup as before, but I am getting a segfault with random.seed. >>> >>> At the start of the NASM program I call a C API program that gets PyObject pointers to ?seed? and ?randrange? in the same way as I did before: >>> >>> int64_t Get_LibModules(int64_t * return_array) >>> { >>> PyObject * pName_random = PyUnicode_FromString("random"); >>> PyObject * pMod_random = PyImport_Import(pName_random); >>> >> Both PyUnicode_FromString and PyImport_Import return new references or null pointers. >> >>> if (pMod_random == 0x0){ >>> PyErr_Print(); >>> >> >> You're leaking a reference here (pName_random). >> >>> return 1;} >>> >>> PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); >>> PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange"); >>> >>> return_array[0] = (int64_t)pAttr_seed; >>> return_array[1] = (int64_t)pAttr_randrange; >>> >> >> You're leaking 2 references here (pName_random and pMod_random). >> >>> return 0; >>> } >>> >>> Later in the same program I call a C API program to call random.seed: >>> >>> int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) >>> { >>> PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_1); >>> >> >> It's expecting all of the arguments to be PyObject*, but value_1 is Py_ssize_t instead of PyObject* (a pointer to a _Python_ int). >> >> The argument list must end with a null pointer. >> >> It returns a new reference or a null pointer. >> >>> >>> if (p_seed_calc == 0x0){ >>> ??? PyErr_Print(); >>> ??? return 1;} >>> >>> //Prepare return values >>> long return_val = PyLong_AsLong(p_seed_calc); >>> >> You're leaking a reference here (p_seed_calc). >> >>> return return_val; >>> } >>> >>> The first program correctly imports ?random? and gets pointers to ?seed? and ?randrange.?? I verified that the same pointer is correctly passed into C_API_2, and the seed value (1234) is passed as? Py_ssize_t value_1.? But I get this segfault: >>> >>> Program received signal SIGSEGV, Segmentation fault. >>> 0x00007ffff64858d5 in _Py_INCREF (op=0x4d2) at ../Include/object.h:459 >>> 459???? ../Include/object.h: No such file or directory. >>> >>> So I tried Py_INCREF in the first program: >>> >>> Py_INCREF(pMod_random); >>> Py_INCREF(pAttr_seed); >>> >>> Then I moved Py_INCREF(pAttr_seed) to the second program.? Same segfault. >>> >>> Finally, I initialized ?random? and ?seed? in the second program, where they are used.? Same segfault. >>> >>> The segfault refers to Py_INCREF, so this seems to do with reference counting, but Py_INCREF didn?t solve it. >>> >>> I?m using Python 3.8 on Ubuntu. >>> >>> Thanks for any ideas on how to solve this. >>> >>> Jen >>> >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > -- > https://mail.python.org/mailman/listinfo/python-list > From python at mrabarnett.plus.com Thu Sep 29 18:31:07 2022 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 29 Sep 2022 23:31:07 +0100 Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: References: Message-ID: On 2022-09-29 21:47, Jen Kris wrote: > To update my previous email, I found the problem, but I have a new > problem. > > Previously I cast PyObject * value_ptr = (PyObject * )value_1 but > that's not correct.? Instead I used PyObject * value_ptr = > PyLong_FromLong(value_1) and that works.? HOWEVER, while > PyObject_CallFunctionObjArgs does work now, it returns -1, which is > not the right answer for random.seed.? I use "long return_val = > PyLong_AsLong(p_seed_calc);" to convert it to a long. > random.seed returns None, so when you call PyObject_CallFunctionObjArgs it returns a new reference to Py_None. If you then pass to PyLong_AsLong a reference to something that's not a PyLong, it'll set an error and return -1. > So my question is why do I get -1 as return value?? When I query > p_seed calc : get: > > (gdb) p p_seed_calc > $2 = (PyObject *) 0x7ffff69be120 <_Py_NoneStruct> > Exactly. It's Py_None, not a PyLong. > Thanks again. > > Jen > > > > > Sep 29, 2022, 13:02 by python-list at python.org: > > Thanks very much to @MRAB for taking time to answer.? I changed my > code to conform to your answer (as best I understand your comments > on references), but I still get the same error.? My comments > continue below the new code immediately below. > > int64_t Get_LibModules(int64_t * return_array) > { > PyObject * pName_random = PyUnicode_FromString("random"); > PyObject * pMod_random = PyImport_Import(pName_random); > > Py_INCREF(pName_random); > Py_INCREF(pMod_random); > > if (pMod_random == 0x0){ > PyErr_Print(); > return 1;} > > PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); > PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, > "randrange"); > > Py_INCREF(pAttr_seed); > Py_INCREF(pAttr_randrange); > > return_array[0] = (int64_t)pAttr_seed; > return_array[1] = (int64_t)pAttr_randrange; > > return 0; > } > > int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) > { > PyObject * value_ptr = (PyObject * )value_1; > PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, > value_ptr, NULL); > > if (p_seed_calc == 0x0){ > ??? PyErr_Print(); > ??? return 1;} > > //Prepare return values > long return_val = PyLong_AsLong(p_seed_calc); > > return return_val; > } > > So I incremented the reference to all objects in Get_LibModules, > but I still get the same segfault at > PyObject_CallFunctionObjArgs.? Unfortunately, reference counting > is not well documented so I?m not clear what?s wrong. > > > > > Sep 29, 2022, 10:06 by python at mrabarnett.plus.com: > > On 2022-09-29 16:54, Jen Kris via Python-list wrote: > > Recently I completed a project where I used > PyObject_CallFunctionObjArgs extensively with the NLTK > library from a program written in NASM, with no problems. > Now I am on a new project where I call the Python random > library.? I use the same setup as before, but I am getting > a segfault with random.seed. > > At the start of the NASM program I call a C API program > that gets PyObject pointers to ?seed? and ?randrange? in > the same way as I did before: > > int64_t Get_LibModules(int64_t * return_array) > { > PyObject * pName_random = PyUnicode_FromString("random"); > PyObject * pMod_random = PyImport_Import(pName_random); > > Both PyUnicode_FromString and PyImport_Import return new > references or null pointers. > > if (pMod_random == 0x0){ > PyErr_Print(); > > > You're leaking a reference here (pName_random). > > return 1;} > > PyObject * pAttr_seed = > PyObject_GetAttrString(pMod_random, "seed"); > PyObject * pAttr_randrange = > PyObject_GetAttrString(pMod_random, "randrange"); > > return_array[0] = (int64_t)pAttr_seed; > return_array[1] = (int64_t)pAttr_randrange; > > > You're leaking 2 references here (pName_random and pMod_random). > > return 0; > } > > Later in the same program I call a C API program to call > random.seed: > > int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) > { > PyObject * p_seed_calc = > PyObject_CallFunctionObjArgs(pAttr_seed, value_1); > > > It's expecting all of the arguments to be PyObject*, but > value_1 is Py_ssize_t instead of PyObject* (a pointer to a > _Python_ int). > > The argument list must end with a null pointer. > > It returns a new reference or a null pointer. > > > if (p_seed_calc == 0x0){ > ??? PyErr_Print(); > ??? return 1;} > > //Prepare return values > long return_val = PyLong_AsLong(p_seed_calc); > > You're leaking a reference here (p_seed_calc). > > return return_val; > } > > The first program correctly imports ?random? and gets > pointers to ?seed? and ?randrange.?? I verified that the > same pointer is correctly passed into C_API_2, and the > seed value (1234) is passed as? Py_ssize_t value_1.? But I > get this segfault: > > Program received signal SIGSEGV, Segmentation fault. > 0x00007ffff64858d5 in _Py_INCREF (op=0x4d2) at > ../Include/object.h:459 > 459???? ../Include/object.h: No such file or directory. > > So I tried Py_INCREF in the first program: > > Py_INCREF(pMod_random); > Py_INCREF(pAttr_seed); > > Then I moved Py_INCREF(pAttr_seed) to the second program.? > Same segfault. > > Finally, I initialized ?random? and ?seed? in the second > program, where they are used.? Same segfault. > > The segfault refers to Py_INCREF, so this seems to do with > reference counting, but Py_INCREF didn?t solve it. > > I?m using Python 3.8 on Ubuntu. > > Thanks for any ideas on how to solve this. > > Jen > > > -- > https://mail.python.org/mailman/listinfo/python-list > > > -- > https://mail.python.org/mailman/listinfo/python-list > > From jenkris at tutanota.com Thu Sep 29 18:41:39 2022 From: jenkris at tutanota.com (Jen Kris) Date: Fri, 30 Sep 2022 00:41:39 +0200 (CEST) Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: References: Message-ID: I just solved this C API problem, and I?m posting the answer to help anyone else who might need it.? The errors were: (1) we must call Py_INCREF on each object when it?s created. (2) in C_API_2 (see below) we don?t cast value_1 as I did before with PyObject * value_ptr = (PyObject * )value_1.? Instead we use PyObject * value_ptr = PyLong_FromLong(value_1); (3) The command string to PyObject_CallFunctionObjArgs must be null terminated. Here?s the revised code: First we load the modules, and increment the reference to each object:? int64_t Get_LibModules(int64_t * return_array) { PyObject * pName_random = PyUnicode_FromString("random"); PyObject * pMod_random = PyImport_Import(pName_random); Py_INCREF(pName_random); Py_INCREF(pMod_random); if (pMod_random == 0x0){ PyErr_Print(); return 1;} PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange"); Py_INCREF(pAttr_seed); Py_INCREF(pAttr_randrange); return_array[0] = (int64_t)pAttr_seed; return_array[1] = (int64_t)pAttr_randrange; return 0; } Next we call a program to initialize the random number generator with random.seed(), and increment the reference to its return value p_seed_calc: int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) { PyObject * value_ptr = PyLong_FromLong(value_1); PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_ptr, NULL); // _________ if (p_seed_calc == 0x0){ ??? PyErr_Print(); ??? return 1;} Py_INCREF(p_seed_calc); return 0; } Now we call another program to get a random number: int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1) { PyObject * value_ptr = PyLong_FromLong(value_1); PyObject * p_randrange_calc = PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL); if (p_randrange_calc == 0x0){ ??? PyErr_Print(); ??? return 1;} //Prepare return values long return_val = PyLong_AsLong(p_randrange_calc); return return_val; } That returns 28, which is what I get from the Python command line.? Thanks again to MRAB for helpful comments.? Jen Sep 29, 2022, 15:31 by python at mrabarnett.plus.com: > On 2022-09-29 21:47, Jen Kris wrote: > >> To update my previous email, I found the problem, but I have a new problem. >> >> Previously I cast PyObject * value_ptr = (PyObject * )value_1 but that's not correct.? Instead I used PyObject * value_ptr = PyLong_FromLong(value_1) and that works.? HOWEVER, while PyObject_CallFunctionObjArgs does work now, it returns -1, which is not the right answer for random.seed.? I use "long return_val = PyLong_AsLong(p_seed_calc);" to convert it to a long. >> > random.seed returns None, so when you call PyObject_CallFunctionObjArgs it returns a new reference to Py_None. > > If you then pass to PyLong_AsLong a reference to something that's not a PyLong, it'll set an error and return -1. > >> So my question is why do I get -1 as return value?? When I query p_seed calc : get: >> >> (gdb) p p_seed_calc >> $2 = (PyObject *) 0x7ffff69be120 <_Py_NoneStruct> >> > Exactly. It's Py_None, not a PyLong. > >> Thanks again. >> >> Jen >> >> >> >> >> Sep 29, 2022, 13:02 by python-list at python.org: >> >> Thanks very much to @MRAB for taking time to answer.? I changed my >> code to conform to your answer (as best I understand your comments >> on references), but I still get the same error.? My comments >> continue below the new code immediately below. >> >> int64_t Get_LibModules(int64_t * return_array) >> { >> PyObject * pName_random = PyUnicode_FromString("random"); >> PyObject * pMod_random = PyImport_Import(pName_random); >> >> Py_INCREF(pName_random); >> Py_INCREF(pMod_random); >> >> if (pMod_random == 0x0){ >> PyErr_Print(); >> return 1;} >> >> PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); >> PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, >> "randrange"); >> >> Py_INCREF(pAttr_seed); >> Py_INCREF(pAttr_randrange); >> >> return_array[0] = (int64_t)pAttr_seed; >> return_array[1] = (int64_t)pAttr_randrange; >> >> return 0; >> } >> >> int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) >> { >> PyObject * value_ptr = (PyObject * )value_1; >> PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, >> value_ptr, NULL); >> >> if (p_seed_calc == 0x0){ >> ??? PyErr_Print(); >> ??? return 1;} >> >> //Prepare return values >> long return_val = PyLong_AsLong(p_seed_calc); >> >> return return_val; >> } >> >> So I incremented the reference to all objects in Get_LibModules, >> but I still get the same segfault at >> PyObject_CallFunctionObjArgs.? Unfortunately, reference counting >> is not well documented so I?m not clear what?s wrong. >> >> >> >> >> Sep 29, 2022, 10:06 by python at mrabarnett.plus.com: >> >> On 2022-09-29 16:54, Jen Kris via Python-list wrote: >> >> Recently I completed a project where I used >> PyObject_CallFunctionObjArgs extensively with the NLTK >> library from a program written in NASM, with no problems. >> Now I am on a new project where I call the Python random >> library.? I use the same setup as before, but I am getting >> a segfault with random.seed. >> >> At the start of the NASM program I call a C API program >> that gets PyObject pointers to ?seed? and ?randrange? in >> the same way as I did before: >> >> int64_t Get_LibModules(int64_t * return_array) >> { >> PyObject * pName_random = PyUnicode_FromString("random"); >> PyObject * pMod_random = PyImport_Import(pName_random); >> >> Both PyUnicode_FromString and PyImport_Import return new >> references or null pointers. >> >> if (pMod_random == 0x0){ >> PyErr_Print(); >> >> >> You're leaking a reference here (pName_random). >> >> return 1;} >> >> PyObject * pAttr_seed = >> PyObject_GetAttrString(pMod_random, "seed"); >> PyObject * pAttr_randrange = >> PyObject_GetAttrString(pMod_random, "randrange"); >> >> return_array[0] = (int64_t)pAttr_seed; >> return_array[1] = (int64_t)pAttr_randrange; >> >> >> You're leaking 2 references here (pName_random and pMod_random). >> >> return 0; >> } >> >> Later in the same program I call a C API program to call >> random.seed: >> >> int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) >> { >> PyObject * p_seed_calc = >> PyObject_CallFunctionObjArgs(pAttr_seed, value_1); >> >> >> It's expecting all of the arguments to be PyObject*, but >> value_1 is Py_ssize_t instead of PyObject* (a pointer to a >> _Python_ int). >> >> The argument list must end with a null pointer. >> >> It returns a new reference or a null pointer. >> >> >> if (p_seed_calc == 0x0){ >> ??? PyErr_Print(); >> ??? return 1;} >> >> //Prepare return values >> long return_val = PyLong_AsLong(p_seed_calc); >> >> You're leaking a reference here (p_seed_calc). >> >> return return_val; >> } >> >> The first program correctly imports ?random? and gets >> pointers to ?seed? and ?randrange.?? I verified that the >> same pointer is correctly passed into C_API_2, and the >> seed value (1234) is passed as? Py_ssize_t value_1.? But I >> get this segfault: >> >> Program received signal SIGSEGV, Segmentation fault. >> 0x00007ffff64858d5 in _Py_INCREF (op=0x4d2) at >> ../Include/object.h:459 >> 459???? ../Include/object.h: No such file or directory. >> >> So I tried Py_INCREF in the first program: >> >> Py_INCREF(pMod_random); >> Py_INCREF(pAttr_seed); >> >> Then I moved Py_INCREF(pAttr_seed) to the second program.? >> Same segfault. >> >> Finally, I initialized ?random? and ?seed? in the second >> program, where they are used.? Same segfault. >> >> The segfault refers to Py_INCREF, so this seems to do with >> reference counting, but Py_INCREF didn?t solve it. >> >> I?m using Python 3.8 on Ubuntu. >> >> Thanks for any ideas on how to solve this. >> >> Jen >> >> >> -- https://mail.python.org/mailman/listinfo/python-list >> >> >> -- https://mail.python.org/mailman/listinfo/python-list >> > -- > https://mail.python.org/mailman/listinfo/python-list > From miked at dewhirst.com.au Thu Sep 29 19:49:59 2022 From: miked at dewhirst.com.au (Mike Dewhirst) Date: Fri, 30 Sep 2022 09:49:59 +1000 Subject: python developer In-Reply-To: References: Message-ID: On 30/09/2022 5:09 am, Walid AlMasri wrote: > Hi all, > > I has been using python in scientific computing for many years > > I want to become a python developer so what is a good reference to follow ? A developer makes products whereas a scientist understands complexities. I would recommend reading about Scrum https://www.scrum.org/resources/what-is-scrum After many years you probably already know enough Python to do anything. Your task as a developer is to understand the relationship between a problem (business or scientific) and a maintainable, perhaps packaged, product which handles the drudgery and leaves expert judgement to the human user(s). Scrum is the easiest way down that road. You need a customer with a problem. You might be your own first customer. Cheers Mike > > Thanks in advance ! > > Walid -- Signed email is an absolute defence against phishing. This email has been signed with my private key. If you import my public key you can automatically decrypt my signature and be sure it came from me. Just ask and I'll send it to you. Your email software can handle signing. -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 495 bytes Desc: OpenPGP digital signature URL: From python at mrabarnett.plus.com Thu Sep 29 20:02:23 2022 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 30 Sep 2022 01:02:23 +0100 Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: References: Message-ID: On 2022-09-29 23:41, Jen Kris wrote: > > I just solved this C API problem, and I?m posting the answer to help > anyone else who might need it. > > The errors were: > > (1) we must call Py_INCREF on each object when it?s created. > Some functions return an object that has already been incref'ed ("new reference"). This occurs when it has either created a new object (the refcount will be 1) or has returned a pointer to an existing object (the refcount will be > 1 because it has been incref'ed). Other functions return an object that hasn't been incref'ed. This occurs when you're looking up something, for example, looking at a member of a list or the value of an attribute. > (2) in C_API_2 (see below) we don?t cast value_1 as I did before with > PyObject * value_ptr = (PyObject * )value_1.? Instead we use PyObject > * value_ptr = PyLong_FromLong(value_1); > > (3) The command string to PyObject_CallFunctionObjArgs must be null > terminated. > Always read the docs carefully! > Here?s the revised code: > > First we load the modules, and increment the reference to each object: > > int64_t Get_LibModules(int64_t * return_array) > { > PyObject * pName_random = PyUnicode_FromString("random"); > PyObject * pMod_random = PyImport_Import(pName_random); > Possible null pointers here. > Py_INCREF(pName_random); > Py_INCREF(pMod_random); > Py_INCREF will fail on a null pointer, which you haven't check for yet. If they aren't null, then they're new references (incref'ed) that you've incref'ed. > if (pMod_random == 0x0){ > PyErr_Print(); Leaks here because of the refcount. > return 1;} > > PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); > PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, > "randrange"); > Possible null pointers here too. > Py_INCREF(pAttr_seed); > Py_INCREF(pAttr_randrange); > Same issue as above. > return_array[0] = (int64_t)pAttr_seed; > return_array[1] = (int64_t)pAttr_randrange; > > return 0; > } > > Next we call a program to initialize the random number generator with > random.seed(), and increment the reference to its return value > p_seed_calc: > > int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) > { > PyObject * value_ptr = PyLong_FromLong(value_1); New reference. > PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, > value_ptr, NULL); > > // _________ > > if (p_seed_calc == 0x0){ > ??? PyErr_Print(); Leak. > ??? return 1;} > > Py_INCREF(p_seed_calc); > p_seed_calc will be a new reference to Py_None, incref'ed again, so leak. > return 0; > } > > Now we call another program to get a random number: > > int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1) > { > PyObject * value_ptr = PyLong_FromLong(value_1); > PyObject * p_randrange_calc = > PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL); > > if (p_randrange_calc == 0x0){ Leak. > ??? PyErr_Print(); > ??? return 1;} > > //Prepare return values > long return_val = PyLong_AsLong(p_randrange_calc); > Leak. > return return_val; > } > > That returns 28, which is what I get from the Python command line. > > Thanks again to MRAB for helpful comments. > > Jen > Getting refcounting right can be difficult! > > Sep 29, 2022, 15:31 by python at mrabarnett.plus.com: > > On 2022-09-29 21:47, Jen Kris wrote: > > To update my previous email, I found the problem, but I have a > new problem. > > Previously I cast PyObject * value_ptr = (PyObject * )value_1 > but that's not correct.? Instead I used PyObject * value_ptr = > PyLong_FromLong(value_1) and that works. HOWEVER, while > PyObject_CallFunctionObjArgs does work now, it returns -1, > which is not the right answer for random.seed.? I use "long > return_val = PyLong_AsLong(p_seed_calc);" to convert it to a long. > > random.seed returns None, so when you call > PyObject_CallFunctionObjArgs it returns a new reference to Py_None. > > If you then pass to PyLong_AsLong a reference to something that's > not a PyLong, it'll set an error and return -1. > > So my question is why do I get -1 as return value?? When I > query p_seed calc : get: > > (gdb) p p_seed_calc > $2 = (PyObject *) 0x7ffff69be120 <_Py_NoneStruct> > > Exactly. It's Py_None, not a PyLong. > > Thanks again. > > Jen > > > > > Sep 29, 2022, 13:02 by python-list at python.org: > > Thanks very much to @MRAB for taking time to answer.? I changed my > code to conform to your answer (as best I understand your comments > on references), but I still get the same error.? My comments > continue below the new code immediately below. > > int64_t Get_LibModules(int64_t * return_array) > { > PyObject * pName_random = PyUnicode_FromString("random"); > PyObject * pMod_random = PyImport_Import(pName_random); > > Py_INCREF(pName_random); > Py_INCREF(pMod_random); > > if (pMod_random == 0x0){ > PyErr_Print(); > return 1;} > > PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, > "seed"); > PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, > "randrange"); > > Py_INCREF(pAttr_seed); > Py_INCREF(pAttr_randrange); > > return_array[0] = (int64_t)pAttr_seed; > return_array[1] = (int64_t)pAttr_randrange; > > return 0; > } > > int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) > { > PyObject * value_ptr = (PyObject * )value_1; > PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, > value_ptr, NULL); > > if (p_seed_calc == 0x0){ > ??? PyErr_Print(); > ??? return 1;} > > //Prepare return values > long return_val = PyLong_AsLong(p_seed_calc); > > return return_val; > } > > So I incremented the reference to all objects in Get_LibModules, > but I still get the same segfault at > PyObject_CallFunctionObjArgs.? Unfortunately, reference counting > is not well documented so I?m not clear what?s wrong. > > > > > Sep 29, 2022, 10:06 by python at mrabarnett.plus.com: > > On 2022-09-29 16:54, Jen Kris via Python-list wrote: > > Recently I completed a project where I used > PyObject_CallFunctionObjArgs extensively with the NLTK > library from a program written in NASM, with no problems. > Now I am on a new project where I call the Python random > library.? I use the same setup as before, but I am getting > a segfault with random.seed. > > At the start of the NASM program I call a C API program > that gets PyObject pointers to ?seed? and ?randrange? in > the same way as I did before: > > int64_t Get_LibModules(int64_t * return_array) > { > PyObject * pName_random = PyUnicode_FromString("random"); > PyObject * pMod_random = PyImport_Import(pName_random); > > Both PyUnicode_FromString and PyImport_Import return new > references or null pointers. > > if (pMod_random == 0x0){ > PyErr_Print(); > > > You're leaking a reference here (pName_random). > > return 1;} > > PyObject * pAttr_seed = > PyObject_GetAttrString(pMod_random, "seed"); > PyObject * pAttr_randrange = > PyObject_GetAttrString(pMod_random, "randrange"); > > return_array[0] = (int64_t)pAttr_seed; > return_array[1] = (int64_t)pAttr_randrange; > > > You're leaking 2 references here (pName_random and pMod_random). > > return 0; > } > > Later in the same program I call a C API program to call > random.seed: > > int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) > { > PyObject * p_seed_calc = > PyObject_CallFunctionObjArgs(pAttr_seed, value_1); > > > It's expecting all of the arguments to be PyObject*, but > value_1 is Py_ssize_t instead of PyObject* (a pointer to a > _Python_ int). > > The argument list must end with a null pointer. > > It returns a new reference or a null pointer. > > > if (p_seed_calc == 0x0){ > ??? PyErr_Print(); > ??? return 1;} > > //Prepare return values > long return_val = PyLong_AsLong(p_seed_calc); > > You're leaking a reference here (p_seed_calc). > > return return_val; > } > > The first program correctly imports ?random? and gets > pointers to ?seed? and ?randrange.?? I verified that the > same pointer is correctly passed into C_API_2, and the > seed value (1234) is passed as? Py_ssize_t value_1.? But I > get this segfault: > > Program received signal SIGSEGV, Segmentation fault. > 0x00007ffff64858d5 in _Py_INCREF (op=0x4d2) at > ../Include/object.h:459 > 459???? ../Include/object.h: No such file or directory. > > So I tried Py_INCREF in the first program: > > Py_INCREF(pMod_random); > Py_INCREF(pAttr_seed); > > Then I moved Py_INCREF(pAttr_seed) to the second program. > Same segfault. > > Finally, I initialized ?random? and ?seed? in the second > program, where they are used.? Same segfault. > > The segfault refers to Py_INCREF, so this seems to do with > reference counting, but Py_INCREF didn?t solve it. > > I?m using Python 3.8 on Ubuntu. > > Thanks for any ideas on how to solve this. > > Jen > > > -- https://mail.python.org/mailman/listinfo/python-list > > > -- https://mail.python.org/mailman/listinfo/python-list > > -- > https://mail.python.org/mailman/listinfo/python-list > > From python at mrabarnett.plus.com Thu Sep 29 20:33:54 2022 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 30 Sep 2022 01:33:54 +0100 Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: References: Message-ID: <4e8fa90c-6f59-1999-93bc-2a31ba14c12e@mrabarnett.plus.com> On 2022-09-30 01:02, MRAB wrote: > On 2022-09-29 23:41, Jen Kris wrote: >> >> I just solved this C API problem, and I?m posting the answer to help >> anyone else who might need it. >> [snip] What I like to do is write comments that state which variables hold a reference, followed by '+' if it's a new reference (incref'ed) and '?' if it could be null. '+?' means that it's probably a new reference but could be null. Once I know that it's not null, I can remove the '?', and once I've decref'ed it (if required) and no longer need it, I remobe it from the comment. Clearing up references, as soon as they're not needed, helps to keep the number of current references more manageable. int64_t Get_LibModules(int64_t * return_array) { PyObject * pName_random = PyUnicode_FromString("random"); //> pName_random+? if (!pName_random) { PyErr_Print(); return 1; } //> pName_random+ PyObject * pMod_random = PyImport_Import(pName_random); //> pName_random+ pMod_random+? Py_DECREF(pName_random); //> pMod_random+? if (!pMod_random) { PyErr_Print(); return 1; } //> pMod_random+ PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); //> pMod_random+ pAttr_seed? if (!pAttr_seed) { Py_DECREF(pMod_random); PyErr_Print(); return 1; } //> pMod_random+ pAttr_seed PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange"); //> pMod_random+ pAttr_seed pAttr_randrange? Py_DECREF(pMod_random); //> pAttr_seed pAttr_randrange? if (!pAttr_randrange) { PyErr_Print(); return 1; } //> pAttr_seed pAttr_randrange return_array[0] = (int64_t)pAttr_seed; return_array[1] = (int64_t)pAttr_randrange; return 0; } int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) { PyObject * value_ptr = PyLong_FromLong(value_1); //> value_ptr+? if (!!value_ptr) { PyErr_Print(); return 1; } //> value_ptr+ PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_ptr, NULL); //> value_ptr+ p_seed_calc+? Py_DECREF(value_ptr); //> p_seed_calc+? if (!p_seed_calc) { PyErr_Print(); return 1; } //> p_seed_calc+ Py_DECREF(p_seed_calc); return 0; } int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1) { PyObject * value_ptr = PyLong_FromLong(value_1); //> value_ptr+? if (!value_ptr) { PyErr_Print(); return 1; } //> value_ptr+ PyObject * p_randrange_calc = PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL); //> value_ptr+ p_randrange_calc+? Py_DECREF(value_ptr); //> p_randrange_calc+? if (!p_randrange_calc) { PyErr_Print(); return 1; } //Prepare return values //> p_randrange_calc+ return_val = PyLong_AsLong(p_randrange_calc); Py_DECREF(p_randrange_calc); return return_val; } From balglaas at dds.nl Fri Sep 30 01:31:15 2022 From: balglaas at dds.nl (Jan van den Broek) Date: Fri, 30 Sep 2022 05:31:15 -0000 (UTC) Subject: python developer References: Message-ID: 2022-09-29, Mike Dewhirst schrieb: > This is an OpenPGP/MIME signed message (RFC 4880 and 3156) Why? [Schnipp] -- Jan v/d Broek balglaas at dds.nl From jenkris at tutanota.com Fri Sep 30 12:02:27 2022 From: jenkris at tutanota.com (Jen Kris) Date: Fri, 30 Sep 2022 18:02:27 +0200 (CEST) Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: <4e8fa90c-6f59-1999-93bc-2a31ba14c12e@mrabarnett.plus.com> References: <4e8fa90c-6f59-1999-93bc-2a31ba14c12e@mrabarnett.plus.com> Message-ID: Thanks very much for your detailed reply.? I have a few followup questions.? You said, ?Some functions return an object that has already been incref'ed ("new reference"). This occurs when it has either created a new object (the refcount will be 1) or has returned a pointer to an existing object (the refcount will be > 1 because it has been incref'ed).? Other functions return an object that hasn't been incref'ed. This occurs when you're looking up something, for example, looking at a member of a list or the value of an attribute.?? In the official docs some functions show ?Return value: New reference? and others do not.? Is there any reason why I should not just INCREF on every new object, regardless of whether it?s a new reference or not, and DECREF when I am finished with it?? The answer at https://stackoverflow.com/questions/59870703/python-c-extension-need-to-py-incref-a-borrowed-reference-if-not-returning-it-to says ?With out-of-order execution, the INCREF/DECREF are basically free operations, so performance is no reason to leave them out.?? Doing so means I don?t have to check each object to see if it needs to be INCREF?d or not, and that is a big help.? Also:? What is a borrowed reference, and how does it effect reference counting?? According to https://jayrambhia.com/blog/pythonc-api-reference-counting, ?Use Py_INCREF on a borrowed PyObject pointer you already have. This increments the reference count on the object, and obligates you to dispose of it properly.?? So I guess it?s yes, but I?m confused by ?pointer you already have.?? What does it mean to steal a reference?? If a function steals a reference does it have to decref it without incref (because it?s stolen)? Finally, you said: if (pMod_random == 0x0){ ??? PyErr_Print(); Leaks here because of the refcount Assuming pMod_random is not null, why would this leak?? Thanks again for your input on this question.? Jen Sep 29, 2022, 17:33 by python at mrabarnett.plus.com: > On 2022-09-30 01:02, MRAB wrote: > >> On 2022-09-29 23:41, Jen Kris wrote: >> >>> >>> I just solved this C API problem, and I?m posting the answer to help anyone else who might need it. >>> > [snip] > > What I like to do is write comments that state which variables hold a reference, followed by '+' if it's a new reference (incref'ed) and '?' if it could be null. '+?' means that it's probably a new reference but could be null. Once I know that it's not null, I can remove the '?', and once I've decref'ed it (if required) and no longer need it, I remobe it from the comment. > > Clearing up references, as soon as they're not needed, helps to keep the number of current references more manageable. > > > int64_t Get_LibModules(int64_t * return_array) { > PyObject * pName_random = PyUnicode_FromString("random"); > //> pName_random+? > if (!pName_random) { > PyErr_Print(); > return 1; > } > > //> pName_random+ > PyObject * pMod_random = PyImport_Import(pName_random); > //> pName_random+ pMod_random+? > Py_DECREF(pName_random); > //> pMod_random+? > if (!pMod_random) { > PyErr_Print(); > return 1; > } > > //> pMod_random+ > PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); > //> pMod_random+ pAttr_seed? > if (!pAttr_seed) { > Py_DECREF(pMod_random); > PyErr_Print(); > return 1; > } > > //> pMod_random+ pAttr_seed > PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange"); > //> pMod_random+ pAttr_seed pAttr_randrange? > Py_DECREF(pMod_random); > //> pAttr_seed pAttr_randrange? > if (!pAttr_randrange) { > PyErr_Print(); > return 1; > } > > //> pAttr_seed pAttr_randrange > return_array[0] = (int64_t)pAttr_seed; > return_array[1] = (int64_t)pAttr_randrange; > > return 0; > } > > int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) { > PyObject * value_ptr = PyLong_FromLong(value_1); > //> value_ptr+? > if (!!value_ptr) { > PyErr_Print(); > return 1; > } > > //> value_ptr+ > PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_ptr, NULL); > //> value_ptr+ p_seed_calc+? > Py_DECREF(value_ptr); > //> p_seed_calc+? > if (!p_seed_calc) { > PyErr_Print(); > return 1; > } > > //> p_seed_calc+ > Py_DECREF(p_seed_calc); > return 0; > } > > int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1) { > PyObject * value_ptr = PyLong_FromLong(value_1); > //> value_ptr+? > if (!value_ptr) { > PyErr_Print(); > return 1; > } > > //> value_ptr+ > PyObject * p_randrange_calc = PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL); > //> value_ptr+ p_randrange_calc+? > Py_DECREF(value_ptr); > //> p_randrange_calc+? > if (!p_randrange_calc) { > PyErr_Print(); > return 1; > } > > //Prepare return values > //> p_randrange_calc+ > return_val = PyLong_AsLong(p_randrange_calc); > Py_DECREF(p_randrange_calc); > > return return_val; > } > > -- > https://mail.python.org/mailman/listinfo/python-list > From heindsight at kruger.dev Fri Sep 30 14:32:17 2022 From: heindsight at kruger.dev (Heinrich Kruger) Date: Fri, 30 Sep 2022 18:32:17 +0000 Subject: Implementation of an lru_cache() decorator that ignores the first argument In-Reply-To: References: Message-ID: On Thursday, September 29th, 2022 at 07:18, Robert Latest via Python-list wrote: > Hi Chris and dh, > > thanks for your --as usually-- thoughtful and interesting answers. Indeed, when > doing these web applications I find that there are several layers of useful, > maybe less useful, and unknown caching. Many of my requests rely on a > notoriously unreliable read-only database outside of my control, so I cache the > required data into a local DB on my server, then I do some in-memory caching of > expensive data plots because I haven't figured out how to reliably exploit the > client-side caching ... then every middleware on that path may or may not > implement its own version of clever or not-so-clever caching. Probably not a > good idea to try and outsmart that by adding yet another thing that may break > or not be up-to-date at the wrong moment. > > That said, the only caching that SQLAlchemy does (to my knowledge) is that it > stores retrieved DB items by their primary keys in the session. Not worth much > since the session gets created and dumped on each request by SQA's unit of work > paradigm. But the DB backend itself may be caching repeated queries. > > Back to Python-theory: The "Cloak" object is the only way I could think of to > sneak changing data past lru_cache's key lookup mechanism. Is there some other > method? Just curious. > > -- > https://mail.python.org/mailman/listinfo/python-list You could use closures. For example, something like this: import functools import time def my_cache(timeout): start = time.monotonic() def cache_decorator(func): wrapper = _my_cache_wrapper(func, timeout, start) return functools.update_wrapper(wrapper, func) return cache_decorator def _my_cache_wrapper(func, timeout, start): first = None @functools.cache def _cached(timeout_factor, *args): print("In the _cached function") return func(first, *args) def wrapper(*args): print("In the wrapper") nonlocal first first, *rest = args elapsed = time.monotonic() - start timeout_factor = elapsed // timeout return _cached(timeout_factor, *rest) return wrapper @my_cache(3) def expensive(first, second, third): print("In the expensive function") return (first, second, third) if __name__ == "__main__": print(expensive(1, 2, 3)) print() time.sleep(2) print(expensive(2, 2, 3)) print() time.sleep(2) print(expensive(3, 2, 3)) This should output the following: In the wrapper In the _cached function In the expensive function (1, 2, 3) In the wrapper (1, 2, 3) In the wrapper In the _cached function In the expensive function (3, 2, 3) It's not necessarily better than your version though. :D Kind regards, Heinrich Kruger From hjp-python at hjp.at Fri Sep 30 14:55:35 2022 From: hjp-python at hjp.at (Peter J. Holzer) Date: Fri, 30 Sep 2022 20:55:35 +0200 Subject: python developer In-Reply-To: References: Message-ID: <20220930185535.sacm6glgrfh2e26a@hjp.at> On 2022-09-30 05:31:15 -0000, Jan van den Broek wrote: > 2022-09-29, Mike Dewhirst schrieb: > > This is an OpenPGP/MIME signed message (RFC 4880 and 3156) > > Why? Why not? Signing an email protects against impersonation and tampering. While you may not be assured that my name is indeed "Peter J. Holzer", you can be sure that all the messages signed with the same key came from the same person and that they were not modified by somebody else (at least not the signed part). I've been signing all my (private) mails for 25 years or so. hp -- _ | Peter J. Holzer | Story must make more sense than reality. |_|_) | | | | | hjp at hjp.at | -- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From python at mrabarnett.plus.com Fri Sep 30 15:18:55 2022 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 30 Sep 2022 20:18:55 +0100 Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: References: <4e8fa90c-6f59-1999-93bc-2a31ba14c12e@mrabarnett.plus.com> Message-ID: <65c6d3ef-c426-d486-565b-549442598eda@mrabarnett.plus.com> On 2022-09-30 17:02, Jen Kris wrote: > > Thanks very much for your detailed reply.? I have a few followup > questions. > > You said, ?Some functions return an object that has already been > incref'ed ("new reference"). This occurs when it has either created a > new object (the refcount will be 1) or has returned a pointer to an > existing object (the refcount will be > 1 because it has been > incref'ed).? Other functions return an object that hasn't been > incref'ed. This occurs when you're looking up something, for example, > looking at a member of a list or the value of an attribute.? > > In the official docs some functions show ?Return value: New reference? > and others do not.? Is there any reason why I should not just INCREF > on every new object, regardless of whether it?s a new reference or > not, and DECREF when I am finished with it?? The answer at > https://stackoverflow.com/questions/59870703/python-c-extension-need-to-py-incref-a-borrowed-reference-if-not-returning-it-to > says ?With out-of-order execution, the INCREF/DECREF are basically > free operations, so performance is no reason to leave them out.?? > Doing so means I don?t have to check each object to see if it needs to > be INCREF?d or not, and that is a big help. > It's OK to INCREF them, provided that you DECREF them when you no longer need them, and remember that if it's a "new reference" you'd need to DECREF it twice. > Also: > > What is a borrowed reference, and how does it effect reference > counting?? According to > https://jayrambhia.com/blog/pythonc-api-reference-counting, ?Use > Py_INCREF on a borrowed PyObject pointer you already have. This > increments the reference count on the object, and obligates you to > dispose of it properly.?? So I guess it?s yes, but I?m confused by > ?pointer you already have.? A borrowed reference is when it hasn't been INCREFed. You can think of INCREFing as a way of indicating ownership, which is often shared ownership (refcount > 1). When you're borrowing a reference, you're using it temporarily, but not claiming ownership. When the last owner releases its ownership (DECREF reduces the refcount to 0), the object can be garbage collected. When, say, you lookup an attribute, or get an object from a list with PyList_GetItem, it won't have been INCREFed. You're using it temporarily, just borrowing a reference. > > What does it mean to steal a reference?? If a function steals a > reference does it have to decref it without incref (because it?s stolen)? When function steals a reference, it's claiming ownership but not INCREFing it. > > Finally, you said: > > if (pMod_random == 0x0){ > ??? PyErr_Print(); > Leaks here because of the refcount > > Assuming pMod_random is not null, why would this leak? > It's pName_random that's the leak. PyUnicode_FromString("random") will either create and return a new object for the string "random" (refcount == 1) or return a reference to an existing object (refcount > 1). You need to DECREF it before returning from the function. Suppose it created a new object. You call the function, it creates an object, you use it, then return from the function. The object still exists, but there's no reference to it. Now call the function again. It creates another object, you use it, then return from the function. You now have 2 objects with no reference to them. > Thanks again for your input on this question. > > Jen > > > > Sep 29, 2022, 17:33 by python at mrabarnett.plus.com: > > On 2022-09-30 01:02, MRAB wrote: > > On 2022-09-29 23:41, Jen Kris wrote: > > > I just solved this C API problem, and I?m posting the > answer to help anyone else who might need it. > > [snip] > > What I like to do is write comments that state which variables > hold a reference, followed by '+' if it's a new reference > (incref'ed) and '?' if it could be null. '+?' means that it's > probably a new reference but could be null. Once I know that it's > not null, I can remove the '?', and once I've decref'ed it (if > required) and no longer need it, I remobe it from the comment. > > Clearing up references, as soon as they're not needed, helps to > keep the number of current references more manageable. > > > int64_t Get_LibModules(int64_t * return_array) { > PyObject * pName_random = PyUnicode_FromString("random"); > //> pName_random+? > if (!pName_random) { > PyErr_Print(); > return 1; > } > > //> pName_random+ > PyObject * pMod_random = PyImport_Import(pName_random); > //> pName_random+ pMod_random+? > Py_DECREF(pName_random); > //> pMod_random+? > if (!pMod_random) { > PyErr_Print(); > return 1; > } > > //> pMod_random+ > PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); > //> pMod_random+ pAttr_seed? > if (!pAttr_seed) { > Py_DECREF(pMod_random); > PyErr_Print(); > return 1; > } > > //> pMod_random+ pAttr_seed > PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, > "randrange"); > //> pMod_random+ pAttr_seed pAttr_randrange? > Py_DECREF(pMod_random); > //> pAttr_seed pAttr_randrange? > if (!pAttr_randrange) { > PyErr_Print(); > return 1; > } > > //> pAttr_seed pAttr_randrange > return_array[0] = (int64_t)pAttr_seed; > return_array[1] = (int64_t)pAttr_randrange; > > return 0; > } > > int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) { > PyObject * value_ptr = PyLong_FromLong(value_1); > //> value_ptr+? > if (!!value_ptr) { > PyErr_Print(); > return 1; > } > > //> value_ptr+ > PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, > value_ptr, NULL); > //> value_ptr+ p_seed_calc+? > Py_DECREF(value_ptr); > //> p_seed_calc+? > if (!p_seed_calc) { > PyErr_Print(); > return 1; > } > > //> p_seed_calc+ > Py_DECREF(p_seed_calc); > return 0; > } > > int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1) { > PyObject * value_ptr = PyLong_FromLong(value_1); > //> value_ptr+? > if (!value_ptr) { > PyErr_Print(); > return 1; > } > > //> value_ptr+ > PyObject * p_randrange_calc = > PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL); > //> value_ptr+ p_randrange_calc+? > Py_DECREF(value_ptr); > //> p_randrange_calc+? > if (!p_randrange_calc) { > PyErr_Print(); > return 1; > } > > //Prepare return values > //> p_randrange_calc+ > return_val = PyLong_AsLong(p_randrange_calc); > Py_DECREF(p_randrange_calc); > > return return_val; > } > > -- > https://mail.python.org/mailman/listinfo/python-list > > From jenkris at tutanota.com Fri Sep 30 16:34:07 2022 From: jenkris at tutanota.com (Jen Kris) Date: Fri, 30 Sep 2022 22:34:07 +0200 (CEST) Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: <65c6d3ef-c426-d486-565b-549442598eda@mrabarnett.plus.com> References: <4e8fa90c-6f59-1999-93bc-2a31ba14c12e@mrabarnett.plus.com> <65c6d3ef-c426-d486-565b-549442598eda@mrabarnett.plus.com> Message-ID: That's great.? It clarifies things a lot for me, particularly re ref count for new references.? I would have had trouble if I didn't decref it twice.? Thanks very much once again.? Sep 30, 2022, 12:18 by python at mrabarnett.plus.com: > On 2022-09-30 17:02, Jen Kris wrote: > >> >> Thanks very much for your detailed reply.? I have a few followup questions. >> >> You said, ?Some functions return an object that has already been incref'ed ("new reference"). This occurs when it has either created a new object (the refcount will be 1) or has returned a pointer to an existing object (the refcount will be > 1 because it has been incref'ed).? Other functions return an object that hasn't been incref'ed. This occurs when you're looking up something, for example, looking at a member of a list or the value of an attribute.? >> >> In the official docs some functions show ?Return value: New reference? and others do not.? Is there any reason why I should not just INCREF on every new object, regardless of whether it?s a new reference or not, and DECREF when I am finished with it?? The answer at https://stackoverflow.com/questions/59870703/python-c-extension-need-to-py-incref-a-borrowed-reference-if-not-returning-it-to says ?With out-of-order execution, the INCREF/DECREF are basically free operations, so performance is no reason to leave them out.?? Doing so means I don?t have to check each object to see if it needs to be INCREF?d or not, and that is a big help. >> > It's OK to INCREF them, provided that you DECREF them when you no longer need them, and remember that if it's a "new reference" you'd need to DECREF it twice. > >> Also: >> >> What is a borrowed reference, and how does it effect reference counting?? According to https://jayrambhia.com/blog/pythonc-api-reference-counting, ?Use Py_INCREF on a borrowed PyObject pointer you already have. This increments the reference count on the object, and obligates you to dispose of it properly.?? So I guess it?s yes, but I?m confused by ?pointer you already have.? >> > > A borrowed reference is when it hasn't been INCREFed. > > You can think of INCREFing as a way of indicating ownership, which is often shared ownership (refcount > 1). When you're borrowing a reference, you're using it temporarily, but not claiming ownership. When the last owner releases its ownership (DECREF reduces the refcount to 0), the object can be garbage collected. > > When, say, you lookup an attribute, or get an object from a list with PyList_GetItem, it won't have been INCREFed. You're using it temporarily, just borrowing a reference. > >> >> What does it mean to steal a reference?? If a function steals a reference does it have to decref it without incref (because it?s stolen)? >> > When function steals a reference, it's claiming ownership but not INCREFing it. > >> >> Finally, you said: >> >> if (pMod_random == 0x0){ >> ??? PyErr_Print(); >> Leaks here because of the refcount >> >> Assuming pMod_random is not null, why would this leak? >> > It's pName_random that's the leak. > > PyUnicode_FromString("random") will either create and return a new object for the string "random" (refcount == 1) or return a reference to an existing object (refcount > 1). You need to DECREF it before returning from the function. > > Suppose it created a new object. You call the function, it creates an object, you use it, then return from the function. The object still exists, but there's no reference to it. Now call the function again. It creates another object, you use it, then return from the function. You now have 2 objects with no reference to them. > >> Thanks again for your input on this question. >> >> Jen >> >> >> >> Sep 29, 2022, 17:33 by python at mrabarnett.plus.com: >> >> On 2022-09-30 01:02, MRAB wrote: >> >> On 2022-09-29 23:41, Jen Kris wrote: >> >> >> I just solved this C API problem, and I?m posting the >> answer to help anyone else who might need it. >> >> [snip] >> >> What I like to do is write comments that state which variables >> hold a reference, followed by '+' if it's a new reference >> (incref'ed) and '?' if it could be null. '+?' means that it's >> probably a new reference but could be null. Once I know that it's >> not null, I can remove the '?', and once I've decref'ed it (if >> required) and no longer need it, I remobe it from the comment. >> >> Clearing up references, as soon as they're not needed, helps to >> keep the number of current references more manageable. >> >> >> int64_t Get_LibModules(int64_t * return_array) { >> PyObject * pName_random = PyUnicode_FromString("random"); >> //> pName_random+? >> if (!pName_random) { >> PyErr_Print(); >> return 1; >> } >> >> //> pName_random+ >> PyObject * pMod_random = PyImport_Import(pName_random); >> //> pName_random+ pMod_random+? >> Py_DECREF(pName_random); >> //> pMod_random+? >> if (!pMod_random) { >> PyErr_Print(); >> return 1; >> } >> >> //> pMod_random+ >> PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed"); >> //> pMod_random+ pAttr_seed? >> if (!pAttr_seed) { >> Py_DECREF(pMod_random); >> PyErr_Print(); >> return 1; >> } >> >> //> pMod_random+ pAttr_seed >> PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, >> "randrange"); >> //> pMod_random+ pAttr_seed pAttr_randrange? >> Py_DECREF(pMod_random); >> //> pAttr_seed pAttr_randrange? >> if (!pAttr_randrange) { >> PyErr_Print(); >> return 1; >> } >> >> //> pAttr_seed pAttr_randrange >> return_array[0] = (int64_t)pAttr_seed; >> return_array[1] = (int64_t)pAttr_randrange; >> >> return 0; >> } >> >> int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1) { >> PyObject * value_ptr = PyLong_FromLong(value_1); >> //> value_ptr+? >> if (!!value_ptr) { >> PyErr_Print(); >> return 1; >> } >> >> //> value_ptr+ >> PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, >> value_ptr, NULL); >> //> value_ptr+ p_seed_calc+? >> Py_DECREF(value_ptr); >> //> p_seed_calc+? >> if (!p_seed_calc) { >> PyErr_Print(); >> return 1; >> } >> >> //> p_seed_calc+ >> Py_DECREF(p_seed_calc); >> return 0; >> } >> >> int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1) { >> PyObject * value_ptr = PyLong_FromLong(value_1); >> //> value_ptr+? >> if (!value_ptr) { >> PyErr_Print(); >> return 1; >> } >> >> //> value_ptr+ >> PyObject * p_randrange_calc = >> PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL); >> //> value_ptr+ p_randrange_calc+? >> Py_DECREF(value_ptr); >> //> p_randrange_calc+? >> if (!p_randrange_calc) { >> PyErr_Print(); >> return 1; >> } >> >> //Prepare return values >> //> p_randrange_calc+ >> return_val = PyLong_AsLong(p_randrange_calc); >> Py_DECREF(p_randrange_calc); >> >> return return_val; >> } >> >> -- https://mail.python.org/mailman/listinfo/python-list >> From greg.ewing at canterbury.ac.nz Fri Sep 30 16:59:00 2022 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 1 Oct 2022 09:59:00 +1300 Subject: PyObject_CallFunctionObjArgs segfaults In-Reply-To: References: <4e8fa90c-6f59-1999-93bc-2a31ba14c12e@mrabarnett.plus.com> <65c6d3ef-c426-d486-565b-549442598eda@mrabarnett.plus.com> Message-ID: On 1/10/22 8:18 am, MRAB wrote: > It's OK to INCREF them, provided that you DECREF them when you no longer > need them, and remember that if it's a "new reference" you'd need to > DECREF it twice. Which means there would usually be no point in doing the extra INCREF/DECREF. You still need to know whether it's a new reference or not and treat it accordingly. -- Greg