From vanderkelen.francois at gmail.com Fri May 16 11:16:28 2014 From: vanderkelen.francois at gmail.com (=?UTF-8?Q?Fran=C3=A7ois_Vanderkelen?=) Date: Fri, 16 May 2014 11:16:28 +0200 Subject: [code-quality] Namespaced packages and relative imports Message-ID: Hi experts ! I am struggling with Pylint with an issue that might be a problem coming from my code. Since I am relatively new to python (~1year) I would like to have your input on the matter. I am trying to split a big python project I am doing for my company into several packages, allowing me to update each part without impacting the others. Doing that, we decided to go with Jenkins to improve our code quality which lead me to implement pylint for code violations checks. Currently my build process is the following : - Create a new virtualenv with Python version 2.7.6 - Install all dependencies needed by my package - Download the last version of the package from the master branch in our internal gitlab - Run nosetests with coverage reports - Run pylint and export the result to have some violations reports in Jenkins. At first I had a lot of pylint errors since I wasn't installing all dependencies, resulting in something like 200 import errors ... Now it is reduced to some 60 errors but when I look a the build output log, I can see that pylint is actually crashing in the middle of the process. ************* Module company.project.core.security.app company/project/core/security/app.py:1: [C0111(missing-docstring), ] Missing module docstring load_entry_point('pylint==1.2.1', 'console_scripts', 'pylint')() File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/__init__.py", line 21, in run_pylint Run(sys.argv[1:]) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/lint.py", line 1051, in __init__ linter.check(args) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/lint.py", line 626, in check self.check_astroid_module(astroid, walker, rawcheckers, tokencheckers) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/lint.py", line 712, in check_astroid_module walker.walk(astroid) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/utils.py", line 715, in walk self.walk(child) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/utils.py", line 712, in walk cb(astroid) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/checkers/imports.py", line 269, in visit_from self._add_imported_module(node, '%s.%s' % (importedmodnode.name, name)) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/checkers/imports.py", line 302, in _add_imported_module importedmodname = get_module_part(importedmodname) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py", line 352, in get_module_part path=path, context_file=context_file) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py", line 297, in file_from_modpath return _file_from_modpath(modpath, path, context) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py", line 556, in _file_from_modpath mtype, mp_filename = _module_file(modpath, path) File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py", line 636, in _module_file return _search_zip(modpath, pic)[:2] File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/logilab/common/modutils.py", line 577, in _search_zip raise ImportError('No module named %s' % '.'.join(modpath)) ImportError: No module named project This is how my package is constructed : company/ -- __init__.py -- project/ -- __init__.py -- core/ -- __init__.py -- app.py -- security/ -- __init__.py -- app.py In company.project.core.security.app I am trying to import company.project.core.app which contains some BaseClass for my project using: from ..app import BaseApplication This is working fine with python, although pylint is crying out loud on this specific line. I tried to replace the relative import by core.app which worked fine with pylint but my code is not working anymore. I tried also to use the full path (company.project.core.app) but I have the exact same import error. Be aware that both company and project are just namespace packages. Also I am forced to use the path to run pylint because it looks like my top-level folder is not a package (even if it contains a __init__ file). pylint company.project.core No module named project.core (fatal) pylint company/project/core is working. When I try to run pylint directly from the top-level package, I have the exact same result as from the path. A workaround I can try is to install the package before running pylint on it but I reckon since my cwd is supposed to be added to the path I can't see how it would help (although I have other company packages installed). What do you think I should do ? Is my design a poorly chosen one and I don't have any solution ? Thanks, Fran?ois -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylvain.thenault at logilab.fr Fri May 16 14:02:49 2014 From: sylvain.thenault at logilab.fr (Sylvain =?utf-8?B?VGjDqW5hdWx0?=) Date: Fri, 16 May 2014 14:02:49 +0200 Subject: [code-quality] Namespaced packages and relative imports In-Reply-To: References: Message-ID: <20140516120249.GR2881@logilab.fr> On 16 mai 11:16, Fran?ois Vanderkelen wrote: > Hi experts ! Hi Fran?ois, > I am struggling with Pylint with an issue that might be a problem coming > from my code. Since I am relatively new to python (~1year) I would like to > have your input on the matter. > > I am trying to split a big python project I am doing for my company into > several packages, allowing me to update each part without impacting the > others. Doing that, we decided to go with Jenkins to improve our code > quality which lead me to implement pylint for code violations checks. > > Currently my build process is the following : > > - Create a new virtualenv with Python version 2.7.6 > - Install all dependencies needed by my package > - Download the last version of the package from the master branch in our > internal gitlab > - Run nosetests with coverage reports > - Run pylint and export the result to have some violations reports in > Jenkins. > > At first I had a lot of pylint errors since I wasn't installing all > dependencies, resulting in something like 200 import errors ... Now it is > reduced to some 60 errors but when I look a the build output log, I can see > that pylint is actually crashing in the middle of the process. > > ************* Module company.project.core.security.app > company/project/core/security/app.py:1: [C0111(missing-docstring), ] > Missing module docstring > load_entry_point('pylint==1.2.1', 'console_scripts', 'pylint')() > File "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/__init__.py", > line 577, in _search_zip > raise ImportError('No module named %s' % '.'.join(modpath)) > ImportError: No module named project Would you create an issue in pylint's bitbucket tracker for this? In any case it shouldn't crash like that? > This is how my package is constructed : > > company/ > -- __init__.py > -- project/ > -- __init__.py > -- core/ > -- __init__.py > -- app.py > -- security/ > -- __init__.py > -- app.py > > In company.project.core.security.app I am trying to import > company.project.core.app which contains some BaseClass for my project using: > > from ..app import BaseApplication This is probably another issue that deserves a ticket as well. > This is working fine with python, although pylint is crying out loud on > this specific line. I tried to replace the relative import by core.app > which worked fine with pylint but my code is not working anymore. I tried > also to use the full path (company.project.core.app) but I have the exact > same import error. > > Be aware that both company and project are just namespace packages. what do you call namespace package ? Do you mean part of them are installed in different directories? > Also I am forced to use the path to run pylint because it looks like my > top-level folder is not a package (even if it contains a __init__ file). > > pylint company.project.core > No module named project.core (fatal) and this work if you set a PYTHONPATH? > pylint company/project/core is working. > > When I try to run pylint directly from the top-level package, I have the > exact same result as from the path. > > A workaround I can try is to install the package before running pylint on > it but I reckon since my cwd is supposed to be added to the path I can't > see how it would help (although I have other company packages installed). It should work. [syt at orion ~]$ mkdir x [syt at orion ~]$ mkdir x/y [syt at orion ~]$ touch x/__init__.py [syt at orion ~]$ touch x/y/__init__.py [syt at orion ~]$ pylint -rn x.y ************* Module x.y C: 1, 0: Missing module docstring (missing-docstring) > What do you think I should do ? Is my design a poorly chosen one and I > don't have any solution ? At a quick glance I would say you seem to abuse a bit from packages, leading to a deep structure which may be over-complicated. For instance, do you really want a 'company' first level package? Anyway, that doesn't seem to be the cause of your problems with pylint. At least, it shouldn't. -- Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42) Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services CubicWeb, the semantic web framework: http://www.cubicweb.org From vanderkelen.francois at gmail.com Fri May 16 14:42:26 2014 From: vanderkelen.francois at gmail.com (=?UTF-8?Q?Fran=C3=A7ois_Vanderkelen?=) Date: Fri, 16 May 2014 14:42:26 +0200 Subject: [code-quality] Namespaced packages and relative imports In-Reply-To: <20140516120249.GR2881@logilab.fr> References: <20140516120249.GR2881@logilab.fr> Message-ID: Hi Sylvain, First of all thanks for the quick answer. I tried to create a dummy project (the real one is confidential and not mine to give to the community) and it helped me narrow down my two issues. The ImportError: No module named project is linked to the relative import, and I could not reproduce the error in a dummy project. I finally found it, it has to do with my namespaces packages. What I mean by namespaces is in my python library I dispose of several packages like this : company.mongodb company.redis company.hbase And also : company.project.core company.project.app.dashboard company.project.app.security Where company and project are all the same things, this allows me and my team to easily identify each package and what it is supposed to do. We try to foster the open-source spirit in my company, hence the company top-level namespace. It is still internal but at least we share it across all teams :) Going back to the issue, if one package of the namespace is installed, then apparently pylint is looking directly inside the python path and can't find my package since it is not yet installed. I have the same issue with the "python setup.py develop" command, so I guess I am doing namespacing wrong, or it is not well defined in python ? The only thing I do is adding *_import__('pkg_resources').declare_namespace(__name__) *in the __init__.py of the namespace package. If I run pylint in another environment without any packages installed, it works like a charm (except I have a bunch of violations for missing imports). Do you want me to open an issue on bitbucket explaining all that or do you think I am doing something I am not supposed to do ? Cheers Fran?ois 2014-05-16 14:02 GMT+02:00 Sylvain Th?nault : > On 16 mai 11:16, Fran?ois Vanderkelen wrote: > > Hi experts ! > > Hi Fran?ois, > > > I am struggling with Pylint with an issue that might be a problem coming > > from my code. Since I am relatively new to python (~1year) I would like > to > > have your input on the matter. > > > > I am trying to split a big python project I am doing for my company into > > several packages, allowing me to update each part without impacting the > > others. Doing that, we decided to go with Jenkins to improve our code > > quality which lead me to implement pylint for code violations checks. > > > > Currently my build process is the following : > > > > - Create a new virtualenv with Python version 2.7.6 > > - Install all dependencies needed by my package > > - Download the last version of the package from the master branch in our > > internal gitlab > > - Run nosetests with coverage reports > > - Run pylint and export the result to have some violations reports in > > Jenkins. > > > > At first I had a lot of pylint errors since I wasn't installing all > > dependencies, resulting in something like 200 import errors ... Now it is > > reduced to some 60 errors but when I look a the build output log, I can > see > > that pylint is actually crashing in the middle of the process. > > > > ************* Module company.project.core.security.app > > company/project/core/security/app.py:1: [C0111(missing-docstring), ] > > Missing module docstring > > load_entry_point('pylint==1.2.1', 'console_scripts', 'pylint')() > > File > "/var/lib/jenkins/.pyenv/versions/2.7.6/lib/python2.7/site-packages/pylint/__init__.py", > > line 577, in _search_zip > > raise ImportError('No module named %s' % '.'.join(modpath)) > > ImportError: No module named project > > Would you create an issue in pylint's bitbucket tracker for this? In any > case > it shouldn't crash like that? > > > This is how my package is constructed : > > > > company/ > > -- __init__.py > > -- project/ > > -- __init__.py > > -- core/ > > -- __init__.py > > -- app.py > > -- security/ > > -- __init__.py > > -- app.py > > > > In company.project.core.security.app I am trying to import > > company.project.core.app which contains some BaseClass for my project > using: > > > > from ..app import BaseApplication > > This is probably another issue that deserves a ticket as well. > > > This is working fine with python, although pylint is crying out loud on > > this specific line. I tried to replace the relative import by core.app > > which worked fine with pylint but my code is not working anymore. I tried > > also to use the full path (company.project.core.app) but I have the exact > > same import error. > > > > Be aware that both company and project are just namespace packages. > > what do you call namespace package ? Do you mean part of them are > installed in > different directories? > > > Also I am forced to use the path to run pylint because it looks like my > > top-level folder is not a package (even if it contains a __init__ file). > > > > pylint company.project.core > > No module named project.core (fatal) > > and this work if you set a PYTHONPATH? > > > pylint company/project/core is working. > > > > When I try to run pylint directly from the top-level package, I have the > > exact same result as from the path. > > > > A workaround I can try is to install the package before running pylint on > > it but I reckon since my cwd is supposed to be added to the path I can't > > see how it would help (although I have other company packages installed). > > It should work. > > [syt at orion ~]$ mkdir x > [syt at orion ~]$ mkdir x/y > [syt at orion ~]$ touch x/__init__.py > [syt at orion ~]$ touch x/y/__init__.py > [syt at orion ~]$ pylint -rn x.y > ************* Module x.y > C: 1, 0: Missing module docstring (missing-docstring) > > > What do you think I should do ? Is my design a poorly chosen one and I > > don't have any solution ? > > At a quick glance I would say you seem to abuse a bit from packages, > leading to > a deep structure which may be over-complicated. For instance, do you > really want > a 'company' first level package? Anyway, that doesn't seem to be the cause > of > your problems with pylint. At least, it shouldn't. > > -- > Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse > (05.62.17.16.42) > Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations > D?veloppement logiciel sur mesure: http://www.logilab.fr/services > CubicWeb, the semantic web framework: http://www.cubicweb.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylvain.thenault at logilab.fr Fri May 16 15:01:54 2014 From: sylvain.thenault at logilab.fr (Sylvain =?utf-8?B?VGjDqW5hdWx0?=) Date: Fri, 16 May 2014 15:01:54 +0200 Subject: [code-quality] Namespaced packages and relative imports In-Reply-To: References: <20140516120249.GR2881@logilab.fr> Message-ID: <20140516130154.GV2881@logilab.fr> On 16 mai 14:42, Fran?ois Vanderkelen wrote: > Hi Sylvain, > > First of all thanks for the quick answer. I tried to create a dummy project > (the real one is confidential and not mine to give to the community) and it > helped me narrow down my two issues. > > The ImportError: No module named project is linked to the relative import, > and I could not reproduce the error in a dummy project. I finally found it, > it has to do with my namespaces packages. > > What I mean by namespaces is in my python library I dispose of several > packages like this : > > company.mongodb > company.redis > company.hbase > > And also : > > company.project.core > company.project.app.dashboard > company.project.app.security > > Where company and project are all the same things, this allows me and my > team to easily identify each package and what it is supposed to do. We try > to foster the open-source spirit in my company, hence the company top-level > namespace. It is still internal but at least we share it across all teams :) > > Going back to the issue, if one package of the namespace is installed, then > apparently pylint is looking directly inside the python path and can't find > my package since it is not yet installed. I have the same issue with the > "python setup.py develop" command, so I guess I am doing namespacing wrong, > or it is not well defined in python ? > > The only thing I do is adding > *_import__('pkg_resources').declare_namespace(__name__) > *in the __init__.py of the namespace package. > > If I run pylint in another environment without any packages installed, it > works like a charm (except I have a bunch of violations for missing > imports). > > Do you want me to open an issue on bitbucket explaining all that or do you > think I am doing something I am not supposed to do ? I think what you're doing is right. The crash has already been supported: https://bitbucket.org/logilab/pylint/issue/203/importing-namespace-packages-crashes It seems to me that the namespace package issue is related to recent version of setuptools and has also been reported but can't find it right away. -- Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42) Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services CubicWeb, the semantic web framework: http://www.cubicweb.org From vanderkelen.francois at gmail.com Fri May 16 15:57:02 2014 From: vanderkelen.francois at gmail.com (=?UTF-8?Q?Fran=C3=A7ois_Vanderkelen?=) Date: Fri, 16 May 2014 15:57:02 +0200 Subject: [code-quality] Namespaced packages and relative imports In-Reply-To: <20140516130154.GV2881@logilab.fr> References: <20140516120249.GR2881@logilab.fr> <20140516130154.GV2881@logilab.fr> Message-ID: I tried multiple things, first even if I install the package before running pylint it won't work except on one of my machines. So I tried to narrow it down with setuptools : Working with 0.9.8 Not working with 2.2 (ImportError) Not working with 3.6 (ImportError) I guess setuptools changed something concerning namespaces packages, bug or not this is another story. I'll keep looking. Thanks a lot Sylvain 2014-05-16 15:01 GMT+02:00 Sylvain Th?nault : > On 16 mai 14:42, Fran?ois Vanderkelen wrote: > > Hi Sylvain, > > > > First of all thanks for the quick answer. I tried to create a dummy > project > > (the real one is confidential and not mine to give to the community) and > it > > helped me narrow down my two issues. > > > > The ImportError: No module named project is linked to the relative > import, > > and I could not reproduce the error in a dummy project. I finally found > it, > > it has to do with my namespaces packages. > > > > What I mean by namespaces is in my python library I dispose of several > > packages like this : > > > > company.mongodb > > company.redis > > company.hbase > > > > And also : > > > > company.project.core > > company.project.app.dashboard > > company.project.app.security > > > > Where company and project are all the same things, this allows me and my > > team to easily identify each package and what it is supposed to do. We > try > > to foster the open-source spirit in my company, hence the company > top-level > > namespace. It is still internal but at least we share it across all > teams :) > > > > Going back to the issue, if one package of the namespace is installed, > then > > apparently pylint is looking directly inside the python path and can't > find > > my package since it is not yet installed. I have the same issue with the > > "python setup.py develop" command, so I guess I am doing namespacing > wrong, > > or it is not well defined in python ? > > > > The only thing I do is adding > > *_import__('pkg_resources').declare_namespace(__name__) > > *in the __init__.py of the namespace package. > > > > If I run pylint in another environment without any packages installed, it > > works like a charm (except I have a bunch of violations for missing > > imports). > > > > Do you want me to open an issue on bitbucket explaining all that or do > you > > think I am doing something I am not supposed to do ? > > I think what you're doing is right. > > The crash has already been supported: > > https://bitbucket.org/logilab/pylint/issue/203/importing-namespace-packages-crashes > > It seems to me that the namespace package issue is related to recent > version of > setuptools and has also been reported but can't find it right away. > > -- > Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse > (05.62.17.16.42) > Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations > D?veloppement logiciel sur mesure: http://www.logilab.fr/services > CubicWeb, the semantic web framework: http://www.cubicweb.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From noisecapella at gmail.com Fri May 16 17:34:00 2014 From: noisecapella at gmail.com (George Schneeloch) Date: Fri, 16 May 2014 11:34:00 -0400 Subject: [code-quality] inference ordering Message-ID: I've been working on a pylint-brain plugin to do inference based on docstring type information. I'd like to set the order of type inference such that, if there is a type already found it uses that, else it uses what's in the docstring. Is there a way to do this? I'd also like to avoid clobbering other pylint-brain plugin results if possible. Is there a mechanism to set priority here? Currently the plugin will override all other inference if types are specified in the docstring, which may be undesirable. Thanks, -George -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylvain.thenault at logilab.fr Thu May 22 06:48:39 2014 From: sylvain.thenault at logilab.fr (Sylvain =?utf-8?B?VGjDqW5hdWx0?=) Date: Thu, 22 May 2014 06:48:39 +0200 Subject: [code-quality] inference ordering In-Reply-To: References: Message-ID: <20140522044839.GA2900@logilab.fr> Hi George, On 16 mai 11:34, George Schneeloch wrote: > I've been working on a pylint-brain plugin to do inference based on > docstring type information. I'd like to set the order of type inference > such that, if there is a type already found it uses that, else it uses > what's in the docstring. Is there a way to do this? I'd also like to avoid > clobbering other pylint-brain plugin results if possible. Is there a > mechanism to set priority here? Currently the plugin will override all > other inference if types are specified in the docstring, which may be > undesirable. Could you provide an example of what you would like to achieve and the problem you're encountering? I'm afraid there are no notion of "type inference priority" mecanism, but by better understanding of your problem one may have other idea. As a side note I would say that it sounds good to me that inference driven by annotation (python 3 or docstring based) would then only consider those types and not those from the "regular" inference. -- Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42) Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services CubicWeb, the semantic web framework: http://www.cubicweb.org From kris at cs.ucsb.edu Sat May 24 00:46:40 2014 From: kris at cs.ucsb.edu (kris kvilekval) Date: Fri, 23 May 2014 15:46:40 -0700 Subject: [code-quality] ImportError: No module named paste Message-ID: <1400885200.14019.64.camel@loup.ece.ucsb.edu> I am using pip to install all dependencies of new packages and am running into a few issues (see error at end of email). The following patch seems to remove the issue. Not sure why imports of names space packages had to be greater than one, but it seems like pip installs namespaces differently than easy_install did. $ hg diff diff --git a/modutils.py b/modutils.py --- a/modutils.py +++ b/modutils.py @@ -612,11 +612,14 @@ except AttributeError: checkeggs = False # pkg_resources support (aka setuptools namespace packages) - if pkg_resources is not None and modpath[0] in pkg_resources._namespace_packages and len(modpath) > 1: +# if pkg_resources is not None and modpath[0] in pkg_resources._namespace_packages and len(modpath) > 1: + if pkg_resources is not None and modpath[0] in pkg_resources._namespace_packages:# # setuptools has added into sys.modules a module object with proper # __path__, get back information from there module = sys.modules[modpath.pop(0)] path = module.__path__ + mtype = PKG_DIRECTORY + mp_filename = path[0] imported = [] while modpath: modname = modpath[0] ======================================================================= $ paver pylint bqserver/bq/module_service/controllers/module_server.py ---> pavement.pylint PYTHONPATH=bqcore/bq:bqserver/bq:bqengine/bq:bqfeature/bq pylint bqserver/bq/module_service/controllers/module_server.py --rcfile=bqcore/pylint.rc ************* Module bq.module_service.controllers.module_server I0011:330,0: Locally disabling dangerous-default-value (W0102) Traceback (most recent call last): File "/home/kgk/work/bisque/bq055/bqenv/bin/pylint", line 9, in load_entry_point('pylint==1.2.1', 'console_scripts', 'pylint')() File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/__init__.py", line 21, in run_pylint Run(sys.argv[1:]) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/lint.py", line 1051, in __init__ linter.check(args) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/lint.py", line 626, in check self.check_astroid_module(astroid, walker, rawcheckers, tokencheckers) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/lint.py", line 712, in check_astroid_module walker.walk(astroid) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/utils.py", line 715, in walk self.walk(child) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/utils.py", line 712, in walk cb(astroid) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/checkers/imports.py", line 269, in visit_from self._add_imported_module(node, '%s.%s' % (importedmodnode.name, name)) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/pylint/checkers/imports.py", line 302, in _add_imported_module importedmodname = get_module_part(importedmodname) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 352, in get_module_part path=path, context_file=context_file) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 297, in file_from_modpath return _file_from_modpath(modpath, path, context) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 556, in _file_from_modpath mtype, mp_filename = _module_file(modpath, path) File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 636, in _module_file return _search_zip(modpath, pic)[:2] File "/home/kgk/work/bisque/bq055/bqenv/lib/python2.6/site-packages/logilab/common/modutils.py", line 577, in _search_zip raise ImportError('No module named %s' % '.'.join(modpath)) ImportError: No module named paste Captured Task Output: From noisecapella at gmail.com Sun May 25 18:35:28 2014 From: noisecapella at gmail.com (George Schneeloch) Date: Sun, 25 May 2014 12:35:28 -0400 Subject: [code-quality] inference ordering In-Reply-To: <20140522044839.GA2900@logilab.fr> References: <20140522044839.GA2900@logilab.fr> Message-ID: This is my code in progress: https://github.com/bostonbusmap/docstring-inference/blob/master/docstring_inference.py Is there any faculty for type mismatches in astroid currently? I am looking to infer a return value from a function the default way, and the return value from a function's docstring, and show a warning if the two types don't match up. -George On Thu, May 22, 2014 at 12:48 AM, Sylvain Th?nault < sylvain.thenault at logilab.fr> wrote: > Hi George, > > On 16 mai 11:34, George Schneeloch wrote: > > I've been working on a pylint-brain plugin to do inference based on > > docstring type information. I'd like to set the order of type inference > > such that, if there is a type already found it uses that, else it uses > > what's in the docstring. Is there a way to do this? I'd also like to > avoid > > clobbering other pylint-brain plugin results if possible. Is there a > > mechanism to set priority here? Currently the plugin will override all > > other inference if types are specified in the docstring, which may be > > undesirable. > > Could you provide an example of what you would like to achieve and the > problem > you're encountering? > > I'm afraid there are no notion of "type inference priority" mecanism, but > by > better understanding of your problem one may have other idea. As a side > note I > would say that it sounds good to me that inference driven by annotation > (python > 3 or docstring based) would then only consider those types and not those > from > the "regular" inference. > -- > Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse > (05.62.17.16.42) > Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations > D?veloppement logiciel sur mesure: http://www.logilab.fr/services > CubicWeb, the semantic web framework: http://www.cubicweb.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikael.pahmp at gmail.com Mon May 26 16:08:04 2014 From: mikael.pahmp at gmail.com (Mikael Pahmp) Date: Mon, 26 May 2014 16:08:04 +0200 Subject: [code-quality] Problem with pylint, python3.4, multiprocessing Message-ID: Hi, I'm having a problem using pylint on Ubuntu 14.04 in a virtualenv based on python3.4. Here are steps to reproduce it: - virtualenv --python=python3.4 VENV - source VENV/bin/activate - pip install pylint==1.2 - echo "import multiprocessing" > test.py - echo "print(multiprocessing.cpu_count())" >> test.py - pylint test.py The problem is that pylint reports: E: 2,22: Module 'multiprocessing' has no 'cpu_count' member (no-member) but python has no problem executing the file. There is no error when run with python3.3. The multiprocessing package has changed in python3.4 regarding how it exposes its methods which might be troublesome for pylint. Could you please confirm there is a problem or if I'm doing something wrong. Regards, Mikael -------------- next part -------------- An HTML attachment was scrubbed... URL: From forspareparts at gmail.com Tue May 27 22:32:06 2014 From: forspareparts at gmail.com (David Jackson) Date: Tue, 27 May 2014 15:32:06 -0500 Subject: [code-quality] Module naming conventions in astroid, and inclusion of __init__.py Message-ID: Hi, guys, I'm trying to build a code-intelligence tool for myself on top of astroid, and I'm confused about the way that astroid decides how to name modules, and when to include __init__ files in the project. I've been creating my astroid projects like this: AstroidManager().project_from_files([project_path]) where project_path is an absolute filesystem path to the root of my project. In some cases, the module names in the resulting project are the absolute path to the file. In other cases, they're the 'dot-notation' path with my project name prepended (e.g. 'my_project.some_root_level_module'), and in still others, they're the dot-notation path without the project name (e.g. 'some_root_level_module'). I've also noticed that only some of the __init__.py files in my project get added as modules. As far as I can tell, this has something to do with whether or not the project source is contained within the current Python path. Even then, I'm still getting inconsistent behavior. Could someone explain to me how the names are supposed to work, and when __init__ files are registered with the project? Thanks for any help you can offer, David Jackson -------------- next part -------------- An HTML attachment was scrubbed... URL: From sylvain.thenault at logilab.fr Wed May 28 06:33:49 2014 From: sylvain.thenault at logilab.fr (Sylvain =?utf-8?B?VGjDqW5hdWx0?=) Date: Wed, 28 May 2014 06:33:49 +0200 Subject: [code-quality] Module naming conventions in astroid, and inclusion of __init__.py In-Reply-To: References: Message-ID: <20140528043349.GA2915@logilab.fr> On 27 mai 15:32, David Jackson wrote: > Hi, guys, Hi, > I'm trying to build a code-intelligence tool for myself on top of astroid, > and I'm confused about the way that astroid decides how to name modules, > and when to include __init__ files in the project. I've been creating my > astroid projects like this: > > AstroidManager().project_from_files([project_path]) > > where project_path is an absolute filesystem path to the root of my project. > > In some cases, the module names in the resulting project are the absolute > path to the file. In other cases, they're the 'dot-notation' path with my > project name prepended (e.g. 'my_project.some_root_level_module'), and in > still others, they're the dot-notation path without the project name (e.g. > 'some_root_level_module'). I've also noticed that only some of the > __init__.py files in my project get added as modules. > > As far as I can tell, this has something to do with whether or not the > project source is contained within the current Python path. Even then, I'm > still getting inconsistent behavior. Could someone explain to me how the > names are supposed to work, and when __init__ files are registered with the > project? Huuum, astroid's Project class is only used by pyreverse and as such is not the more active part of astroid... Pylint does such job by itself, take a look at the pylint.utils.expand_modules function. If there is some interest for external use, we could backport it to astroid, and even use it for Project. -- Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (05.62.17.16.42) Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services CubicWeb, the semantic web framework: http://www.cubicweb.org From forspareparts at gmail.com Wed May 28 23:53:15 2014 From: forspareparts at gmail.com (David Jackson) Date: Wed, 28 May 2014 16:53:15 -0500 Subject: [code-quality] Module naming conventions in astroid, and inclusion of __init__.py In-Reply-To: <20140528043349.GA2915@logilab.fr> References: <20140528043349.GA2915@logilab.fr> Message-ID: Ah, thanks! That makes more sense. I have no problem pulling pylint into my project and using expand_modules to build out my tree of modules/packages, but it does seem like a useful thing to have in astroid. Wherever that specific function belongs, it'd be nice to have a little documentation for astroid to help guide newbies like myself to the right functionality. Tangentially, has anybody looked at my ticket about line info for the end of list/bracket elements ( https://bitbucket.org/logilab/astroid/issue/31/astroid-sometimes-reports-the-wrong)? If it matters to anybody else, I'd be happy to take a stab at fixing it -- if not, I might handle it on my end in my own project. On Tue, May 27, 2014 at 11:33 PM, Sylvain Th?nault < sylvain.thenault at logilab.fr> wrote: > On 27 mai 15:32, David Jackson wrote: > > Hi, guys, > > Hi, > > > I'm trying to build a code-intelligence tool for myself on top of > astroid, > > and I'm confused about the way that astroid decides how to name modules, > > and when to include __init__ files in the project. I've been creating my > > astroid projects like this: > > > > AstroidManager().project_from_files([project_path]) > > > > where project_path is an absolute filesystem path to the root of my > project. > > > > In some cases, the module names in the resulting project are the absolute > > path to the file. In other cases, they're the 'dot-notation' path with my > > project name prepended (e.g. 'my_project.some_root_level_module'), and in > > still others, they're the dot-notation path without the project name > (e.g. > > 'some_root_level_module'). I've also noticed that only some of the > > __init__.py files in my project get added as modules. > > > > As far as I can tell, this has something to do with whether or not the > > project source is contained within the current Python path. Even then, > I'm > > still getting inconsistent behavior. Could someone explain to me how the > > names are supposed to work, and when __init__ files are registered with > the > > project? > > Huuum, astroid's Project class is only used by pyreverse and as such is > not the > more active part of astroid... Pylint does such job by itself, take a look > at > the pylint.utils.expand_modules function. If there is some interest for > external > use, we could backport it to astroid, and even use it for Project. > > -- > Sylvain Th?nault, LOGILAB, Paris (01.45.32.03.12) - Toulouse > (05.62.17.16.42) > Formations Python, Debian, M?th. Agiles: http://www.logilab.fr/formations > D?veloppement logiciel sur mesure: http://www.logilab.fr/services > CubicWeb, the semantic web framework: http://www.cubicweb.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: