Python 2.7, on windows7 64 bit development machine, inconsistent issue on similar machines

[Jacob Kruger]

Ok, and just tested MySQLdb connection to both XAMPP server instance on same 
machine, as well as slightly remote connection to other machine over wifi, 
and same error - so, seems issue is invoked/caused by MySQLdb connection 
closing - if just put process to sleep for 30 seconds, nothing happens, but, 
the moment I in fact try to then just close the connection, that's when 
error dialogue immediately pops up.

Stay well

Jacob Kruger
Blind Biker
Skype: BlindZA
"Roger Wilco wants to welcome you...to the space janitor's closet..."

----- Original Message ----- 
From: "Jacob Kruger" <jacob at blindza.co.za>
To: "Dennis Lee Bieber" <wlfraed at ix.netcom.com>
Cc: <python-list at python.org>
Sent: Tuesday, January 13, 2015 8:37 PM
Subject: Re: Python 2.7, on windows7 64 bit development machine, 
inconsistent issue on similar machines


> See answers below.
>
> Jacob Kruger
> Blind Biker
> Skype: BlindZA
> "Roger Wilco wants to welcome you...to the space janitor's closet..."
>
> ----- Original Message ----- 
> From: "Dennis Lee Bieber" <wlfraed at ix.netcom.com>
> <snip />
>
>> The very first hit /I/ get is:
>> https://social.technet.microsoft.com/Forums/windows/en-US/3932e3eb-c034-4eb7-aa06-4a0a8e6ea493/fault-module-namestackhash0a9e
>>
>
> Played around with data expraction policy changes/settings now, and didn't 
> make it change behaviour.
>
>>>
> <snip />
>>>      s_host = str(d_mysql["host"])
>>>      s_user = str(d_mysql["user"])
>>>      s_pass = str(d_mysql["password"])
>>>      s_db = self.s_target_value
>>
>> ONE: Have you confirmed all of these are coming in correctly
>>
> Yes - that mysql.set pickle dump is just a way of storing some string 
> values relating to mysql server connection to sort of hide it from an 
> end-user, but, yes, the values work fine in that sense.
>
>>>      cn = MySQLdb.connect(s_host, s_user, s_pass, s_db)
>>>      if cn:
>>>        cn.autocommit = True
>>
>> TWO-A: you've turned on autocommit here, yet...
>
> Sorry - that was sort of left-over from when was trying to make sure that 
> other parts were happening/working - along with trying out specific commit 
> call lower down - have tried taking eitehr of them out one by one, to see 
> if made a difference, but, no-go.
>
>>
>>>        cur = cn.cursor()
>>>        i_struct = 0
>>>        i_data = 0
>>>        f = open(self.s_sql_struct_file, "rb")
>>>        s = f.read()
>>>        f.close()
>>>        l_struct = s.split(";")
>>
>> THREE-A: here you split some string on semicolons, just to...
>>
>>>        f = open(self.s_sql_data_file, "rb")
>>>        s = f.read()
>>>        f.close()
>>>        l_data = unidecode(s).split(";")
>>>        l_struct_errors = []
>>>        l_data_errors = []
>>>        i_start_time = int(time.time())
>>>        s_status = "/{0} structs".format(str(len(l_struct)))
>>>        for I in range(len(l_struct)):
>>
>> Very UN-Pythonic...
>
> I know - that was also since originally used this code in a sort of GUI 
> version, and meant to use one by one counter to update status bar text at 
> sort of intervals of 10 records, to let guys track progress - but, FWIW, 
> it seemed to overload my screen reader with too many changes, partly since 
> some of these collections of insert statements are processing over 40000 
> record insertions - not currently with test data, but, some of the other 
> test database files I used were operating in that region of the number of 
> data record numbers. Have taken that part out again now.
>
>>
>> for a_struct in l_struct:
>> # do stuff with a_struct directly, no indexing into l_struct
>>
>>>          try:
>>>            i_struct = I
>>>            if str(l_struct[I]).strip() != "": res = 
>>> cur.execute(l_struct[I] + ";")
>>
>> THREE-B: ... reappend the semicolon here... which probably isn't needed
>> -- cur.execute() assumes, as I recall, that the provided argument is a
>> complete statement; unlike a command line interface where the semicolon 
>> is
>> needed to tell the CLI that the statement is complete and can be sent to
>> the DBMS for processing.
>>
>> And very confusing names... l_struct and i_struct, where the latter is
>> an index into the former (and I don't see it used anywhere).
>
> Have taken out counter now, and, yes, cursor seems happy to execute 
> statements without ; character at end.
>
>>
>>>            print(l_struct[I])
>>>          #except Warning as wrn:
>>>          #  print("warning: " + str(wrn.args))
>>>          except Exception as exc:
>>>            l_struct_errors.append([l_struct[I], copy.copy(exc)])
>>>          finally:
>>>            pass
>>>        cn.commit()
>>
>> TWO-B: ... here you attempt to force a commit.
>
> That was also during test phase when was just trying to make sure parts of 
> this were being executed/committed - taken out additional commits now.
>
>>
>>>        time.sleep(2.0)
>>>        sNada = raw_input("hit enter to continue with data")
>>>        s_status = "/{0} data".format(str(len(l_data)))
>>>        for I in range(len(l_data)):
>>>          try:
>>>            #self.SetStatusText(str(I) + s_status)
>>>            i_data = I
>>>            if str(l_data[I]).strip() != "": res = 
>>> cur.execute(l_data[I][l_data[I].index("INSERT"):] + ";")
>>>            print(l_data[I][l_data[I].index("INSERT"):] + ";")
>>>          except Exception as exc:
>>>            l_data_errors.append([l_data[I], copy.copy(exc)])
>>>          finally:
>>>            pass
>>>        i_end_time = int(time.time())
>>>        s_time_taken = sTimeDiffFormat(i_start_time, i_end_time)
>>>        cn.commit()
>>
>> TWO-C: ... and here too.
>>
>>>        cur.close()
>>>        cn.close()
>>>        print("cn and cur closed")
>>>        fPickle = open("testDataErrors.pickle", "wb")
>>>        pickle.dump(l_struct_errors, fPickle, 2)
>>>        pickle.dump(l_data_errors, fPickle, 2)
>>>        fPickle.close()
>>>        print("pickled")
>>>        print("MySQL Results - {0} structure queries, and {1} data 
>>> queries completed - total of {2}".format(str(i_struct), str(i_data), 
>>> s_time_taken))
>>>        print("MySQL Errors - {0} structure query errors, and {1} data 
>>> query errors".format(str(len(l_struct_errors)), 
>>> str(len(l_data_errors))))
>>>        print("Done!")
>>>        sys.exit()
>>>      else:
>>>        print("Connection issue - There was a problem connecting to MySQL 
>>> server")
>>>        sys.exit()
>>>    except Exception as exc:
>>>      s_exc = str(exc.args)
>>>      #lbc.DialogShow(title="errorMessage", message=s_exc)
>>>      print(s_exc)
>>>      exc_type, exc_obj, tb = sys.exc_info()
>>>      print(str(exc_obj))
>>>      print("line number: " + str(tb.tb_lineno))
>>>  #end of convertExport function
>>>
>>>#---end code---
>>>
>>
>> What is this affair with pickling everything? Mistakes in a pickle
>> could lead to anything happening <G>
>
> More to do with tracking errors later on, since person executing this app, 
> on site, might not be too technical, and idea would be that, aside from 
> sql statements, they might be able to just copy .pickle files onto flash 
> drive to bring back for later review, and it was also my form of 
> post-execution debugging to let me try figure out what was causing issues 
> with regards to actual statements, etc.
>
>>
>> I'm not even going to try to figure out what your attempted SQL
>> operations turn into. Only that, from what I can see, there is no attempt
>> at preventing SQL injection attacks -- other than the use of pickled data
>> (and I still wouldn't trust that the pickle hasn't been tampered with).
>>
>
> I am pulling data out of MS access .mdb files, using pyodbc, and then 
> generating the relevant text files, with all the SQL statements inside 
> them, and one of the command line options indicates if the user wants to 
> just generate the SQL statement files for copying, or wants to directly 
> execute these statements against an instance of a mySQL database, so, yes, 
> in theory, the worries of sql injection shouldn't be too much of an issue, 
> and when generating the sql statements higher up, before writing them into 
> these text files am reading in here, I do things like remove any ; 
> characters, from inside actual data values, etc. etc. to work with forms 
> of sql injection, but, this would also, in theory not be interacting with 
> anything other than sort of interim instances of mysql databases that we 
> then want to process the transformation of using some other code, etc. - 
> but later's problem.
>
>>>
>>>
>>>If this is operating system specific, sorry, and will maybe also try 
>>>debugging code via VS.Net 2013 shortly, but, thought someone might have 
>>>an off-hand thought regarding something could try use to figure out this 
>>>issue in meantime...?
>>>
>> Yes, it is OS specific.
>>
>> Either the accounts have different privileges of the different machines
>> (presuming a DEP problem, and something you are doing is executing data),
>> or http://winwiki.org/stackhash_0a9e-fix/ indicates potentially corrupted
>> system files.
>
> Will have a look at that one, and another thing found mention of with 
> regards to someone's wxPython code was sort of inserting different 
> controls, with same names onto wx.Frame parents, which they were guessing 
> might have had something to with application garbage collection at end of 
> execution, and could thus also be causing issues, but, am pretty sure 
> haven't done something like that here - might also try taking out taking 
> copies of parts of dictionary objects, since that was also more to do with 
> testing, and not wanting to mutilate/manipulate original values during 
> test process, but anyway.
>
> In terms of user privileges, both of these machines, I just run as 
> administrator, and the machine that's not currently giving me these 
> hassles is windows7 pro 64 bit, where machine with issues is windows7 
> ultimate 64 bit, but, it's my test/play-time machine, so, yes, I do 
> install/uninstall a lot more of things on/off it at times, but, I also 
> reinstalled operating system from scratch roundabout a month ago, so 
> wouldn't have thought it could have too many issues as of yet, but anyway.
>
> Did also try figuring out debug process using MS visual studio.net 2013, 
> community edition, but, just got pretty much same info/error message, and 
> pdb is not something have ever really worked with, and not sure it would 
> do too much more than what I'm already working with.
>
> Other thing tried doing now was just taking out chunks of functionality in 
> that function/method, to try see if could track down cause of issue like 
> that, and, actually seems like trigger is even if I just connect to mysql 
> server/database, since if I then just wait 1 second, and close the 
> connection, and then just try invoke sys.exit(), same thing happens, 
> whereas if I comment out db connection, then it just exits happily.
>
> Will have to look for more specific examples of something like MySQLdb 
> connections that might cause something like this.
>
> Might also try running it against a different version/instance of MySQL 
> server since have both WAMP and XAMPP on this machine, but, make sure they 
> only run one at a time, or might fire up both machines, turn on public 
> access to 'happy' machine, and execute code from this machine against it's 
> MySQL server instance, and see if that affects it as well.
>
>> -- 
>> Wulfraed                 Dennis Lee Bieber         AF6VN
>
> Thanks
>
>>    wlfraed at ix.netcom.com    HTTP://wlfraed.home.netcom.com/
>>
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>