Python 2.7, on windows7 64 bit development machine, inconsistent issue on similar machines

[Jacob Kruger]

See answers below.

Jacob Kruger
Blind Biker
Skype: BlindZA
"Roger Wilco wants to welcome you...to the space janitor's closet..."

----- Original Message ----- 
From: "Dennis Lee Bieber" <wlfraed at ix.netcom.com>
<snip />

> The very first hit /I/ get is:
> https://social.technet.microsoft.com/Forums/windows/en-US/3932e3eb-c034-4eb7-aa06-4a0a8e6ea493/fault-module-namestackhash0a9e
>

Played around with data expraction policy changes/settings now, and didn't 
make it change behaviour.

>>
<snip />
>>      s_host = str(d_mysql["host"])
>>      s_user = str(d_mysql["user"])
>>      s_pass = str(d_mysql["password"])
>>      s_db = self.s_target_value
>
> ONE: Have you confirmed all of these are coming in correctly
>
Yes - that mysql.set pickle dump is just a way of storing some string values 
relating to mysql server connection to sort of hide it from an end-user, 
but, yes, the values work fine in that sense.

>>      cn = MySQLdb.connect(s_host, s_user, s_pass, s_db)
>>      if cn:
>>        cn.autocommit = True
>
> TWO-A: you've turned on autocommit here, yet...

Sorry - that was sort of left-over from when was trying to make sure that 
other parts were happening/working - along with trying out specific commit 
call lower down - have tried taking eitehr of them out one by one, to see if 
made a difference, but, no-go.

>
>>        cur = cn.cursor()
>>        i_struct = 0
>>        i_data = 0
>>        f = open(self.s_sql_struct_file, "rb")
>>        s = f.read()
>>        f.close()
>>        l_struct = s.split(";")
>
> THREE-A: here you split some string on semicolons, just to...
>
>>        f = open(self.s_sql_data_file, "rb")
>>        s = f.read()
>>        f.close()
>>        l_data = unidecode(s).split(";")
>>        l_struct_errors = []
>>        l_data_errors = []
>>        i_start_time = int(time.time())
>>        s_status = "/{0} structs".format(str(len(l_struct)))
>>        for I in range(len(l_struct)):
>
> Very UN-Pythonic...

I know - that was also since originally used this code in a sort of GUI 
version, and meant to use one by one counter to update status bar text at 
sort of intervals of 10 records, to let guys track progress - but, FWIW, it 
seemed to overload my screen reader with too many changes, partly since some 
of these collections of insert statements are processing over 40000 record 
insertions - not currently with test data, but, some of the other test 
database files I used were operating in that region of the number of data 
record numbers. Have taken that part out again now.

>
> for a_struct in l_struct:
> # do stuff with a_struct directly, no indexing into l_struct
>
>>          try:
>>            i_struct = I
>>            if str(l_struct[I]).strip() != "": res = 
>> cur.execute(l_struct[I] + ";")
>
> THREE-B: ... reappend the semicolon here... which probably isn't needed
> -- cur.execute() assumes, as I recall, that the provided argument is a
> complete statement; unlike a command line interface where the semicolon is
> needed to tell the CLI that the statement is complete and can be sent to
> the DBMS for processing.
>
> And very confusing names... l_struct and i_struct, where the latter is
> an index into the former (and I don't see it used anywhere).

Have taken out counter now, and, yes, cursor seems happy to execute 
statements without ; character at end.

>
>>            print(l_struct[I])
>>          #except Warning as wrn:
>>          #  print("warning: " + str(wrn.args))
>>          except Exception as exc:
>>            l_struct_errors.append([l_struct[I], copy.copy(exc)])
>>          finally:
>>            pass
>>        cn.commit()
>
> TWO-B: ... here you attempt to force a commit.

That was also during test phase when was just trying to make sure parts of 
this were being executed/committed - taken out additional commits now.

>
>>        time.sleep(2.0)
>>        sNada = raw_input("hit enter to continue with data")
>>        s_status = "/{0} data".format(str(len(l_data)))
>>        for I in range(len(l_data)):
>>          try:
>>            #self.SetStatusText(str(I) + s_status)
>>            i_data = I
>>            if str(l_data[I]).strip() != "": res = 
>> cur.execute(l_data[I][l_data[I].index("INSERT"):] + ";")
>>            print(l_data[I][l_data[I].index("INSERT"):] + ";")
>>          except Exception as exc:
>>            l_data_errors.append([l_data[I], copy.copy(exc)])
>>          finally:
>>            pass
>>        i_end_time = int(time.time())
>>        s_time_taken = sTimeDiffFormat(i_start_time, i_end_time)
>>        cn.commit()
>
> TWO-C: ... and here too.
>
>>        cur.close()
>>        cn.close()
>>        print("cn and cur closed")
>>        fPickle = open("testDataErrors.pickle", "wb")
>>        pickle.dump(l_struct_errors, fPickle, 2)
>>        pickle.dump(l_data_errors, fPickle, 2)
>>        fPickle.close()
>>        print("pickled")
>>        print("MySQL Results - {0} structure queries, and {1} data queries 
>> completed - total of {2}".format(str(i_struct), str(i_data), 
>> s_time_taken))
>>        print("MySQL Errors - {0} structure query errors, and {1} data 
>> query errors".format(str(len(l_struct_errors)), str(len(l_data_errors))))
>>        print("Done!")
>>        sys.exit()
>>      else:
>>        print("Connection issue - There was a problem connecting to MySQL 
>> server")
>>        sys.exit()
>>    except Exception as exc:
>>      s_exc = str(exc.args)
>>      #lbc.DialogShow(title="errorMessage", message=s_exc)
>>      print(s_exc)
>>      exc_type, exc_obj, tb = sys.exc_info()
>>      print(str(exc_obj))
>>      print("line number: " + str(tb.tb_lineno))
>>  #end of convertExport function
>>
>>#---end code---
>>
>
> What is this affair with pickling everything? Mistakes in a pickle
> could lead to anything happening <G>

More to do with tracking errors later on, since person executing this app, 
on site, might not be too technical, and idea would be that, aside from sql 
statements, they might be able to just copy .pickle files onto flash drive 
to bring back for later review, and it was also my form of post-execution 
debugging to let me try figure out what was causing issues with regards to 
actual statements, etc.

>
> I'm not even going to try to figure out what your attempted SQL
> operations turn into. Only that, from what I can see, there is no attempt
> at preventing SQL injection attacks -- other than the use of pickled data
> (and I still wouldn't trust that the pickle hasn't been tampered with).
>

I am pulling data out of MS access .mdb files, using pyodbc, and then 
generating the relevant text files, with all the SQL statements inside them, 
and one of the command line options indicates if the user wants to just 
generate the SQL statement files for copying, or wants to directly execute 
these statements against an instance of a mySQL database, so, yes, in 
theory, the worries of sql injection shouldn't be too much of an issue, and 
when generating the sql statements higher up, before writing them into these 
text files am reading in here, I do things like remove any ; characters, 
from inside actual data values, etc. etc. to work with forms of sql 
injection, but, this would also, in theory not be interacting with anything 
other than sort of interim instances of mysql databases that we then want to 
process the transformation of using some other code, etc. - but later's 
problem.

>>
>>
>>If this is operating system specific, sorry, and will maybe also try 
>>debugging code via VS.Net 2013 shortly, but, thought someone might have an 
>>off-hand thought regarding something could try use to figure out this 
>>issue in meantime...?
>>
> Yes, it is OS specific.
>
> Either the accounts have different privileges of the different machines
> (presuming a DEP problem, and something you are doing is executing data),
> or http://winwiki.org/stackhash_0a9e-fix/ indicates potentially corrupted
> system files.

Will have a look at that one, and another thing found mention of with 
regards to someone's wxPython code was sort of inserting different controls, 
with same names onto wx.Frame parents, which they were guessing might have 
had something to with application garbage collection at end of execution, 
and could thus also be causing issues, but, am pretty sure haven't done 
something like that here - might also try taking out taking copies of parts 
of dictionary objects, since that was also more to do with testing, and not 
wanting to mutilate/manipulate original values during test process, but 
anyway.

In terms of user privileges, both of these machines, I just run as 
administrator, and the machine that's not currently giving me these hassles 
is windows7 pro 64 bit, where machine with issues is windows7 ultimate 64 
bit, but, it's my test/play-time machine, so, yes, I do install/uninstall a 
lot more of things on/off it at times, but, I also reinstalled operating 
system from scratch roundabout a month ago, so wouldn't have thought it 
could have too many issues as of yet, but anyway.

Did also try figuring out debug process using MS visual studio.net 2013, 
community edition, but, just got pretty much same info/error message, and 
pdb is not something have ever really worked with, and not sure it would do 
too much more than what I'm already working with.

Other thing tried doing now was just taking out chunks of functionality in 
that function/method, to try see if could track down cause of issue like 
that, and, actually seems like trigger is even if I just connect to mysql 
server/database, since if I then just wait 1 second, and close the 
connection, and then just try invoke sys.exit(), same thing happens, whereas 
if I comment out db connection, then it just exits happily.

Will have to look for more specific examples of something like MySQLdb 
connections that might cause something like this.

Might also try running it against a different version/instance of MySQL 
server since have both WAMP and XAMPP on this machine, but, make sure they 
only run one at a time, or might fire up both machines, turn on public 
access to 'happy' machine, and execute code from this machine against it's 
MySQL server instance, and see if that affects it as well.

> -- 
> Wulfraed                 Dennis Lee Bieber         AF6VN

Thanks

>    wlfraed at ix.netcom.com    HTTP://wlfraed.home.netcom.com/
>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>