[IronPython] Problem with 'obsoleted' functions and warnings.

[Dino Viehland]

I'd say this is technically a bug in IronPython - but one which people would rather have us not fix (at least naively) simply because they do like parallel import.

CPython holds a big lock while doing imports so this problem doesn't exist there - but CPython's importing is much faster so parallelization is less necessary.  We could add a per-module lock which is different from CPython but still allows parallel imports.  Something where importing a module marks some internal state in a PythonModule object and when we import a module we check for that and block for completion.  Of course that could lead to deadlocks in the face of circular imports performed in parallel :(  So we could then add some sort of deadlock aware lock and break the deadlock by allowing one module to continue and see uninitialized state - intuitively I think that'll be the same observable behavior as the circular import case in CPython but I'm not a 100% certain.   Now someone could try to implement this and contribute it if they were interested :)

An alternate workaround to removing warnings.py might be to import warnings on startup (so it's initialized before you hit this issue).

From: users-bounces at lists.ironpython.com [mailto:users-bounces at lists.ironpython.com] On Behalf Of Leighton Haynes
Sent: Tuesday, November 02, 2010 7:46 PM
To: users at lists.ironpython.com
Subject: [IronPython] Problem with 'obsoleted' functions and warnings.

Hi Guys,
  We have a problem that seems to be related to a combination of multi-threaded code, the use of the python standard library and some obsoleted functions in one of our libraries. We have a C# library, call it DeprecatedLib, which has some function marked as [Obsolete] (this should result in warnings at runtime). We have a threaded section of code (running in IronPython) which can result in half a dozen of these method calls happening at the same time. We have the python standard libraries available to our IronPython interpreter. What seems to happen is that IronPython hits the Obsolete attribute, decides it needs to warn, imports warnings.py from the standard library (on 2 threads at the same time) and then it fails with a MissingMemberException of "'module' object has no attribute XXX" where XXX is either '_getframe' or 'warn'. I'm using IronPython 2.6.1 (.net 2 version) for my testing. It fails about 90% of the time in my test app (and much closer to 100% in our real app).

The section of IronPython where it imports warnings.py has a fallback method for doing the warning which it uses if it can't find a 'warnings' module. This appears to work fine. It only fails when the standard library is available and it tries to use the CPython warnings.py.

What I'm trying to work out is: is this a problem with IronPython or something with the standard warnings.py? I can 'fix' the error just by removing warnings.py from our standard library, but I'm a bit nervous that this might result in me hitting a similar problem in the future when something else triggers parallel imports (I thought this supposed to work in 2.6.1?).

I'd appreciate any guidance on this.

Regards,
Leighton Haynes...

Details of my test:

My sample python file (tester.py):
import System
import clr
clr.AddReference("DeprecatedLib")
import DeprecatedLib

class Tester():
  def __init__(self):
    pass
  def go(self):
    for i in range(0,100):
      System.Threading.ThreadPool.QueueUserWorkItem(self.doError, None)
  def doError(self, object):
    d = DeprecatedLib.Deprecated()
    d.DoSomething()

t = Tester()
t.go()

DeprecatedLib is a simple C# dll with a single class:
using System;

namespace DeprecatedLib
{
    public class Deprecated
    {
        [Obsolete]
        public void DoSomething()
        {

        }
    }
}

I'm running it using the ipy.exe command line:
IronPython 2.6.1 (2.6.10920.0) on .NET 2.0.50727.4952
Type "help", "copyright", "credits" or "license" for more information.
>>> import tester
>>> warning: DeprecationWarning: Deprecated.DoSomething has been obsoleted.
...
<crash>

And a full stack trace of the error:
System.MissingMemberException: 'module' object has no attribute '_getframe'
   at Microsoft.Scripting.Interpreter.ThrowInstruction.Run(InterpretedFrame frame)
   at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
   at Microsoft.Scripting.Interpreter.LightLambda.Run4[T0,T1,T2,T3,TRet](T0 arg0, T1 arg1, T2 arg2, T3 arg3)
   at IronPython.Compiler.PythonCallTargets.OriginalCallTarget3(PythonFunction function, Object arg0, Object arg1, Object arg2)
   at CallSite.Target(Closure , CallSite , CodeContext , Object , Object[] )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2)
   at IronPython.Runtime.PythonContext.CallSplat(Object func, Object[] args)
   at IronPython.Runtime.Operations.PythonOps.Warn(CodeContext context, PythonType category, String message, Object[] args)
   at IronPython.Runtime.Binding.WarningInfo.<>c__DisplayClass3.<AddWarning>b__1(Object[] callArgs, Boolean& shouldOptimize)
   at IronPython.Runtime.Types.BuiltinFunction.BuiltinMethodCaller`1.Call0(CallSite site, CodeContext context, TFuncType func)
   at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)
   at CallSite.Target(Closure , CallSite , CodeContext , Object )
   at IronPython.Compiler.Ast.CallExpression.Invoke0Instruction.Run(InterpretedFrame frame)
   at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
   at Microsoft.Scripting.Interpreter.LightLambda.Run3[T0,T1,T2,TRet](T0 arg0, T1 arg1, T2 arg2)
   at IronPython.Runtime.PythonFunction.FunctionCaller`2.Call2(CallSite site, CodeContext context, Object func, T0 arg0, T1 arg1)
   at CallSite.Target(Closure , CallSite , Object , Object )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)
   at _Scripting_(Object[] , Object )
   at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
   at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)

If anyone's especially keen, I can provide this as a zip file.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ironpython-users/attachments/20101103/e92309ed/attachment.html>