SetNoSiteFlag is ignored on Windows unless another PythonEngine call precedes it #1517
Description
Environment
- Pythonnet version: Self-built from commit ec65efe 8/11/2021
- Python version: 3.8.10
- Operating System: Windows 10 Pro 20H2 19042.1110
- .NET Runtime: .NET Core 3.1
Details
-
Describe what you were trying to get done.
I am trying to disable automatic site run in order to setup a custom virtualenv path.
-
What commands did you run to trigger this issue? If you can provide a
Minimal, Complete, and Verifiable example
this will help us understand the issue.
using System;
using System.Collections.Generic;
using Python.Runtime;
namespace TestPythonnet
{
class Program
{
static void Main(string[] args)
{
// my base environment
// also added this to top of PATH in project settings
var pathToBaseEnv = @"C:\Users\myuser\AppData\Local\Programs\Python\Python38";
Environment.SetEnvironmentVariable("PYTHONNET_PYVER", "3.8", EnvironmentVariableTarget.Process);
// this call has no effect
PythonEngine.SetNoSiteFlag();
// if you swap this call with previous line, it will work
PythonEngine.PythonHome = pathToBaseEnv;
PythonEngine.Initialize();
using (Py.GIL())
{
Console.WriteLine($"sys.executable: {sys.executable}");
Console.WriteLine($"sys.prefix: {sys.prefix}");
Console.WriteLine($"sys.base_prefix: {sys.base_prefix}");
Console.WriteLine($"sys.exec_prefix: {sys.exec_prefix}");
Console.WriteLine($"sys.base_exec_prefix: {sys.base_exec_prefix}");
Console.WriteLine("sys.path:");
foreach (var p in sys.path)
{
Console.WriteLine(p);
}
Console.WriteLine();
}
PythonEngine.Shutdown();
}
}
}
Result:
sys.executable: C:\Users\myuser\source\repos\TestPythonnet\TestPythonnet\bin\Debug\netcoreapp3.1\TestPythonnet.exe
sys.prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.base_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.exec_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.base_exec_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.path:
C:\Users\myuser\OneDrive\Documents\Python Scripts
C:\Users\myuser\AppData\Local\Programs\Python\Python38\python38.zip
C:\Users\myuser\AppData\Local\Programs\Python\Python38\DLLs
C:\Users\myuser\AppData\Local\Programs\Python\Python38\lib
C:\Users\myuser\source\repos\TestPythonnet\TestPythonnet\bin\Debug\netcoreapp3.1
C:\Users\myuser\AppData\Local\Programs\Python\Python38
C:\Users\myuser\AppData\Local\Programs\Python\Python38\lib\site-packages
C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.18\
The site packages are still added.
If you swap order of PythonEngine.SetNoSiteFlag(); and PythonEngine.PythonHome = pathToBaseEnv;, it works
sys.executable: C:\Users\myuser\source\repos\TestPythonnet\TestPythonnet\bin\Debug\netcoreapp3.1\TestPythonnet.exe
sys.prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.base_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.exec_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.base_exec_prefix: C:\Users\myuser\AppData\Local\Programs\Python\Python38
sys.path:
C:\Users\myuser\OneDrive\Documents\Python Scripts
C:\Users\myuser\AppData\Local\Programs\Python\Python38\python38.zip
C:\Users\myuser\AppData\Local\Programs\Python\Python38\DLLs
C:\Users\myuser\AppData\Local\Programs\Python\Python38\lib
C:\Users\myuser\source\repos\TestPythonnet\TestPythonnet\bin\Debug\netcoreapp3.1
C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.18\
No site folders added to the path.
In fact, one can workaround the issue by adding any access to the PythonEngine API which uses the DLL before setting the flag, e.g.
var version = PythonEngine.Version;
PythonEngine.SetNoSiteFlag();
The root cause appears to be that the reference count on the DLL is sent to zero after setting Py_NoSiteFlag, so the DLL is unloaded and reloaded by the Delegate class with the original value of Py_NoSiteFlag=0.
Snip from Python.Runtime.Runtime (runtime.cs) with my added comments.
internal static void SetNoSiteFlag()
{
var loader = LibraryLoader.Instance;
IntPtr dllLocal = IntPtr.Zero;
if (_PythonDll != "__Internal")
{
dllLocal = loader.Load(_PythonDll);
if (dllLocal == IntPtr.Zero)
{
throw new Exception($"Cannot load {_PythonDll}");
}
}
try
{
Py_NoSiteFlag = loader.GetFunction(dllLocal, "Py_NoSiteFlag");
Marshal.WriteInt32(Py_NoSiteFlag, 1); // This works, I can Marshal.ReadInt32 with Immediate window in Debug and get 1
}
finally
{
if (dllLocal != IntPtr.Zero)
{
loader.Free(dllLocal); // if nobody else loaded _PythonDll, it will potentially get unloaded here :-(
}
}
}
- If there was a crash, please include the traceback here.
N/A