-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Provide detailed error for circular from
imports
#5972
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide detailed error for circular from
imports
#5972
Conversation
Warning Rate limit exceeded@ever0de has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 6 minutes and 36 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
WalkthroughThe Changes
Poem
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
9c09aef
to
dfb4db1
Compare
dfb4db1
to
d86644e
Compare
from
importsfrom
imports
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
📜 Review details
Configuration used: .coderabbit.yml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
Lib/test/test_import/__init__.py
is excluded by!Lib/**
📒 Files selected for processing (3)
vm/src/builtins/module.rs
(3 hunks)vm/src/frame.rs
(2 hunks)vm/src/import.rs
(2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs
Instructions used from:
Sources:
📄 CodeRabbit Inference Engine
- .github/copilot-instructions.md
🧠 Learnings (2)
vm/src/builtins/module.rs (1)
Learnt from: CR
PR: RustPython/RustPython#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-06-30T10:08:48.858Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
vm/src/import.rs (1)
Learnt from: CR
PR: RustPython/RustPython#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-06-30T10:08:48.858Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Run snippets and cpython tests (windows-latest)
🔇 Additional comments (4)
vm/src/builtins/module.rs (2)
1-1
: LGTM! Atomic boolean field properly added for initialization tracking.The
AtomicBool
import and field addition are appropriate for tracking module initialization state in a thread-safe manner.Also applies to: 57-57
80-80
: Field initialization is correct in both constructors.The
initializing
field is properly initialized tofalse
in bothnew()
andfrom_def()
constructors, ensuring modules start in a non-initializing state.Also applies to: 88-88
vm/src/import.rs (2)
3-3
: Imports correctly added for atomic operations.The
Ordering
import andPyModule
addition are necessary for the initialization tracking implementation.Also applies to: 7-7
161-178
: Well-implemented guard pattern for initialization tracking.The guard ensures the
initializing
flag is always reset, even if module code execution fails or panics. The use ofOrdering::Relaxed
is appropriate here.However, this sets an internal field that isn't exposed to Python code. The
frame.rs
code checks__spec__._initializing
, which is a different attribute. Consider exposing this state to Python.
vm/src/frame.rs
Outdated
fn import_from(&mut self, vm: &VirtualMachine, idx: bytecode::NameIdx) -> PyResult { | ||
let module = self.top_value(); | ||
let name = self.code.names[idx as usize]; | ||
let err = || vm.new_import_error(format!("cannot import name '{name}'"), name.to_owned()); | ||
|
||
// Load attribute, and transform any error into import error. | ||
if let Some(obj) = vm.get_attribute_opt(module.to_owned(), name)? { | ||
return Ok(obj); | ||
} | ||
// fallback to importing '{module.__name__}.{name}' from sys.modules | ||
let mod_name = module | ||
.get_attr(identifier!(vm, __name__), vm) | ||
.map_err(|_| err())?; | ||
let mod_name = mod_name.downcast::<PyStr>().map_err(|_| err())?; | ||
let full_mod_name = format!("{mod_name}.{name}"); | ||
let sys_modules = vm.sys_module.get_attr("modules", vm).map_err(|_| err())?; | ||
sys_modules.get_item(&full_mod_name, vm).map_err(|_| err()) | ||
|
||
let fallback_result: Option<PyResult> = module | ||
.get_attr(&vm.ctx.new_str("__name__"), vm) | ||
.ok() | ||
.and_then(|mod_name| mod_name.downcast_ref::<PyStr>().map(|s| s.to_owned())) | ||
.and_then(|mod_name_str| { | ||
let full_mod_name = format!("{}.{}", mod_name_str.as_str(), name.as_str()); | ||
vm.sys_module | ||
.get_attr("modules", vm) | ||
.ok() | ||
.and_then(|sys_modules| sys_modules.get_item(&full_mod_name, vm).ok()) | ||
}) | ||
.map(Ok); | ||
|
||
if let Some(Ok(sub_module)) = fallback_result { | ||
return Ok(sub_module); | ||
} | ||
|
||
if is_module_initializing(module, vm) { | ||
let module_name = module | ||
.get_attr(&vm.ctx.new_str("__name__"), vm) | ||
.ok() | ||
.and_then(|n| n.downcast_ref::<PyStr>().map(|s| s.as_str().to_owned())) | ||
.unwrap_or_else(|| "<unknown>".to_owned()); | ||
|
||
let msg = format!( | ||
"cannot import name '{}' from partially initialized module '{}' (most likely due to a circular import)", | ||
name.as_str(), | ||
module_name | ||
); | ||
Err(vm.new_import_error(msg, name.to_owned())) | ||
} else { | ||
Err(vm.new_import_error( | ||
format!("cannot import name '{}'", name.as_str()), | ||
name.to_owned(), | ||
)) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Verification agent
🧩 Analysis chain
Critical: Disconnect between internal initializing
field and Python-level __spec__._initializing
attribute.
The refactored import logic looks correct, but there's a fundamental issue: this code checks __spec__._initializing
(line 1389 via is_module_initializing
), but the import_code_obj
function in import.rs
only sets the internal PyModule.initializing
field. There's no code that exposes this internal state to Python as __spec__._initializing
.
Without this connection, the circular import detection won't work as intended.
Consider either:
- Setting
__spec__._initializing
inimport_code_obj
when setting the internal field - Modifying
is_module_initializing
to check the internalPyModule.initializing
field instead
Run the following script to verify if __spec__._initializing
is set anywhere in the codebase:
🏁 Script executed:
#!/bin/bash
# Search for any code that sets _initializing on spec objects
rg -A 5 "_initializing.*="
# Search for any references to the initializing field on PyModule
ast-grep --pattern 'initializing'
Length of output: 62
🏁 Script executed:
#!/bin/bash
set -e
# Search for any occurrences of the internal `initializing` field in Rust code
rg -n "initializing" -t rs
# Search for any usage of `__spec__` or `_initializing` on spec objects
rg -n "__spec__" -t rs
rg -n "_initializing" -t rs
Length of output: 62
🏁 Script executed:
#!/bin/bash
set -e
# Search for occurrences of 'initializing' in all Rust source files
rg -n "initializing" --glob '*.rs'
# Search for any usage of '__spec__' in Rust source
rg -n "__spec__" --glob '*.rs'
# Search for any explicit '_initializing' assignments or checks in Rust source
rg -n "_initializing" --glob '*.rs'
Length of output: 1656
Critical: Circular‐import detection is broken—spec._initializing
is never set
The new is_module_initializing
path in vm/src/frame.rs (lines 1389, 2405–2415) checks the Python‐level __spec__._initializing
flag, but no code ever assigns _initializing
on the module’s __spec__
. We only toggle the internal PyModule.initializing
field in vm/src/import.rs (lines 168, 172), so the “partially initialized module” branch will never fire.
To resolve this, choose one of the following fixes:
• In vm/src/import.rs (around lines 168–172), after calling
module.initializing.store(true/false, Ordering::Relaxed);
also set the Python‐level flag:
let spec = module.get_attr(&vm.ctx.new_str("__spec__"), vm)?;
spec.set_attr(&vm.ctx.new_str("_initializing"), vm.ctx.new_bool(true), vm)?;
(and similarly for false
)
• Or in vm/src/frame.rs’s is_module_initializing
(lines 2405–2415), downcast module
to PyModule
and read its internal initializing
flag directly instead of querying __spec__
.
Add appropriate tests to verify circular‐import error messages now appear.
🤖 Prompt for AI Agents
In vm/src/frame.rs lines 1363 to 1408, the circular-import detection relies on
the Python-level __spec__._initializing flag, but this flag is never set,
causing the detection to fail. To fix this, either update vm/src/import.rs
around lines 168-172 to set the _initializing attribute on the module's __spec__
object whenever the internal PyModule.initializing flag is changed, or modify
the is_module_initializing function in vm/src/frame.rs (lines 2405-2415) to
downcast the module to PyModule and check the internal initializing field
directly. Additionally, add tests to confirm that circular-import errors are
correctly detected and reported.
fn is_module_initializing(module: &PyObject, vm: &VirtualMachine) -> bool { | ||
let Ok(spec) = module.get_attr(&vm.ctx.new_str("__spec__"), vm) else { | ||
return false; | ||
}; | ||
if vm.is_none(&spec) { | ||
return false; | ||
} | ||
let Ok(initializing_attr) = spec.get_attr(&vm.ctx.new_str("_initializing"), vm) else { | ||
return false; | ||
}; | ||
initializing_attr.try_to_bool(vm).unwrap_or(false) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Verification agent
🧩 Analysis chain
Helper function implementation is sound but relies on unset attribute.
The is_module_initializing
function has proper error handling, but it checks for __spec__._initializing
which isn't being set by the import system. This function needs to either:
- Check the internal
PyModule.initializing
field if the module is aPyModule
instance - Ensure that
__spec__._initializing
is properly set during import
Consider checking the internal field directly:
fn is_module_initializing(module: &PyObject, vm: &VirtualMachine) -> bool {
+ // First try to check the internal field if this is a PyModule
+ if let Ok(py_module) = module.downcast_ref::<PyModule>() {
+ use std::sync::atomic::Ordering;
+ return py_module.initializing.load(Ordering::Relaxed);
+ }
+
+ // Fallback to checking __spec__._initializing
let Ok(spec) = module.get_attr(&vm.ctx.new_str("__spec__"), vm) else {
return false;
};
if vm.is_none(&spec) {
return false;
}
let Ok(initializing_attr) = spec.get_attr(&vm.ctx.new_str("_initializing"), vm) else {
return false;
};
initializing_attr.try_to_bool(vm).unwrap_or(false)
}
Run the following script to check how CPython handles module initialization tracking:
🌐 Web query:
How does CPython track module initialization state for circular import detection? Does it use __spec__._initializing?
💡 Result:
CPython tracks module initialization state for circular import detection by leveraging the sys.modules
cache. When a module is first imported, a module object is placed into sys.modules
immediately—before its code is executed. This ensures that if the module is imported again (such as via a circular import), Python will return this already-existing (but possibly only partially initialized) module object, rather than starting a new import. This mechanism is essential for breaking import cycles and preventing infinite recursion during import[3][7].
The __spec__._initializing
attribute (or similar such as __spec__._initialized
) is an internal implementation detail and is not the primary mechanism for detection of circular imports in CPython. There have been discussions and experimental ideas about using such an attribute to mark a module’s initialization state for more fine-grained control or for advanced import hacks like async imports[6][8]. However, in normal CPython operation and as of current mainstream Python versions, this attribute is either not present or not used by the import machinery for circular import detection.
Instead, here is how it works:
- When a module is being imported, it is immediately inserted into
sys.modules
, signaling "import in progress." - If another module tries to import it during this time, it receives a reference to this partially initialized module[3][7].
- If the code tries to access names that haven’t been defined yet (because initialization hasn't finished), you get errors like:
ImportError: cannot import name 'X' from 'Y' (most likely due to a circular import)[7]
- There is no robust, public-facing per-module "initializing" flag used by the import system; the presence of the module object in
sys.modules
before its code has finished running is the primary state signal.
Summary Table
Mechanism | Used by CPython for circular import detection? | Notes |
---|---|---|
sys.modules early insert |
Yes | Inserted immediately, acts as the signal for "import in progress" |
__spec__._initializing |
No (not in standard import) | May be used in experimental or advanced import schemes, not standard in CPython as of 2025 |
Explicit module state flag | No | Not part of public API or normal workflow |
In conclusion:
CPython does not use __spec__._initializing
to track module initialization state for the purpose of circular import detection; it relies on the presence of the module in sys.modules
during import.[3][7]
Citations:
- 1: https://github.com/bndr/pycycle
- 2: https://github.com/python/cpython/blob/master/Lib/test/test_import/__init__.py
- 3: https://rollbar.com/blog/how-to-fix-circular-import-in-python/
- 4: https://realpython.com/python-import/
- 5: https://www.geeksforgeeks.org/python/python-circular-imports/
- 6: https://discuss.python.org/t/async-imports-to-reduce-startup-times/69732
- 7: https://www.datacamp.com/tutorial/python-circular-import
- 8: Improve AttributeError message for partially initialized module python/cpython#77418
- 9: https://labex.io/tutorials/python-how-to-resolve-circular-imports-in-python-418812
- 10: https://docs.kanaries.net/topics/Python/python-circular-import
Streamline is_module_initializing
to use internal flag and drop __spec__._initializing
fallback.
- File: vm/src/frame.rs (lines 2405–2416)
- CPython doesn’t use
__spec__._initializing
for import tracking—modules insys.modules
signal “in progress.” - RustPython’s
PyModule
already has aninitializing
flag; use that and returnfalse
for all other objects.
fn is_module_initializing(module: &PyObject, vm: &VirtualMachine) -> bool {
- // Fallback to checking __spec__._initializing
- let Ok(spec) = module.get_attr(&vm.ctx.new_str("__spec__"), vm) else {
- return false;
- };
- if vm.is_none(&spec) {
- return false;
- }
- let Ok(initializing_attr) = spec.get_attr(&vm.ctx.new_str("_initializing"), vm) else {
- return false;
- };
- initializing_attr.try_to_bool(vm).unwrap_or(false)
+ // Use internal initializing flag on PyModule
+ if let Ok(py_module) = module.downcast_ref::<PyModule>() {
+ use std::sync::atomic::Ordering;
+ return py_module.initializing.load(Ordering::Relaxed);
+ }
+ false
}
🤖 Prompt for AI Agents
In vm/src/frame.rs around lines 2405 to 2416, refactor the
is_module_initializing function to remove the fallback logic that checks
__spec__._initializing. Instead, check if the given module is a PyModule and
directly return its initializing flag. For all other object types, return false.
This aligns with CPython's approach and simplifies the function by relying on
the internal initializing flag of PyModule.
2d99040
to
69c0294
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice feature! Thanks!
Summary by CodeRabbit
Bug Fixes
New Features