Skip to content

Provide detailed error for circular from imports #5972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 14, 2025

Conversation

ever0de
Copy link
Contributor

@ever0de ever0de commented Jul 14, 2025

Summary by CodeRabbit

  • Bug Fixes

    • Improved error messages when importing names from a module, providing clearer information in cases of circular imports or partial initialization.
  • New Features

    • Enhanced detection of modules that are still initializing, helping users identify circular import issues more easily.

Copy link
Contributor

coderabbitai bot commented Jul 14, 2025

Warning

Rate limit exceeded

@ever0de has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 6 minutes and 36 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 9d6912c and 69c0294.

📒 Files selected for processing (1)
  • vm/src/frame.rs (2 hunks)

Walkthrough

The import_from method in the ExecutingFrame struct was revised to improve error handling and fallback logic during module imports. A new helper function, is_module_initializing, was introduced to check if a module is still initializing, allowing for more precise import error messages, especially in cases of circular imports or partial initialization.

Changes

File(s) Change Summary
vm/src/frame.rs Refactored import_from to improve fallback logic and error reporting; added is_module_initializing helper function.

Poem

In the warren of code, a module peeks through,
Importing its friends, as rabbits often do.
If caught in a loop, or half-awake yawn,
A new error message hops helpfully on.
With clearer advice, imports now sing—
No more confusion in the Python spring! 🐇

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@ever0de ever0de force-pushed the fix/circular-import-error branch 2 times, most recently from 9c09aef to dfb4db1 Compare July 14, 2025 15:04
@ever0de ever0de force-pushed the fix/circular-import-error branch from dfb4db1 to d86644e Compare July 14, 2025 15:07
@ever0de ever0de changed the title fix(vm): provide detailed error for circular from imports Provide detailed error for circular from imports Jul 14, 2025
@ever0de ever0de marked this pull request as ready for review July 14, 2025 15:22
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fd35c7a and d86644e.

⛔ Files ignored due to path filters (1)
  • Lib/test/test_import/__init__.py is excluded by !Lib/**
📒 Files selected for processing (3)
  • vm/src/builtins/module.rs (3 hunks)
  • vm/src/frame.rs (2 hunks)
  • vm/src/import.rs (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

Instructions used from:

Sources:
📄 CodeRabbit Inference Engine

  • .github/copilot-instructions.md
🧠 Learnings (2)
vm/src/builtins/module.rs (1)
Learnt from: CR
PR: RustPython/RustPython#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-06-30T10:08:48.858Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
vm/src/import.rs (1)
Learnt from: CR
PR: RustPython/RustPython#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-06-30T10:08:48.858Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Run snippets and cpython tests (windows-latest)
🔇 Additional comments (4)
vm/src/builtins/module.rs (2)

1-1: LGTM! Atomic boolean field properly added for initialization tracking.

The AtomicBool import and field addition are appropriate for tracking module initialization state in a thread-safe manner.

Also applies to: 57-57


80-80: Field initialization is correct in both constructors.

The initializing field is properly initialized to false in both new() and from_def() constructors, ensuring modules start in a non-initializing state.

Also applies to: 88-88

vm/src/import.rs (2)

3-3: Imports correctly added for atomic operations.

The Ordering import and PyModule addition are necessary for the initialization tracking implementation.

Also applies to: 7-7


161-178: Well-implemented guard pattern for initialization tracking.

The guard ensures the initializing flag is always reset, even if module code execution fails or panics. The use of Ordering::Relaxed is appropriate here.

However, this sets an internal field that isn't exposed to Python code. The frame.rs code checks __spec__._initializing, which is a different attribute. Consider exposing this state to Python.

vm/src/frame.rs Outdated
Comment on lines 1363 to 1408
fn import_from(&mut self, vm: &VirtualMachine, idx: bytecode::NameIdx) -> PyResult {
let module = self.top_value();
let name = self.code.names[idx as usize];
let err = || vm.new_import_error(format!("cannot import name '{name}'"), name.to_owned());

// Load attribute, and transform any error into import error.
if let Some(obj) = vm.get_attribute_opt(module.to_owned(), name)? {
return Ok(obj);
}
// fallback to importing '{module.__name__}.{name}' from sys.modules
let mod_name = module
.get_attr(identifier!(vm, __name__), vm)
.map_err(|_| err())?;
let mod_name = mod_name.downcast::<PyStr>().map_err(|_| err())?;
let full_mod_name = format!("{mod_name}.{name}");
let sys_modules = vm.sys_module.get_attr("modules", vm).map_err(|_| err())?;
sys_modules.get_item(&full_mod_name, vm).map_err(|_| err())

let fallback_result: Option<PyResult> = module
.get_attr(&vm.ctx.new_str("__name__"), vm)
.ok()
.and_then(|mod_name| mod_name.downcast_ref::<PyStr>().map(|s| s.to_owned()))
.and_then(|mod_name_str| {
let full_mod_name = format!("{}.{}", mod_name_str.as_str(), name.as_str());
vm.sys_module
.get_attr("modules", vm)
.ok()
.and_then(|sys_modules| sys_modules.get_item(&full_mod_name, vm).ok())
})
.map(Ok);

if let Some(Ok(sub_module)) = fallback_result {
return Ok(sub_module);
}

if is_module_initializing(module, vm) {
let module_name = module
.get_attr(&vm.ctx.new_str("__name__"), vm)
.ok()
.and_then(|n| n.downcast_ref::<PyStr>().map(|s| s.as_str().to_owned()))
.unwrap_or_else(|| "<unknown>".to_owned());

let msg = format!(
"cannot import name '{}' from partially initialized module '{}' (most likely due to a circular import)",
name.as_str(),
module_name
);
Err(vm.new_import_error(msg, name.to_owned()))
} else {
Err(vm.new_import_error(
format!("cannot import name '{}'", name.as_str()),
name.to_owned(),
))
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Critical: Disconnect between internal initializing field and Python-level __spec__._initializing attribute.

The refactored import logic looks correct, but there's a fundamental issue: this code checks __spec__._initializing (line 1389 via is_module_initializing), but the import_code_obj function in import.rs only sets the internal PyModule.initializing field. There's no code that exposes this internal state to Python as __spec__._initializing.

Without this connection, the circular import detection won't work as intended.

Consider either:

  1. Setting __spec__._initializing in import_code_obj when setting the internal field
  2. Modifying is_module_initializing to check the internal PyModule.initializing field instead

Run the following script to verify if __spec__._initializing is set anywhere in the codebase:


🏁 Script executed:

#!/bin/bash
# Search for any code that sets _initializing on spec objects
rg -A 5 "_initializing.*=" 

# Search for any references to the initializing field on PyModule
ast-grep --pattern 'initializing'

Length of output: 62


🏁 Script executed:

#!/bin/bash
set -e

# Search for any occurrences of the internal `initializing` field in Rust code
rg -n "initializing" -t rs

# Search for any usage of `__spec__` or `_initializing` on spec objects
rg -n "__spec__" -t rs
rg -n "_initializing" -t rs

Length of output: 62


🏁 Script executed:

#!/bin/bash
set -e

# Search for occurrences of 'initializing' in all Rust source files
rg -n "initializing" --glob '*.rs'

# Search for any usage of '__spec__' in Rust source
rg -n "__spec__" --glob '*.rs'

# Search for any explicit '_initializing' assignments or checks in Rust source
rg -n "_initializing" --glob '*.rs'

Length of output: 1656


Critical: Circular‐import detection is broken—spec._initializing is never set

The new is_module_initializing path in vm/src/frame.rs (lines 1389, 2405–2415) checks the Python‐level __spec__._initializing flag, but no code ever assigns _initializing on the module’s __spec__. We only toggle the internal PyModule.initializing field in vm/src/import.rs (lines 168, 172), so the “partially initialized module” branch will never fire.

To resolve this, choose one of the following fixes:

• In vm/src/import.rs (around lines 168–172), after calling

module.initializing.store(true/false, Ordering::Relaxed);

also set the Python‐level flag:

let spec = module.get_attr(&vm.ctx.new_str("__spec__"), vm)?;
spec.set_attr(&vm.ctx.new_str("_initializing"), vm.ctx.new_bool(true), vm)?;

(and similarly for false)

• Or in vm/src/frame.rs’s is_module_initializing (lines 2405–2415), downcast module to PyModule and read its internal initializing flag directly instead of querying __spec__.

Add appropriate tests to verify circular‐import error messages now appear.

🤖 Prompt for AI Agents
In vm/src/frame.rs lines 1363 to 1408, the circular-import detection relies on
the Python-level __spec__._initializing flag, but this flag is never set,
causing the detection to fail. To fix this, either update vm/src/import.rs
around lines 168-172 to set the _initializing attribute on the module's __spec__
object whenever the internal PyModule.initializing flag is changed, or modify
the is_module_initializing function in vm/src/frame.rs (lines 2405-2415) to
downcast the module to PyModule and check the internal initializing field
directly. Additionally, add tests to confirm that circular-import errors are
correctly detected and reported.

Comment on lines +2405 to +2416
fn is_module_initializing(module: &PyObject, vm: &VirtualMachine) -> bool {
let Ok(spec) = module.get_attr(&vm.ctx.new_str("__spec__"), vm) else {
return false;
};
if vm.is_none(&spec) {
return false;
}
let Ok(initializing_attr) = spec.get_attr(&vm.ctx.new_str("_initializing"), vm) else {
return false;
};
initializing_attr.try_to_bool(vm).unwrap_or(false)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Helper function implementation is sound but relies on unset attribute.

The is_module_initializing function has proper error handling, but it checks for __spec__._initializing which isn't being set by the import system. This function needs to either:

  1. Check the internal PyModule.initializing field if the module is a PyModule instance
  2. Ensure that __spec__._initializing is properly set during import

Consider checking the internal field directly:

fn is_module_initializing(module: &PyObject, vm: &VirtualMachine) -> bool {
+    // First try to check the internal field if this is a PyModule
+    if let Ok(py_module) = module.downcast_ref::<PyModule>() {
+        use std::sync::atomic::Ordering;
+        return py_module.initializing.load(Ordering::Relaxed);
+    }
+    
+    // Fallback to checking __spec__._initializing
    let Ok(spec) = module.get_attr(&vm.ctx.new_str("__spec__"), vm) else {
        return false;
    };
    if vm.is_none(&spec) {
        return false;
    }
    let Ok(initializing_attr) = spec.get_attr(&vm.ctx.new_str("_initializing"), vm) else {
        return false;
    };
    initializing_attr.try_to_bool(vm).unwrap_or(false)
}

Run the following script to check how CPython handles module initialization tracking:


🌐 Web query:

How does CPython track module initialization state for circular import detection? Does it use __spec__._initializing?

💡 Result:

CPython tracks module initialization state for circular import detection by leveraging the sys.modules cache. When a module is first imported, a module object is placed into sys.modules immediately—before its code is executed. This ensures that if the module is imported again (such as via a circular import), Python will return this already-existing (but possibly only partially initialized) module object, rather than starting a new import. This mechanism is essential for breaking import cycles and preventing infinite recursion during import[3][7].

The __spec__._initializing attribute (or similar such as __spec__._initialized) is an internal implementation detail and is not the primary mechanism for detection of circular imports in CPython. There have been discussions and experimental ideas about using such an attribute to mark a module’s initialization state for more fine-grained control or for advanced import hacks like async imports[6][8]. However, in normal CPython operation and as of current mainstream Python versions, this attribute is either not present or not used by the import machinery for circular import detection.

Instead, here is how it works:

  • When a module is being imported, it is immediately inserted into sys.modules, signaling "import in progress."
  • If another module tries to import it during this time, it receives a reference to this partially initialized module[3][7].
  • If the code tries to access names that haven’t been defined yet (because initialization hasn't finished), you get errors like:

    ImportError: cannot import name 'X' from 'Y' (most likely due to a circular import)[7]

  • There is no robust, public-facing per-module "initializing" flag used by the import system; the presence of the module object in sys.modules before its code has finished running is the primary state signal.

Summary Table

Mechanism Used by CPython for circular import detection? Notes
sys.modules early insert Yes Inserted immediately, acts as the signal for "import in progress"
__spec__._initializing No (not in standard import) May be used in experimental or advanced import schemes, not standard in CPython as of 2025
Explicit module state flag No Not part of public API or normal workflow

In conclusion:
CPython does not use __spec__._initializing to track module initialization state for the purpose of circular import detection; it relies on the presence of the module in sys.modules during import.[3][7]

Citations:


Streamline is_module_initializing to use internal flag and drop __spec__._initializing fallback.

  • File: vm/src/frame.rs (lines 2405–2416)
  • CPython doesn’t use __spec__._initializing for import tracking—modules in sys.modules signal “in progress.”
  • RustPython’s PyModule already has an initializing flag; use that and return false for all other objects.
 fn is_module_initializing(module: &PyObject, vm: &VirtualMachine) -> bool {
-    // Fallback to checking __spec__._initializing
-    let Ok(spec) = module.get_attr(&vm.ctx.new_str("__spec__"), vm) else {
-        return false;
-    };
-    if vm.is_none(&spec) {
-        return false;
-    }
-    let Ok(initializing_attr) = spec.get_attr(&vm.ctx.new_str("_initializing"), vm) else {
-        return false;
-    };
-    initializing_attr.try_to_bool(vm).unwrap_or(false)
+    // Use internal initializing flag on PyModule
+    if let Ok(py_module) = module.downcast_ref::<PyModule>() {
+        use std::sync::atomic::Ordering;
+        return py_module.initializing.load(Ordering::Relaxed);
+    }
+    false
 }
🤖 Prompt for AI Agents
In vm/src/frame.rs around lines 2405 to 2416, refactor the
is_module_initializing function to remove the fallback logic that checks
__spec__._initializing. Instead, check if the given module is a PyModule and
directly return its initializing flag. For all other object types, return false.
This aligns with CPython's approach and simplifies the function by relying on
the internal initializing flag of PyModule.

@ever0de
Copy link
Contributor Author

ever0de commented Jul 14, 2025

@ever0de ever0de force-pushed the fix/circular-import-error branch from 2d99040 to 69c0294 Compare July 14, 2025 16:09
Copy link
Member

@youknowone youknowone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice feature! Thanks!

@youknowone youknowone merged commit d4f85cf into RustPython:main Jul 14, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants