Skip to content

Fix eval type check #5908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 7, 2025
Merged

Fix eval type check #5908

merged 2 commits into from
Jul 7, 2025

Conversation

youknowone
Copy link
Member

@youknowone youknowone commented Jul 7, 2025

Summary by CodeRabbit

  • Documentation
    • Updated repository guidelines to clarify that the Lib/ directory should not be edited directly, except for copying files from CPython.
  • Tests
    • Added comprehensive tests for the eval function, focusing on error handling and validation of globals and locals arguments.
  • Bug Fixes
    • Improved validation for the globals argument in eval and exec, ensuring stricter type requirements and clearer error messages.

Copy link
Contributor

coderabbitai bot commented Jul 7, 2025

Walkthrough

The changes update documentation to clarify restrictions on editing the Lib/ directory, add comprehensive tests for the eval built-in regarding globals and locals handling, and refactor the scope creation logic in the virtual machine to enforce stricter validation and error reporting for eval and exec functions.

Changes

File(s) Change Summary
.github/copilot-instructions.md Updated description of Lib/ to explicitly prohibit direct edits and clarify its intended usage.
extra_tests/snippets/builtin_eval.py Added extensive tests for eval validating types and behaviors of globals and locals parameters.
vm/src/stdlib/builtins.rs Refactored scope creation: stricter type validation for globals, updated function signatures, improved error messages, and adjusted control flow for eval/exec.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Builtins
    participant ScopeArgs
    participant Scope
    participant VM

    User->>Builtins: call eval(source, scope_args, vm)
    Builtins->>ScopeArgs: make_scope(vm, "eval")
    ScopeArgs->>ScopeArgs: validate_globals_dict (must be real dict)
    ScopeArgs->>Scope: construct Scope with validated globals/locals
    Builtins->>VM: run_code(source, scope, mode, "eval")
    VM-->>User: result or error
Loading

Poem

In the warren of code, a rule now stands tall,
"Don't touch Lib/ directly!"—the clearest of all.
Eval's now picky, with globals checked right,
Tests hop in, ensuring no bugs in sight.
With stricter scopes and error delight,
The code hops forward—oh, what a night! 🐇✨


📜 Recent review details

Configuration used: .coderabbit.yml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b803b0f and c8f36aa.

📒 Files selected for processing (2)
  • extra_tests/snippets/builtin_eval.py (1 hunks)
  • vm/src/stdlib/builtins.rs (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • vm/src/stdlib/builtins.rs
  • extra_tests/snippets/builtin_eval.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
  • GitHub Check: Run snippets and cpython tests (ubuntu-latest)
  • GitHub Check: Check Rust code with rustfmt and clippy
  • GitHub Check: Run rust tests (windows-latest)
  • GitHub Check: Run snippets and cpython tests (macos-latest)
  • GitHub Check: Run snippets and cpython tests on wasm-wasi
  • GitHub Check: Run snippets and cpython tests (windows-latest)
  • GitHub Check: Run rust tests (ubuntu-latest)
  • GitHub Check: Ensure compilation on various targets
  • GitHub Check: Check the WASM package and demo
  • GitHub Check: Run rust tests (macos-latest)
  • GitHub Check: Run tests under miri
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @youknowone, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the eval and exec built-in functions in RustPython by implementing stricter type validation for their globals argument, aligning its behavior and error messages more closely with CPython. This change improves compatibility and robustness, supported by a comprehensive set of new tests that verify various edge cases and correct error handling. Additionally, a minor documentation update clarifies guidelines for contributing to the Lib/ directory.

Highlights

  • Stricter eval/exec globals type checking: The globals argument for eval and exec now undergoes more rigorous type validation, ensuring it is a true dict as required by CPython, rather than just any mapping.
  • Improved error messages for globals: Error messages for invalid globals arguments in eval and exec have been refined to match CPython's specific wording, distinguishing between non-mapping types and mapping types that are not true dictionaries.
  • Comprehensive eval test suite: A significant number of new test cases have been added for builtin_eval.py to thoroughly cover various scenarios for globals and locals arguments, including UserDict, None, __builtins__ injection, and free variable handling.
  • Documentation clarification: A note was added to the copilot-instructions.md file, emphasizing that the Lib/ directory should only be updated by copying files from CPython, not by direct editing.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request fixes a type check for the globals argument in the eval function. The changes include modifying ScopeArgs to accept any Python object for globals and then performing a detailed type validation. The logic has been refactored into a validate_globals_dict helper function inside make_scope, which now also handles exec with more specific error messages. A comprehensive set of tests for eval has been added in extra_tests/snippets/builtin_eval.py. I've provided a suggestion to fix an assertion in the new test file.

eval("x", [])
assert False, "eval with list globals should fail"
except TypeError as e:
assert "globals must be a real dict" in str(e), e

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The assertion for eval with a list as globals is not quite right. Since a list is not a mapping, eval should raise a TypeError indicating that globals must be a dict, not a "real dict". The "real dict" message is for mapping-like objects that aren't actual dictionaries (e.g., collections.UserDict).

Suggested change
assert "globals must be a real dict" in str(e), e
assert "globals must be a dict" in str(e)
assert "real dict" not in str(e), e

@youknowone youknowone merged commit 7ebe018 into RustPython:main Jul 7, 2025
12 checks passed
@youknowone youknowone deleted the eval-typecheck branch July 7, 2025 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant