Skip to content

OTEL attribute count limit not respected, causing columns dropped #4677

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

giongto35
Copy link

@giongto35 giongto35 commented Jul 10, 2025

Description

Fixes #4671

This PR fixes a bug where the LoggingHandler in the OpenTelemetry Python SDK does not respect the OTEL_ATTRIBUTE_COUNT_LIMIT environment variable. When this environment variable is set, it should limit the number of attributes in log records, but the LoggingHandler was ignoring this setting entirely and only use the default value 128.
It means:

  • If we set to small count limit like 3, logging 10 attributes still works without any attributes dropped.
  • If we set to high count limit like 256, logging 200 attributes has (200-128= 72) attributes dropped.

Changes Made

Fixed the Root Cause Bug

The core issue was that LogRecord was created with _UnsetLogLimits which used UNSET values, and the LogLimits._from_env_if_absent method had a bug where it returned None immediately when value == cls.UNSET, never checking environment variables.

# Before (buggy):
_UnsetLogLimits = LogLimits(max_attributes=LogLimits.UNSET, ...)
# This caused _from_env_if_absent to return None immediately without reading env vars

# After (fixed):
# Removed _UnsetLogLimits entirely, now use None as default
limits = limits or LogLimits()  # LogLimits() correctly reads from env vars

Implementation Benefits

This implementation allows users to:

  • Rely on environment variables for configuration (OTEL_ATTRIBUTE_COUNT_LIMIT)
  • Use a cleaner, more intuitive API without special UNSET values

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Code refactoring (improves code quality without changing functionality)

How Has This Been Tested?

You can run the tests using pytest:

# Install the package in development mode
pip install -e .

# Run all the LoggingHandler tests
pytest opentelemetry-sdk/tests/logs/test_handler.py -v

# Run all logging tests
pytest opentelemetry-sdk/tests/logs/ -v

# Run only the attribute count limit tests
pytest opentelemetry-sdk/tests/logs/test_handler.py::TestLoggingHandler -k "otel_attribute_count_limit" -v

Test Coverage

Added comprehensive tests to verify the fix works correctly:

  1. test_otel_attribute_count_limit_respected_in_logging_handler - Tests that when OTEL_ATTRIBUTE_COUNT_LIMIT=3, only 3 attributes are kept and the rest are dropped
  2. test_otel_attribute_count_limit_includes_code_attributes - Tests that when OTEL_ATTRIBUTE_COUNT_LIMIT=5, only 5 attributes are kept and the limit applies to all attribute types including code attributes
  3. test_logging_handler_without_env_var_uses_default_limit - Tests that without the environment variable, the default limit of 128 is applied

All tests pass, confirming the fix works correctly across all these scenarios.

Does This PR Require a Contrib Repo Change?

  • No.

Checklist:

  • Followed the style guidelines of this project
  • Unit tests have been added
  • Documentation has been updated
  • Fixed the original bug with OTEL_ATTRIBUTE_COUNT_LIMIT
  • Refactored code for better maintainability
  • Removed deprecated/unnecessary code patterns
  • All existing tests continue to pass

@giongto35 giongto35 requested a review from a team as a code owner July 10, 2025 02:37
Copy link

linux-foundation-easycla bot commented Jul 10, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

@giongto35 giongto35 force-pushed the otel-attribute-count-not-respected branch from 889a3ec to d2ac2c3 Compare July 10, 2025 05:09
Copy link
Member

@yurishkuro yurishkuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@giongto35
Copy link
Author

Hi @pmcollins , can this change be merged so we can patch our code?

@pmcollins
Copy link
Member

Hi @pmcollins , can this change be merged so we can patch our code?

Hi @giongto35 -- we'll need one of the maintainers to review the change. Once a maintainer has approved, then the change can be merged.

In the meantime, I have approved the workflows to run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

LoggingHandler does not respect OTEL_ATTRIBUTE_COUNT_LIMIT environment variable
3 participants