Skip to content

Interrupt exporter retry backoff sleeps when shutdown is called. Update BatchSpan/LogRecordProcessor.shutdown to complete in 30 seconds #4638

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 50 commits into
base: main
Choose a base branch
from

Conversation

DylanRussell
Copy link
Contributor

@DylanRussell DylanRussell commented Jun 16, 2025

Description

This PR updates the HTTP/gRPC OTLP Exporters, so that the retry backoff sleep in export gets immediately interrupted (and failure returned) when shutdown is called, instead of sleeping / retrying to completion.

This PR also updates the BatchSpan/LogRecordProcessor's shutdown method to complete after 30 seconds instead of continuing to run until all telemetry was flushed from the queue.

Fixes an issue where shutdown would hang or stall indefinitely, especially when export was failing inside a retry loop.

Fixes: #3309, #4043, #2663

Type of change

  • [ x] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Lots of unit tests.

Does This PR Require a Contrib Repo Change?

  • Yes. - Link to PR:
  • [ x] No.

Checklist:

  • [x ] Followed the style guidelines of this project
  • [ x] Changelogs have been updated
  • [ x] Unit tests have been added
  • [ x] Documentation has been updated

@aabmass
Copy link
Member

aabmass commented Jun 23, 2025

Also im not sure if this fixes #4623, since it's still not configurable

@DylanRussell
Copy link
Contributor Author

Ok. Removed 4623 from the list of fixed

Copy link
Member

@aabmass aabmass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% sure on the Exporter.shutdown comment, I'll think on it a little

Copy link
Member

@aabmass aabmass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One bug but otherwise LGTM

DylanRussell and others added 5 commits July 18, 2025 19:47
Co-authored-by: Emídio Neto <9735060+emdneto@users.noreply.github.com>
Co-authored-by: Emídio Neto <9735060+emdneto@users.noreply.github.com>
Copy link
Member

@emdneto emdneto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed offline to rename the shutdown event variable to _shutdown_in_progress.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Exporters shutdown takes longer then a minute when failing to send metrics/traces
3 participants