Skip to content

Reduce memory allocations in event loops #20306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
May 21, 2025
Merged

Reduce memory allocations in event loops #20306

merged 8 commits into from
May 21, 2025

Conversation

stelfrag
Copy link
Collaborator

@stelfrag stelfrag commented May 19, 2025 •
edited
Loading

Summary
  • Add Worker pool to metadata/ aclk
    • uv_queue_work will reuse worker to avoid memory allocations
  • Add cmd pool for metadata / aclk event loops
    • Avoid memory allocations, use a circular buffer
  • Do not queue commands from the event loop (operation will block if cmd queue is full)
    • Use flags to signal appropriate processing once the event loop is stopped
  • Cleanup ACLK sync commands
    • Unused or removed operations
  • Add unittest for command pool

@stelfrag
Add Worker pool to metadata/ aclk
517a34c 
Add cmd pool for metadata / aclk event loops
Do not queue commands from the event loop
Cleanup ACLK sync commands
Add unittest test for cmd_pool
@stelfrag stelfrag marked this pull request as ready for review May 20, 2025 14:32
thiagoftsm
thiagoftsm reviewed May 20, 2025
View reviewed changes
thiagoftsm
thiagoftsm reviewed May 20, 2025
View reviewed changes
thiagoftsm
thiagoftsm reviewed May 20, 2025
View reviewed changes
thiagoftsm
thiagoftsm reviewed May 20, 2025
View reviewed changes
@thiagoftsm
Copy link
Contributor

During compilation time, we have this warning:

[758/1157] Building C object CMakeFiles/netdata.dir/src/daemon/libuv_workers.c.o
/home/thiago/Netdata/tests_netdata/src/daemon/libuv_workers.c: In function 'test_cmd_pool_fifo':
/home/thiago/Netdata/tests_netdata/src/daemon/libuv_workers.c:261:23: warning: comparison of integer expressions of different signedness: 'int' a
nd 'long unsigned int' [-Wsign-compare]
  261 |     for (int i = 0; i < sizeof(pool_sizes) / sizeof(pool_sizes[0]); ++i) {

thiagoftsm
thiagoftsm previously approved these changes May 20, 2025
View reviewed changes
Copy link
Contributor

@thiagoftsm thiagoftsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After adjusts, PR is working as expected, LGTM!

@stelfrag stelfrag requested a review from thiagoftsm May 20, 2025 18:34
@stelfrag stelfrag merged commit 70c3457 into netdata:master May 21, 2025
107 of 108 checks passed
@stelfrag stelfrag deleted the reduce_memory_allocations branch May 21, 2025 05:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thiagoftsm thiagoftsm thiagoftsm approved these changes

@vkalintiris vkalintiris Awaiting requested review from vkalintiris vkalintiris is a code owner

Assignees
No one assigned
Labels
area/aclk area/daemon area/database
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@stelfrag @thiagoftsm