Labels: simpler/faster stringlabels encoding #16069
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dgl
reviewed
Feb 24, 2025
|
I like the idea, we just need to agree that Also, just thinking out loud, we could take this a step further. This way, it's incremental. (actually, with the current approach, I think we can make the 3 bytes store number from |
|
Every branch slows it down. This is why I went for a very simple scheme. |
|
I missed that, I even re-ran the benchmarks, but I have an Arm x) |
|
Found myself looking at the assembly for regressions, apparently hoisting (I thought helps speed up some of them, the change also makes that 1-byte case "identical" with the code in main. Still Note that only the 1-byte case in I also need to run this on ARM. |
Meh, I don't much care how fast it goes if you abuse the Prometheus data model. |
Instead of using varint to encode the size of each label, use a single byte for size 0-254, or a flag value of 255 followed by the size in 3 bytes little-endian. This reduces the amount of code, and also the number of branches in commonly-executed code, so it runs faster. The maximum allowed label name or value length is now 2^24 or 16MB. Memory used by labels changes as follows: * Labels from 0 to 127 bytes length: same * From 128 to 254: 1 byte less * From 255 to 16383: 2 bytes more * From 16384 to 2MB: 1 byte more * From 2MB to 16MB: same Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Goes noticeably faster on AMD64 architecture. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Also add test that shows the problem. Credit to @dgl. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Slightly more user-friendly than encoding bad data and finding out when we decode. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
bboreham
force-pushed
the
simpler-stringlabels
branch
from
b835d3b to
78ab02a
Compare
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
|
Added @machine424's change which makes a decent improvement on amd64. Latest benchmark comparison against main (8487ed8). |
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
Yes some of the cases results are intriguing, I even got completely different results on different AMD CPUs (lost the diff unfortunately). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of using varint to encode the size of each label, use a single byte for size 0-254, or a flag value of 255 followed by the size in 3 bytes little-endian.
This reduces the amount of code, and also the number of branches in commonly-executed code, so it runs faster.
The maximum allowed label name or value length is now 2^24 or 16MB.
Memory used by labels changes as follows:
Benchmarks - Go 1.24.0, x64:
Benchmarks - Go 1.24.0, ARM64: