[CORE-10975] Add QoS packet rate limit #9938
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add bandwidth limiting for ingress and egress, implemented via `tc qdisc`
configuration based on the {ingress,egress} bandwidth and burst fields of
the QoSControls struct in WorkloadEndpoint. In kubernetes, they come from
annotations on the kubernetes pods.
Felix FV tests added to verify that the `tc qdisc` configuration is
properly added and that the bandwidth is properly limited with `iperf3`.
Some code borrowed from the kubernetes bandwidth plugin (credit given).
- Improve error messages - Fix ingress bandwidth testing - Other misc corrections
Guard QoS setup against BPF mode. Add cleanup of QoS for when BPF is enabled. Update QoS bandwidth in place instead of remove+readd. Minor test fixes.
Change logic of maybeUpdateQoSBandwidth() to consider the dataplane state (actual `tc qdisc` config) instead of the datastore values. Fix QoS cleanup on BPF startup to not short-circuit on errors.
- Generally use TokenBucketState pointers - Use *TokenBucketState more widely
- Merge suggested changes. A separate 'qos' package still needed to be split from 'intdataplane' in order to avoid an import loop.
- Properly attribute and copyright code from the CNI bandwidth plugin
- Add a proper update function for kdd workloads to the felix FV infra
- Adjust test expectations (using 'Consistently()' where appropriate)
- Correctly use '%w' for outputting errors
- Inline calculations used for converting rate and burst in bits {per second,} to TokenBucketState{} values
- Other misc changes
…e spurious errors, etc
coutinhop
added
release-note-required
coutinhop
force-pushed
the
pedro-CORE-10975
branch
from
fbef100 to
0a7c304
Compare
Add packet rate limiting QoS control implementation, using iptables (using '-m limit' and a new mark) and nftables (using 'limit rate over') rules to limit ingress and/or egress packet rate based on workload QoSControls (from pod annotations in k8s).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add packet rate limiting QoS control implementation, using iptables
(using '-m limit' and a new mark) and nftables (using 'limit rate over')
rules to limit ingress and/or egress packet rate based on workload
QoSControls (from pod annotations in k8s).
Branched off of #9881, until that merges the diff will show changes from both PRs.
Related issues/PRs
Todos
Release Note
Reminder for the reviewer
Make sure that this PR has the correct labels and milestone set.
Every PR needs one
docs-*label.docs-pr-required: This change requires a change to the documentation that has not been completed yet.docs-completed: This change has all necessary documentation completed.docs-not-required: This change has no user-facing impact and requires no docs.Every PR needs one
release-note-*label.release-note-required: This PR has user-facing changes. Most PRs should have this label.release-note-not-required: This PR has no user-facing changes.Other optional labels:
cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.