Commit bdb5d248 authored by Jakub Kicinski's avatar Jakub Kicinski
Browse files

Merge branch 'net-introduce-tx-h-w-shaping-api'

Paolo Abeni says:

====================
net: introduce TX H/W shaping API

We have a plurality of shaping-related drivers API, but none flexible
enough to meet existing demand from vendors[1].

This series introduces new device APIs to configure in a flexible way
TX H/W shaping. The new functionalities are exposed via a newly
defined generic netlink interface and include introspection
capabilities. Some self-tests are included, on top of a dummy
netdevsim implementation. Finally a basic implementation for the iavf
driver is provided.

Some usage examples:

* Configure shaping on a given queue:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do set --json '{"ifindex": '$IFINDEX',
			  "shaper": {"handle":
				     {"scope": "queue", "id":'$QUEUEID'},
			  "bw-max": 2000000}}'

* Container B/W sharing

The orchestration infrastructure wants to group the
container-related queues under a RR scheduling and limit the aggregate
bandwidth:

./tools/net/ynl/cli.py --spec Documentation/netlink/specs/shaper.yaml \
	--do group --json '{"ifindex": '$IFINDEX',
			"leaves": [
			  {"handle": {"scope": "queue", "id":'$QID1'},
			   "weight": '$W1'},
			  {"handle": {"scope": "queue", "id":'$QID2'},
			   "weight": '$W2'}],
			  {"handle": {"scope": "queue", "id":'$QID3'},
			   "weight": '$W3'}],
			"handle": {"scope":"node"},
			"bw-max": 10000000}'
{'ifindex': $IFINDEX, 'handle': {'scope': 'node', 'id': 0}}

Q1 \
    \
Q2 -- node 0 -------  netdev
    / (bw-max: 10M)
Q3 /

* Delegation

A containers wants to limit the aggregate B/W bandwidth of 2 of the 3
queues it owns - the starting configuration is the one from the
previous point:

SPEC=Documentation/netlink/specs/net_shaper.yaml
./tools/net/ynl/cli.py --spec $SPEC \
	--do group --json '{"ifindex": '$IFINDEX',
			"leaves": [
			  {"handle": {"scope": "queue", "id":'$QID1'},
			   "weight": '$W1'},
			  {"handle": {"scope": "queue", "id":'$QID2'},
			   "weight": '$W2'}],
			"handle": {"scope": "node"},
			"bw-max": 5000000 }'
{'ifindex': $IFINDEX, 'handle': {'scope': 'node', 'id': 1}}

Q1 -- node 1 --------\
    / (bw-max: 5M)    \
Q2 /                   node 0 -------  netdev
                      /(bw-max: 10M)
Q3 ------------------/

In a group operation, when prior to the op itself, the leaves have
different parents, the user must specify the parent handle for the
group. I.e., starting from the previous config:

./tools/net/ynl/cli.py --spec $SPEC \
	--do group --json '{"ifindex": '$IFINDEX',
			"leaves": [
			  {"handle": {"scope": "queue", "id":'$QID1'},
			   "weight": '$W1'},
			  {"handle": {"scope": "queue", "id":'$QID3'},
			   "weight": '$W3'}],
			"handle": {"scope": "node"},
			"bw-max": 3000000 }'
Netlink error: Invalid argument
nl_len = 96 (80) nl_flags = 0x300 nl_type = 2
	error: -22
	extack: {'msg': 'All the leaves shapers must have the same old parent'}

./tools/net/ynl/cli.py --spec $SPEC \
	--do group --json '{"ifindex": '$IFINDEX',
			"leaves": [
			  {"handle": {"scope": "queue", "id":'$QID1'},
			   "weight": '$W1'},
			  {"handle": {"scope": "queue", "id":'$QID3'},
			   "weight": '$W3'}],
			"handle": {"scope": "node"},
			"parent": {"scope": "node", "id": 1},
			"bw-max": 3000000 }
{'ifindex': $IFINDEX, 'handle': {'scope': 'node', 'id': 2}}

Q1 -- node 2 ---
    /(bw-max:3M)\
Q3 /             \
         ---- node 1 \
        / (bw-max: 5M)\
      Q2              node 0 -------  netdev
                      (bw-max: 10M)

* Cleanup:

Still starting from config 1To delete a single queue shaper

./tools/net/ynl/cli.py --spec $SPEC --do delete --json \
	'{"ifindex": '$IFINDEX',
	  "handle": {"scope": "queue", "id":'$QID3'}}'

Q1 -- node 2 ---
     (bw-max:3M)\
                 \
         ---- node 1 \
        / (bw-max: 5M)\
      Q2              node 0 -------  netdev
                      (bw-max: 10M)

Deleting a node shaper relinks all its leaves to the node's parent:

./tools/net/ynl/cli.py --spec $SPEC --do delete --json \
	'{"ifindex": '$IFINDEX',
	  "handle": {"scope": "node", "id":2}}'

Q1 ---\
       \
        node 1----- \
       / (bw-max: 5M)\
Q2----/              node 0 -------  netdev
                     (bw-max: 10M)

Deleting the last shaper under a node shaper deletes the node, too:

./tools/net/ynl/cli.py --spec $SPEC --do delete --json \
	'{"ifindex": '$IFINDEX',
	  "handle": {"scope": "queue", "id":'$QID1'}}'
./tools/net/ynl/cli.py --spec $SPEC --do delete --json \
	'{"ifindex": '$IFINDEX',
	  "handle": {"scope": "queue", "id":'$QID2'}}'
./tools/net/ynl/cli.py --spec $SPEC --do get --json \
	'{"ifindex": '$IFINDEX',
	  "handle": {"scope": "node", "id": 1}}'
Netlink error: No such file or directory
nl_len = 44 (28) nl_flags = 0x300 nl_type = 2
	error: -2
	extack: {'bad-attr': '.handle'}

Such delete recurses on parents that are left over with no leaves:

./tools/net/ynl/cli.py --spec $SPEC --do get --json \
	'{"ifindex": '$IFINDEX',
	  "handle": {"scope": "node", "id": 0}}'
Netlink error: No such file or directory
nl_len = 44 (28) nl_flags = 0x300 nl_type = 2
	error: -2
	extack: {'bad-attr': '.handle'}

v8: https://lore.kernel.org/cover.1727704215.git.pabeni@redhat.com
v7: https://lore.kernel.org/cover.1725919039.git.pabeni@redhat.com
v6: https://lore.kernel.org/cover.1725457317.git.pabeni@redhat.com
v5: https://lore.kernel.org/cover.1724944116.git.pabeni@redhat.com
v4: https://lore.kernel.org/cover.1724165948.git.pabeni@redhat.com
v3: https://lore.kernel.org/cover.1722357745.git.pabeni@redhat.com
RFC v2: https://lore.kernel.org/cover.1721851988.git.pabeni@redhat.com
RFC v1: https://lore.kernel.org/cover.1719518113.git.pabeni@redhat.com
====================

Link: https://patch.msgid.link/cover.1728460186.git.pabeni@redhat.com


Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
parents 16aef666 4c1a457c
Loading
Loading
Loading
Loading
+362 −0
Original line number Diff line number Diff line
# SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)
name: net-shaper

doc: |
  Networking HW rate limiting configuration.

  This API allows configuring HW shapers available on the network
  devices at different levels (queues, network device) and allows
  arbitrary manipulation of the scheduling tree of the involved
  shapers.

  Each @shaper is identified within the given device, by a @handle,
  comprising both a @scope and an @id.

  Depending on the @scope value, the shapers are attached to specific
  HW objects (queues, devices) or, for @node scope, represent a
  scheduling group, that can be placed in an arbitrary location of
  the scheduling tree.

  Shapers can be created with two different operations: the @set
  operation, to create and update a single "attached" shaper, and
  the @group operation, to create and update a scheduling
  group. Only the @group operation can create @node scope shapers.

  Existing shapers can be deleted/reset via the @delete operation.

  The user can query the running configuration via the @get operation.

  Different devices can provide different feature sets, e.g. with no
  support for complex scheduling hierarchy, or for some shaping
  parameters. The user can introspect the HW capabilities via the
  @cap-get operation.

definitions:
  -
    type: enum
    name: scope
    doc: Defines the shaper @id interpretation.
    render-max: true
    entries:
      - name: unspec
        doc: The scope is not specified.
      -
        name: netdev
        doc: The main shaper for the given network device.
      -
        name: queue
        doc: |
            The shaper is attached to the given device queue,
            the @id represents the queue number.
      -
        name: node
        doc: |
             The shaper allows grouping of queues or other
             node shapers; can be nested in either @netdev
             shapers or other @node shapers, allowing placement
             in any location of the scheduling tree, except
             leaves and root.
  -
    type: enum
    name: metric
    doc: Different metric supported by the shaper.
    entries:
      -
        name: bps
        doc: Shaper operates on a bits per second basis.
      -
        name: pps
        doc: Shaper operates on a packets per second basis.

attribute-sets:
  -
    name: net-shaper
    attributes:
      -
        name: handle
        type: nest
        nested-attributes: handle
        doc: Unique identifier for the given shaper inside the owning device.
      -
        name: metric
        type: u32
        enum: metric
        doc: Metric used by the given shaper for bw-min, bw-max and burst.
      -
        name: bw-min
        type: uint
        doc: Guaranteed bandwidth for the given shaper.
      -
        name: bw-max
        type: uint
        doc: Maximum bandwidth for the given shaper or 0 when unlimited.
      -
        name: burst
        type: uint
        doc: |
          Maximum burst-size for shaping. Should not be interpreted
          as a quantum.
      -
        name: priority
        type: u32
        doc: |
          Scheduling priority for the given shaper. The priority
          scheduling is applied to sibling shapers.
      -
        name: weight
        type: u32
        doc: |
          Relative weight for round robin scheduling of the
          given shaper.
          The scheduling is applied to all sibling shapers
          with the same priority.
      -
        name: ifindex
        type: u32
        doc: Interface index owning the specified shaper.
      -
        name: parent
        type: nest
        nested-attributes: handle
        doc: |
          Identifier for the parent of the affected shaper.
          Only needed for @group operation.
      -
        name: leaves
        type: nest
        multi-attr: true
        nested-attributes: leaf-info
        doc: |
           Describes a set of leaves shapers for a @group operation.
  -
    name: handle
    attributes:
      -
        name: scope
        type: u32
        enum: scope
        doc: Defines the shaper @id interpretation.
      -
        name: id
        type: u32
        doc: |
          Numeric identifier of a shaper. The id semantic depends on
          the scope. For @queue scope it's the queue id and for @node
          scope it's the node identifier.
  -
    name: leaf-info
    subset-of: net-shaper
    attributes:
      -
        name: handle
      -
        name: priority
      -
        name: weight
  -
    name: caps
    attributes:
      -
        name: ifindex
        type: u32
        doc: Interface index queried for shapers capabilities.
      -
        name: scope
        type: u32
        enum: scope
        doc: The scope to which the queried capabilities apply.
      -
        name: support-metric-bps
        type: flag
        doc: The device accepts 'bps' metric for bw-min, bw-max and burst.
      -
        name: support-metric-pps
        type: flag
        doc: The device accepts 'pps' metric for bw-min, bw-max and burst.
      -
        name: support-nesting
        type: flag
        doc: |
          The device supports nesting shaper belonging to this scope
          below 'node' scoped shapers. Only 'queue' and 'node'
          scope can have flag 'support-nesting'.
      -
        name: support-bw-min
        type: flag
        doc: The device supports a minimum guaranteed B/W.
      -
        name: support-bw-max
        type: flag
        doc: The device supports maximum B/W shaping.
      -
        name: support-burst
        type: flag
        doc: The device supports a maximum burst size.
      -
        name: support-priority
        type: flag
        doc: The device supports priority scheduling.
      -
        name: support-weight
        type: flag
        doc: The device supports weighted round robin scheduling.

operations:
  list:
    -
      name: get
      doc: |
        Get information about a shaper for a given device.
      attribute-set: net-shaper

      do:
        pre: net-shaper-nl-pre-doit
        post: net-shaper-nl-post-doit
        request:
          attributes: &ns-binding
            - ifindex
            - handle
        reply:
          attributes: &ns-attrs
            - ifindex
            - parent
            - handle
            - metric
            - bw-min
            - bw-max
            - burst
            - priority
            - weight

      dump:
        pre: net-shaper-nl-pre-dumpit
        post: net-shaper-nl-post-dumpit
        request:
          attributes:
            - ifindex
        reply:
          attributes: *ns-attrs
    -
      name: set
      doc: |
        Create or update the specified shaper.
        The set operation can't be used to create a @node scope shaper,
        use the @group operation instead.
      attribute-set: net-shaper
      flags: [ admin-perm ]

      do:
        pre: net-shaper-nl-pre-doit
        post: net-shaper-nl-post-doit
        request:
          attributes:
            - ifindex
            - handle
            - metric
            - bw-min
            - bw-max
            - burst
            - priority
            - weight

    -
      name: delete
      doc: |
        Clear (remove) the specified shaper. When deleting
        a @node shaper, reattach all the node's leaves to the
        deleted node's parent.
        If, after the removal, the parent shaper has no more
        leaves and the parent shaper scope is @node, the parent
        node is deleted, recursively.
        When deleting a @queue shaper or a @netdev shaper,
        the shaper disappears from the hierarchy, but the
        queue/device can still send traffic: it has an implicit
        node with infinite bandwidth. The queue's implicit node
        feeds an implicit RR node at the root of the hierarchy.
      attribute-set: net-shaper
      flags: [ admin-perm ]

      do:
        pre: net-shaper-nl-pre-doit
        post: net-shaper-nl-post-doit
        request:
          attributes: *ns-binding

    -
      name: group
      doc: |
        Create or update a scheduling group, attaching the specified
        @leaves shapers under the specified node identified by @handle.
        The @leaves shapers scope must be @queue and the node shaper
        scope must be either @node or @netdev.
        When the node shaper has @node scope, if the @handle @id is not
        specified, a new shaper of such scope is created, otherwise the
        specified node must already exist.
        When updating an existing node shaper, the specified @leaves are
        added to the existing node; such node will also retain any preexisting
        leave.
        The @parent handle for a new node shaper defaults to the parent
        of all the leaves, provided all the leaves share the same parent.
        Otherwise @parent handle must be specified.
        The user can optionally provide shaping attributes for the node
        shaper.
        The operation is atomic, on failure no change is applied to
        the device shaping configuration, otherwise the @node shaper
        full identifier, comprising @binding and @handle, is provided
        as the reply.
      attribute-set: net-shaper
      flags: [ admin-perm ]

      do:
        pre: net-shaper-nl-pre-doit
        post: net-shaper-nl-post-doit
        request:
          attributes:
            - ifindex
            - parent
            - handle
            - metric
            - bw-min
            - bw-max
            - burst
            - priority
            - weight
            - leaves
        reply:
          attributes: *ns-binding

    -
      name: cap-get
      doc: |
        Get the shaper capabilities supported by the given device
        for the specified scope.
      attribute-set: caps

      do:
        pre: net-shaper-nl-cap-pre-doit
        post: net-shaper-nl-cap-post-doit
        request:
          attributes:
            - ifindex
            - scope
        reply:
          attributes: &cap-attrs
            - ifindex
            - scope
            - support-metric-bps
            - support-metric-pps
            - support-nesting
            - support-bw-min
            - support-bw-max
            - support-burst
            - support-priority
            - support-weight

      dump:
        pre: net-shaper-nl-cap-pre-dumpit
        post: net-shaper-nl-cap-post-dumpit
        request:
          attributes:
            - ifindex
        reply:
          attributes: *cap-attrs
+3 −0
Original line number Diff line number Diff line
@@ -104,6 +104,9 @@ Driver Support
.. kernel-doc:: include/linux/netdevice.h
   :internal:

.. kernel-doc:: include/net/net_shaper.h
   :internal:

PHY Support
-----------

+1 −0
Original line number Diff line number Diff line
@@ -16116,6 +16116,7 @@ F: include/linux/platform_data/wiznet.h
F:	include/uapi/linux/cn_proc.h
F:	include/uapi/linux/ethtool_netlink.h
F:	include/uapi/linux/if_*
F:	include/uapi/linux/net_shaper.h
F:	include/uapi/linux/netdev*
F:	tools/testing/selftests/drivers/net/
X:	Documentation/devicetree/bindings/net/bluetooth/
+1 −0
Original line number Diff line number Diff line
@@ -641,6 +641,7 @@ config NETDEVSIM
	depends on PTP_1588_CLOCK_MOCK || PTP_1588_CLOCK_MOCK=n
	select NET_DEVLINK
	select PAGE_POOL
	select NET_SHAPER
	help
	  This driver is a developer testing tool and software model that can
	  be used to test various control path networking APIs, especially
+1 −0
Original line number Diff line number Diff line
@@ -258,6 +258,7 @@ config I40E_DCB
config IAVF
	tristate
	select LIBIE
	select NET_SHAPER

config I40EVF
	tristate "Intel(R) Ethernet Adaptive Virtual Function support"
Loading