Commit 9fb3dc1e authored by David S. Miller's avatar David S. Miller
Browse files

Merge branch 'phy-listing-link_topology-tracking' 



Maxime Chevallier says:

====================
Introduce PHY listing and link_topology tracking

Here's a V5 of the multi-PHY support series.

At a glance, besides some minor fixes and R'd-by from Andrew, one of the
thing this series does is remove the ASSERT_RTNL() from the
topo_add_phy/del_phy operations.

These operations will take a PHY device and put it into the list of
devices associated to a netdevice. The main thing to protect here is the
list itself, but since we use xarrays, my naive understanding of it is
that it contains its own protection scheme. There shouldn't be a need
for more locking, as the insertion/deletion paths are already hooked
into the PHY connection to a netdev, or disconnection from it.

Now for the rest of the cover :

As a remainder, this ongoing work aims ultimately at supporting complex
link topologies that involve multiplexing multiple PHYs/SFPs on a single
netdevice. As a first step, it's required that we are able to enumerate the
PHYs on a given ethernet interface.

By just doing so, we also improve already-existing use-cases, namely the
copper SFP modules support when a media-converter is used (as we have 2
PHYs on the link, but only one is referenced by net_device.phydev, which
is used on a variety of netlink commands).

The series is architectured as follows :

- The first patch adds the notion of phy_link_topology, which tracks
all PHYs attached to a netdevice.

- Patches 2, 3 and 4 adds some plumbing into SFP and phylib to be able
  to connect the dots when building the topology tree, to know which PHY
  is connected to which SFP bus, trying not to be too invasive on phylib.

- Patch 5 allows passing a PHY_INDEX to ethnl commands. I'm uncertain about
  this, as there are at least 4 netlink commands ( 5 with the one introduced
  in patch 7 ) that targets PHYs directly or indirectly, which to me makes
  it worth-it to have a generic way to pass a PHY index to commands, however
  the approach taken may be too generic.

- Patch 6 is the netlink spec update + ethtool-user.c|h autogenerated code
update (the autogenerated code triggers checkpatch warning though)

- Patch 7 introduces a new netlink command set to list PHYs on a netdevice.
It implements a custom DUMP and GET operation to allow filtered dumps,
that lists all PHYs on a given netdevice. I couldn't use most of ethnl's
plumbing though.

- Patch 8 is the netlink spec update + ethtool-user.c|h update for that
new command

- Patch 8,9,10 and 11 updates the PLCA, strset, cable-test and pse netlink
commands to use the user-provided PHY instead of net_device.phydev.

- Finally patch 12 adds some documentation for this whole work.

Examples
========

Here's a short overview of the kind of operations you can have regarding
the PHY topology. These tests were performed on a MacchiatoBin, which
has 3 interfaces :

eth0 and eth1 have the following layout:

MAC - PHY - SFP

eth2 has this more classic topology :

MAC - PHY - RJ45

finally eth3 has the following topology :

MAC - SFP

When performing a dump with all interfaces down, we don't get any
result, as no PHY has been attached to their respective net_device :

None

The following output is with eth0, eth2 and eth3 up, but no SFP module
inserted in none of the interfaces :

[{'downstream-sfp-name': 'sfp-eth0',
  'drvname': 'mv88x3310',
  'header': {'dev-index': 2, 'dev-name': 'eth0'},
  'id': 0,
  'index': 1,
  'name': 'f212a600.mdio-mii:00',
  'upstream-type': 'mac'},
 {'drvname': 'Marvell 88E1510',
  'header': {'dev-index': 4, 'dev-name': 'eth2'},
  'id': 21040593,
  'index': 1,
  'name': 'f212a200.mdio-mii:00',
  'upstream-type': 'mac'}]

And now is a dump operation with a copper SFP in the eth0 port :

[{'downstream-sfp-name': 'sfp-eth0',
  'drvname': 'mv88x3310',
  'header': {'dev-index': 2, 'dev-name': 'eth0'},
  'id': 0,
  'index': 1,
  'name': 'f212a600.mdio-mii:00',
  'upstream-type': 'mac'},
 {'drvname': 'Marvell 88E1111',
  'header': {'dev-index': 2, 'dev-name': 'eth0'},
  'id': 21040322,
  'index': 2,
  'name': 'i2c:sfp-eth0:16',
  'upstream': {'index': 1, 'sfp-name': 'sfp-eth0'},
  'upstream-type': 'phy'},
 {'drvname': 'Marvell 88E1510',
  'header': {'dev-index': 4, 'dev-name': 'eth2'},
  'id': 21040593,
  'index': 1,
  'name': 'f212a200.mdio-mii:00',
  'upstream-type': 'mac'}]

 -- Note that this shouldn't actually work as the 88x3310 PHY doesn't allow
a 1G SFP to be connected to its SFP interface, and I don't have a 10G copper SFP,
so for the sake of the demo I applied the following modification, which
of courses gives a non-functionnal link, but the PHY attach still works,
which is what I want to demonstrate :

@@ -488,7 +488,7 @@ static int mv3310_sfp_insert(void *upstream, const struct sfp_eeprom_id *id)

        if (iface != PHY_INTERFACE_MODE_10GBASER) {
                dev_err(&phydev->mdio.dev, "incompatible SFP module inserted\n");
-               return -EINVAL;
+               //return -EINVAL;
        }
        return 0;
 }

Finally an example of the filtered DUMP operation that Jakub suggested
in V1 :

[{'downstream-sfp-name': 'sfp-eth0',
  'drvname': 'mv88x3310',
  'header': {'dev-index': 2, 'dev-name': 'eth0'},
  'id': 0,
  'index': 1,
  'name': 'f212a600.mdio-mii:00',
  'upstream-type': 'mac'},
 {'drvname': 'Marvell 88E1111',
  'header': {'dev-index': 2, 'dev-name': 'eth0'},
  'id': 21040322,
  'index': 2,
  'name': 'i2c:sfp-eth0:16',
  'upstream': {'index': 1, 'sfp-name': 'sfp-eth0'},
  'upstream-type': 'phy'}]

And a classic GET operation allows querying a single PHY's info :

{'drvname': 'Marvell 88E1111',
 'header': {'dev-index': 2, 'dev-name': 'eth0'},
 'id': 21040322,
 'index': 2,
 'name': 'i2c:sfp-eth0:16',
 'upstream': {'index': 1, 'sfp-name': 'sfp-eth0'},
 'upstream-type': 'phy'}

Changed in V5:
- Removed the RTNL assertion in the topology ops
- Made the phy_topo_get_phy inline
- Fixed the PSE-PD multi-PHY support by re-adding a wrongly dropped
  check
- Fixed some typos in the documentation
- Fixed reverse xmas trees

Changes in V4:
- Dropped the RFC flag
- Made the net_device integration independent to having phylib enabled
- Removed the autogenerated ethtool-user code for the YNL specs

Changes in V3:
- Added RTNL assertions where needed
- Fixed issues in the DUMP code for PHY_GET, which crashed when running it
  twice in a row
- Added the documentation, and moved in-source docs around
- renamed link_topology to phy_link_topology

Changes in V2:
- Added the DUMP operation
- Added much more information in the reported data, to be able to reconstruct
  precisely the topology tree
- renamed phy_list to link_topology
====================

Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
parents 109bf4cf 32bb4515

Documentation/netlink/specs/ethtool.yaml

+68 −0
Original line number Diff line number Diff line
@@ -16,6 +16,11 @@ definitions: 
    name: stringset

    type: enum

    entries: []

  -

    name: phy-upstream-type

    enum-name:

    type: enum

    entries: [ mac, phy ]



attribute-sets:

  -
@@ -30,6 +35,9 @@ attribute-sets: 
      -

        name: flags

        type: u32

      -

        name: phy-index

        type: u32



  -

    name: bitset-bit
@@ -942,6 +950,45 @@ attribute-sets: 
      -

        name: burst-tmr

        type: u32

  -

    name: phy-upstream

    attributes:

      -

        name: index

        type: u32

      -

        name: sfp-name

        type: string

  -

    name: phy

    attributes:

      -

        name: header

        type: nest

        nested-attributes: header

      -

        name: index

        type: u32

      -

        name: drvname

        type: string

      -

        name: name

        type: string

      -

        name: upstream-type

        type: u8

        enum: phy-upstream-type

      -

        name: upstream

        type: nest

        nested-attributes: phy-upstream

      -

        name: downstream-sfp-name

        type: string

      -

        name: id

        type: u32



operations:

  enum-model: directional
@@ -1693,3 +1740,24 @@ operations: 
      name: mm-ntf

      doc: Notification for change in MAC Merge configuration.

      notify: mm-get

    -

      name: phy-get

      doc: Get PHY devices attached to an interface



      attribute-set: phy



      do: &phy-get-op

        request:

          attributes:

            - header

        reply:

          attributes:

            - header

            - index

            - drvname

            - name

            - upstream-type

            - upstream

            - downstream-sfp-name

            - id

      dump: *phy-get-op

Documentation/networking/ethtool-netlink.rst

+51 −0
Original line number Diff line number Diff line
@@ -57,6 +57,7 @@ Structure of this header is 
  ``ETHTOOL_A_HEADER_DEV_INDEX``  u32     device ifindex

  ``ETHTOOL_A_HEADER_DEV_NAME``   string  device name

  ``ETHTOOL_A_HEADER_FLAGS``      u32     flags common for all requests

  ``ETHTOOL_A_HEADER_PHY_INDEX``  u32     phy device index

  ==============================  ======  =============================



``ETHTOOL_A_HEADER_DEV_INDEX`` and ``ETHTOOL_A_HEADER_DEV_NAME`` identify the
@@ -81,6 +82,12 @@ the behaviour is backward compatible, i.e. requests from old clients not aware 
of the flag should be interpreted the way the client expects. A client must

not set flags it does not understand.



``ETHTOOL_A_HEADER_PHY_INDEX`` identify the ethernet PHY the message relates to.

As there are numerous commands that are related to PHY configuration, and because

we can have more than one PHY on the link, the PHY index can be passed in the

request for the commands that needs it. It is however not mandatory, and if it

is not passed for commands that target a PHY, the net_device.phydev pointer

is used, as a fallback that keeps the legacy behaviour.



Bit sets

========
@@ -2004,6 +2011,49 @@ The attributes are propagated to the driver through the following structure: 
.. kernel-doc:: include/linux/ethtool.h

    :identifiers: ethtool_mm_cfg



PHY_GET

=======



Retrieve information about a given Ethernet PHY sitting on the link. As there

can be more than one PHY, the DUMP operation can be used to list the PHYs

present on a given interface, by passing an interface index or name in

the dump request



Request contents:



  ====================================  ======  ==========================

  ``ETHTOOL_A_PHY_HEADER``              nested  request header

  ====================================  ======  ==========================



Kernel response contents:



  ===================================== ======  ==========================

  ``ETHTOOL_A_PHY_HEADER``              nested  request header

  ``ETHTOOL_A_PHY_INDEX``               u32     the phy's unique index, that can

                                                be used for phy-specific requests

  ``ETHTOOL_A_PHY_DRVNAME``             string  the phy driver name

  ``ETHTOOL_A_PHY_NAME``                string  the phy device name

  ``ETHTOOL_A_PHY_UPSTREAM_TYPE``       u32     the type of device this phy is

                                                connected to

  ``ETHTOOL_A_PHY_UPSTREAM_PHY``        nested  if the phy is connected to another

                                                phy, this nest contains info on

                                                that connection

  ``ETHTOOL_A_PHY_DOWNSTREAM_SFP_NAME`` string  if the phy controls an sfp bus,

                                                the name of the sfp bus

  ``ETHTOOL_A_PHY_ID``                  u32     the phy id if the phy is C22

  ===================================== ======  ==========================



When ``ETHTOOL_A_PHY_UPSTREAM_TYPE`` is PHY_UPSTREAM_PHY, the PHY's parent is

another PHY. Information on the parent PHY will be set in the

``ETHTOOL_A_PHY_UPSTREAM_PHY`` nest, which has the following structure :



  =================================== ======  ==========================

  ``ETHTOOL_A_PHY_UPSTREAM_INDEX``    u32     the PHY index of the upstream PHY

  ``ETHTOOL_A_PHY_UPSTREAM_SFP_NAME`` string  if this PHY is connected to it's

                                                parent PHY through an SFP bus, the

                                                name of this sfp bus

  =================================== ======  ==========================



Request translation

===================
@@ -2110,4 +2160,5 @@ are netlink only. 
  n/a                                 ``ETHTOOL_MSG_PLCA_GET_STATUS``

  n/a                                 ``ETHTOOL_MSG_MM_GET``

  n/a                                 ``ETHTOOL_MSG_MM_SET``

  n/a                                 ``ETHTOOL_MSG_PHY_GET``

  =================================== =====================================

Documentation/networking/index.rst

+1 −0
Original line number Diff line number Diff line
@@ -88,6 +88,7 @@ Contents: 
   operstates

   packet_mmap

   phonet

   phy-link-topology

   pktgen

   plip

   ppp_generic

Documentation/networking/phy-link-topology.rst

0 → 100644
+121 −0
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: GPL-2.0



=================

PHY link topology

=================



Overview

========



The PHY link topology representation in the networking stack aims at representing

the hardware layout for any given Ethernet link.



An Ethernet Interface from userspace's point of view is nothing but a

:c:type:`struct net_device <net_device>`, which exposes configuration options

through the legacy ioctls and the ethool netlink commands. The base assumption

when designing these configuration channels were that the link looked

something like this ::



  +-----------------------+        +----------+      +--------------+

  | Ethernet Controller / |        | Ethernet |      | Connector /  |

  |       MAC             | ------ |   PHY    | ---- |    Port      | ---... to LP

  +-----------------------+        +----------+      +--------------+

  struct net_device               struct phy_device



Commands that needs to configure the PHY will go through the net_device.phydev

field to reach the PHY and perform the relevant configuration.



This assumption falls apart in more complex topologies that can arise when,

for example, using SFP transceivers (although that's not the only specific case).



Here, we have 2 basic scenarios. Either the MAC is able to output a serialized

interface, that can directly be fed to an SFP cage, such as SGMII, 1000BaseX,

10GBaseR, etc.



The link topology then looks like this (when an SFP module is inserted) ::



  +-----+  SGMII  +------------+

  | MAC | ------- | SFP Module |

  +-----+         +------------+



Knowing that some modules embed a PHY, the actual link is more like ::



  +-----+  SGMII   +--------------+

  | MAC | -------- | PHY (on SFP) |

  +-----+          +--------------+



In this case, the SFP PHY is handled by phylib, and registered by phylink through

its SFP upstream ops.



Now some Ethernet controllers aren't able to output a serialized interface, so

we can't directly connect them to an SFP cage. However, some PHYs can be used

as media-converters, to translate the non-serialized MAC MII interface to a

serialized MII interface fed to the SFP ::



  +-----+  RGMII  +-----------------------+  SGMII  +--------------+

  | MAC | ------- | PHY (media converter) | ------- | PHY (on SFP) |

  +-----+         +-----------------------+         +--------------+



This is where the model of having a single net_device.phydev pointer shows its

limitations, as we now have 2 PHYs on the link.



The phy_link topology framework aims at providing a way to keep track of every

PHY on the link, for use by both kernel drivers and subsystems, but also to

report the topology to userspace, allowing to target individual PHYs in configuration

commands.



API

===



The :c:type:`struct phy_link_topology <phy_link_topology>` is a per-netdevice

resource, that gets initialized at netdevice creation. Once it's initialized,

it is then possible to register PHYs to the topology through :



:c:func:`phy_link_topo_add_phy`



Besides registering the PHY to the topology, this call will also assign a unique

index to the PHY, which can then be reported to userspace to refer to this PHY

(akin to the ifindex). This index is a u32, ranging from 1 to U32_MAX. The value

0 is reserved to indicate the PHY doesn't belong to any topology yet.



The PHY can then be removed from the topology through



:c:func:`phy_link_topo_del_phy`



These function are already hooked into the phylib subsystem, so all PHYs that

are linked to a net_device through :c:func:`phy_attach_direct` will automatically

join the netdev's topology.



PHYs that are on a SFP module will also be automatically registered IF the SFP

upstream is phylink (so, no media-converter).



PHY drivers that can be used as SFP upstream need to call :c:func:`phy_sfp_attach_phy`

and :c:func:`phy_sfp_detach_phy`, which can be used as a

.attach_phy / .detach_phy implementation for the

:c:type:`struct sfp_upstream_ops <sfp_upstream_ops>`.



UAPI

====



There exist a set of netlink commands to query the link topology from userspace,

see ``Documentation/networking/ethtool-netlink.rst``.



The whole point of having a topology representation is to assign the phyindex

field in :c:type:`struct phy_device <phy_device>`. This index is reported to

userspace using the ``ETHTOOL_MSG_PHY_GET`` ethtnl command. Performing a DUMP operation

will result in all PHYs from all net_device being listed. The DUMP command

accepts either a ``ETHTOOL_A_HEADER_DEV_INDEX`` or ``ETHTOOL_A_HEADER_DEV_NAME``

to be passed in the request to filter the DUMP to a single net_device.



The retrieved index can then be passed as a request parameter using the

``ETHTOOL_A_HEADER_PHY_INDEX`` field in the following ethnl commands :



* ``ETHTOOL_MSG_STRSET_GET`` to get the stats string set from a given PHY

* ``ETHTOOL_MSG_CABLE_TEST_ACT`` and ``ETHTOOL_MSG_CABLE_TEST_ACT``, to perform

  cable testing on a given PHY on the link (most likely the outermost PHY)

* ``ETHTOOL_MSG_PSE_SET`` and ``ETHTOOL_MSG_PSE_GET`` for PHY-controlled PoE and PSE settings

* ``ETHTOOL_MSG_PLCA_GET_CFG``, ``ETHTOOL_MSG_PLCA_SET_CFG`` and ``ETHTOOL_MSG_PLCA_GET_STATUS``

  to set the PLCA (Physical Layer Collision Avoidance) parameters



Note that the PHY index can be passed to other requests, which will silently

ignore it if present and irrelevant.

MAINTAINERS

+2 −0
Original line number Diff line number Diff line
@@ -7871,6 +7871,8 @@ F: include/linux/mii.h 
F:	include/linux/of_net.h

F:	include/linux/phy.h

F:	include/linux/phy_fixed.h

F:	include/linux/phy_link_topology.h

F:	include/linux/phy_link_topology_core.h

F:	include/linux/phylib_stubs.h

F:	include/linux/platform_data/mdio-bcm-unimac.h

F:	include/linux/platform_data/mdio-gpio.h