Merge branch 'phc-support-in-ena-driver' (a5b4e4f4) · Commits · git / linux-net

Documentation/networking/device_drivers/ethernet/amazon/ena.rst

+108 −0

Original line number	Diff line number	Diff line
		@@ -56,6 +56,9 @@ ena_netdev.[ch] Main Linux kernel driver.
		ena_ethtool.c ethtool callbacks.
		ena_xdp.[ch] XDP files
		ena_pci_id_tbl.h Supported device IDs.
		ena_phc.[ch] PTP hardware clock infrastructure (see `PHC`_ for more info)
		ena_devlink.[ch] devlink files.
		ena_debugfs.[ch] debugfs files.
		================= ======================================================

		Management Interface:
		@@ -221,6 +224,99 @@ descriptor it was received on would be recycled. When a packet smaller
		than RX copybreak bytes is received, it is copied into a new memory
		buffer and the RX descriptor is returned to HW.

		.. _`PHC`:

		PTP Hardware Clock (PHC)
		========================
		.. _`ptp-userspace-api`: https://docs.kernel.org/driver-api/ptp.html#ptp-hardware-clock-user-space-api
		.. _`testptp`: https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests/ptp/testptp.c

		ENA Linux driver supports PTP hardware clock providing timestamp reference to achieve nanosecond resolution.

		PHC support

		PHC depends on the PTP module, which needs to be either loaded as a module or compiled into the kernel.

		Verify if the PTP module is present:

		.. code-block:: shell

		grep -w '^CONFIG_PTP_1588_CLOCK=[ym]' /boot/config-`uname -r`

		- If no output is provided, the ENA driver cannot be loaded with PHC support.

		PHC activation

		The feature is turned off by default, in order to turn the feature on, the ENA driver
		can be loaded in the following way:

		- devlink:

		.. code-block:: shell

		sudo devlink dev param set pci/<domain:bus:slot.function> name enable_phc value true cmode driverinit
		sudo devlink dev reload pci/<domain:bus:slot.function>
		# for example:
		sudo devlink dev param set pci/0000:00:06.0 name enable_phc value true cmode driverinit
		sudo devlink dev reload pci/0000:00:06.0

		All available PTP clock sources can be tracked here:

		.. code-block:: shell

		ls /sys/class/ptp

		PHC support and capabilities can be verified using ethtool:

		.. code-block:: shell

		ethtool -T <interface>

		PHC timestamp

		To retrieve PHC timestamp, use `ptp-userspace-api`_, usage example using `testptp`_:

		.. code-block:: shell

		testptp -d /dev/ptp$(ethtool -T <interface> \| awk '/PTP Hardware Clock:/ {print $NF}') -k 1

		PHC get time requests should be within reasonable bounds,
		avoid excessive utilization to ensure optimal performance and efficiency.
		The ENA device restricts the frequency of PHC get time requests to a maximum
		of 125 requests per second. If this limit is surpassed, the get time request
		will fail, leading to an increment in the phc_err_ts statistic.

		PHC statistics

		PHC can be monitored using debugfs (if mounted):

		.. code-block:: shell

		sudo cat /sys/kernel/debug/<domain:bus:slot.function>/phc_stats

		# for example:
		sudo cat /sys/kernel/debug/0000:00:06.0/phc_stats

		PHC errors must remain below 1% of all PHC requests to maintain the desired level of accuracy and reliability

		================= ======================================================
		phc_cnt \| Number of successful retrieved timestamps (below expire timeout).
		phc_exp \| Number of expired retrieved timestamps (above expire timeout).
		phc_skp \| Number of skipped get time attempts (during block period).
		phc_err_dv \| Number of failed get time attempts due to device errors (entering into block state).
		phc_err_ts \| Number of failed get time attempts due to timestamp errors (entering into block state),
		\| This occurs if driver exceeded the request limit or device received an invalid timestamp.
		================= ======================================================

		PHC timeouts:

		================= ======================================================
		expire \| Max time for a valid timestamp retrieval, passing this threshold will fail
		\| the get time request and block new requests until block timeout.
		block \| Blocking period starts once get time request expires or fails,
		\| all get time requests during block period will be skipped.
		================= ======================================================

		Statistics
		==========

		@@ -268,6 +364,18 @@ RSS
		- The user can provide a hash key, hash function, and configure the
		indirection table through `ethtool(8)`.

		DEVLINK SUPPORT
		===============
		.. _`devlink`: https://www.kernel.org/doc/html/latest/networking/devlink/index.html

		`devlink`_ supports reloading the driver and initiating re-negotiation with the ENA device

		.. code-block:: shell

		sudo devlink dev reload pci/<domain:bus:slot.function>
		# for example:
		sudo devlink dev reload pci/0000:00:06.0

		DATA PATH
		=========

Documentation/networking/devlink/devlink-params.rst

+3 −0

Original line number	Diff line number	Diff line
		@@ -137,3 +137,6 @@ own name.
		* - ``event_eq_size``
		- u32
		- Control the size of asynchronous control events EQ.
		* - ``enable_phc``
		- Boolean
		- Enable PHC (PTP Hardware Clock) functionality in the device.

drivers/net/ethernet/amazon/Kconfig

+2 −0

Original line number	Diff line number	Diff line
		@@ -19,7 +19,9 @@ if NET_VENDOR_AMAZON
		config ENA_ETHERNET
		tristate "Elastic Network Adapter (ENA) support"
		depends on PCI_MSI && !CPU_BIG_ENDIAN
		depends on PTP_1588_CLOCK_OPTIONAL
		select DIMLIB
		select NET_DEVLINK
		help
		This driver supports Elastic Network Adapter (ENA)"

drivers/net/ethernet/amazon/ena/Makefile

+1 −1

Original line number	Diff line number	Diff line
		@@ -5,4 +5,4 @@

		obj-$(CONFIG_ENA_ETHERNET) += ena.o

		ena-y := ena_netdev.o ena_com.o ena_eth_com.o ena_ethtool.o ena_xdp.o
		ena-y := ena_netdev.o ena_com.o ena_eth_com.o ena_ethtool.o ena_xdp.o ena_phc.o ena_devlink.o ena_debugfs.o

drivers/net/ethernet/amazon/ena/ena_admin_defs.h

+73 −1

Original line number	Diff line number	Diff line
		@@ -60,6 +60,7 @@ enum ena_admin_aq_feature_id {
		ENA_ADMIN_AENQ_CONFIG = 26,
		ENA_ADMIN_LINK_CONFIG = 27,
		ENA_ADMIN_HOST_ATTR_CONFIG = 28,
		ENA_ADMIN_PHC_CONFIG = 29,
		ENA_ADMIN_FEATURES_OPCODE_NUM = 32,
		};

		@@ -127,6 +128,14 @@ enum ena_admin_get_stats_scope {
		ENA_ADMIN_ETH_TRAFFIC = 1,
		};

		enum ena_admin_phc_type {
		ENA_ADMIN_PHC_TYPE_READLESS = 0,
		};

		enum ena_admin_phc_error_flags {
		ENA_ADMIN_PHC_ERROR_FLAG_TIMESTAMP = BIT(0),
		};

		/* ENA SRD configuration for ENI */
		enum ena_admin_ena_srd_flags {
		/* Feature enabled */
		@@ -943,7 +952,9 @@ struct ena_admin_host_info {
		* 4 : rss_configurable_function_key
		* 5 : reserved
		* 6 : rx_page_reuse
		* 31:7 : reserved
		* 7 : reserved
		* 8 : phc
		* 31:9 : reserved
		*/
		u32 driver_supported_features;
		};
		@@ -1023,6 +1034,43 @@ struct ena_admin_queue_ext_feature_desc {
		};
		};

		struct ena_admin_feature_phc_desc {
		/* PHC type as defined in enum ena_admin_get_phc_type,
		* used only for GET command.
		*/
		u8 type;

		/* Reserved - MBZ */
		u8 reserved1[3];

		/* PHC doorbell address as an offset to PCIe MMIO REG BAR,
		* used only for GET command.
		*/
		u32 doorbell_offset;

		/* Max time for valid PHC retrieval, passing this threshold will
		* fail the get-time request and block PHC requests for
		* block_timeout_usec, used only for GET command.
		*/
		u32 expire_timeout_usec;

		/* PHC requests block period, blocking starts if PHC request expired
		* in order to prevent floods on busy device,
		* used only for GET command.
		*/
		u32 block_timeout_usec;

		/* Shared PHC physical address (ena_admin_phc_resp),
		* used only for SET command.
		*/
		struct ena_common_mem_addr output_address;

		/* Shared PHC Size (ena_admin_phc_resp),
		* used only for SET command.
		*/
		u32 output_length;
		};

		struct ena_admin_get_feat_resp {
		struct ena_admin_acq_common_desc acq_common_desc;

		@@ -1052,6 +1100,8 @@ struct ena_admin_get_feat_resp {
		struct ena_admin_feature_intr_moder_desc intr_moderation;

		struct ena_admin_ena_hw_hints hw_hints;

		struct ena_admin_feature_phc_desc phc;
		} u;
		};

		@@ -1085,6 +1135,9 @@ struct ena_admin_set_feat_cmd {

		/* LLQ configuration */
		struct ena_admin_feature_llq_desc llq;

		/* PHC configuration */
		struct ena_admin_feature_phc_desc phc;
		} u;
		};

		@@ -1162,6 +1215,23 @@ struct ena_admin_ena_mmio_req_read_less_resp {
		u32 reg_val;
		};

		struct ena_admin_phc_resp {
		/* Request Id, received from DB register */
		u16 req_id;

		u8 reserved1[6];

		/* PHC timestamp (nsec) */
		u64 timestamp;

		u8 reserved2[12];

		/* Bit field of enum ena_admin_phc_error_flags */
		u32 error_flags;

		u8 reserved3[32];
		};

		/* aq_common_desc */
		#define ENA_ADMIN_AQ_COMMON_DESC_COMMAND_ID_MASK GENMASK(11, 0)
		#define ENA_ADMIN_AQ_COMMON_DESC_PHASE_MASK BIT(0)
		@@ -1260,6 +1330,8 @@ struct ena_admin_ena_mmio_req_read_less_resp {
		#define ENA_ADMIN_HOST_INFO_RSS_CONFIGURABLE_FUNCTION_KEY_MASK BIT(4)
		#define ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_SHIFT 6
		#define ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_MASK BIT(6)
		#define ENA_ADMIN_HOST_INFO_PHC_SHIFT 8
		#define ENA_ADMIN_HOST_INFO_PHC_MASK BIT(8)

		/* aenq_common_desc */
		#define ENA_ADMIN_AENQ_COMMON_DESC_PHASE_MASK BIT(0)