Loading Documentation/block/ublk.rst +60 −4 Original line number Diff line number Diff line Loading @@ -260,9 +260,12 @@ The following IO commands are communicated via io_uring passthrough command, and each command is only for forwarding the IO and committing the result with specified IO tag in the command data: - ``UBLK_IO_FETCH_REQ`` Traditional Per-I/O Commands ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Sent from the server IO pthread for fetching future incoming IO requests - ``UBLK_U_IO_FETCH_REQ`` Sent from the server I/O pthread for fetching future incoming I/O requests destined to ``/dev/ublkb*``. This command is sent only once from the server IO pthread for ublk driver to setup IO forward environment. Loading @@ -278,7 +281,7 @@ with specified IO tag in the command data: supported by the driver, daemons must be per-queue instead - i.e. all I/Os associated to a single qid must be handled by the same task. - ``UBLK_IO_COMMIT_AND_FETCH_REQ`` - ``UBLK_U_IO_COMMIT_AND_FETCH_REQ`` When an IO request is destined to ``/dev/ublkb*``, the driver stores the IO's ``ublksrv_io_desc`` to the specified mapped area; then the Loading @@ -293,7 +296,7 @@ with specified IO tag in the command data: requests with the same IO tag. That is, ``UBLK_IO_COMMIT_AND_FETCH_REQ`` is reused for both fetching request and committing back IO result. - ``UBLK_IO_NEED_GET_DATA`` - ``UBLK_U_IO_NEED_GET_DATA`` With ``UBLK_F_NEED_GET_DATA`` enabled, the WRITE request will be firstly issued to ublk server without data copy. Then, IO backend of ublk server Loading Loading @@ -322,6 +325,59 @@ with specified IO tag in the command data: ``UBLK_IO_COMMIT_AND_FETCH_REQ`` to the server, ublkdrv needs to copy the server buffer (pages) read to the IO request pages. Batch I/O Commands (UBLK_F_BATCH_IO) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``UBLK_F_BATCH_IO`` feature provides an alternative high-performance I/O handling model that replaces the traditional per-I/O commands with per-queue batch commands. This significantly reduces communication overhead and enables better load balancing across multiple server tasks. Key differences from traditional mode: - **Per-queue vs Per-I/O**: Commands operate on queues rather than individual I/Os - **Batch processing**: Multiple I/Os are handled in single operations - **Multishot commands**: Use io_uring multishot for reduced submission overhead - **Flexible task assignment**: Any task can handle any I/O (no per-I/O daemons) - **Better load balancing**: Tasks can adjust their workload dynamically Batch I/O Commands: - ``UBLK_U_IO_PREP_IO_CMDS`` Prepares multiple I/O commands in batch. The server provides a buffer containing multiple I/O descriptors that will be processed together. This reduces the number of individual command submissions required. - ``UBLK_U_IO_COMMIT_IO_CMDS`` Commits results for multiple I/O operations in batch, and prepares the I/O descriptors to accept new requests. The server provides a buffer containing the results of multiple completed I/Os, allowing efficient bulk completion of requests. - ``UBLK_U_IO_FETCH_IO_CMDS`` **Multishot command** for fetching I/O commands in batch. This is the key command that enables high-performance batch processing: * Uses io_uring multishot capability for reduced submission overhead * Single command can fetch multiple I/O requests over time * Buffer size determines maximum batch size per operation * Multiple fetch commands can be submitted for load balancing * Only one fetch command is active at any time per queue * Supports dynamic load balancing across multiple server tasks It is one typical multishot io_uring request with provided buffer, and it won't be completed until any failure is triggered. Each task can submit ``UBLK_U_IO_FETCH_IO_CMDS`` with different buffer sizes to control how much work it handles. This enables sophisticated load balancing strategies in multi-threaded servers. Migration: Applications using traditional commands (``UBLK_U_IO_FETCH_REQ``, ``UBLK_U_IO_COMMIT_AND_FETCH_REQ``) cannot use batch mode simultaneously. Zero copy --------- Loading Loading
Documentation/block/ublk.rst +60 −4 Original line number Diff line number Diff line Loading @@ -260,9 +260,12 @@ The following IO commands are communicated via io_uring passthrough command, and each command is only for forwarding the IO and committing the result with specified IO tag in the command data: - ``UBLK_IO_FETCH_REQ`` Traditional Per-I/O Commands ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Sent from the server IO pthread for fetching future incoming IO requests - ``UBLK_U_IO_FETCH_REQ`` Sent from the server I/O pthread for fetching future incoming I/O requests destined to ``/dev/ublkb*``. This command is sent only once from the server IO pthread for ublk driver to setup IO forward environment. Loading @@ -278,7 +281,7 @@ with specified IO tag in the command data: supported by the driver, daemons must be per-queue instead - i.e. all I/Os associated to a single qid must be handled by the same task. - ``UBLK_IO_COMMIT_AND_FETCH_REQ`` - ``UBLK_U_IO_COMMIT_AND_FETCH_REQ`` When an IO request is destined to ``/dev/ublkb*``, the driver stores the IO's ``ublksrv_io_desc`` to the specified mapped area; then the Loading @@ -293,7 +296,7 @@ with specified IO tag in the command data: requests with the same IO tag. That is, ``UBLK_IO_COMMIT_AND_FETCH_REQ`` is reused for both fetching request and committing back IO result. - ``UBLK_IO_NEED_GET_DATA`` - ``UBLK_U_IO_NEED_GET_DATA`` With ``UBLK_F_NEED_GET_DATA`` enabled, the WRITE request will be firstly issued to ublk server without data copy. Then, IO backend of ublk server Loading Loading @@ -322,6 +325,59 @@ with specified IO tag in the command data: ``UBLK_IO_COMMIT_AND_FETCH_REQ`` to the server, ublkdrv needs to copy the server buffer (pages) read to the IO request pages. Batch I/O Commands (UBLK_F_BATCH_IO) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``UBLK_F_BATCH_IO`` feature provides an alternative high-performance I/O handling model that replaces the traditional per-I/O commands with per-queue batch commands. This significantly reduces communication overhead and enables better load balancing across multiple server tasks. Key differences from traditional mode: - **Per-queue vs Per-I/O**: Commands operate on queues rather than individual I/Os - **Batch processing**: Multiple I/Os are handled in single operations - **Multishot commands**: Use io_uring multishot for reduced submission overhead - **Flexible task assignment**: Any task can handle any I/O (no per-I/O daemons) - **Better load balancing**: Tasks can adjust their workload dynamically Batch I/O Commands: - ``UBLK_U_IO_PREP_IO_CMDS`` Prepares multiple I/O commands in batch. The server provides a buffer containing multiple I/O descriptors that will be processed together. This reduces the number of individual command submissions required. - ``UBLK_U_IO_COMMIT_IO_CMDS`` Commits results for multiple I/O operations in batch, and prepares the I/O descriptors to accept new requests. The server provides a buffer containing the results of multiple completed I/Os, allowing efficient bulk completion of requests. - ``UBLK_U_IO_FETCH_IO_CMDS`` **Multishot command** for fetching I/O commands in batch. This is the key command that enables high-performance batch processing: * Uses io_uring multishot capability for reduced submission overhead * Single command can fetch multiple I/O requests over time * Buffer size determines maximum batch size per operation * Multiple fetch commands can be submitted for load balancing * Only one fetch command is active at any time per queue * Supports dynamic load balancing across multiple server tasks It is one typical multishot io_uring request with provided buffer, and it won't be completed until any failure is triggered. Each task can submit ``UBLK_U_IO_FETCH_IO_CMDS`` with different buffer sizes to control how much work it handles. This enables sophisticated load balancing strategies in multi-threaded servers. Migration: Applications using traditional commands (``UBLK_U_IO_FETCH_REQ``, ``UBLK_U_IO_COMMIT_AND_FETCH_REQ``) cannot use batch mode simultaneously. Zero copy --------- Loading