Commit f7128262 authored by Trond Myklebust's avatar Trond Myklebust Committed by Anna Schumaker
Browse files

nfs: add FAQ section to Documentation/filesystems/nfs/localio.rst



Add a FAQ section to give answers to questions that have been raised
during review of the localio feature.

Signed-off-by: default avatarTrond Myklebust <trond.myklebust@hammerspace.com>
Co-developed-by: default avatarMike Snitzer <snitzer@kernel.org>
Signed-off-by: default avatarMike Snitzer <snitzer@kernel.org>
Reviewed-by: default avatarNeilBrown <neilb@suse.de>
Reviewed-by: default avatarJeff Layton <jlayton@kernel.org>
Signed-off-by: default avatarAnna Schumaker <anna.schumaker@oracle.com>
parent 92945bd8
Loading
Loading
Loading
Loading
+86 −0
Original line number Diff line number Diff line
@@ -64,6 +64,92 @@ fio for 20 secs with directio, qd of 8, 1 libaio thread:
  128K read:  IOPS=24.4k, BW=3050MiB/s (3198MB/s)(59.6GiB/20001msec)
  128K write: IOPS=11.4k, BW=1430MiB/s (1500MB/s)(27.9GiB/20001msec)

FAQ
===

1. What are the use cases for LOCALIO?

   a. Workloads where the NFS client and server are on the same host
      realize improved IO performance. In particular, it is common when
      running containerised workloads for jobs to find themselves
      running on the same host as the knfsd server being used for
      storage.

2. What are the requirements for LOCALIO?

   a. Bypass use of the network RPC protocol as much as possible. This
      includes bypassing XDR and RPC for open, read, write and commit
      operations.
   b. Allow client and server to autonomously discover if they are
      running local to each other without making any assumptions about
      the local network topology.
   c. Support the use of containers by being compatible with relevant
      namespaces (e.g. network, user, mount).
   d. Support all versions of NFS. NFSv3 is of particular importance
      because it has wide enterprise usage and pNFS flexfiles makes use
      of it for the data path.

3. Why doesn’t LOCALIO just compare IP addresses or hostnames when
   deciding if the NFS client and server are co-located on the same
   host?

   Since one of the main use cases is containerised workloads, we cannot
   assume that IP addresses will be shared between the client and
   server. This sets up a requirement for a handshake protocol that
   needs to go over the same connection as the NFS traffic in order to
   identify that the client and the server really are running on the
   same host. The handshake uses a secret that is sent over the wire,
   and can be verified by both parties by comparing with a value stored
   in shared kernel memory if they are truly co-located.

4. Does LOCALIO improve pNFS flexfiles?

   Yes, LOCALIO complements pNFS flexfiles by allowing it to take
   advantage of NFS client and server locality.  Policy that initiates
   client IO as closely to the server where the data is stored naturally
   benefits from the data path optimization LOCALIO provides.

5. Why not develop a new pNFS layout to enable LOCALIO?

   A new pNFS layout could be developed, but doing so would put the
   onus on the server to somehow discover that the client is co-located
   when deciding to hand out the layout.
   There is value in a simpler approach (as provided by LOCALIO) that
   allows the NFS client to negotiate and leverage locality without
   requiring more elaborate modeling and discovery of such locality in a
   more centralized manner.

6. Why is having the client perform a server-side file OPEN, without
   using RPC, beneficial?  Is the benefit pNFS specific?

   Avoiding the use of XDR and RPC for file opens is beneficial to
   performance regardless of whether pNFS is used. Especially when
   dealing with small files its best to avoid going over the wire
   whenever possible, otherwise it could reduce or even negate the
   benefits of avoiding the wire for doing the small file I/O itself.
   Given LOCALIO's requirements the current approach of having the
   client perform a server-side file open, without using RPC, is ideal.
   If in the future requirements change then we can adapt accordingly.

7. Why is LOCALIO only supported with UNIX Authentication (AUTH_UNIX)?

   Strong authentication is usually tied to the connection itself. It
   works by establishing a context that is cached by the server, and
   that acts as the key for discovering the authorisation token, which
   can then be passed to rpc.mountd to complete the authentication
   process. On the other hand, in the case of AUTH_UNIX, the credential
   that was passed over the wire is used directly as the key in the
   upcall to rpc.mountd. This simplifies the authentication process, and
   so makes AUTH_UNIX easier to support.

8. How do export options that translate RPC user IDs behave for LOCALIO
   operations (eg. root_squash, all_squash)?

   Export options that translate user IDs are managed by nfsd_setuser()
   which is called by nfsd_setuser_and_check_port() which is called by
   __fh_verify().  So they get handled exactly the same way for LOCALIO
   as they do for non-LOCALIO.

RPC
===