Commit 7856a565 authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge tag 'mm-nonmm-stable-2024-09-21-07-52' of... 

Merge tag 'mm-nonmm-stable-2024-09-21-07-52' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

Pull non-MM updates from Andrew Morton:
 "Many singleton patches - please see the various changelogs for
  details.

  Quite a lot of nilfs2 work this time around.

  Notable patch series in this pull request are:

   - "mul_u64_u64_div_u64: new implementation" by Nicolas Pitre, with
     assistance from Uwe Kleine-König. Reimplement mul_u64_u64_div_u64()
     to provide (much) more accurate results. The current implementation
     was causing Uwe some issues in the PWM drivers.

   - "xz: Updates to license, filters, and compression options" from
     Lasse Collin. Miscellaneous maintenance and kinor feature work to
     the xz decompressor.

   - "Fix some GDB command error and add some GDB commands" from
     Kuan-Ying Lee. Fixes and enhancements to the gdb scripts.

   - "treewide: add missing MODULE_DESCRIPTION() macros" from Jeff
     Johnson. Adds lots of MODULE_DESCRIPTIONs, thus fixing lots of
     warnings about this.

   - "nilfs2: add support for some common ioctls" from Ryusuke Konishi.
     Adds various commonly-available ioctls to nilfs2.

   - "This series fixes a number of formatting issues in kernel doc
     comments" from Ryusuke Konishi does that.

   - "nilfs2: prevent unexpected ENOENT propagation" from Ryusuke
     Konishi. Fix issues where -ENOENT was being unintentionally and
     inappropriately returned to userspace.

   - "nilfs2: assorted cleanups" from Huang Xiaojia.

   - "nilfs2: fix potential issues with empty b-tree nodes" from Ryusuke
     Konishi fixes some issues which can occur on corrupted nilfs2
     filesystems.

   - "scripts/decode_stacktrace.sh: improve error reporting and
     usability" from Luca Ceresoli does those things"

* tag 'mm-nonmm-stable-2024-09-21-07-52' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (103 commits)
  list: test: increase coverage of list_test_list_replace*()
  list: test: fix tests for list_cut_position()
  proc: use __auto_type more
  treewide: correct the typo 'retun'
  ocfs2: cleanup return value and mlog in ocfs2_global_read_info()
  nilfs2: remove duplicate 'unlikely()' usage
  nilfs2: fix potential oob read in nilfs_btree_check_delete()
  nilfs2: determine empty node blocks as corrupted
  nilfs2: fix potential null-ptr-deref in nilfs_btree_insert()
  user_namespace: use kmemdup_array() instead of kmemdup() for multiple allocation
  tools/mm: rm thp_swap_allocator_test when make clean
  squashfs: fix percpu address space issues in decompressor_multi_percpu.c
  lib: glob.c: added null check for character class
  nilfs2: refactor nilfs_segctor_thread()
  nilfs2: use kthread_create and kthread_stop for the log writer thread
  nilfs2: remove sc_timer_task
  nilfs2: do not repair reserved inode bitmap in nilfs_new_inode()
  nilfs2: eliminate the shared counter and spinlock for i_generation
  nilfs2: separate inode type information from i_state field
  nilfs2: use the BITS_PER_LONG macro
  ...
parents 617a814f 5e06e089

Documentation/ABI/testing/sysfs-devices-memory

+3 −3
Original line number Diff line number Diff line
@@ -115,6 +115,6 @@ What: /sys/devices/system/memory/crash_hotplug 
Date:		Aug 2023

Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>

Description:

		(RO) indicates whether or not the kernel directly supports

		modifying the crash elfcorehdr for memory hot un/plug and/or

		on/offline changes.
 
		(RO) indicates whether or not the kernel updates relevant kexec

		segments on memory hot un/plug and/or on/offline events, avoiding the

		need to reload kdump kernel.

Documentation/ABI/testing/sysfs-devices-system-cpu

+3 −3
Original line number Diff line number Diff line
@@ -704,9 +704,9 @@ What: /sys/devices/system/cpu/crash_hotplug 
Date:		Aug 2023

Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>

Description:

		(RO) indicates whether or not the kernel directly supports

		modifying the crash elfcorehdr for CPU hot un/plug and/or

		on/offline changes.

		(RO) indicates whether or not the kernel updates relevant kexec

		segments on memory hot un/plug and/or on/offline events, avoiding the

		need to reload kdump kernel.



What:		/sys/devices/system/cpu/enabled

Date:		Nov 2022

Documentation/admin-guide/mm/memory-hotplug.rst

+3 −2
Original line number Diff line number Diff line
@@ -294,8 +294,9 @@ The following files are currently defined: 
``crash_hotplug``      read-only: when changes to the system memory map

		       occur due to hot un/plug of memory, this file contains

		       '1' if the kernel updates the kdump capture kernel memory

		       map itself (via elfcorehdr), or '0' if userspace must update

		       the kdump capture kernel memory map.

		       map itself (via elfcorehdr and other relevant kexec

		       segments), or '0' if userspace must update the kdump

		       capture kernel memory map.



		       Availability depends on the CONFIG_MEMORY_HOTPLUG kernel

		       configuration option.

Documentation/core-api/cpu_hotplug.rst

+6 −4
Original line number Diff line number Diff line
@@ -737,8 +737,9 @@ can process the event further. 


When changes to the CPUs in the system occur, the sysfs file

/sys/devices/system/cpu/crash_hotplug contains '1' if the kernel

updates the kdump capture kernel list of CPUs itself (via elfcorehdr),

or '0' if userspace must update the kdump capture kernel list of CPUs.

updates the kdump capture kernel list of CPUs itself (via elfcorehdr and

other relevant kexec segment), or '0' if userspace must update the kdump

capture kernel list of CPUs.



The availability depends on the CONFIG_HOTPLUG_CPU kernel configuration

option.
@@ -750,8 +751,9 @@ file can be used in a udev rule as follows: 
 SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"



For a CPU hot un/plug event, if the architecture supports kernel updates

of the elfcorehdr (which contains the list of CPUs), then the rule skips

the unload-then-reload of the kdump capture kernel.

of the elfcorehdr (which contains the list of CPUs) and other relevant

kexec segments, then the rule skips the unload-then-reload of the kdump

capture kernel.



Kernel Inline Documentations Reference

======================================

Documentation/staging/xz.rst

+64 −93
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: 0BSD



============================

XZ data compression in Linux

============================
@@ -6,62 +8,55 @@ Introduction 
============



XZ is a general purpose data compression format with high compression

ratio and relatively fast decompression. The primary compression

algorithm (filter) is LZMA2. Additional filters can be used to improve

compression ratio even further. E.g. Branch/Call/Jump (BCJ) filters

improve compression ratio of executable data.



The XZ decompressor in Linux is called XZ Embedded. It supports

the LZMA2 filter and optionally also BCJ filters. CRC32 is supported

for integrity checking. The home page of XZ Embedded is at

<https://tukaani.org/xz/embedded.html>, where you can find the

latest version and also information about using the code outside

the Linux kernel.



For userspace, XZ Utils provide a zlib-like compression library

and a gzip-like command line tool. XZ Utils can be downloaded from

<https://tukaani.org/xz/>.

ratio. The XZ decompressor in Linux is called XZ Embedded. It supports

the LZMA2 filter and optionally also Branch/Call/Jump (BCJ) filters

for executable code. CRC32 is supported for integrity checking.



See the `XZ Embedded`_ home page for the latest version which includes

a few optional extra features that aren't required in the Linux kernel

and information about using the code outside the Linux kernel.



For userspace, `XZ Utils`_ provide a zlib-like compression library

and a gzip-like command line tool.



.. _XZ Embedded: https://tukaani.org/xz/embedded.html

.. _XZ Utils: https://tukaani.org/xz/



XZ related components in the kernel

===================================



The xz_dec module provides XZ decompressor with single-call (buffer

to buffer) and multi-call (stateful) APIs. The usage of the xz_dec

module is documented in include/linux/xz.h.



The xz_dec_test module is for testing xz_dec. xz_dec_test is not

useful unless you are hacking the XZ decompressor. xz_dec_test

allocates a char device major dynamically to which one can write

.xz files from userspace. The decompressed output is thrown away.

Keep an eye on dmesg to see diagnostics printed by xz_dec_test.

See the xz_dec_test source code for the details.

to buffer) and multi-call (stateful) APIs in include/linux/xz.h.



For decompressing the kernel image, initramfs, and initrd, there

is a wrapper function in lib/decompress_unxz.c. Its API is the

same as in other decompress_*.c files, which is defined in

include/linux/decompress/generic.h.



scripts/xz_wrap.sh is a wrapper for the xz command line tool found

from XZ Utils. The wrapper sets compression options to values suitable

for compressing the kernel image.

For kernel makefiles, three commands are provided for use with

``$(call if_changed)``. They require the xz tool from XZ Utils.



- ``$(call if_changed,xzkern)`` is for compressing the kernel image.

  It runs the script scripts/xz_wrap.sh which uses arch-optimized

  options and a big LZMA2 dictionary.



- ``$(call if_changed,xzkern_with_size)`` is like ``xzkern`` above but

  this also appends a four-byte trailer containing the uncompressed size

  of the file. The trailer is needed by the boot code on some archs.



For kernel makefiles, two commands are provided for use with

$(call if_needed). The kernel image should be compressed with

$(call if_needed,xzkern) which will use a BCJ filter and a big LZMA2

dictionary. It will also append a four-byte trailer containing the

uncompressed size of the file, which is needed by the boot code.

Other things should be compressed with $(call if_needed,xzmisc)

- Other things can be compressed with ``$(call if_needed,xzmisc)``

  which will use no BCJ filter and 1 MiB LZMA2 dictionary.



Notes on compression options

============================



Since the XZ Embedded supports only streams with no integrity check or

CRC32, make sure that you don't use some other integrity check type

when encoding files that are supposed to be decoded by the kernel. With

liblzma, you need to use either LZMA_CHECK_NONE or LZMA_CHECK_CRC32

when encoding. With the xz command line tool, use --check=none or

--check=crc32.

Since the XZ Embedded supports only streams with CRC32 or no integrity

check, make sure that you don't use some other integrity check type

when encoding files that are supposed to be decoded by the kernel.

With liblzma from XZ Utils, you need to use either ``LZMA_CHECK_CRC32``

or ``LZMA_CHECK_NONE`` when encoding. With the ``xz`` command line tool,

use ``--check=crc32`` or ``--check=none`` to override the default

``--check=crc64``.



Using CRC32 is strongly recommended unless there is some other layer

which will verify the integrity of the uncompressed data anyway.
@@ -71,57 +66,33 @@ by the decoder; you can only change the integrity check type (or 
disable it) for the actual uncompressed data.



In userspace, LZMA2 is typically used with dictionary sizes of several

megabytes. The decoder needs to have the dictionary in RAM, thus big

dictionaries cannot be used for files that are intended to be decoded

by the kernel. 1 MiB is probably the maximum reasonable dictionary

size for in-kernel use (maybe more is OK for initramfs). The presets

in XZ Utils may not be optimal when creating files for the kernel,

so don't hesitate to use custom settings. Example::



	xz --check=crc32 --lzma2=dict=512KiB inputfile



An exception to above dictionary size limitation is when the decoder

is used in single-call mode. Decompressing the kernel itself is an

example of this situation. In single-call mode, the memory usage

doesn't depend on the dictionary size, and it is perfectly fine to

use a big dictionary: for maximum compression, the dictionary should

be at least as big as the uncompressed data itself.



Future plans

============

megabytes. The decoder needs to have the dictionary in RAM:



- In multi-call mode the dictionary is allocated as part of the

  decoder state. The reasonable maximum dictionary size for in-kernel

  use will depend on the target hardware: a few megabytes is fine for

  desktop systems while 64 KiB to 1 MiB might be more appropriate on

  some embedded systems.



- In single-call mode the output buffer is used as the dictionary

  buffer. That is, the size of the dictionary doesn't affect the

  decompressor memory usage at all. Only the base data structures

  are allocated which take a little less than 30 KiB of memory.

  For the best compression, the dictionary should be at least

  as big as the uncompressed data. A notable example of single-call

  mode is decompressing the kernel itself (except on PowerPC).



The compression presets in XZ Utils may not be optimal when creating

files for the kernel, so don't hesitate to use custom settings to,

for example, set the dictionary size. Also, xz may produce a smaller

file in single-threaded mode so setting that explicitly is recommended.

Example::



    xz --threads=1 --check=crc32 --lzma2=dict=512KiB inputfile



xz_dec API

==========



This is available with ``#include <linux/xz.h>``.



Creating a limited XZ encoder may be considered if people think it is

useful. LZMA2 is slower to compress than e.g. Deflate or LZO even at

the fastest settings, so it isn't clear if LZMA2 encoder is wanted

into the kernel.



Support for limited random-access reading is planned for the

decompression code. I don't know if it could have any use in the

kernel, but I know that it would be useful in some embedded projects

outside the Linux kernel.



Conformance to the .xz file format specification

================================================



There are a couple of corner cases where things have been simplified

at expense of detecting errors as early as possible. These should not

matter in practice all, since they don't cause security issues. But

it is good to know this if testing the code e.g. with the test files

from XZ Utils.



Reporting bugs

==============



Before reporting a bug, please check that it's not fixed already

at upstream. See <https://tukaani.org/xz/embedded.html> to get the

latest code.



Report bugs to <lasse.collin@tukaani.org> or visit #tukaani on

Freenode and talk to Larhzu. I don't actively read LKML or other

kernel-related mailing lists, so if there's something I should know,

you should email to me personally or use IRC.



Don't bother Igor Pavlov with questions about the XZ implementation

in the kernel or about XZ Utils. While these two implementations

include essential code that is directly based on Igor Pavlov's code,

these implementations aren't maintained nor supported by him.
 
.. kernel-doc:: include/linux/xz.h