docs: verify/bisect: proper headlines and more spacing

========================================

*[If you are new to building or bisecting Linux, ignore this section and head

over to the* ":ref:`step-by-step guide<introguide_bissbs>`" *below. It utilizes

over to the* ':ref:`step-by-step guide <introguide_bissbs>`' *below. It utilizes

the same commands as this section while describing them in brief fashion. The

steps are nevertheless easy to follow and together with accompanying entries

in a reference section mention many alternatives, pitfalls, and additional

*segment 3* to **perform a bisection** for a full-fledged regression report

developers are obliged to act upon.

 :ref:`Preparations: set up everything to build your own kernels.<introprep_bissbs>`

 :ref:`Preparations: set up everything to build your own kernels <introprep_bissbs>`.

 :ref:`Segment 1: try to reproduce the problem with the latest codebase.<introlatestcheck_bissbs>`

 :ref:`Segment 1: try to reproduce the problem with the latest codebase <introlatestcheck_bissbs>`.

 :ref:`Segment 2: check if the kernels you build work fine.<introworkingcheck_bissbs>`

 :ref:`Segment 2: check if the kernels you build work fine <introworkingcheck_bissbs>`.

 :ref:`Segment 3: perform a bisection and validate the result.<introbisect_bissbs>`

 :ref:`Segment 3: perform a bisection and validate the result <introbisect_bissbs>`.

 :ref:`Supplementary tasks: cleanup during and after following this guide.<introclosure_bissbs>`

 :ref:`Supplementary tasks: cleanup during and after following this guide <introclosure_bissbs>`.

The steps in each segment illustrate the important aspects of the process, while

a comprehensive reference section holds additional details for almost all of the

Preparations: set up everything to build your own kernels

---------------------------------------------------------

The following steps lay the groundwork for all further tasks.

.. _backup_bissbs:

* Create a fresh backup and put system repair and restore tools at hand, just

  [:ref:`details <finishingtouch_bisref>`]

.. _submit_improvements:

This concludes the step-by-step guide.

Conclusion

----------

You have reached the end of the step-by-step guide.

Did you run into trouble following any of the above steps not cleared up by the

reference section below? Did you spot errors? Or do you have ideas how to

This section holds additional information for almost all the items in the above

step-by-step guide.

Preparations for building your own kernels

------------------------------------------

  *The steps in this section lay the groundwork for all further tests.*

  [:ref:`... <introprep_bissbs>`]

The steps in all later sections of this guide depend on those described here.

[:ref:`back to step-by-step guide <introprep_bissbs>`].

.. _backup_bisref:

Prepare for emergencies

-----------------------

~~~~~~~~~~~~~~~~~~~~~~~

  *Create a fresh backup and put system repair and restore tools at hand.*

  [:ref:`... <backup_bissbs>`]

.. _vanilla_bisref:

Remove anything related to externally maintained kernel modules

---------------------------------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Remove all software that depends on externally developed kernel drivers or

  builds them automatically.* [:ref:`...<vanilla_bissbs>`]

.. _secureboot_bisref:

Deal with techniques like Secure Boot

-------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *On platforms with 'Secure Boot' or similar techniques, prepare everything to

  ensure the system will permit your self-compiled kernel to boot later.*

.. _bootworking_bisref:

Boot the last kernel that was working

-------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Boot into the last working kernel and briefly recheck if the feature that

  regressed really works.* [:ref:`...<bootworking_bissbs>`]

.. _diskspace_bisref:

Space requirements

------------------

~~~~~~~~~~~~~~~~~~

  *Ensure to have enough free space for building Linux.*

  [:ref:`... <diskspace_bissbs>`]

.. _rangecheck_bisref:

Bisection range

---------------

~~~~~~~~~~~~~~~

  *Determine the kernel versions considered 'good' and 'bad' throughout this

  guide.* [:ref:`...<rangecheck_bissbs>`]

.. _buildrequires_bisref:

Install build requirements

--------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Install all software required to build a Linux kernel.*

  [:ref:`...<buildrequires_bissbs>`]

.. _sources_bisref:

Download the sources using Git

------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Retrieve the Linux mainline sources.*

  [:ref:`...<sources_bissbs>`]

.. _sources_bundle_bisref:

Downloading Linux mainline sources using a bundle

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

"""""""""""""""""""""""""""""""""""""""""""""""""

Use the following commands to retrieve the Linux mainline sources using a

bundle::

.. _oldconfig_bisref:

Start defining the build configuration for your kernel

------------------------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Start preparing a kernel build configuration (the '.config' file).*

  [:ref:`... <oldconfig_bissbs>`]

.. _localmodconfig_bisref:

Trim the build configuration for your kernel

--------------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Disable any kernel modules apparently superfluous for your setup.*

  [:ref:`... <localmodconfig_bissbs>`]

.. _tagging_bisref:

Tag the kernels about to be build

---------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Ensure all the kernels you will build are clearly identifiable using a

  special tag and a unique version identifier.* [:ref:`... <tagging_bissbs>`]

.. _debugsymbols_bisref:

Decide to enable or disable debug symbols

-----------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Decide how to handle debug symbols.* [:ref:`... <debugsymbols_bissbs>`]

.. _configmods_bisref:

Adjust build configuration

--------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Check if you may want or need to adjust some other kernel configuration

  options:*

.. _configmods_distros_bisref:

Distro specific adjustments

~~~~~~~~~~~~~~~~~~~~~~~~~~~

"""""""""""""""""""""""""""

  *Are you running* [:ref:`... <configmods_bissbs>`]

.. _configmods_individual_bisref:

Individual adjustments

~~~~~~~~~~~~~~~~~~~~~~

""""""""""""""""""""""

  *If you want to influence the other aspects of the configuration, do so

  now.* [:ref:`... <configmods_bissbs>`]

.. _saveconfig_bisref:

Put the .config file aside

--------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Reprocess the .config after the latest changes and store it in a safe place.*

  [:ref:`... <saveconfig_bissbs>`]

.. _introlatestcheck_bisref:

Try to reproduce the regression

-----------------------------------------

Try to reproduce the problem with the latest codebase

-----------------------------------------------------

  *Verify the regression is not caused by some .config change and check if it

  still occurs with the latest codebase.* [:ref:`... <introlatestcheck_bissbs>`]

.. _checkoutmaster_bisref:

Check out the latest Linux codebase

-----------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Check out the latest Linux codebase.*

  [:ref:`... <introlatestcheck_bissbs>`]

  [:ref:`... <checkoutmaster_bissbs>`]

In case you later want to recheck if an ever newer codebase might fix the

problem, remember to run that ``git fetch --shallow-exclude [...]`` command

again mentioned earlier to update your local Git repository.

[:ref:`back to step-by-step guide <introlatestcheck_bissbs>`]

[:ref:`back to step-by-step guide <checkoutmaster_bissbs>`]

.. _build_bisref:

Build your kernel

-----------------

~~~~~~~~~~~~~~~~~

  *Build the image and the modules of your first kernel using the config file

  you prepared.* [:ref:`... <build_bissbs>`]

deb, rpm or tar file.

Dealing with build errors

~~~~~~~~~~~~~~~~~~~~~~~~~

"""""""""""""""""""""""""

When a build error occurs, it might be caused by some aspect of your machine's

setup that often can be fixed quickly; other times though the problem lies in

a solution (e.g. a patch) or workaround for your issue, too.

Package your kernel up

~~~~~~~~~~~~~~~~~~~~~~

""""""""""""""""""""""

The step-by-step guide uses the default make targets (e.g. 'bzImage' and

'modules' on x86) to build the image and the modules of your kernel, which later

.. _install_bisref:

Put the kernel in place

-----------------------

~~~~~~~~~~~~~~~~~~~~~~~

  *Install the kernel you just built.* [:ref:`... <install_bissbs>`]

.. _storagespace_bisref:

Storage requirements per kernel

-------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Check how much storage space the kernel, its modules, and other related files

  like the initramfs consume.* [:ref:`... <storagespace_bissbs>`]

.. _tainted_bisref:

Check if your newly built kernel considers itself 'tainted'

-----------------------------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Check if the kernel marked itself as 'tainted'.*

  [:ref:`... <tainted_bissbs>`]

.. _recheckbroken_bisref:

Check the kernel built from a recent mainline codebase

------------------------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Verify if your bug occurs with the newly built kernel.*

  [:ref:`... <recheckbroken_bissbs>`]

.. _recheckstablebroken_bisref:

Check the kernel built from the latest stable/longterm codebase

---------------------------------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Are you facing a regression within a stable/longterm release, but failed to

  reproduce it with the kernel you just built using the latest mainline sources?

.. _recheckworking_bisref:

Build your own version of the 'good' kernel

-------------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Build your own variant of the working kernel and check if the feature that

  regressed works as expected with it.* [:ref:`... <recheckworking_bissbs>`]

[:ref:`back to step-by-step guide <recheckworking_bissbs>`]

Perform a bisection and validate the result

-------------------------------------------

  *With all the preparations and precaution builds taken care of, you are now

  ready to begin the bisection.* [:ref:`... <introbisect_bissbs>`]

The steps in this segment perform and validate the bisection.

[:ref:`back to step-by-step guide <introbisect_bissbs>`].

.. _bisectstart_bisref:

Start the bisection

-------------------

~~~~~~~~~~~~~~~~~~~

  *Start the bisection and tell Git about the versions earlier established as

  'good' and 'bad'.* [:ref:`... <bisectstart_bissbs>`]

.. _bisectbuild_bisref:

Build a kernel from the bisection point

---------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Build, install, and boot a kernel from the code Git checked out using the

  same commands you used earlier.* [:ref:`... <bisectbuild_bissbs>`]

.. _bisecttest_bisref:

Bisection checkpoint

--------------------

~~~~~~~~~~~~~~~~~~~~

  *Check if the feature that regressed works in the kernel you just built.*

  [:ref:`... <bisecttest_bissbs>`]

.. _bisectlog_bisref:

Put the bisection log away

--------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~

  *Store Git's bisection log and the current .config file in a safe place.*

  [:ref:`... <bisectlog_bissbs>`]

.. _revert_bisref:

Try reverting the culprit

-------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~

  *Try reverting the culprit on top of the latest codebase to see if this fixes

  your regression.* [:ref:`... <revert_bissbs>`]

[:ref:`back to step-by-step guide <revert_bissbs>`]

Cleanup steps during and after following this guide

---------------------------------------------------

Supplementary tasks: cleanup during and after the bisection

-----------------------------------------------------------

  *During and after following this guide you might want or need to remove some

  of the kernels you installed.* [:ref:`... <introclosure_bissbs>`]

The steps in this section describe clean-up procedures.

[:ref:`back to step-by-step guide <introclosure_bissbs>`].

.. _makeroom_bisref:

Cleaning up during the bisection

--------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  *To remove one of the kernels you installed, look up its 'kernelrelease'

  identifier.* [:ref:`... <makeroom_bissbs>`]

[:ref:`back to step-by-step guide <makeroom_bissbs>`]

Cleaning up after the bisection

-------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. _finishingtouch_bisref: