Commit 6fd600d7 authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge tag 'media/v6.10-1' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media 

Pull media updates from Mauro Carvalho Chehab:

 - New V4L2 ioctl VIDIOC_REMOVE_BUFS

 - experimental support for using generic metaformats on V4L2 core

 - New drivers: Intel IPU6 controller driver, Broadcom BCM283x/BCM271x

 - More cleanups at atomisp driver

 - Usual bunch of driver cleanups, improvements and fixes

* tag 'media/v6.10-1' of git://git.kernel.org/pub/scm/linux/kernel/git/mchehab/linux-media: (328 commits)
  media: bcm2835-unicam: Depend on COMMON_CLK
  Revert "media: v4l2-ctrls: show all owned controls in log_status"
  media: ov2740: Ensure proper reset sequence on probe()
  media: intel/ipu6: Don't print user-triggerable errors to kernel log
  media: bcm2835-unicam: Fix driver path in MAINTAINERS
  media: bcm2835-unicam: Fix a NULL vs IS_ERR() check
  media: bcm2835-unicam: Do not print error when irq not found
  media: bcm2835-unicam: Do not replace IRQ retcode during probe
  media: bcm2835-unicam: Convert to platform remove callback returning void
  media: media: intel/ipu6: Fix spelling mistake "remappinp" -> "remapping"
  media: intel/ipu6: explicitly include vmalloc.h
  media: cec.h: Fix kerneldoc
  media: uvcvideo: Refactor iterators
  media: v4l: async: refactor v4l2_async_create_ancillary_links
  media: intel/ipu6: Don't re-allocate memory for firmware
  media: dvb-frontends: tda10048: Fix integer overflow
  media: tc358746: Use the correct div_ function
  media: i2c: st-mipid02: Use the correct div function
  media: tegra-vde: Refactor timeout handling
  media: stk1160: Use min macro
  ...
parents 972a2543 8771b7f3

Documentation/admin-guide/media/ipu6-isys.rst

0 → 100644
+161 −0
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: GPL-2.0



.. include:: <isonum.txt>



========================================================

Intel Image Processing Unit 6 (IPU6) Input System driver

========================================================



Copyright |copy| 2023--2024 Intel Corporation



Introduction

============



This file documents the Intel IPU6 (6th generation Image Processing Unit)

Input System (MIPI CSI2 receiver) drivers located under

drivers/media/pci/intel/ipu6.



The Intel IPU6 can be found in certain Intel SoCs but not in all SKUs:



* Tiger Lake

* Jasper Lake

* Alder Lake

* Raptor Lake

* Meteor Lake



Intel IPU6 is made up of two components - Input System (ISYS) and Processing

System (PSYS).



The Input System mainly works as MIPI CSI-2 receiver which receives and

processes the image data from the sensors and outputs the frames to memory.



There are 2 driver modules - intel-ipu6 and intel-ipu6-isys. intel-ipu6 is an

IPU6 common driver which does PCI configuration, firmware loading and parsing,

firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit)

configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2

sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected

to the IPU6 ISYS through V4L2 sub-device sensor drivers.



.. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more

	  information about the IPU6 hardware.



Input system driver

===================



The Input System driver mainly configures CSI-2 D-PHY, constructs the firmware

stream configuration, sends commands to firmware, gets response from hardware

and firmware and then returns buffers to user.  The ISYS is represented as

several V4L2 sub-devices as well as video nodes.



.. kernel-figure::  ipu6_isys_graph.svg

   :alt: ipu6 isys media graph with multiple streams support



   IPU6 ISYS media graph with multiple streams support



The graph has been produced using the following command:



.. code-block:: none



   fdp -Gsplines=true -Tsvg < dot > dot.svg



Capturing frames with IPU6 ISYS

-------------------------------



IPU6 ISYS is used to capture frames from the camera sensors connected to the

CSI2 ports. The supported input formats of ISYS are listed in table below:



.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|



.. flat-table::

    :header-rows: 1



    * - IPU6 ISYS supported input formats



    * - RGB565, RGB888



    * - UYVY8, YUYV8



    * - RAW8, RAW10, RAW12



.. _ipu6_isys_capture_examples:



Examples

~~~~~~~~



Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this

machine, ov01a10 sensor is connected to IPU ISYS CSI-2 port 2, which can

generate images at sBGGR10 with resolution 1280x800.



Using the media controller APIs, we can configure ov01a10 sensor by

media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS.



.. code-block:: none



    # Example 1 capture frame from ov01a10 camera sensor

    # This example assumes /dev/media0 as the IPU ISYS media device

    export MDEV=/dev/media0



    # Establish the link for the media devices using media-ctl

    media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]"



    # Set the format for the media devices

    media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]"

    media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]"

    media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]"



Once the media pipeline is configured, desired sensor specific settings

(such as exposure and gain settings) can be set, using the yavta tool.



e.g



.. code-block:: none



    # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36

    export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036")



    yavta -w 0x009e0903 400 $SDEV

    yavta -w 0x009e0913 1000 $SDEV

    yavta -w 0x009e0911 2000 $SDEV



Once the desired sensor settings are set, frame captures can be done as below.



e.g



.. code-block:: none



    yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \

            -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0")



With the above command, 10 frames are captured at 1280x800 resolution with

sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files.



Here is another example of IPU6 ISYS RAW and metadata capture from camera

sensor ov2740 on Lenovo X1 Yoga laptop.



.. code-block:: none



    media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]"

    media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[5]"

    media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[5]"



    # set routing

    media-ctl -v -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]"



    media-ctl -v "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]"

    media-ctl -v "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]"

    media-ctl -v "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]"

    media-ctl -v "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]"



    CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0")

    ./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \

        -f SGRBG10 ${CAPTURE_DEV}



    CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1")

    ./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \

        --file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META}



References

==========



.. [#f1] https://git.ideasonboard.org/media-ctl.git

.. [#f2] https://git.ideasonboard.org/yavta.git

Documentation/admin-guide/media/ipu6_isys_graph.svg

0 → 100644
+548 −0

File added.

Preview size limit exceeded, changes collapsed.

Documentation/admin-guide/media/mgb4.rst

+15 −20
Original line number Diff line number Diff line 
.. SPDX-License-Identifier: GPL-2.0



====================

mgb4 sysfs interface

====================

The mgb4 driver

===============



sysfs interface

---------------



The mgb4 driver provides a sysfs interface, that is used to configure video

stream related parameters (some of them must be set properly before the v4l2
@@ -12,9 +14,8 @@ There are two types of parameters - global / PCI card related, found under 
``/sys/class/video4linux/videoX/device`` and module specific found under

``/sys/class/video4linux/videoX``.





Global (PCI card) parameters

============================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~



**module_type** (R):

    Module type.
@@ -42,9 +43,8 @@ Global (PCI card) parameters 


    where each component is a 8b number.





Common FPDL3/GMSL input parameters

==================================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



**input_id** (R):

    Input number ID, zero based.
@@ -190,9 +190,8 @@ Common FPDL3/GMSL input parameters 
    *Note: This parameter can not be changed while the input v4l2 device is

    open.*





Common FPDL3/GMSL output parameters

===================================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



**output_id** (R):

    Output number ID, zero based.
@@ -282,9 +281,8 @@ Common FPDL3/GMSL output parameters 
    Number of video lines between the end of the last valid pixel line (marked

    by DE=1) and assertion of the VSYNC signal. The default value is 2.





FPDL3 specific input parameters

===============================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



**fpdl3_input_width** (RW):

    Number of deserializer input lines.
@@ -294,7 +292,7 @@ FPDL3 specific input parameters 
    | 2 - dual



FPDL3 specific output parameters

================================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



**fpdl3_output_width** (RW):

    Number of serializer output lines.
@@ -304,7 +302,7 @@ FPDL3 specific output parameters 
    | 2 - dual



GMSL specific input parameters

==============================

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



**gmsl_mode** (RW):

    GMSL speed mode.
@@ -328,10 +326,8 @@ GMSL specific input parameters 
    | 0 - disabled

    | 1 - enabled (default)





====================

mgb4 mtd partitions

====================

MTD partitions

--------------



The mgb4 driver creates a MTD device with two partitions:

 - mgb4-fw.X - FPGA firmware.
@@ -344,9 +340,8 @@ also have a third partition named *mgb4-flash* available in the system. This 
partition represents the whole, unpartitioned, card's FLASH memory and one should

not fiddle with it...



====================

mgb4 iio (triggers)

====================

IIO (triggers)

--------------



The mgb4 driver creates an Industrial I/O (IIO) device that provides trigger and

signal level status capability. The following scan elements are available:

Documentation/admin-guide/media/v4l-drivers.rst

+1 −0
Original line number Diff line number Diff line
@@ -16,6 +16,7 @@ Video4Linux (V4L) driver-specific documentation 
	imx

	imx7

	ipu3

	ipu6-isys

	ivtv

	mgb4

	omap3isp

Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml

0 → 100644
+127 −0
Original line number Diff line number Diff line 
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)

%YAML 1.2

---

$id: http://devicetree.org/schemas/media/brcm,bcm2835-unicam.yaml#

$schema: http://devicetree.org/meta-schemas/core.yaml#



title: Broadcom BCM283x Camera Interface (Unicam)



maintainers:

  - Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>



description: |-

  The Unicam block on BCM283x SoCs is the receiver for either

  CSI-2 or CCP2 data from image sensors or similar devices.



  The main platform using this SoC is the Raspberry Pi family of boards.  On

  the Pi the VideoCore firmware can also control this hardware block, and

  driving it from two different processors will cause issues.  To avoid this,

  the firmware checks the device tree configuration during boot. If it finds

  device tree nodes whose name starts with 'csi' then it will stop the firmware

  accessing the block, and it can then safely be used via the device tree

  binding.



properties:

  compatible:

    const: brcm,bcm2835-unicam



  reg:

    items:

      - description: Unicam block.

      - description: Clock Manager Image (CMI) block.



  reg-names:

    items:

      - const: unicam

      - const: cmi



  interrupts:

    maxItems: 1



  clocks:

    items:

      - description: Clock to drive the LP state machine of Unicam.

      - description: Clock for the VPU (core clock).



  clock-names:

    items:

      - const: lp

      - const: vpu



  power-domains:

    items:

      - description: Unicam power domain



  brcm,num-data-lanes:

    $ref: /schemas/types.yaml#/definitions/uint32

    enum: [ 2, 4 ]

    description: |

      Number of CSI-2 data lanes supported by this Unicam instance. The number

      of data lanes actively used is specified with the data-lanes endpoint

      property.



  port:

    $ref: /schemas/graph.yaml#/$defs/port-base

    unevaluatedProperties: false



    properties:

      endpoint:

        $ref: /schemas/media/video-interfaces.yaml#

        additionalProperties: false



        properties:

          bus-type:

            enum: [ 3, 4 ]



          clock-noncontinuous: true

          data-lanes: true

          remote-endpoint: true



        required:

          - bus-type

          - data-lanes

          - remote-endpoint



    required:

      - endpoint



required:

  - compatible

  - reg

  - reg-names

  - interrupts

  - clocks

  - clock-names

  - power-domains

  - brcm,num-data-lanes

  - port



additionalProperties: False



examples:

  - |

    #include <dt-bindings/clock/bcm2835.h>

    #include <dt-bindings/interrupt-controller/arm-gic.h>

    #include <dt-bindings/media/video-interfaces.h>

    #include <dt-bindings/power/raspberrypi-power.h>



    csi1: csi@7e801000 {

        compatible = "brcm,bcm2835-unicam";

        reg = <0x7e801000 0x800>,

              <0x7e802004 0x4>;

        reg-names = "unicam", "cmi";

        interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>;

        clocks = <&clocks BCM2835_CLOCK_CAM1>,

                 <&firmware_clocks 4>;

        clock-names = "lp", "vpu";

        power-domains = <&power RPI_POWER_DOMAIN_UNICAM1>;

        brcm,num-data-lanes = <2>;

        port {

                csi1_ep: endpoint {

                        remote-endpoint = <&imx219_0>;

                        bus-type = <MEDIA_BUS_TYPE_CSI2_DPHY>;

                        data-lanes = <1 2>;

                };

        };

    };

...