Compilation error conflicting types

lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   -  
[Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   -

Hello everyone :),

Warning: since an unfortunate click on a link in the preview of the previous version of this thread caused me to lose all its content :(, I won't detail the avenues I've explored (I'll do it later if needed)!

I'm trying to install this driver on Linux (Debian 11 kernel 5.10.0-20-amd64) with make (no README and the project seems to be inactive) which should install without any issues, but I'm getting this:

root@vm-bullseye-xfce:~/ms912x-main# make make CHECK="/usr/bin/sparse" -C /lib/modules/5.10.0-20-amd64/build M=/root/ms912x-main modules make[1] : entering directory « /usr/src/linux-headers-5.10.0-20-amd64 » CC [M] /root/ms912x-main/ms912x_registers.o In file included from /root/ms912x-main/ms912x_registers.c:4: /root/ms912x-main/ms912x.h:113:19: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration 113 | const struct dma_buf_map *map, | ^~~~~~~~~~~ CC [M] /root/ms912x-main/ms912x_connector.o In file included from /root/ms912x-main/ms912x_connector.c:7: /root/ms912x-main/ms912x.h:113:19: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration 113 | const struct dma_buf_map *map, | ^~~~~~~~~~~ CC [M] /root/ms912x-main/ms912x_transfer.o In file included from /root/ms912x-main/ms912x_transfer.c:7: /root/ms912x-main/ms912x.h:113:19: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration 113 | const struct dma_buf_map *map, | ^~~~~~~~~~~ /root/ms912x-main/ms912x_transfer.c:160:19: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration 160 | const struct dma_buf_map *map, | ^~~~~~~~~~~ /root/ms912x-main/ms912x_transfer.c:159:6: error: conflicting types for ‘ms912x_fb_send_rect’ 159 | void ms912x_fb_send_rect(struct drm_framebuffer *fb, | ^~~~~~~~~~~~~~~~~~~ In file included from /root/ms912x-main/ms912x_transfer.c:7: /root/ms912x-main/ms912x.h:112:6: note: previous declaration of ‘ms912x_fb_send_rect’ was here 112 | void ms912x_fb_send_rect(struct drm_framebuffer *fb, | ^~~~~~~~~~~~~~~~~~~ /root/ms912x-main/ms912x_transfer.c: In function ‘ms912x_fb_send_rect’: /root/ms912x-main/ms912x_transfer.c:164:19: error: invalid use of undefined type ‘const struct dma_buf_map’ 164 | void *vaddr = map->vaddr; | ^~ /root/ms912x-main/ms912x_transfer.c:195:8: error: implicit declaration of function ‘drm_gem_fb_begin_cpu_access’; did you mean ‘dma_buf_begin_cpu_access’? [-Werror=implicit-function-declaration] 195 | ret = drm_gem_fb_begin_cpu_access(fb, DMA_FROM_DEVICE); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ | dma_buf_begin_cpu_access /root/ms912x-main/ms912x_transfer.c:237:2: error: implicit declaration of function ‘drm_gem_fb_end_cpu_access’; did you mean ‘dma_buf_end_cpu_access’? [-Werror=implicit-function-declaration] 237 | drm_gem_fb_end_cpu_access(fb, DMA_FROM_DEVICE); | ^~~~~~~~~~~~~~~~~~~~~~~~~ | dma_buf_end_cpu_access cc1: some warnings being treated as errors make[3]: *** [/usr/src/linux-headers-5.10.0-20-common/scripts/Makefile.build:291 : /root/ms912x-main/ms912x_transfer.o] Error 1 make[2]: *** [/usr/src/linux-headers-5.10.0-20-common/Makefile:1861 : /root/ms912x-main] Error 2 make[1]: *** [/usr/src/linux-headers-5.10.0-20-common/Makefile:185 : __sub-make] Error 2 make[1] : leaving directory « /usr/src/linux-headers-5.10.0-20-amd64 » make: *** [Makefile:15 : modules] Error 2 

I'm trying to resolve the blocking errors that can't be ignored with -Wxxx for the compilation.

The first one:

 /root/ms912x-main/ms912x_transfer.c:159:6: error: conflicting types for ‘ms912x_fb_send_rect’

would indicate a conflict between the declaration and the definition of the function ms912x_fb_send_rect().

However, in ms912x-main/ms912x.h (its declaration) and ms912x_transfer.c (its definition) the types of the arguments and return are identical BUT it does not take into account that the arguments are using structures provided in the #include and in particular those from the current kernel headers (yes I have vague returns of understanding from the time when I was a developer ;) but I can't reconnect everything :D ).

In short! I don't know what is wrong and if I'm using the right compiler (there's even the insmod.sh script but running it leads to the same result).

Does anyone see where the problem is or have a lead to suggest?

With affection,

lnj



15 answers

  1. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108
     

    Hello lenainjaune,

    I don't know which history from a previous thread you're talking about :-)

    That said, I think your problem is related to the fact that you don't have a Linux system with a kernel and corresponding headers that are compatible with the module you're trying to compile.

    In general, in compilation errors, the first warning or error is the most relevant.

    A search on elixir.bootlin.com shows that, in the Linux kernel, it's the header dma-buf-map.h that defines struct dma_buf_map (see my edit below*).

    Your make output shows that you have a kernel 5.10.0-20-amd64.

    In this version of the kernel, dma-buf-map.h does not exist and struct dma_buf_map is defined nowhere.

    https://elixir.bootlin.com/linux/v5.10.194/source/include/linux/dma-buf-map.h

    This header seems to exist and define struct dma_buf_map starting from kernel 5.11.

    https://elixir.bootlin.com/linux/v5.11/source/include/linux/dma-buf-map.h#L115

    I think you need a more up-to-date version of the kernel and its headers to compile this module.

    ---

    * Edit: after a more detailed search, it seems that it's the kernel versions 5.11 to 5.17 that contain dma-buf-map.h

    struct dma_buf_map is not defined in kernel versions 5.18 and 5.19, nor in the 6 kernel versions which is the latest major version currently in development.

    1
    1. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108
       

      I don't think there's an official Debian package for these kernel versions, and... it's a bit tricky to compile and install your own kernel version the Debian way (https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html).

      What exactly do you need this module for?

      Have you checked if your hardware is not directly supported in Debian bookworm (you are apparently on bullseye, this could be an opportunity to upgrade)?

      0
    2. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
       

      Hello [Dal] and thank you for your detailed response :)

      I'm replying in the main thread, as the readability isn't great in the comments.

      0
  2. mamiemando Posted messages 33228 Registration date   Status Moderator Last intervention   7 940
     

    Hello,

    In theory, you should install your kernel headers and the necessary tools to compile as follows:

    sudo apt update sudo apt install build-essential make linux-headers-amd64

    Regarding [Dal]'s remark, this structure still seems to exist in the current kernel headers (6.4.0-3) in Debian testing:

    /usr/src/linux-headers-6.4.0-3-common/include/linux/dma-buf.h

    struct dma_buf { /** * @size: * * Size of the buffer; invariant over the lifetime of the buffer. */ size_t size; /** * @file: * * File pointer used for sharing buffers across, and for refcounting. * See dma_buf_get() and dma_buf_put(). */ struct file *file; /** * @attachments: * * List of dma_buf_attachment that denotes all devices attached, * protected by &dma_resv lock @resv. */ struct list_head attachments; /** @ops: dma_buf_ops associated with this buffer object. */ const struct dma_buf_ops *ops; /** * @vmapping_counter: * * Used internally to refcnt the vmaps returned by dma_buf_vmap(). * Protected by @lock. */ unsigned vmapping_counter; /** * @vmap_ptr: * The current vmap ptr if @vmapping_counter > 0. Protected by @lock. */ struct iosys_map vmap_ptr; /** * @exp_name: * * Name of the exporter; useful for debugging. See the * DMA_BUF_SET_NAME IOCTL. */ const char *exp_name; /** * @name: * * Userspace-provided name; useful for accounting and debugging, * protected by dma_resv_lock() on @resv and @name_lock for read access. */ const char *name; /** @name_lock: Spinlock to protect name access for read access. */ spinlock_t name_lock; /** * @owner: * * Pointer to exporter module; used for refcounting when exporter is a * kernel module. */ struct module *owner; /** @list_node: node for dma_buf accounting and debugging. */ struct list_head list_node; /** @priv: exporter specific private data for this buffer object. */ void *priv; /** * @resv: * * Reservation object linked to this dma-buf. * * IMPLICIT SYNCHRONIZATION RULES: * * Drivers which support implicit synchronization of buffer access as * e.g. exposed in `Implicit Fence Poll Support`_ must follow the * below rules. * * - Drivers must add a read fence through dma_resv_add_fence() with the * DMA_RESV_USAGE_READ flag for anything the userspace API considers a * read access. This highly depends upon the API and window system. * * - Similarly drivers must add a write fence through * dma_resv_add_fence() with the DMA_RESV_USAGE_WRITE flag for * anything the userspace API considers write access. * * - Drivers may just always add a write fence, since that only * causes unnecessary synchronization, but no correctness issues. * * - Some drivers only expose a synchronous userspace API with no * pipelining across drivers. These do not set any fences for their * access. An example here is v4l. * * - Driver should use dma_resv_usage_rw() when retrieving fences as * dependency for implicit synchronization. * * DYNAMIC IMPORTER RULES: * * Dynamic importers, see dma_buf_attachment_is_dynamic(), have * additional constraints on how they set up fences: * * - Dynamic importers must obey the write fences and wait for them to * signal before allowing access to the buffer's underlying storage * through the device. * * - Dynamic importers should set fences for any access that they can't * disable immediately from their &dma_buf_attach_ops.move_notify * callback. * * IMPORTANT: * * All drivers and memory management related functions must obey the * struct dma_resv rules, specifically the rules for updating and * obeying fences. See enum dma_resv_usage for further descriptions. */ struct dma_resv *resv; /** @poll: for userspace poll support */ wait_queue_head_t poll; /** @cb_in: for userspace poll support */ /** @cb_out: for userspace poll support */ struct dma_buf_poll_cb_t { struct dma_fence_cb cb; wait_queue_head_t *poll; __poll_t active; } cb_in, cb_out; #ifdef CONFIG_DMABUF_SYSFS_STATS /** * @sysfs_entry: * * For exposing information about this buffer in sysfs. See also * `DMA-BUF statistics`_ for the uapi this enables. */ struct dma_buf_sysfs_entry { struct kobject kobj; struct dma_buf *dmabuf; } *sysfs_entry; #endif }; 

    Assuming this structure is what your driver expects, there are two possible solutions:

    1) Migrate to a more recent Debian

    Be careful, this is not a trivial operation, so be sure to back up any important documents before proceeding.

    If you're ready to migrate, correct /etc/apt/sources.list, for example as follows to switch to Debian testing (= trixie, currently):

    deb http://ftp.fr.debian.org/debian/ testing main contrib non-free-firmware deb http://security.debian.org testing-security main contrib non-free-firmware deb http://ftp.fr.debian.org/debian/ testing-updates main contrib non-free-firmware

    ... then run:

    sudo apt update sudo apt upgrade

    In any case... it will have to be done sooner or later :-)

    2) Install only a newer kernel

    If migration scares you, you can download the packages manually from packages.debian.org and then install them with dpkg -i.

    wget http://ftp.fr.debian.org/debian/pool/main/l/linux-signed-amd64/linux-image-amd64_6.4.11-1_amd64.deb wget http://ftp.fr.debian.org/debian/pool/main/l/linux-signed-amd64/linux-headers-amd64_6.4.11-1_amd64.deb sudo dpkg -i linux-*-amd64.deb

    (hoping that all dependencies are already installed!)

    3) Intermediate solution: /etc/apt/preferences

    The difficulty with solution (2) is that sometimes retrieving the dependencies is particularly tedious (this shouldn't happen in your case). The idea is then to reference in /etc/apt/sources.list the current repositories and also additional repositories (e.g. those from debian testing). This means copying and pasting the lines I've mentioned in solution (1) at the end of the current content of your file.

    To tell APT to prioritize the packages provided by debian testing under certain conditions, we define a file in /etc/apt/preferences.d/ with an arbitrary but sensible name (in your case, it could be /etc/apt/preferences.d/ms9120 since it's the name of the module you are trying to compile.

    Good luck

    1
    1. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108
       

      The definition of the struct dma_buf_map is missing and is causing the first warning reported, not that of dma_buf (which is indeed still in the kernel code).

      Installing testing if it has a stable version is indeed not trivial...

      But in his case, I don't see a definition of the struct dma_buf_map in kernel 6.4.0 and I don't think it would resolve his issue.

      https://elixir.bootlin.com/linux/v6.4/A/ident/dma_buf_map

      (returns: "Identifier not used")

      This struct is defined in kernel v5.11.22 for example:

      https://elixir.bootlin.com/linux/v5.11.22/A/ident/dma_buf_map

      (returns: "Defined in 1 files as a struct: include/linux/dma-buf-map.h, line 115")

      and it is in versions 5.11 to 5.17 of the Linux kernel.

      There is a good chance that the developer of this module that lenainjaune is trying to compile has developed his code for one of these versions of the kernel.

      1
    2. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
       

      Hello mamiemando and once again thank you for your (very) active participation in the questions :D

      I'm replying in the main thread, as the readability is not great in the comments

      0
  3. mamiemando Posted messages 33228 Registration date   Status Moderator Last intervention   7 940
     

    Hello,

    I agree with your feedback / responses / conclusions [Dal] and lenainjaune. Just one point:

    So if I understand correctly, I can't adapt the code to work with my current kernel

    Yes, but you need to understand how to replace the missing structure dma_buf_map with its replacement. This requires knowledge of kernel programming, understanding why this structure has disappeared, and how to deal with a more recent kernel.

    If I believe this exchange, it's apparently because the structure has been renamed to iosys_map (similarly, all functions prefixed with dma_buf_map_* have their prefix corrected accordingly). If that's the case, you would just need to fix your driver code to check the kernel version and abstract these symbols.

    It’s been a while since I did any kernel programming, but by referencing this discussion, you could correct each occurrence of:

    #include <linux/dma-buf-map.h>

    to:

    #ifdef __linux__ # if LINUX_VERSION_CODE < KERNEL_VERSION(6, 0, 0) # include <linux/dma-buf-map.h> # else # include <linux/iosys-map.h> # define dma_buf_map iosys_map # define DMA_BUF_MAP_INIT_VADDR IOSYS_MAP_INIT_VADDR # define dma_buf_map_set_vaddr iosys_map_set_vaddr # define dma_buf_map_set_vaddr_iomem iosys_map_set_vaddr_iomem # define dma_buf_map_is_equal iosys_map_is_equal # define dma_buf_map_is_null iosys_map_is_null # define dma_buf_map_is_set iosys_map_is_set # define dma_buf_map_clear iosys_map_clear # define dma_buf_map_memcpy_to iosys_map_memcpy_to # define dma_buf_map_incr iosys_map_incr # endif #endif

    For something cleaner, you could move this section of code into a header in the module folder you’re trying to compile (let's call it my-dfa-buf-map.h) and then replace:

    #include <linux/dma-buf-map.h>

    with:

    #include "my-dma-buf-map.h"

    All of this assumes it's just a matter of renaming...

    Good luck

    1
    1. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
       

      Great, it looks promising, I will test it as soon as possible. On the other hand, [Dal]'s proposal worked on Ubuntu 20.04 but for now the procedure is not very clear. I need to redo everything from scratch.

      0
  4. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108
     

    If you find the task of editing the project's files to apply the changes proposed by mamiemando in #10 daunting or if you're unsure of yourself, you can use find and sed to make the changes directly by running this line.

    This results in a single line, using GNU sed and find from your Linux system:

    find . -type f -not -path "./.git" -name "*.*" -exec sed -i 's/dma-buf-map.h/iosys-map.h/g; s/dma_buf_map/iosys_map/g; s/DMA_BUF_MAP_INIT_VADDR/IOSYS_MAP_INIT_VADDR/g; s/dma_buf_map_set_vaddr/iosys_map_set_vaddr/g; s/dma_buf_map_set_vaddr_iomem/iosys_map_set_vaddr_iomem/g; s/dma_buf_map_is_equal/iosys_map_is_equal/g; s/dma_buf_map_is_null/iosys_map_is_null/g; s/dma_buf_map_is_set/iosys_map_is_set/g; s/dma_buf_map_clear/iosys_map_clear/g; s/dma_buf_map_memcpy_to/iosys_map_memcpy_to/g; s/dma_buf_map_incr/iosys_map_incr/g; ' {} \; 

    Theoretically, the result is code that should compile on a recent system with a kernel and kernel headers supporting the new API iosys-map.h (starting from v5.18), provided that the semantic changes are correctly identified and that the changes are limited to that.

    However, the modified code would not compile on a system using the old API dma-buf-map.h (unlike mamiemando's solution which proposes compilation directives distinguishing between kernel versions).

    1
  5. mamiemando Posted messages 33228 Registration date   Status Moderator Last intervention   7 940
     

    Hello

    I created a multi-line command mainly to apply the fixes you propose without having to manually edit the files (I'm keeping the template for future projects and I'll try to refine it to find a less cumbersome version and hopefully with better readability :D). If you have more practical writing or command solutions, I'm all ears.

    As I mentioned in #10, the cleanest/simple solution would be to put the #defines I propose in a header and ensure that these #defines are applied to all the files involved in your module.

    I suggested "my-dma-buf-map.h" (arbitrary name). In the specific case of your module, since the file ms912x.h is included everywhere, you just need to add the directive at the beginning of "ms912x.h":

    #include "my-dma-buf-map.h"

    Note that you could also simply place the content of "my-dma-buf-map.h" directly in "ms912x.h". If you plan to propose your changes in a GitHub pull request, that would probably be the best, as there's really no reason to separate these considerations from "ms912x.h". The only advantage of keeping the separation is to isolate all our corrections from the existing code instead of hacking it up here and there.

    I propose the approach that almost worked for installation, taking into account the proposals from mamiemando and [Dal], which yield roughly the same result, but I'm not knowledgeable enough to determine the differences.

    The two solutions are equivalent but different in their execution. As a reminder, when compiling a program, there are actually several steps:

    1. precompilation (which only resolves instructions prefixed with a #): these are just "simple" tasks (#include = copy-paste; #define = renaming, etc.)
    2. compilation: each precompiled .c file is converted to .o form
    3. linking: the .o files are gathered to form the final binary (for example, an executable, or a library (.so or .a), or a module (.ko)).

    Returning to the two solutions we proposed to you:

    • In #10, I propose using #define to let the precompiler handle renaming;
    • In #17, [Dal] proposes to do the precompiler's job by correcting the sources with sed (so we're still before precompilation since we haven't even called gcc at this stage);
    • In both cases, once the precompilation step is passed, we end up with the same source code, which is why from a compilation perspective.

    Let's briefly discuss the advantages/disadvantages of the two solutions.

    • The disadvantage of #17 is that you have to perform the manipulation or not depending on the version of the kernel, while in #10, we allow the compiler to check the kernel version and based on that, decide if renaming is necessary or not. It's also an opportunity to do
      "#include <drm/drm_edid.h>"

      ... only if the kernel is at a version that provides this header (I'll let you check).

    • The disadvantage of #10 is that you have to add the extra file and include it everywhere, which can be tedious if the module involves a large number of files. But in this case, it's not a big problem; you just need to include "my-dma-buf-map.h" in "ms912x.h".
     Sep. 09 01:00:53 vm-bookworm kernel: ms912x: module verification failed: signature and/or required key missing - tainting kernel

    From what I see here, it looks like you haven't signed your module. I don't know if this is what's triggering the subsequent errors, but it's possible.

    This error is legitimate if you are using secure boot. Indeed, Linux then refuses to load a module (.ko) if it hasn't been signed by an authorized key in the BIOS.

    In practice, this requires creating a key pair, registering it in the BIOS (enroll), and using it to sign the module (sign), then rebooting. I've detailed the entire process in this tutorial (see the "Creating the key pair" and "Signing the module" sections).

    The issue of signing a module arises for any module you might compile, including those you would compile through APT packages *-dkms. To make it less tedious, recent Debians seem to generate a key pair (which needs to be enrolled), but this isn't very clear yet. In short, at worst, you can fall back on the previous paragraph.

    Good luck

    1
  6. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108
     

    Hello lenainjaune,

    The module loading fails, and it is due to the fact that the symbols used during the attempt to load the module drm_gem_shmem_dumb_create and drm_gem_shmem_prime_import_sg_table are unknown or inaccessible.

    These symbols are indeed exported by the Linux kernel 6.1, with EXPORT_SYMBOL_GPL() directives which are macros that make the symbols readily accessible to any module indicating a GPL-compatible license.

    https://elixir.bootlin.com/linux/v6.1/source/drivers/gpu/drm/drm_gem_shmem_helper.c#L771

    https://elixir.bootlin.com/linux/v6.1/source/drivers/gpu/drm/drm_gem_shmem_helper.c#L771

    (from kernel versions 5.2)

    I assume you are indeed trying to load the module on a 6.1 kernel.

    On its side, the module does indeed include a MODULE_LICENSE("GPL"); macro in ms912x_drv.c

    One hypothesis for your issue could then be that the loading fails because your module is trying to access a part of the kernel that is not loaded, and which would be one or more modules that constitute a dependency for your type of hardware.

    Can you run /usr/sbin/modinfo ./ms912x.ko to see what it says, especially in the "depends:" section?

    1
    1. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
       

      Hey [Dal]!

      I wanted to start over to understand what's blocking and also test what mamiemando suggested, but I haven't made much progress on the subject, so I haven't reported back (I was planning to get back to it this weekend, and I'm not giving up because I find it super interesting given the number of times I've thrown in the towel in front of sources that wouldn't compile without errors).

      To answer your question that I skimmed over. Kernel 6.1.0-10-amd64

      root@vm-bookworm:~/ms912x-main# /usr/sbin/modinfo ./ms912x.ko filename: /root/ms912x-main/./ms912x.ko license: GPL alias: usb:v534Dp0821d*dc*dsc*dp*icFFisc00ip00in* alias: usb:v534Dp6021d*dc*dsc*dp*icFFisc00ip00in* depends: drm_kms_helper,drm,usbcore,drm_shmem_helper retpoline: Y name: ms912x vermagic: 6.1.0-10-amd64 SMP preempt mod_unload modversions 

      Interestingly, drm_shmem_helper seems to be the dependency related to the Unknown symbol drm_gem_shmem_* messages from the kernel right after it was noted that it was tainted (after loading with insmod).

      And the currently loaded dependencies are:

      root@vm-bookworm:~/ms912x-main# lsmod \ | grep -Eo \ "^(drm_kms_helper|drm|usbcore|drm_shmem_helper)" \ | sort -u drm drm_kms_helper usbcore root@vm-bookworm:~/ms912x-main# modprobe drm_shmem_helper root@vm-bookworm:~/ms912x-main# lsmod \ | grep -c drm_shmem_helper 2 root@vm-bookworm:~/ms912x-main# insmod ./ms912x.ko # => no message, no error root@vm-bookworm:~/ms912x-main# journalctl -k | grep ms912x sept. 23 00:23:08 vm-bookworm kernel: ms912x: \\ loading out-of-tree module taints kernel. sept. 23 00:23:08 vm-bookworm kernel: ms912x: \\ module verification failed: \\ signature and/or required key missing - tainting kernel sept. 23 00:23:08 vm-bookworm kernel: \\ usbcore: registered new interface driver ms912x 

      Sweet :D! I'm going to tackle all of this again when I have a clear head :D!!!

      0
      1. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108 > lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention  
         

        Cool :-)

        0
  7. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
     

    @[Dal] StatusContributor

    I don't know which history of a previous thread you're talking about :-)

    No, obviously since it only existed until I stupidly clicked on a link during its preview :D, which opened the link in the same tab and thus made me lose all the content of the post I was writing (for those who encounter this, there supposedly exists a solution that consists of saving the data in RAM for the browser process - possible lead here on Linux and requires the package gdb; furthermore, there are also addons like this one for Firefox that keep form data).

    That said, I think your problem is related to the fact that you do not have a Linux system with a kernel and the corresponding headers compatible with the module you are trying to compile.

    I am more and more convinced of this. The kernel 5.10.0-20-amd64 under Debian 11 gives as the first error error: conflicting types for dma_buf_map according to your analysis as it is unhandled by the kernel and I just attempted to compile under Debian 12 with the kernel 6.1.0-10-amd64 (yes I have virtual machines all set up for tests, no need to migrate a system ;-) ) :

    root@vm-bookworm:~/ms912x-main# make make CHECK="/usr/bin/sparse" -C /lib/modules/6.1.0-10-amd64/build M=/root/ms912x-main modules make[1] : entering directory « /usr/src/linux-headers-6.1.0-10-amd64 » CC [M] /root/ms912x-main/ms912x_registers.o In file included from /root/ms912x-main/ms912x_registers.c:4: /root/ms912x-main/ms912x.h:113:47: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration 113 | const struct dma_buf_map *map, | ^~~~~~~~~~~ CC [M] /root/ms912x-main/ms912x_connector.o In file included from /root/ms912x-main/ms912x_connector.c:7: /root/ms912x-main/ms912x.h:113:47: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration 113 | const struct dma_buf_map *map, | ^~~~~~~~~~~ /root/ms912x-main/ms912x_connector.c: In function ‘ms912x_read_edid’: /root/ms912x-main/ms912x_connector.c:12:30: error: ‘EDID_LENGTH’ undeclared (first use in this function) 12 | int offset = block * EDID_LENGTH; | ^~~~~~~~~~~ /root/ms912x-main/ms912x_connector.c:12:30: note: each undeclared identifier is reported only once for each function it appears in /root/ms912x-main/ms912x_connector.c: In function ‘ms912x_connector_get_modes’: /root/ms912x-main/ms912x_connector.c:26:16: error: implicit declaration of function ‘drm_do_get_edid’ [-Werror=implicit-function-declaration] 26 | edid = drm_do_get_edid(connector, ms912x_read_edid, ms912x); | ^~~~~~~~~~~~~~~ /root/ms912x-main/ms912x_connector.c:26:14: warning: assignment to ‘struct edid *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] 26 | edid = drm_do_get_edid(connector, ms912x_read_edid, ms912x); | ^ /root/ms912x-main/ms912x_connector.c:28:15: error: implicit declaration of function ‘drm_add_edid_modes’ [-Werror=implicit-function-declaration] 28 | ret = drm_add_edid_modes(connector, edid); | ^~~~~~~~~~~~~~~~~~ cc1: some warnings being treated as errors make[2]: *** [/usr/src/linux-headers-6.1.0-10-common/scripts/Makefile.build:255 : /root/ms912x-main/ms912x_connector.o] Error 1 make[1]: *** [/usr/src/linux-headers-6.1.0-10-common/Makefile:2037 : /root/ms912x-main] Error 2 make[1] : leaving directory « /usr/src/linux-headers-6.1.0-10-amd64 » make: *** [Makefile:15 : modules] Error 2 

    and I note that the first error is error: ‘EDID_LENGTH’ undeclared different from that of Debian 11 and the previous warning is warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration

    => this clearly indicates that the kernel change has resolved the problem mentioned in my first post but that not everything is resolved yet!

    Moreover, in the section of questions about the ms912x project, after compilation someone faces this error insmod: ERROR: could not insert module ms912x.ko: Unknown symbol in module
    from dmseg:
    and asks for the version of the kernel to be used to compile the program successfully, but as I said, there is little activity on the project and it clearly lacks information. I wouldn't say it is abandoned but certainly on pause.

    We're making progress ...

    A search on elixir.bootlin.com shows that, in the Linux kernel, it is the header dma-buf-map.h that defines struct dma_buf_map (see my edit below*).

    I didn't know about the tool elixir.bootlin.com so I went to take a look and tested which kernels use EDID_LENGTH, and from what I understand it seems to have always been supported (it seems logical to me since EDID is - if I remember correctly, a protocol used for screens to identify themselves to a system using the video cable as a transmission medium, so even for VGA) but now I don’t understand why it wouldn’t include the header edid.h either directly or indirectly through another header, but I don’t know how to determine that...

    Also, according to what the person says in the project questions, we could replace dma_buf_map with iosys_map and since I didn’t understand why I did some research and found this and by correlating with your tool, I conclude this:

    dma_buf_map (v5.11-rc1 -> v5.18-rc3)

    iosys_map (v5.18-rc1 -> v6.5.1 currently)

    This tool is amazing :D !

    Moreover, in the section of questions about the project there is also a person who asks for the possibility of being compiled with kernel v6+ (which should yield a result similar to mine - see above).

    => by deduction: since the code uses dma_buf_map, I should test all kernels between v5.11-rc1 and v5.18-rc3.

    This seems to confirm what you are suggesting ...

    So the idea is to test them one by one?

    If I'm not mistaken, there are 308 candidate kernels where dma_buf_map exists ...

    Is there no way to reduce and target this list because I have other things to do than test 300 kernels :( ?

    What exactly do you need this module for?

    I didn't want to bring this up, as I think it can be controversial. I acquired a cheap USB/VGA adapter to easily connect an additional VGA screen via USB. I managed to get it working on Windows 10 in a VM with the drivers provided on disc (today I can no longer do it, I don’t understand why and I’m not going to waste time on a proprietary system); I was hoping to do the same under Linux.

    root@vm-bookworm:~/ms912x-main# lsusb | grep -i vga Bus 001 Device 003: ID 534d:6021 MacroSilicon VGA Display Adapter

    There and there I read that it cannot be done under Linux, that no one is developing a driver, that you need to spend money on equipment or change hardware solution ... until I found this which seems to perfectly fit my needs since as indicated in the first sentence Linux kernel driver for MacroSilicon USB to VGA/HDMI adapter. VID/PID is 534d:6021. Device is USB2.0

    Hey! It seems that it's exactly what I need ... :D

    So the first step is to successfully compile, the second will be to load it, and the third is to understand why ... the display still doesn’t work :D (I’m used to that under Linux) and maybe one day it will work but in any case I would have learned a lot to get there.

    There you go ... :D

    @mamiemando StatusModerator

    1) Migrate to a newer Debian

    2) Install only a newer kernel

    No need, for now I’m testing in VM

    In any case... it will have to be done sooner or later :-)

    Or ... not ;D! I’m half joking but as long as I haven’t found the right method to migrate applications, settings, I will stay on Debian 11. It must be said that I have a lot of things to migrate and that well, the idea that "since it doesn't work, you have to upgrade to version n+1" yes in theory, in practice, not so much! For the record before installing Debian 11 on my personal PC I was on Ubuntu 12.04 LTS and overall it worked very well and I was able to live with it until about 2 years ago I think when software requirements forced me to change systems!

    wget http://ftp.fr.debian.org/debian/pool/main/l/linux-signed-amd64/linux-image-amd64_6.4.11-1_amd64.deb
    wget http://ftp.fr.debian.org/debian/pool/main/l/linux-signed-amd64/linux-headers-amd64_6.4.11-1_amd64.deb
    sudo dpkg -i linux-*-amd64.deb


    (hoping that all dependencies are already installed!)

    Little typo:

    sudo dpkg -i linux-*-amd64.deb

    ->

    sudo dpkg -i linux-*amd64.deb

    no hyphen to the left of amd64.deb otherwise dpkg won’t install!

    2) Install only a newer kernel

    3) Intermediate solution: /etc/apt/preferences

    Yes, as I said to [Dal], I will try to target the candidate kernel versions.

    0
    1. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108
       

      Hello lenainjaune,

      In principle, I will try with the latest version to integrate the structure definition, which is a 5.11 (5.11.22), but this is a kernel version that has not been adopted by a distribution (which means you would be working on a system not supported by your distribution) and installing a specific version of the Linux kernel is not easy.

      So, I have another lead. The module code was posted in May 2022.

      According to https://en.wikipedia.org/wiki/Linux_kernel_version_history#Releases_5.x.y, at that date, the following distributions included a 5.15 version of the Linux kernel:

      * Ubuntu 22.04 LTS,
      * Slackware 15,
      * UEK 7

      It is possible that the developer used one of these distributions :-)

      You could create a virtual machine with Ubuntu 22.04 LTS to test.

      1
      1. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62 > [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention  
         

        It's a ... good find :D! Before clarifying and detailing ... As you suggested, I tested it on Ubuntu 22.04 LTS with the original kernel at v5.15.0-43 (note: if you choose to update the system, you'll get a different kernel that isn't compatible with the module; for my part, I had v6.2.0-32 and I had to permanently change the boot kernel in Grub2). In the end, Ubuntu does recognize a 2nd screen that we can use in clone mode for example, BUT be careful, it's difficult to use in extended desktop mode, as the mouse can't reach it (I think it's a matter of adding a 2nd virtual screen or something like that, I'll need to look into it).

        0
  8. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
     

      @[Dal] StatusContributor

    You could create a virtual machine with Ubuntu 22.04 LTS to test.

    I installed the Ubuntu VM this morning and I will test the kernel installation this evening when I get back.

    So if I understand correctly, I can't adapt the code to work with my current kernel, or I would need to adapt parts of one operational kernel to another kernel, but then all the dependencies would need to be installed and not conflict. That seems really tricky and perhaps not possible :( (I don't think anything can be truly IMpossible :D) ...

    I have an inspiration that connects several ideas and technologies ...

    Roughly speaking: I test the operational kernel on Ubuntu and see if the compilation succeeds, ultimately if the screen is detected and it works, then I try to install this operational kernel on the "right" Debian and see if it works too. If I find a functional association, maybe I could consider going through a container (among other things if passing this USB3 device in passthrough is possible) to ultimately have an autonomous operational kernel (from what I've understood containers are more responsive and resource-efficient than VMs, in terms of storage space, CPU, RAM) to then be theoretically installed on any distribution.

    This direction excites me a lot as it would resolve many of my other projects :D

    To be continued...

    Note: if you're wondering why I'm so eager to install it on a Debian distribution rather than Ubuntu (its baby, you know ;) ), I would reply (without starting a controversy) that I am extremely disappointed with the commercial direction Canonical/Ubuntu has taken in recent years and that I am boycotting this distribution (even though in the past I sang its praises).


    0
    1. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108
       

      Hi lenainjaune,

      Containers typically share the same kernel as the host, so I don't think you can use this process.

      As mamiemando found the original post from the kernel maintainers that addresses changes to the API:

      https://lore.kernel.org/lkml/aa1312fc-197b-c1ab-6a18-369d49c1e8f8@xs4all.nl/t/

      we have a correspondence list of all the types modified during the change and there's a good chance you can modify the module code following the recommendations from mamiemando to try to get it working with your current Debian kernel, if there haven't been any changes other than semantic ones.

      This seems to be the least "Rube Goldberg" way to go :-)

      0
  9. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
     

    Roughly speaking: ... I'm trying to install this operating kernel on the "right" Debian and see if it works too,

    I also managed to compile the module on a Debian 11 VM after compiling the v5.15.0-43 kernel by following this tutorial adapted for v5.15.0-43 and after 3-4 hours of compilation (I think I'm going to study parallelization seriously and add CPUs to the VM because it’s really slow ;) ), I find that I can load the module without errors BUT that the screen is not detected either in GUI or CLI. But I'm not sure yet if the problem is solely due to virtualization (USB2 or 3 Controller, X/Wayland display server, etc.). I will test on a physical PC to see.

    => so it's "almost" a success!

    @[Dal] StatusContributor

    Containers normally share the same kernel as the host, so I don't think you can use this process.

    Oops, you're right :( ! I was so caught up in the excitement that I made shortcuts. Apparently, containers do not carry their own kernel (to verify if it’s entirely impossible). So we go back to virtualization ... and to take advantage of an external screen doesn't seem the most practical way (we'll see!).

    As mamiemando found the original post from the kernel maintainers that implements changes to the API:

    https://lore.kernel.org/lkml/aa1312fc-197b-c1ab-6a18-369d49c1e8f8@xs4all.nl/t/

    Yes, I had also mentioned it in my earlier posts because I needed to dig into why the person had replaced dma_buf_map with iosys_map in the module's code with kernel v5.19.0. To be honest, I had no idea what that could be useful for and how to use it. So I'm going to try mamiemando's suggestion

    => by deduction: since we use dma_buf_map in the code, I should test all kernels between v5.11-rc1 and v5.18-rc3.

    I received feedback in the section of the questions of the ms912x project. It would apparently be possible to compile with all kernels between v5.11 and v5.17.

    Moreover, according to this contributor, the X server would not be the best for allowing the management of an additional screen with this module (again, I'm getting information bit by bit) and it would be better to use Wayland from what I understand.

    In my case, Ubuntu uses Wayland and Debian uses X11, that’s probably the underlying problem!

    The catch is that my preferred desktop environment is XFCE4 and it would only handle Wayland starting from XFCE4 v4.18 and even more specifically v4.20 (under Debian 11 XFCE4 is at v4.16 and we cannot apparently upgrade), and moreover I haven't managed to get lightdm to use xwayland (I've only skimmed the topic).

    I think I've encountered the tip of the iceberg ...

    From what I understand, for it to work, goodbye XFCE4/lightdm ...

    I still need to dig deeper ...


    0
    1. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108
       

      I received feedback in the questions section of the ms912x project. It should be possible to compile with all kernels between v5.11 and v5.17.

      Yes :-)

      That's what I suspected given my analysis of the first warning in your compilation, which could only mean that your kernel did not include the definition of the struct dma_buf_map used as an argument by the module functions, which allowed me to identify these versions 5.11 and v5.17 thanks to Elixir.

      To return to the answer to your initial question, as mentioned in my first message, the first warning or error is generally the most relevant.

      When the compiler sends this kind of warning:

      "In file included from /root/ms912x-main/ms912x_registers.c:4:
      /root/ms912x-main/ms912x.h:113:19: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration"

      where it warns you that it thinks you are declaring a struct in a function's parameters, it actually means that the compiled code does not understand the definition of the struct that should be located elsewhere (normally, in a header).

      Your compilation problem is solved :-)

      Now, whether the module works well with a particular desktop environment or graphical service may be related to the limitations of the module itself, or even to inherent problems with the Linux kernel used that may have been resolved in more recent versions. These issues are a bit far from the topic of C programming.

      You should try modifying the module code to attempt to make it work on a recent kernel with different desktop environments or graphical services, as the fact that you are working on a recent kernel and system can also influence the final functioning of your hardware with everything else.

      You could then contribute the modified module code and make a few other users with the same hardware as you happy :-).

      1
      1. mamiemando Posted messages 33228 Registration date   Status Moderator Last intervention   7 940 > [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention  
         

        Hello

        after compiling the kernel v5.15.0-43

        • As a rule, you have no reason to compile the kernel yourself; that's the whole point of the combination offered by the linux-image-amd64 and linux-headers-amd64 packages. It deploys a compiled kernel and the headers needed to compile a module compatible with that kernel.
        • If you really want to compile your own kernel (which I would not recommend), you can look at the -j option of make (see this discussion). Moreover, compilation performance is degraded on a VM. Once the compilation is done, it is interesting to package with make deb-pkg which replaces make-kpkg (see this discussion).
        • Returning to the initial problem, the most sustainable/proper/simple/green approach remains (assuming it's correct) the one I recommended in #10.

        Good luck

        1
  10. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
     

    From what I understand, for it to work, exit XFCE4/lightdm ...

    For those interested, the installation on Debian 11 with a supported kernel (tested with a v5.15.43 compiled kernel) and the Gnome/GDM/xWayland trio was successful without much surprise, and the screen is detected; however, in my virtual environment (Qemu/KVM), there is a bug because once detected, the mouse behaves strangely.

    @[Dal] StatusContributor

    When the compiler gives you this kind of warning:

    "In file included from /root/ms912x-main/ms912x_registers.c:4:
    /root/ms912x-main/ms912x.h:113:19: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration"

    where it warns you that it thinks you are declaring a struct in the parameters of a function, it actually means that the compiled code does not understand the definition of the struct which should be located elsewhere (usually in a header).

    OK, I will keep this information carefully because the message is confusing!

    Now, the fact that the module works well, with this or that desktop manager or graphical service, may be related to the limitations of the module itself, or even to inherent problems in the used Linux kernel that may have been resolved in more recent versions. These issues are somewhat far from the topic of C programming.

    Yes completely, I will continue my investigations elsewhere and refocus on compiling the module ...

    You could then contribute the modified module code and make some other users with the same hardware as you happy :-).

    If you find the task of editing the project files to apply the changes proposed by mamiemando in #10 daunting or if you're not sure of yourself, you can use find and sed to apply the changes by executing this line.

    Proposition #10 from mamiemando :

    root@vm-bookworm:~# diff ~/ms912x-main/ms912x.h ~/ms912x-main_origin/ms912x.h 8,26d7 < #include <linux/version.h> < #ifdef __linux__ < # if LINUX_VERSION_CODE < KERNEL_VERSION(6, 0, 0) < # include <linux/dma-buf-map.h> < # else < # include <linux/iosys-map.h> < # define dma_buf_map iosys_map < # define DMA_BUF_MAP_INIT_VADDR IOSYS_MAP_INIT_VADDR < # define dma_buf_map_set_vaddr iosys_map_set_vaddr < # define dma_buf_map_set_vaddr_iomem iosys_map_set_vaddr_iomem < # define dma_buf_map_is_equal iosys_map_is_equal < # define dma_buf_map_is_null iosys_map_is_null < # define dma_buf_map_is_set iosys_map_is_set < # define dma_buf_map_clear iosys_map_clear < # define dma_buf_map_memcpy_to iosys_map_memcpy_to < # define dma_buf_map_incr iosys_map_incr < # endif < #endif < 137c118 < #endif --- > #endif \ No newline at end of file 

    Note: LINUX_CODE_VERSION implies #include <linux/version.h> otherwise it will not be defined and we will get an equally confusing error error: missing binary operator before token "("

    root@vm-bookworm:~# uname -r 6.1.0-10-amd64 root@vm-bookworm:~# make ... /root/ms912x-main/ms912x_connector.c:12:30: error: ‘EDID_LENGTH’ undeclared (first use in this function) 12 | int offset = block * EDID_LENGTH; ...

    => we get the same error as #6

    The proposal from [Dal] #17 gives the same result

    I therefore searched for the EDID_LENGTH error and found this and by quickly scanning the exchange, I found this which suggests adding the header #include <drm/drm_edid.h>; I tested adding it to ms912.h and this time I got:

    root@vm-bookworm:~/ms912x-main# make make CHECK="/usr/bin/sparse" -C /lib/modules/6.1.0-10-amd64/build M=/root/ms912x-main modules make[1] : entering directory « /usr/src/linux-headers-6.1.0-10-amd64 » CC [M] /root/ms912x-main/ms912x_registers.o CC [M] /root/ms912x-main/ms912x_connector.o CC [M] /root/ms912x-main/ms912x_transfer.o CC [M] /root/ms912x-main/ms912x_drv.o LD [M] /root/ms912x-main/ms912x.o MODPOST /root/ms912x-main/Module.symvers CC [M] /root/ms912x-main/ms912x.mod.o LD [M] /root/ms912x-main/ms912x.ko BTF [M] /root/ms912x-main/ms912x.ko /bin/sh: 1: ./tools/bpf/resolve_btfids/resolve_btfids: not found make[2]: *** [/usr/src/linux-headers-6.1.0-10-common/scripts/Makefile.modfinal:63 : /root/ms912x-main/ms912x.ko] Error 127 make[2]: *** Deleting file « /root/ms912x-main/ms912x.ko » make[1]: *** [/usr/src/linux-headers-6.1.0-10-common/Makefile:1954 : modules] Error 2 make[1] : leaving directory « /usr/src/linux-headers-6.1.0-10-amd64 » make: *** [Makefile:15 : modules] Error 2 

    Well anyway! I don't know where I'm going, I’ll have to review this when I have a clear mind.

    @mamiemando StatusModerator

    after compiling kernel v5.15.0-43

    • In principle, you shouldn't have any reason to compile the kernel yourself; that's the whole point of the tandem offered by the linux-image-amd64 and linux-headers-amd64 packages. It deploys a compiled kernel and the headers necessary to compile a module compatible with that kernel.

    Yes, I agree, but since some distributions (for example, my current one, Debian 11) I don't really have a choice (reminder: my_buf_map v5.11-rc1 -> v5.18-rc3 and iosys_map v5.18-rc1 -> v6.5.1 currently).

    root@vm-bullseye-xfce:~# apt-cache search linux-image | grep headers linux-headers-5.10.0-20-amd64 - Header files for Linux 5.10.0-20-amd64 linux-headers-5.10.0-20-cloud-amd64 - Header files for Linux 5.10.0-20-cloud-amd64 linux-headers-5.10.0-20-rt-amd64 - Header files for Linux 5.10.0-20-rt-amd64 linux-headers-5.10.0-22-amd64 - Header files for Linux 5.10.0-22-amd64 linux-headers-5.10.0-22-cloud-amd64 - Header files for Linux 5.10.0-22-cloud-amd64 linux-headers-5.10.0-22-rt-amd64 - Header files for Linux 5.10.0-22-rt-amd64 linux-headers-5.10.0-25-amd64 - Header files for Linux 5.10.0-25-amd64 linux-headers-5.10.0-25-cloud-amd64 - Header files for Linux 5.10.0-25-cloud-amd64 linux-headers-5.10.0-25-rt-amd64 - Header files for Linux 5.10.0-25-rt-amd64 

    0
  11. mamiemando Posted messages 33228 Registration date   Status Moderator Last intervention   7 940
     

    Hello,

    Note: LINUX_CODE_VERSION requires #include <linux/version.h> otherwise it will not be defined and we will get a confusing error error: missing binary operator before token "("

    Indeed.

     /bin/sh: 1: ./tools/bpf/resolve_btfids/resolve_btfids: not found

    Strange this error. The package libbpf-tools doesn’t seem to provide this executable. Check this discussion just in case.

    I really don't have a choice (reminder: ma_buf_map v5.11-rc1 -> v5.18-rc3 and iosys_map v5.18-rc1 -> v6.5.1 currently)

    The purpose of the #define I mentioned is precisely to lift this constraint by replacing the obsolete symbols with the current ones.

    Good luck

    0
  12. [Dal] Posted messages 6122 Registration date   Status Contributor Last intervention   1 108
     

    Hello lenainjaune,

    "resolve_btfids" is a tool used during kernel compilation and apparently for modules.

    It is indeed present in Linux v6.1.0 that you are using:

    https://elixir.bootlin.com/linux/v6.1/source/tools/bpf/resolve_btfids

    but I haven't seen any Debian packages that provide it (neither in source nor in binary).

    With the kernel code corresponding to Linux v6.1.0, you could compile this binary.

    https://github.com/torvalds/linux/tree/v6.1

    The code and the Makefile are located in tools/bpf/resolve_btfids

    Then, you can copy the binary into the module build directory at the expected location, as suggested here:

    https://aur.archlinux.org/packages/linux-git-headers?O=10#comment-836241

    mkdir -p /lib/modules/$(uname -r)/build/tools/bpf/resolve_btfids/ cp /tmp/mybuilddir/linux-git/linux-git/src/linux/tools/bpf/resolve_btfids/resolve_btfids /lib/modules/$(uname -r)/build/tools/bpf/resolve_btfids/ 

    (replacing the 1st argument of the cp with the location of your binary)

    0
  13. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
     

    News ...

    Installing the ms912x module on Debian 12 with the default kernel (v6.1.0-10)

    @[Dal] StatusContributor

    Skipping BTF generation for /root/ms912x-main/ms912x.ko due to unavailability of vmlinux

    ...

    resolve_btfids" is a tool used during kernel and apparently module compilation. [...] With the kernel code corresponding to Linux v6.1.0, you could compile this binary.

    I followed your recommendations (requires the package libelf-dev) but it was not enough to eliminate the warning, I had to follow this procedure (I successfully tested solution 1) and then the warning disappeared. Moreover, this warning does not seem to be blocking, so we can presumably ignore it (see below).

    In the end, the module was compiled successfully BUT when loading it, I encountered an "Unknown symbol in module" error that I have not yet been able to resolve, and as a result, the module does not load (see details below).

    @mamiemando StatusModerator

    I created a multi-line command primarily to apply the fixes you propose without having to manually edit the files (I keep the template for future projects and I will try to refine it to find a less cumbersome version, hopefully with better readability :D). If you have more practical writing or command solutions, I'm all ears.

    ---

    I suggest the process that almost succeeded for installation, taking into account the proposals from mamiemando and [Dal], which yield approximately the same result, but I'm not savvy enough to determine the differences.

    Preparations and source code retrieval

    root@vm-bookworm:~# apt update root@vm-bookworm:~# apt install -y build-essential gcc make cmake gettext git g++ intltool dbus-user-session vim sshfs root@vm-bookworm:~# wget https://github.com/rhgndf/ms912x/archive/refs/heads/main.zip root@vm-bookworm:~# unzip main.zip Archive: main.zip 86aac85d4347af8893c74cb8b7023187fc2e2f8b creating: ms912x-main/ inflating: ms912x-main/.clang-format inflating: ms912x-main/.gitignore inflating: ms912x-main/Makefile inflating: ms912x-main/README.md creating: ms912x-main/captures/ inflating: ms912x-main/captures/resolutions inflating: ms912x-main/insmod.sh inflating: ms912x-main/ms912x.h inflating: ms912x-main/ms912x_connector.c inflating: ms912x-main/ms912x_drv.c inflating: ms912x-main/ms912x_registers.c inflating: ms912x-main/ms912x_transfer.c inflating: ms912x-main/registers root@vm-bookworm:~# cp -a ms912x-main ms912x-main_origin root@vm-bookworm:~# cd ms912x-main

    The kernel headers

    root@vm-bookworm:~/ms912x-main# make make CHECK="/usr/bin/sparse" -C /lib/modules/6.1.0-10-amd64/build M=/root/ms912x-main modules make[1]: *** /lib/modules/6.1.0-10-amd64/build : No such file or directory. Stopping. make: *** [Makefile:15 : modules] Error 2 # => kernel headers missing # solution : https://devicetests.com/fixing-ubuntu-error-no-such-file-or-directory root@vm-bookworm:~# dpkg -l | grep $( uname -r ) ii linux-image-6.1.0-10-amd64 6.1.38-1 amd64 Linux 6.1 for 64-bit PCs (signed) root@vm-bookworm:~# apt install -y linux-headers-$( uname -r ) root@vm-bookworm:~# dpkg -l | grep $( uname -r ) ii linux-headers-6.1.0-10-amd64 6.1.38-1 amd64 Header files for Linux 6.1.0-10-amd64 ii linux-image-6.1.0-10-amd64 6.1.38-1 amd64 Linux 6.1 for 64-bit PCs (signed)

    Missing header for dma_buf_map

    root@vm-bookworm:~/ms912x-main# make make CHECK="/usr/bin/sparse" -C /lib/modules/6.1.0-10-amd64/build M=/root/ms912x-main modules make[1] : entering directory « /usr/src/linux-headers-6.1.0-10-amd64 » CC [M] /root/ms912x-main/ms912x_registers.o In file included from /root/ms912x-main/ms912x_registers.c:4: /root/ms912x-main/ms912x.h:113:47: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration 113 | const struct dma_buf_map *map, | ^~~~~~~~~~~ CC [M] /root/ms912x-main/ms912x_connector.o In file included from /root/ms912x-main/ms912x_connector.c:7: /root/ms912x-main/ms912x.h:113:47: warning: ‘struct dma_buf_map’ declared inside parameter list will not be visible outside of this definition or declaration 113 | const struct dma_buf_map *map, | ^~~~~~~~~~~ /root/ms912x-main/ms912x_connector.c: In function ‘ms912x_read_edid’: /root/ms912x-main/ms912x_connector.c:12:30: error: ‘EDID_LENGTH’ undeclared (first use in this function) 12 | int offset = block * EDID_LENGTH; | ^~~~~~~~~~~ /root/ms912x-main/ms912x_connector.c:12:30: note: each undeclared identifier is reported only once for each function it appears in /root/ms912x-main/ms912x_connector.c: In function ‘ms912x_connector_get_modes’: /root/ms912x-main/ms912x_connector.c:26:16: error: implicit declaration of function ‘drm_do_get_edid’ [-Werror=implicit-function-declaration] 26 | edid = drm_do_get_edid(connector, ms912x_read_edid, ms912x); | ^~~~~~~~~~~~~~~ /root/ms912x-main/ms912x_connector.c:26:14: warning: assignment to ‘struct edid *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion] 26 | edid = drm_do_get_edid(connector, ms912x_read_edid, ms912x); | ^ /root/ms912x-main/ms912x_connector.c:28:15: error: implicit declaration of function ‘drm_add_edid_modes’ [-Werror=implicit-function-declaration] 28 | ret = drm_add_edid_modes(connector, edid); | ^~~~~~~~~~~~~~~~~~ cc1: some warnings being treated as errors make[2]: *** [/usr/src/linux-headers-6.1.0-10-common/scripts/Makefile.build:255 : /root/ms912x-main/ms912x_connector.o] Error 1 make[1]: *** [/usr/src/linux-headers-6.1.0-10-common/Makefile:2037 : /root/ms912x-main] Error 2 make[1] : leaving directory « /usr/src/linux-headers-6.1.0-10-amd64 » make: *** [Makefile:15 : modules] Error 2 root@vm-bookworm:~/ms912x-main# make clean make -C /lib/modules/6.1.0-10-amd64/build M=/root/ms912x-main clean make[1] : entering directory « /usr/src/linux-headers-6.1.0-10-amd64 » make[1] : leaving directory « /usr/src/linux-headers-6.1.0-10-amd64 » rm -f /root/ms912x-main/Module.symvers /root/ms912x-main/*.ur-safe

    Supporting the dma_buf_map header

    Solution from mamiemando

    Selective header based on kernel version (see #10)

    Objective of the multi-line command: reproducibility and to avoid errors

    Note: \n = newline

    Notes on writing a complex multi-line command:

    • ... | ... | ... : a piped multi-command
    • ( ... \n ... ) > temp : output of a multi-command into a file
    • echo "..." > /dev/null : my DIY comment :D !
    • ... \\n ... : multi-line command (\ strict last before \n)
    • cat <<'EOF' \n ... \n ... \n EOF : heredoc (simplifies writing)
       WARNING: EOF must be strictly the only text on the line
    • \ strict end of line required on lines NOT in heredoc

    Notes on commands:

    • tac | ... | tac : allows handling by last match
    • tac -b : resolves the strange handling of the last line
        (avoids having to manually add a line at the end of the
        input command)
    • awk ...  { ... next } : prevents a next match
    • awk ... '...'\n'...' : multi-line (NO spaces around the \n)
    • ; end of line mandatory between piped command blocks
    • if [[ $( ! grep ... ) ]] ; then ... \n fi : prevents an additional
        insertion if it is already in place

    Note: to understand the structure of the command, download the ms912x.h file from its ZIP archive

    root@vm-bookworm:~/ms912x-main# \ src=ms912x.h ; \ ( echo "start preceded by \n -> last #include <linux> /dev/null ; \ echo ; \ cat $src | tac -b \ | awk '/^#include <linux>/ { print ; r = 1 ; next } r' \ | tac ; \ \ echo "insertion \n and header for *VERSION* needed" > /dev/null ; \ if [[ ! $( grep "#include <linux>" $src ) ]] ; then \ cat <<'EOF' #include <linux> EOF fi \ echo "other #include -> last #define DRIVER_*" > /dev/null ; \ cat $src | tac -b \ | awk \ ' /^#define DRIVER_/ { print ; r = 1 ; next } '\ ' /^#include <linux>/ { exit } '\ ' r' \ | tac ; \ \ echo "adding kernel version handling" > /dev/null ; \ if [[ ! $( grep "include <linux>" $src \ ) ]] ; then cat <<'EOF' #ifdef __linux__ # if LINUX_VERSION_CODE < KERNEL_VERSION(6, 0, 0) # include <linux> # else # include <linux> # define dma_buf_map iosys_map # define DMA_BUF_MAP_INIT_VADDR IOSYS_MAP_INIT_VADDR # define dma_buf_map_set_vaddr iosys_map_set_vaddr # define dma_buf_map_set_vaddr_iomem iosys_map_set_vaddr_iomem # define dma_buf_map_is_equal iosys_map_is_equal # define dma_buf_map_is_null iosys_map_is_null # define dma_buf_map_is_set iosys_map_is_set # define dma_buf_map_clear iosys_map_clear # define dma_buf_map_memcpy_to iosys_map_memcpy_to # define dma_buf_map_incr iosys_map_incr # endif #endif EOF fi \ echo "... rest untouched" > /dev/null ; \ cat $src | tac -b \ | awk '/^#define DRIVER_/ { exit } 1' \ | tac \ ) > temp</linux></linux></linux></linux></linux></linux></linux></linux>

    Preliminary check that the above command gives the correct result and replace the contents of ms912x.h with that of temp

    root@vm-bookworm:~/ms912x-main# diff ms912x.h temp 7a8,9 > #include <linux> > 21a24,42 > #ifdef __linux__ > # if LINUX_VERSION_CODE < KERNEL_VERSION(6, 0, 0) > # include <linux> > # else > # include <linux> ... > # define dma_buf_map_incr iosys_map_incr > # endif > #endif > > 118c139,140 < #endif \ No newline at end of file --- > #endif > root@vm-bookworm:~/ms912x-main# mv temp ms912x.h</linux></linux></linux>

    Solution from [Dal]

    See #17

    root@vm-bookworm:~/ms912x-main# find . -type f -not -path "./.git" -name "*.*" -exec sed -i 's/dma-buf-map.h/iosys-map.h/g; s/dma_buf_map/iosys_map/g; s/DMA_BUF_MAP_INIT_VADDR/IOSYS_MAP_INIT_VADDR/g; s/dma_buf_map_set_vaddr/iosys_map_set_vaddr/g; s/dma_buf_map_set_vaddr_iomem/iosys_map_set_vaddr_iomem/g; s/dma_buf_map_is_equal/iosys_map_is_equal/g; s/dma_buf_map_is_null/iosys_map_is_null/g; s/dma_buf_map_is_set/iosys_map_is_set/g; s/dma_buf_map_clear/iosys_map_clear/g; s/dma_buf_map_memcpy_to/iosys_map_memcpy_to/g; s/dma_buf_map_incr/iosys_map_incr/g; ' {} \; root@vm-bookworm:~/ms912x-main# diff -qr ~/ms912x-main ~/ms912x-main_origin/ The files /root/ms912x-main/ms912x.h and /root/ms912x-main_origin/ms912x.h are different The files /root/ms912x-main/ms912x_transfer.c and /root/ms912x-main_origin/ms912x_transfer.c are different root@vm-bookworm:~/ms912x-main# diff ~/ms912x-main/ms912x.h ~/ms912x-main_origin/ms912x.h 113c113 < const struct iosys_map *map, --- > const struct dma_buf_map *map, root@vm-bookworm:~/ms912x-main# diff ~/ms912x-main/ms912x_transfer.c ~/ms912x-main_origin/ms912x_transfer.c 160c160 < const struct iosys_map *map, --- > const struct dma_buf_map *map,

    Missing header for EDID handling

    Note: for the details of the multi-line command see the notes above in mamiemando's solution

    root@vm-bookworm:~/ms912x-main# make ... /root/ms912x-main/ms912x_connector.c:12:30: error: ‘EDID_LENGTH’ undeclared (first use in this function) 12 | int offset = block * EDID_LENGTH; | ^~~~~~~~~~~ root@vm-bookworm:~/ms912x-main# make clean root@vm-bookworm:~/ms912x-main# \ src=ms912x.h ; \ ( echo "start preceded by \n -> last #include" > /dev/null ; \ echo ; \ cat $src | tac -b \ | awk '/^#include.+/ { print ; r = 1 ; next } r' \ | tac ; \ \ echo "insertion \n and header for EDID" > /dev/null ; \ if [[ ! $( grep "#include <drm>" $src ) ]] ; then cat <<'EOF' #include <drm> EOF fi \ echo "... rest untouched" > /dev/null ; \ cat $src | tac -b \ | awk '/^#include <drm>/ { exit } 1' \ | tac \ ) > temp root@vm-bookworm:~/ms912x-main# grep drm/drm_edid.h temp #include <drm> root@vm-bookworm:~/ms912x-main# mv temp ms912x.h root@vm-bookworm:~/ms912x-main# make ... Skipping BTF generation for /root/ms912x-main/ms912x.ko due to unavailability of vmlinux make[1] : leaving directory « /usr/src/linux-headers-6.1.0-10-amd64 »</drm></drm></drm></drm>

    The missing vmlinux problem

    On the advice of [Dal], I followed his instructions from #20

    Warning: I am not sure if this part is necessary since the compiler's message is a warning and NOT an error. Furthermore, after testing, its absence does not prevent the compiled module from being generated. However, I noticed that the resulting files are different depending on whether vmlinux is managed or not. So it's up to the experts ...

    root@vm-bookworm:~/ms912x-main# cd root@vm-bookworm:~# wget https://github.com/torvalds/linux/archive/refs/tags/v6.1.zip root@vm-bookworm:~# unzip v6.1.zip root@vm-bookworm:~# cd linux-6.1/tools/bpf/resolve_btfids/ root@vm-bookworm:~/linux-6.1/tools/bpf/resolve_btfids# make MKDIR /root/linux-6.1/tools/bpf/resolve_btfids/libbpf/ GEN /root/linux-6.1/tools/bpf/resolve_btfids/libbpf/bpf_helper_defs.h MKDIR /root/linux-6.1/tools/bpf/resolve_btfids/libbpf/staticobjs/ CC /root/linux-6.1/tools/bpf/resolve_btfids/libbpf/staticobjs/libbpf.o libbpf.c:46:10: fatal error: libelf.h: No such file or directory 46 | #include <libelf.h> | ^~~~~~~~~~ # => libelf.h header missing # solution : https://github.com/buserror/simavr/issues/170 root@vm-bookworm:~/linux-6.1/tools/bpf/resolve_btfids# apt install -y libelf-dev root@vm-bookworm:~/linux-6.1/tools/bpf/resolve_btfids# make # => OK resolve_btfids file compiled successfully! # # Back to the #20 # root@vm-bookworm:~/linux-6.1/tools/bpf/resolve_btfids# ls -dl /lib/modules/$(uname -r)/build/tools/bpf ls: cannot access '/lib/modules/6.1.0-10-amd64/build/tools/bpf': No such file or directory root@vm-bookworm:~/linux-6.1/tools/bpf/resolve_btfids# mkdir -p /lib/modules/$(uname -r)/build/tools/bpf/resolve_btfids/ root@vm-bookworm:~/linux-6.1/tools/bpf/resolve_btfids# cp resolve_btfids /lib/modules/$(uname -r)/build/tools/bpf/resolve_btfids/ # # Back to compiling the module # root@vm-bookworm:~/linux-6.1/tools/bpf/resolve_btfids# cd ~/ms912x-main root@vm-bookworm:~/ms912x-main# make clean root@vm-bookworm:~/ms912x-main# make ... Skipping BTF generation for /root/ms912x-main/ms912x.ko due to unavailability of vmlinux make[1] : leaving directory « /usr/src/linux-headers-6.1.0-10-amd64 » # => same error! # solution : https://devicetests.com/fixing-skipping-btf-generation-error-ubuntu-kernel-module-build root@vm-bookworm:~/ms912x-main# apt install -y dwarves root@vm-bookworm:~/ms912x-main# ls -ld /sys/kernel/btf drwxr-xr-x 2 root root 0 13 Sep 17:58 /sys/kernel/btf root@vm-bookworm:~/ms912x-main# ls -ld /usr/lib/modules/$(uname -r)/build lrwxrwxrwx 1 root root 37 14 Jul 05:46 /usr/lib/modules/6.1.0-10-amd64/build -> /usr/src/linux-headers-6.1.0-10-amd64 root@vm-bookworm:~/ms912x-main# cp -a /sys/kernel/btf/vmlinux /usr/lib/modules/6.1.0-10-amd64/build/ root@vm-bookworm:~/ms912x-main# make clean root@vm-bookworm:~/ms912x-main# make make CHECK="/usr/bin/sparse" -C /lib/modules/6.1.0-10-amd64/build M=/root/ms912x-main modules make[1] : entering directory « /usr/src/linux-headers-6.1.0-10-amd64 » CC [M] /root/ms912x-main/ms912x_registers.o CC [M] /root/ms912x-main/ms912x_connector.o CC [M] /root/ms912x-main/ms912x_transfer.o CC [M] /root/ms912x-main/ms912x_drv.o LD [M] /root/ms912x-main/ms912x.o MODPOST /root/ms912x-main/Module.symvers CC [M] /root/ms912x-main/ms912x.mod.o LD [M] /root/ms912x-main/ms912x.ko BTF [M] /root/ms912x-main/ms912x.ko make[1] : leaving directory « /usr/src/linux-headers-6.1.0-10-amd64 » # => OK no warning about vmlinux</libelf.h>

    Loading the module

    root@vm-bookworm:~/ms912x-main# insmod /root/ms912x-main/ms912x.ko insmod: ERROR: could not
    0
  14. lenainjaune Posted messages 726 Registration date   Status Contributor Last intervention   62
     

    If you are considering submitting your modifications in a GitHub pull request, that would probably be the best option, as there is really no reason to move these considerations outside of "ms912x.h". The only advantage of keeping the separation is to separate all our corrections from the existing code rather than chopping it up here and there.

    I will come back to this point, which interests me a lot... ;)

    The two solutions are equivalent, but different in their implementation.

    My question was about the fact that the 2 resulting binaries are not identical (diff) when using the same kernel, whereas in my logic they should be.

    Concretely, this requires creating a key pair, registering it in the BIOS (enroll), and using it to sign the module (sign), then rebooting. I detailed the entire process in this tutorial (see the "Creating the key pair" and "Signing the module" sections).

    I had a doubt about the type of BIOS I was using (source), in any case in virtual it gives:

    root@vm-bookworm:~# \ [ -d /sys/firmware/efi ] && echo BIOS UEFI || echo BIOS Legacy BIOS Legacy

    In general, I boot my systems (physical and virtual) in BIOS legacy and NOT in BIOS EFI or MOK which seems to be for EFI.

    Since I have not yet found out how to sign the module, I wanted to bypass by disabling signature verification to see if the module would ultimately load, but in my case it did not work (yet here it seems to have worked).

    To be continued ...


    I have questions about all your answers. (Woody Allen)
    Knowledge and ideas belong to everyone (noosphere)!

    0
  15. mamiemando Posted messages 33228 Registration date   Status Moderator Last intervention   7 940
     

    Hello,

    My question was about the fact that the 2 resulting binaries are not identical (diff) when using the same kernel, while logically they should be.

    I don't know. To make the comparison, you should rather look at the output of the preprocessor (what I called above the precompilation) with gcc -E to only execute the preprocessor. This way, you will only have source code to compare.

    In general, I boot my systems (physical and virtual) in legacy BIOS and NOT in EFI BIOS, whereas MOK is for EFI, I believe.

    Indeed, in legacy BIOS, you don't have to worry about signing your module. Secure boot is a feature specific to UEFI. When enabled and the Linux kernel has been compiled "normally", secure boot requires signing modules compiled by yourself, otherwise the Linux kernel usually refuses to load them.

    Since I haven't found out how to sign the module yet, I wanted to bypass by disabling signature verification to see if the module would load in the end, but in my case, it didn't work (yet here it seems to have worked).

    To start with, since you are in legacy BIOS, you normally shouldn't have to worry about signing your module.

    Otherwise, the signing of a module is detailed in the NVIDIA driver tutorial that I already indicated to you (the section "the module is not signed"). What is discussed there is not specific to NVIDIA, but to any module compiled manually or by DKMS. Once the key is created, the proposed script handles signing all the concerned modules, which can then be loaded once you have enrolled the key.

    In the links you provided, which I have never tested, there is mention of the CONFIG_MODULE_SIG option. According to what is explained here, it is a kernel compilation option (so it has nothing to do with the module you are compiling).

    • By default, this variable is set to "y" and the generated kernel will refuse to load an unsigned DKMS module or manually compiled module if secure boot is enabled. This is typically the choice made on a standard kernel (like the one deployed by linux-image-amd64, the choice has already been made).
    • Setting it to "n" allows you to no longer have to sign manually compiled modules or those via DKMS, but at the same time makes your kernel more vulnerable. In short, it's a workaround that is not very elegant and not very clean. It's better to create your key and sign your modules properly.

    Good luck

    0