Linux kernel Bug Fixing Spring Unpaid 2024 - Guidance to solve the given Tasks

csengineer90

Need Guidance to complete the given task in the Linux kernel Bug Fixing Spring Unpaid 2024.?

sacco

Hi, I just did the "Build & boot kernel" task. I was wondering what kind of proof can I upload.

Thank you,
Vincenzo

csengineer90

You can provide steps of the command and a screenshot of the updated Build/boot Linus's latest tree.

sacco

@csengineer90 said:
You can provide steps of the command and a screenshot of the updated Build/boot Linus's latest tree.

Thanks

richardorido

I get the following error when trying to build the kernel. Any assistance on how I can solve it will be greatly appreciated.

aolawani

@richardorido said:
I get the following error when trying to build the kernel. Any assistance on how I can solve it will be greatly appreciated.

I ran the following in the kernel directory:

• cp -v /boot/config-$(uname -r) .config
• make menuconfig (Save then exit)
• make

richardorido

@aolawani said:

@richardorido said:
I get the following error when trying to build the kernel. Any assistance on how I can solve it will be greatly appreciated.

I ran the following in the kernel directory:

• cp -v /boot/config-$(uname -r) .config
• make menuconfig (Save then exit)
• make

I still get the same error. Do I need to undo the configurations I already made?

sacco

I still get the same error. Do I need to undo the configurations I already made?

The screenshot suggests that you are using the mainline kernel (I see the folder is "linux" and not "linux-stable"), right?
If so, I'd suggest to use linux-stable and try again. Not sure this solves the issue, but maybe it's worth trying.

richardorido

Thanks for all your help. I actually solved it. I used make mrproper and make clean than ran the commands suggested above again and it worked. I thought that the makefiles had generated bad dependencies which were still saved in the cache so I ran the above commands and it fixed the errors

richardorido

How do I send the email to show proof for the Build and Boot task? I have already set up my .gitconfig but I have never used git send-email and I didn't understand the Email configuration section of the course. Any resources will be helpful. Thanks

richardorido

How do I send the email to show proof for the Build and Boot task? I have already set up my .gitconfig but I have never used git send-email and I didn't understand the Email configuration section of the course. Any resources will be helpful. Thanks

sacco

Have anyone received the confirmation of the subscription to the linux-kernel-mentees mailing list?

I submitted the request the 28 of January, but I haven't received a confirmation yet. Is it normal?

Thanks,
Vincenzo

amoghmc

@sacco
Check your spam folder because mine was in the spam.

sacco

@amoghmc said:
@sacco
Check your spam folder because mine was in the spam.

I did, but there's nothing..

I mean, I received the confirmation mail "We have received a request from [...] To
confirm that you want to be added to this mailing list, simply reply
to this message [...] Or visit this
web page: [...] " and I followed the link to confirm. But after that I never received a confirmation by the admin of the mailing list. Did you?

amoghmc

@sacco
You are right. I checked again and all it said was "Your request has been forwarded to the list moderator for approval." after I sent the confirmation string.

sacco

@amoghmc said:
@sacco
You are right. I checked again and all it said was "Your request has been forwarded to the list moderator for approval." after I sent the confirmation string.

So, besides that email, you haven't received any confirmation yet, correct?

Thanks

venuchoudhary001

Hi, I am also facing the same error > @richardorido said:

Thanks for all your help. I actually solved it. I used make mrproper and make clean than ran the commands suggested above again and it worked. I thought that the makefiles had generated bad dependencies which were still saved in the cache so I ran the above commands and it fixed the errors

I tried using these commands still the same error.
Commands I executed -

make mrproper
make clean
make menuconfig
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem" make V=1 -j8 all

The CONFIG_MODULE_SIG_KEY addition was from this thread - here

When i ran using without CONFIG_MODULE_SIG_KEY the error was same.There was no proper description of the error, it was happening on the same line 1913.
I am using linux 6.6. And in the menuconfig just keeping it as it is, no modifications.

mikernl

What does Build and boot Linus's latest tree mean?

Do we need to use the last Linux-stable tree or the mainline tree(i.e., v6.8-rc3)?

richardorido

@mikernl said:
What does Build and boot Linus's latest tree mean?

Do we need to use the last Linux-stable tree or the mainline tree(i.e., v6.8-rc3)?

The mainline tree

yttrium32

What should I upload for the documentation task? It doesn't mention anything in the task instructions.

venuchoudhary001

@exlud > @venuchoudhary001 said:

Hi, I am also facing the same error > @richardorido said:

Thanks for all your help. I actually solved it. I used make mrproper and make clean than ran the commands suggested above again and it worked. I thought that the makefiles had generated bad dependencies which were still saved in the cache so I ran the above commands and it fixed the errors

I tried using these commands still the same error.
Commands I executed -

make mrproper
make clean
make menuconfig
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem" make V=1 -j8 all

The CONFIG_MODULE_SIG_KEY addition was from this thread - here

When i ran using without CONFIG_MODULE_SIG_KEY the error was same.There was no proper description of the error, it was happening on the same line 1913.
I am using linux 6.6. And in the menuconfig just keeping it as it is, no modifications.

Unable to solve this issue. Can anyone please help? Here is the log file

Also this log is generated with just using these commands :
make mrproper make menuconfig make -j8 > build.log
Also @exlud I have read your comment about the config issue but I am unable to resolve

richardorido

@venuchoudhary001 said:
@exlud > @venuchoudhary001 said:

Hi, I am also facing the same error > @richardorido said:

Thanks for all your help. I actually solved it. I used make mrproper and make clean than ran the commands suggested above again and it worked. I thought that the makefiles had generated bad dependencies which were still saved in the cache so I ran the above commands and it fixed the errors

I tried using these commands still the same error.
Commands I executed -

make mrproper
make clean
make menuconfig
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem" make V=1 -j8 all

The CONFIG_MODULE_SIG_KEY addition was from this thread - here

When i ran using without CONFIG_MODULE_SIG_KEY the error was same.There was no proper description of the error, it was happening on the same line 1913.
I am using linux 6.6. And in the menuconfig just keeping it as it is, no modifications.

Unable to solve this issue. Can anyone please help? Here is the log file

Also this log is generated with just using these commands :
make mrproper make menuconfig make -j8 > build.log
Also @exlud I have read your comment about the config issue but I am unable to resolve

Have you tried this: https://askubuntu.com/questions/1329538/compiling-kernel-5-11-11-and-later. It worked for me

venuchoudhary001

@richardorido said:

@venuchoudhary001 said:
@exlud > @venuchoudhary001 said:

Hi, I am also facing the same error > @richardorido said:

Thanks for all your help. I actually solved it. I used make mrproper and make clean than ran the commands suggested above again and it worked. I thought that the makefiles had generated bad dependencies which were still saved in the cache so I ran the above commands and it fixed the errors

I tried using these commands still the same error.
Commands I executed -

make mrproper
make clean
make menuconfig
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem" make V=1 -j8 all

The CONFIG_MODULE_SIG_KEY addition was from this thread - here

When i ran using without CONFIG_MODULE_SIG_KEY the error was same.There was no proper description of the error, it was happening on the same line 1913.
I am using linux 6.6. And in the menuconfig just keeping it as it is, no modifications.

Unable to solve this issue. Can anyone please help? Here is the log file

Also this log is generated with just using these commands :
make mrproper make menuconfig make -j8 > build.log
Also @exlud I have read your comment about the config issue but I am unable to resolve

Have you tried this: https://askubuntu.com/questions/1329538/compiling-kernel-5-11-11-and-later. It worked for me

Thanks @richardorido It worked

richardorido

@venuchoudhary001 said:

@richardorido said:

@venuchoudhary001 said:
@exlud > @venuchoudhary001 said:

Hi, I am also facing the same error > @richardorido said:

Thanks for all your help. I actually solved it. I used make mrproper and make clean than ran the commands suggested above again and it worked. I thought that the makefiles had generated bad dependencies which were still saved in the cache so I ran the above commands and it fixed the errors

I tried using these commands still the same error.
Commands I executed -

make mrproper
make clean
make menuconfig
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem" make V=1 -j8 all

The CONFIG_MODULE_SIG_KEY addition was from this thread - here

When i ran using without CONFIG_MODULE_SIG_KEY the error was same.There was no proper description of the error, it was happening on the same line 1913.
I am using linux 6.6. And in the menuconfig just keeping it as it is, no modifications.

Unable to solve this issue. Can anyone please help? Here is the log file

Also this log is generated with just using these commands :
make mrproper make menuconfig make -j8 > build.log
Also @exlud I have read your comment about the config issue but I am unable to resolve

Have you tried this: https://askubuntu.com/questions/1329538/compiling-kernel-5-11-11-and-later. It worked for me

Thanks @richardorido It worked

Awesome

dorinetipo

Hi everyone,

I'm encountering an issue with the Generate Panic task. When my patch triggers a panic, the system freezes until I forcibly reboot it. However, despite the panic occurring, no panic message is being logged. I've attempted to use the kernel crash dump mechanism with kexec and kdump, but even after rebooting, no crash logs are generated.

My system is running Ubuntu Focal on x86_64 architecture, and I've verified that the kernel panic mechanism is enabled in the kernel configuration.

Has anyone experienced a similar issue or have any insights into why the panic message isn't being logged despite the system freeze?

Any help or suggestions would be greatly appreciated.

abhinavjain

@dorinetipo said:
Hi everyone,

I'm encountering an issue with the Generate Panic task. When my patch triggers a panic, the system freezes until I forcibly reboot it. However, despite the panic occurring, no panic message is being logged. I've attempted to use the kernel crash dump mechanism with kexec and kdump, but even after rebooting, no crash logs are generated.

My system is running Ubuntu Focal on x86_64 architecture, and I've verified that the kernel panic mechanism is enabled in the kernel configuration.

Has anyone experienced a similar issue or have any insights into why the panic message isn't being logged despite the system freeze?

Any help or suggestions would be greatly appreciated.

I spent ~6 days stressing over this, the hint is for you to look into other log levels and use appropriate print variant with it. I don't believe we can share straight answers here but I hope this helps.

richardorido

@abhinavjain said:

@dorinetipo said:
Hi everyone,

I'm encountering an issue with the Generate Panic task. When my patch triggers a panic, the system freezes until I forcibly reboot it. However, despite the panic occurring, no panic message is being logged. I've attempted to use the kernel crash dump mechanism with kexec and kdump, but even after rebooting, no crash logs are generated.

My system is running Ubuntu Focal on x86_64 architecture, and I've verified that the kernel panic mechanism is enabled in the kernel configuration.

Has anyone experienced a similar issue or have any insights into why the panic message isn't being logged despite the system freeze?

Any help or suggestions would be greatly appreciated.

I spent ~6 days stressing over this, the hint is for you to look into other log levels and use appropriate print variant with it. I don't believe we can share straight answers here but I hope this helps.

which other log levels?

abhinavjain

@richardorido said:

@abhinavjain said:

@dorinetipo said:
Hi everyone,

I'm encountering an issue with the Generate Panic task. When my patch triggers a panic, the system freezes until I forcibly reboot it. However, despite the panic occurring, no panic message is being logged. I've attempted to use the kernel crash dump mechanism with kexec and kdump, but even after rebooting, no crash logs are generated.

My system is running Ubuntu Focal on x86_64 architecture, and I've verified that the kernel panic mechanism is enabled in the kernel configuration.

Has anyone experienced a similar issue or have any insights into why the panic message isn't being logged despite the system freeze?

Any help or suggestions would be greatly appreciated.

I spent ~6 days stressing over this, the hint is for you to look into other log levels and use appropriate print variant with it. I don't believe we can share straight answers here but I hope this helps.

which other log levels?

I can't comment more - we are not supposed to discuss the tasks amongst us :

https://forum.linuxfoundation.org/discussion/859086/do-not-discuss-challenge-related-topics-with-anyone#latest

dorinetipo

@abhinavjain Thanks for pointing that out. I have my debug level set to WARNING(level 30), and kernel panic being a servere event, I'm assuming it should generate log messages at the CRITICAL level or higher, which means it should have been logged in my case. If you don't mind me asking which level did you set to get the message logged?
Also, I got my acceptance email today, I would love to collaborate with you and others, please let me know if that's OK.> @abhinavjain said:

@dorinetipo said:
Hi everyone,

I'm encountering an issue with the Generate Panic task. When my patch triggers a panic, the system freezes until I forcibly reboot it. However, despite the panic occurring, no panic message is being logged. I've attempted to use the kernel crash dump mechanism with kexec and kdump, but even after rebooting, no crash logs are generated.

My system is running Ubuntu Focal on x86_64 architecture, and I've verified that the kernel panic mechanism is enabled in the kernel configuration.

Has anyone experienced a similar issue or have any insights into why the panic message isn't being logged despite the system freeze?

Any help or suggestions would be greatly appreciated.

I spent ~6 days stressing over this, the hint is for you to look into other log levels and use appropriate print variant with it. I don't believe we can share straight answers here but I hope this helps.

picoteando

@dorinetipo I assume you don't see any .crash files under /var/crash, right?

I just tested it with Ubuntu 23.10 on VirtualBox and after installing kdump-tools, I triggered a crash with echo c > /proc/sysrq-trigger, and the *.crash file was generated as expected. Moreover, the dmesg.(date) shows the expected call trace:

[  214.275933] sysrq: Trigger a crash
[  214.275938] Kernel panic - not syncing: sysrq triggered crash
[  214.275941] CPU: 2 PID: 2511 Comm: bash Kdump: loaded Tainted: G           OE      6.5.0-21-generic #21-Ubuntu
[  214.275944] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[  214.275946] Call Trace:
[  214.275948]  <TASK>
[  214.275964]  dump_stack_lvl+0x48/0x70
[  214.275971]  dump_stack+0x10/0x20
[  214.275973]  panic+0x1bb/0x3a0
[  214.275977]  sysrq_handle_crash+0x1a/0x20
...

Did you check kdump-config show? KDUMP_COREDIR should be /var/crash.

abhinavjain

@dorinetipo Congratulations, I got accepted too. Looking forward to collaborate and learn.
The log level I had to use was EMERG with the help of pr_emerg() - only then I was seeing the dmesg log for the module being printed in the console of my VM just before the panic strikes. When I was using pr_info(), it was not getting printed.

@picoteando in my case, while I could see the dmesg log for the module insert before panic using pr_emerg(), I was not able to (and am still not able to get the vmcore). kdump is working fine however I verified with the console (virsh console) of the VM that the dump kernel was failing to boot as the system always get deadlocked on memory(with no killable processes). It shows that node 0 DMA has 0 pages at the time.

[    2.993322] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB
[    3.154854] Out of memory and no killable processes...
[    3.160934] Kernel panic - not syncing: System is deadlocked on memory
[    3.164931] CPU: 0 PID: 23 Comm: kworker/u2:1 Not tainted 6.6.0 #4

I have read around and suspected that it might be that since the initrd being used by kdump for debugging enabled kernel (DWARF4) is big in size, I might need more low memory area reserve and thus I experimented with crashkernel=Y,low. It did not work and any kind of panic results in this, even with echo c > /proc/sysrq-trigger.

abhinav@dev0:~$ dmesg | grep -i crash | grep Reserving
[    0.008069] Reserving 600MB of low memory at 112MB for crashkernel (low RAM limit: 4096MB)
[    0.008071] Reserving 1024MB of memory at 11248MB for crashkernel (System RAM: 10239MB)

In terms of kernel configuration, I have confirmed that kdump-config shows ready to kdump however due to some reason, I am not seeing any crashkernel address, just 0x:

abhinav@dev0:~$ kdump-config show
DUMP_MODE:        kdump
USE_KDUMP:        1
KDUMP_SYSCTL:     kernel.panic_on_oops=1
KDUMP_COREDIR:    /src
crashkernel addr: 0x
0x
   /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-6.6.0
kdump initrd: 
   /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-6.6.0
current state:    ready to kdump

kexec command:
  /sbin/kexec -p --elf64-core-headers --command-line="BOOT_IMAGE=/vmlinuz-6.6.0 root=/dev/mapper/ubuntu--vg-ubuntu--lv ro maybe-ubiquity console=ttyS0 console=tty0 nmi_watchdog=1 reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz

I am still looking into this as I am not sure yet if the suspect I have is the actual cause. (debugging enabled initrd has huge size causing dump kernel boot failure with memory deadlock).

abhinav@dev0:~$ ls -lah /var/lib/kdump/
total 803M
drwxr-xr-x  3 root root 4.0K Feb 27 07:03 .
drwxr-xr-x 52 root root 4.0K Feb  3 14:05 ..
drwxr-xr-x  5 root root 4.0K Mar 14  2023 initramfs-tools
lrwxrwxrwx  1 root root   31 Feb 27 07:03 initrd.img -> /var/lib/kdump/initrd.img-6.6.0
-rw-r--r--  1 root root  42M Jan 31 15:50 initrd.img-5.4.0-170-generic
-rw-r--r--  1 root root  43M Feb 14 06:05 initrd.img-5.4.0-171-generic
-rw-r--r--  1 root root 175M Feb  3 17:41 initrd.img-5.4.0.new
-rw-r--r--  1 root root 545M Feb  7 17:02 initrd.img-6.6.0
lrwxrwxrwx  1 root root   19 Feb 27 07:03 vmlinuz -> /boot/vmlinuz-6.6.0

As can be seen, the initrd being used is 545M in size which is why I configured crashkernel low with 600M but the issue occurs even then.