[ 1958.476071][ T1442] pktgen: Packet Generator for packet performance testing. Version: 2.75 [ 1990.031136][ T12] ------------[ cut here ]------------ [ 1990.031504][ T12] conntrack cleanup blocked for 60s [ 1990.031507][ T12] WARNING: net/netfilter/nf_conntrack_core.c:2511 at 0x0, CPU#2: kworker/u16:0/12 [ 1990.032026][ T12] Modules linked in: pktgen(+) xt_conntrack nft_chain_nat xt_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables cls_bpf netdevsim psample cls_matchall act_gact cls_flower sch_ingress vxlan [ 1990.032666][ T12] CPU: 2 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 6.18.0-virtme #1 PREEMPT(full) [ 1990.032958][ T12] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 1990.033190][ T12] Workqueue: netns cleanup_net [ 1990.033383][ T12] RIP: 0010:nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack] [ 1990.033645][ T12] Code: 00 48 8b 54 24 18 48 8b 05 c3 28 68 ec 48 81 c2 60 ea 00 00 48 39 c2 78 0a e8 42 ae 57 eb e9 25 fd ff ff 48 8d 3d 56 46 da ff <67> 48 0f b9 3a eb e8 48 8b 5c 24 28 48 b8 00 00 00 00 00 fc ff df [ 1990.034175][ T12] RSP: 0018:ffa00000000c7980 EFLAGS: 00010297 [ 1990.034385][ T12] RAX: 000000010019c7db RBX: ffa00000000c7af8 RCX: 0000000000000001 [ 1990.034629][ T12] RDX: 000000010019c7da RSI: 0000000000000004 RDI: ffffffffc0533030 [ 1990.034864][ T12] RBP: 0000000000000001 R08: ffffffffc078e7aa R09: ffe21c0001681d84 [ 1990.035108][ T12] R10: ffe21c0001681d85 R11: 0000000000000001 R12: dffffc0000000000 [ 1990.035364][ T12] R13: ff1100000b40ec20 R14: fffffbfff80a8007 R15: ff11000004f3b400 [ 1990.035610][ T12] FS: 0000000000000000(0000) GS:ff110000bdf0b000(0000) knlGS:0000000000000000 [ 1990.035879][ T12] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 1990.036092][ T12] CR2: 0000562ff79a21a0 CR3: 0000000063148006 CR4: 0000000000771ef0 [ 1990.036340][ T12] PKRU: 55555554 [ 1990.036796][ T12] Call Trace: [ 1990.036942][ T12] [ 1990.037058][ T12] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 1990.037279][ T12] ? kfree+0x119/0x580 [ 1990.037419][ T12] ? lock_acquire+0x10a/0x150 [ 1990.037601][ T12] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 1990.037807][ T12] ops_undo_list+0x2d7/0x8f0 [ 1990.037980][ T12] ? netns_install+0x280/0x280 [ 1990.038168][ T12] cleanup_net+0x3bc/0x890 [ 1990.038340][ T12] ? process_one_work+0xd16/0x1390 [ 1990.038511][ T12] ? net_passive_dec+0x190/0x190 [ 1990.038688][ T12] ? rcu_is_watching+0x15/0xd0 [ 1990.038858][ T12] ? process_one_work+0xd16/0x1390 [ 1990.039031][ T12] ? lock_acquire+0x10a/0x150 [ 1990.039214][ T12] ? rcu_is_watching+0x15/0xd0 [ 1990.039384][ T12] process_one_work+0xd57/0x1390 [ 1990.039563][ T12] ? pwq_dec_nr_in_flight+0x700/0x700 [ 1990.039733][ T12] ? lock_acquire.part.0+0xbc/0x260 [ 1990.039905][ T12] ? assign_work+0x152/0x380 [ 1990.040076][ T12] worker_thread+0x4d6/0xd40 [ 1990.040265][ T12] ? process_one_work+0x1390/0x1390 [ 1990.040437][ T12] kthread+0x355/0x5b0 [ 1990.040581][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1990.040750][ T12] ? __lock_release.isra.0+0x59/0x170 [ 1990.040920][ T12] ? rcu_is_watching+0x15/0xd0 [ 1990.041089][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1990.041271][ T12] ret_from_fork+0x3fb/0x510 [ 1990.041441][ T12] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 1990.041650][ T12] ? __switch_to+0x602/0xd00 [ 1990.041822][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 1990.041993][ T12] ret_from_fork_asm+0x11/0x20 [ 1990.042181][ T12] [ 1990.042321][ T12] irq event stamp: 35188405 [ 1990.042493][ T12] hardirqs last enabled at (35188415): [] __up_console_sem+0x5a/0x70 [ 1990.042759][ T12] hardirqs last disabled at (35188424): [] __up_console_sem+0x3f/0x70 [ 1990.043022][ T12] softirqs last enabled at (35187922): [] handle_softirqs+0x56f/0x840 [ 1990.043295][ T12] softirqs last disabled at (35187915): [] irq_exit_rcu+0xa2/0xf0 [ 1990.043562][ T12] ---[ end trace 0000000000000000 ]--- [ 2090.420218][ T44] INFO: task modprobe:1442 blocked for more than 122 seconds. [ 2090.420582][ T44] Tainted: G W 6.18.0-virtme #1 [ 2090.420762][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2090.420984][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2090.421291][ T44] Call Trace: [ 2090.421385][ T44] [ 2090.421454][ T44] __schedule+0xae2/0x1720 [ 2090.421586][ T44] ? io_schedule_timeout+0x130/0x130 [ 2090.421725][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.421851][ T44] schedule+0xd4/0x260 [ 2090.421941][ T44] schedule_preempt_disabled+0x18/0x30 [ 2090.422059][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2090.422193][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2090.422316][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.422436][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2090.422550][ T44] ? register_pernet_subsys+0x19/0x40 [ 2090.422666][ T44] ? lock_acquire+0x10a/0x150 [ 2090.422783][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 2090.422908][ T44] down_write+0x1c6/0x1f0 [ 2090.422995][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2090.423153][ T44] register_pernet_subsys+0x19/0x40 [ 2090.423272][ T44] pg_init+0x29/0x1000 [pktgen] [ 2090.423388][ T44] do_one_initcall+0xa3/0x2e0 [ 2090.423506][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2090.423652][ T44] ? kasan_unpoison+0x44/0x70 [ 2090.423772][ T44] do_init_module+0x27b/0x820 [ 2090.423894][ T44] ? free_module+0x450/0x450 [ 2090.424006][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2090.424405][ T44] init_module_from_file+0x15d/0x180 [ 2090.424520][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 2090.424636][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.424747][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2090.424862][ T44] idempotent_init_module+0x226/0x760 [ 2090.424972][ T44] ? init_module_from_file+0x180/0x180 [ 2090.425086][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.425208][ T44] ? cap_capable+0x181/0x3f0 [ 2090.425319][ T44] __x64_sys_finit_module+0xcd/0x150 [ 2090.425428][ T44] do_syscall_64+0xbd/0xfc0 [ 2090.425540][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2090.425675][ T44] RIP: 0033:0x7f073f4a034d [ 2090.425794][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2090.425961][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 2090.426129][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 2090.426299][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 2090.426462][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 2090.426625][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 2090.426797][ T44] [ 2090.426882][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2090.427121][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2090.427391][ T44] Workqueue: netns cleanup_net [ 2090.427501][ T44] Call Trace: [ 2090.427584][ T44] [ 2090.427642][ T44] ? rcu_read_lock_sched_held+0x3c/0x70 [ 2090.427755][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.427864][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2090.427979][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2090.428125][ T44] ? lock_acquire+0x10a/0x150 [ 2090.428238][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.428347][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.428459][ T44] ? trace_contention_end+0xd6/0x110 [ 2090.428569][ T44] ? __mutex_lock+0x285/0x1f20 [ 2090.428679][ T44] ? asm_common_interrupt+0x26/0x40 [ 2090.428789][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2090.428926][ T44] ? mark_held_locks+0x40/0x70 [ 2090.429033][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.429152][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 2090.429291][ T44] ? common_interrupt+0xa6/0x100 [ 2090.429403][ T44] ? asm_common_interrupt+0x26/0x40 [ 2090.429513][ T44] ? trace_contention_end+0x30/0x110 [ 2090.429622][ T44] ? nf_ct_iterate_cleanup+0xa6/0x350 [nf_conntrack] [ 2090.429760][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2090.429899][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2090.430067][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2090.430244][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2090.430386][ T44] ? kfree+0x119/0x580 [ 2090.430468][ T44] ? lock_acquire+0x10a/0x150 [ 2090.430580][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2090.430719][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2090.430833][ T44] ? netns_install+0x280/0x280 [ 2090.430945][ T44] ? cleanup_net+0x3bc/0x890 [ 2090.431054][ T44] ? process_one_work+0xd16/0x1390 [ 2090.431176][ T44] ? net_passive_dec+0x190/0x190 [ 2090.431284][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.431392][ T44] ? process_one_work+0xd16/0x1390 [ 2090.431504][ T44] ? lock_acquire+0x10a/0x150 [ 2090.431612][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.431724][ T44] ? process_one_work+0xd57/0x1390 [ 2090.431837][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2090.431946][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2090.432056][ T44] ? assign_work+0x152/0x380 [ 2090.432176][ T44] ? worker_thread+0x4d6/0xd40 [ 2090.432290][ T44] ? process_one_work+0x1390/0x1390 [ 2090.432397][ T44] ? kthread+0x355/0x5b0 [ 2090.432480][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2090.432588][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2090.432699][ T44] ? rcu_is_watching+0x15/0xd0 [ 2090.432807][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2090.432916][ T44] ? ret_from_fork+0x3fb/0x510 [ 2090.433028][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2090.433178][ T44] ? __switch_to+0x602/0xd00 [ 2090.433287][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2090.433397][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2090.433511][ T44] [ 2090.433599][ T44] [ 2090.433599][ T44] Showing all locks held in the system: [ 2090.433758][ T44] 4 locks held by kworker/u16:0/12: [ 2090.433870][ T44] 1 lock held by khungtaskd/44: [ 2090.433974][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2090.434296][ T44] 1 lock held by modprobe/1442: [ 2090.434400][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2090.434620][ T44] [ 2090.434677][ T44] ============================================= [ 2090.434677][ T44] [ 2213.300173][ T44] INFO: task modprobe:1442 blocked for more than 245 seconds. [ 2213.300457][ T44] Tainted: G W 6.18.0-virtme #1 [ 2213.300625][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2213.300845][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2213.301199][ T44] Call Trace: [ 2213.301302][ T44] [ 2213.301374][ T44] __schedule+0xae2/0x1720 [ 2213.301503][ T44] ? io_schedule_timeout+0x130/0x130 [ 2213.301624][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.301751][ T44] schedule+0xd4/0x260 [ 2213.301839][ T44] schedule_preempt_disabled+0x18/0x30 [ 2213.301961][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2213.302084][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2213.302212][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.302333][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2213.302454][ T44] ? register_pernet_subsys+0x19/0x40 [ 2213.302575][ T44] ? lock_acquire+0x10a/0x150 [ 2213.302691][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 2213.302811][ T44] down_write+0x1c6/0x1f0 [ 2213.302897][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2213.303047][ T44] register_pernet_subsys+0x19/0x40 [ 2213.303164][ T44] pg_init+0x29/0x1000 [pktgen] [ 2213.303278][ T44] do_one_initcall+0xa3/0x2e0 [ 2213.303397][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2213.303535][ T44] ? kasan_unpoison+0x44/0x70 [ 2213.303648][ T44] do_init_module+0x27b/0x820 [ 2213.303762][ T44] ? free_module+0x450/0x450 [ 2213.303874][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2213.303995][ T44] init_module_from_file+0x15d/0x180 [ 2213.304107][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 2213.304228][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.304338][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2213.304448][ T44] idempotent_init_module+0x226/0x760 [ 2213.304556][ T44] ? init_module_from_file+0x180/0x180 [ 2213.304672][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.304782][ T44] ? cap_capable+0x181/0x3f0 [ 2213.304893][ T44] __x64_sys_finit_module+0xcd/0x150 [ 2213.305007][ T44] do_syscall_64+0xbd/0xfc0 [ 2213.305138][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2213.305275][ T44] RIP: 0033:0x7f073f4a034d [ 2213.305393][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2213.305564][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 2213.305725][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 2213.305886][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 2213.306052][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 2213.306224][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 2213.306398][ T44] [ 2213.306482][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2213.306710][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2213.306979][ T44] Workqueue: netns cleanup_net [ 2213.307089][ T44] Call Trace: [ 2213.307175][ T44] [ 2213.307231][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2213.307371][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.307480][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2213.307593][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2213.307733][ T44] ? update_curr+0x411/0x670 [ 2213.307841][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.307949][ T44] ? trace_contention_end+0xd6/0x110 [ 2213.308063][ T44] ? __mutex_lock+0x285/0x1f20 [ 2213.308180][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2213.308318][ T44] ? mark_held_locks+0x40/0x70 [ 2213.308428][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.308537][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 2213.308673][ T44] ? common_interrupt+0xa6/0x100 [ 2213.308781][ T44] ? asm_common_interrupt+0x26/0x40 [ 2213.308891][ T44] ? trace_contention_end+0x30/0x110 [ 2213.309003][ T44] ? nf_ct_iterate_cleanup+0x77/0x350 [nf_conntrack] [ 2213.309156][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2213.309297][ T44] ? schedule+0xd4/0x260 [ 2213.309380][ T44] ? nf_conntrack_cleanup_net_list+0x3ce/0x550 [nf_conntrack] [ 2213.309545][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2213.309684][ T44] ? kfree+0x119/0x580 [ 2213.309768][ T44] ? lock_acquire+0x10a/0x150 [ 2213.309880][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2213.310022][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2213.310140][ T44] ? netns_install+0x280/0x280 [ 2213.310253][ T44] ? cleanup_net+0x3bc/0x890 [ 2213.310362][ T44] ? process_one_work+0xd16/0x1390 [ 2213.310472][ T44] ? net_passive_dec+0x190/0x190 [ 2213.310585][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.310694][ T44] ? process_one_work+0xd16/0x1390 [ 2213.310804][ T44] ? lock_acquire+0x10a/0x150 [ 2213.310915][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.311026][ T44] ? process_one_work+0xd57/0x1390 [ 2213.311145][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2213.311258][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2213.311371][ T44] ? assign_work+0x152/0x380 [ 2213.311481][ T44] ? worker_thread+0x4d6/0xd40 [ 2213.311595][ T44] ? process_one_work+0x1390/0x1390 [ 2213.311706][ T44] ? kthread+0x355/0x5b0 [ 2213.311789][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2213.311897][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2213.312014][ T44] ? rcu_is_watching+0x15/0xd0 [ 2213.312129][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2213.312239][ T44] ? ret_from_fork+0x3fb/0x510 [ 2213.312351][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2213.312489][ T44] ? __switch_to+0x602/0xd00 [ 2213.312599][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2213.312708][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2213.312820][ T44] [ 2213.312916][ T44] [ 2213.312916][ T44] Showing all locks held in the system: [ 2213.313082][ T44] 4 locks held by kworker/u16:0/12: [ 2213.313279][ T44] 1 lock held by khungtaskd/44: [ 2213.313384][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2213.313609][ T44] 1 lock held by modprobe/1442: [ 2213.313711][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2213.313922][ T44] [ 2213.313980][ T44] ============================================= [ 2213.313980][ T44] [ 2336.180223][ T44] INFO: task modprobe:1442 blocked for more than 368 seconds. [ 2336.180540][ T44] Tainted: G W 6.18.0-virtme #1 [ 2336.180716][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2336.181018][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2336.181349][ T44] Call Trace: [ 2336.181451][ T44] [ 2336.181523][ T44] __schedule+0xae2/0x1720 [ 2336.181667][ T44] ? io_schedule_timeout+0x130/0x130 [ 2336.181813][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.182194][ T44] schedule+0xd4/0x260 [ 2336.182286][ T44] schedule_preempt_disabled+0x18/0x30 [ 2336.182409][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2336.182545][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2336.182670][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.182794][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2336.182915][ T44] ? register_pernet_subsys+0x19/0x40 [ 2336.183030][ T44] ? lock_acquire+0x10a/0x150 [ 2336.183153][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 2336.183271][ T44] down_write+0x1c6/0x1f0 [ 2336.183356][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2336.183500][ T44] register_pernet_subsys+0x19/0x40 [ 2336.183615][ T44] pg_init+0x29/0x1000 [pktgen] [ 2336.183736][ T44] do_one_initcall+0xa3/0x2e0 [ 2336.183854][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2336.183992][ T44] ? kasan_unpoison+0x44/0x70 [ 2336.184106][ T44] do_init_module+0x27b/0x820 [ 2336.184225][ T44] ? free_module+0x450/0x450 [ 2336.184334][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2336.184448][ T44] init_module_from_file+0x15d/0x180 [ 2336.184564][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 2336.184681][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.184797][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2336.184907][ T44] idempotent_init_module+0x226/0x760 [ 2336.185018][ T44] ? init_module_from_file+0x180/0x180 [ 2336.185146][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.185256][ T44] ? cap_capable+0x181/0x3f0 [ 2336.185370][ T44] __x64_sys_finit_module+0xcd/0x150 [ 2336.185480][ T44] do_syscall_64+0xbd/0xfc0 [ 2336.185591][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2336.185728][ T44] RIP: 0033:0x7f073f4a034d [ 2336.185853][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2336.186019][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 2336.186202][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 2336.186366][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 2336.186530][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 2336.186694][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 2336.186873][ T44] [ 2336.186958][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2336.187195][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2336.187462][ T44] Workqueue: netns cleanup_net [ 2336.187578][ T44] Call Trace: [ 2336.187661][ T44] [ 2336.187718][ T44] ? rcu_read_lock_sched_held+0x3c/0x70 [ 2336.187834][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.187942][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2336.188052][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2336.188199][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.188313][ T44] ? trace_contention_end+0xd6/0x110 [ 2336.188431][ T44] ? __mutex_lock+0x285/0x1f20 [ 2336.188546][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2336.188686][ T44] ? ww_mutex_lock+0x160/0x160 [ 2336.188798][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 2336.188938][ T44] ? common_interrupt+0xa6/0x100 [ 2336.189050][ T44] ? asm_common_interrupt+0x26/0x40 [ 2336.189172][ T44] ? trace_contention_end+0x30/0x110 [ 2336.189286][ T44] ? nf_ct_iterate_cleanup+0x77/0x350 [nf_conntrack] [ 2336.189424][ T44] ? nf_conntrack_cleanup_net_list+0x1aa/0x550 [nf_conntrack] [ 2336.189594][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2336.189763][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2336.189934][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2336.190079][ T44] ? kfree+0x119/0x580 [ 2336.190177][ T44] ? lock_acquire+0x10a/0x150 [ 2336.190294][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2336.190434][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2336.190547][ T44] ? netns_install+0x280/0x280 [ 2336.190663][ T44] ? cleanup_net+0x3bc/0x890 [ 2336.190773][ T44] ? process_one_work+0xd16/0x1390 [ 2336.190894][ T44] ? net_passive_dec+0x190/0x190 [ 2336.191003][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.191121][ T44] ? process_one_work+0xd16/0x1390 [ 2336.191230][ T44] ? lock_acquire+0x10a/0x150 [ 2336.191337][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.191453][ T44] ? process_one_work+0xd57/0x1390 [ 2336.191564][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2336.191682][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2336.191795][ T44] ? assign_work+0x152/0x380 [ 2336.191907][ T44] ? worker_thread+0x4d6/0xd40 [ 2336.192019][ T44] ? process_one_work+0x1390/0x1390 [ 2336.192132][ T44] ? kthread+0x355/0x5b0 [ 2336.192217][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2336.192326][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2336.192437][ T44] ? rcu_is_watching+0x15/0xd0 [ 2336.192546][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2336.192656][ T44] ? ret_from_fork+0x3fb/0x510 [ 2336.192764][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2336.192904][ T44] ? __switch_to+0x602/0xd00 [ 2336.193013][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2336.193131][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2336.193246][ T44] [ 2336.193358][ T44] [ 2336.193358][ T44] Showing all locks held in the system: [ 2336.193518][ T44] 4 locks held by kworker/u16:0/12: [ 2336.193629][ T44] 1 lock held by khungtaskd/44: [ 2336.193733][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2336.193959][ T44] 1 lock held by modprobe/1442: [ 2336.194064][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2336.194374][ T44] [ 2336.194428][ T44] ============================================= [ 2336.194428][ T44] [ 2459.060186][ T44] INFO: task modprobe:1442 blocked for more than 491 seconds. [ 2459.060476][ T44] Tainted: G W 6.18.0-virtme #1 [ 2459.060653][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2459.060883][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2459.061214][ T44] Call Trace: [ 2459.061317][ T44] [ 2459.061383][ T44] __schedule+0xae2/0x1720 [ 2459.061530][ T44] ? io_schedule_timeout+0x130/0x130 [ 2459.061667][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.061796][ T44] schedule+0xd4/0x260 [ 2459.061890][ T44] schedule_preempt_disabled+0x18/0x30 [ 2459.062013][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2459.062134][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2459.062255][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.062375][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2459.062490][ T44] ? register_pernet_subsys+0x19/0x40 [ 2459.062606][ T44] ? lock_acquire+0x10a/0x150 [ 2459.062726][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 2459.062843][ T44] down_write+0x1c6/0x1f0 [ 2459.062929][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2459.063075][ T44] register_pernet_subsys+0x19/0x40 [ 2459.063192][ T44] pg_init+0x29/0x1000 [pktgen] [ 2459.063302][ T44] do_one_initcall+0xa3/0x2e0 [ 2459.063414][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2459.063553][ T44] ? kasan_unpoison+0x44/0x70 [ 2459.063670][ T44] do_init_module+0x27b/0x820 [ 2459.063786][ T44] ? free_module+0x450/0x450 [ 2459.063896][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2459.064009][ T44] init_module_from_file+0x15d/0x180 [ 2459.064126][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 2459.064242][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.064352][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2459.064460][ T44] idempotent_init_module+0x226/0x760 [ 2459.064570][ T44] ? init_module_from_file+0x180/0x180 [ 2459.064686][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.064798][ T44] ? cap_capable+0x181/0x3f0 [ 2459.064909][ T44] __x64_sys_finit_module+0xcd/0x150 [ 2459.065018][ T44] do_syscall_64+0xbd/0xfc0 [ 2459.065140][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2459.065277][ T44] RIP: 0033:0x7f073f4a034d [ 2459.065394][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2459.065558][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 2459.065726][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 2459.065893][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 2459.066055][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 2459.066224][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 2459.066394][ T44] [ 2459.066479][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2459.066716][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2459.066980][ T44] Workqueue: netns cleanup_net [ 2459.067093][ T44] Call Trace: [ 2459.067180][ T44] [ 2459.067239][ T44] ? rcu_read_lock_sched_held+0x3c/0x70 [ 2459.067353][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.067461][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2459.067570][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2459.067714][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.067824][ T44] ? trace_contention_end+0xd6/0x110 [ 2459.067935][ T44] ? irqentry_enter+0x2a/0xd0 [ 2459.068042][ T44] ? __mutex_lock+0x285/0x1f20 [ 2459.068158][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2459.068298][ T44] ? mark_held_locks+0x40/0x70 [ 2459.068408][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.068516][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 2459.068658][ T44] ? common_interrupt+0xa6/0x100 [ 2459.068767][ T44] ? nf_conntrack_cleanup_net_list+0x3df/0x550 [nf_conntrack] [ 2459.068933][ T44] ? asm_common_interrupt+0x26/0x40 [ 2459.069044][ T44] ? trace_contention_end+0x30/0x110 [ 2459.069164][ T44] ? nf_ct_iterate_cleanup+0x9d/0x350 [nf_conntrack] [ 2459.069303][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2459.069443][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2459.069609][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2459.069777][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2459.069915][ T44] ? kfree+0x119/0x580 [ 2459.069998][ T44] ? lock_acquire+0x10a/0x150 [ 2459.070109][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2459.070259][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2459.070372][ T44] ? netns_install+0x280/0x280 [ 2459.070483][ T44] ? cleanup_net+0x3bc/0x890 [ 2459.070591][ T44] ? process_one_work+0xd16/0x1390 [ 2459.070703][ T44] ? net_passive_dec+0x190/0x190 [ 2459.070811][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.070918][ T44] ? process_one_work+0xd16/0x1390 [ 2459.071028][ T44] ? lock_acquire+0x10a/0x150 [ 2459.071146][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.071257][ T44] ? process_one_work+0xd57/0x1390 [ 2459.071370][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2459.071479][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2459.071589][ T44] ? assign_work+0x152/0x380 [ 2459.071704][ T44] ? worker_thread+0x4d6/0xd40 [ 2459.071816][ T44] ? process_one_work+0x1390/0x1390 [ 2459.071927][ T44] ? kthread+0x355/0x5b0 [ 2459.072011][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2459.072125][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2459.072234][ T44] ? rcu_is_watching+0x15/0xd0 [ 2459.072344][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2459.072453][ T44] ? ret_from_fork+0x3fb/0x510 [ 2459.072562][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2459.072704][ T44] ? __switch_to+0x602/0xd00 [ 2459.072820][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2459.072928][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2459.073042][ T44] [ 2459.073155][ T44] [ 2459.073155][ T44] Showing all locks held in the system: [ 2459.073312][ T44] 4 locks held by kworker/u16:0/12: [ 2459.073422][ T44] 1 lock held by khungtaskd/44: [ 2459.073528][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2459.073754][ T44] 1 lock held by modprobe/1442: [ 2459.073857][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2459.074072][ T44] [ 2459.074211][ T44] ============================================= [ 2459.074211][ T44] [ 2581.940196][ T44] INFO: task modprobe:1442 blocked for more than 614 seconds. [ 2581.940483][ T44] Tainted: G W 6.18.0-virtme #1 [ 2581.940652][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2581.940872][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2581.941227][ T44] Call Trace: [ 2581.941318][ T44] [ 2581.941381][ T44] __schedule+0xae2/0x1720 [ 2581.941513][ T44] ? io_schedule_timeout+0x130/0x130 [ 2581.941636][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.941767][ T44] schedule+0xd4/0x260 [ 2581.941862][ T44] schedule_preempt_disabled+0x18/0x30 [ 2581.941982][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2581.942123][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2581.942255][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.942369][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2581.942483][ T44] ? register_pernet_subsys+0x19/0x40 [ 2581.942597][ T44] ? lock_acquire+0x10a/0x150 [ 2581.942716][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 2581.942835][ T44] down_write+0x1c6/0x1f0 [ 2581.942920][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2581.943061][ T44] register_pernet_subsys+0x19/0x40 [ 2581.943206][ T44] pg_init+0x29/0x1000 [pktgen] [ 2581.943319][ T44] do_one_initcall+0xa3/0x2e0 [ 2581.943432][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2581.943573][ T44] ? kasan_unpoison+0x44/0x70 [ 2581.943688][ T44] do_init_module+0x27b/0x820 [ 2581.943801][ T44] ? free_module+0x450/0x450 [ 2581.943911][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2581.944023][ T44] init_module_from_file+0x15d/0x180 [ 2581.944147][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 2581.944262][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.944372][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2581.944485][ T44] idempotent_init_module+0x226/0x760 [ 2581.944595][ T44] ? init_module_from_file+0x180/0x180 [ 2581.944711][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.944819][ T44] ? cap_capable+0x181/0x3f0 [ 2581.944930][ T44] __x64_sys_finit_module+0xcd/0x150 [ 2581.945041][ T44] do_syscall_64+0xbd/0xfc0 [ 2581.945162][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2581.945300][ T44] RIP: 0033:0x7f073f4a034d [ 2581.945419][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2581.945591][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 2581.945755][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 2581.945916][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 2581.946080][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 2581.946253][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 2581.946421][ T44] [ 2581.946516][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2581.946752][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2581.947017][ T44] Workqueue: netns cleanup_net [ 2581.947135][ T44] Call Trace: [ 2581.947219][ T44] [ 2581.947276][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2581.947419][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.947533][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2581.947641][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2581.947778][ T44] ? lock_acquire+0x10a/0x150 [ 2581.947886][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.947994][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.948103][ T44] ? trace_contention_end+0xd6/0x110 [ 2581.948217][ T44] ? __mutex_lock+0x285/0x1f20 [ 2581.948325][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 2581.948462][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2581.948602][ T44] ? mark_held_locks+0x40/0x70 [ 2581.948712][ T44] ? __schedule+0xd3e/0x1720 [ 2581.948822][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 2581.948956][ T44] ? common_interrupt+0xa6/0x100 [ 2581.949064][ T44] ? __schedule+0xd3e/0x1720 [ 2581.949181][ T44] ? trace_contention_end+0x30/0x110 [ 2581.949289][ T44] ? nf_ct_iterate_cleanup+0x8e/0x350 [nf_conntrack] [ 2581.949427][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2581.949564][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2581.949729][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2581.949895][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2581.950035][ T44] ? kfree+0x119/0x580 [ 2581.950123][ T44] ? lock_acquire+0x10a/0x150 [ 2581.950233][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2581.950373][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2581.950486][ T44] ? netns_install+0x280/0x280 [ 2581.950603][ T44] ? cleanup_net+0x3bc/0x890 [ 2581.950716][ T44] ? process_one_work+0xd16/0x1390 [ 2581.950828][ T44] ? net_passive_dec+0x190/0x190 [ 2581.950936][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.951044][ T44] ? process_one_work+0xd16/0x1390 [ 2581.951154][ T44] ? lock_acquire+0x10a/0x150 [ 2581.951263][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.951372][ T44] ? process_one_work+0xd57/0x1390 [ 2581.951485][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2581.951595][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2581.951705][ T44] ? assign_work+0x152/0x380 [ 2581.951816][ T44] ? worker_thread+0x4d6/0xd40 [ 2581.951927][ T44] ? process_one_work+0x1390/0x1390 [ 2581.952037][ T44] ? kthread+0x355/0x5b0 [ 2581.952126][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2581.952234][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2581.952344][ T44] ? rcu_is_watching+0x15/0xd0 [ 2581.952451][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2581.952563][ T44] ? ret_from_fork+0x3fb/0x510 [ 2581.952672][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2581.952811][ T44] ? __switch_to+0x602/0xd00 [ 2581.952919][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2581.953029][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2581.953155][ T44] [ 2581.953265][ T44] [ 2581.953265][ T44] Showing all locks held in the system: [ 2581.953427][ T44] 5 locks held by kworker/u16:0/12: [ 2581.953540][ T44] 1 lock held by khungtaskd/44: [ 2581.953643][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2581.953870][ T44] 1 lock held by modprobe/1442: [ 2581.953976][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2581.954294][ T44] [ 2581.954351][ T44] ============================================= [ 2581.954351][ T44] [ 2704.820194][ T44] INFO: task modprobe:1442 blocked for more than 737 seconds. [ 2704.820479][ T44] Tainted: G W 6.18.0-virtme #1 [ 2704.820647][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2704.820874][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2704.821208][ T44] Call Trace: [ 2704.821315][ T44] [ 2704.821387][ T44] __schedule+0xae2/0x1720 [ 2704.821537][ T44] ? io_schedule_timeout+0x130/0x130 [ 2704.821663][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.821791][ T44] schedule+0xd4/0x260 [ 2704.821890][ T44] schedule_preempt_disabled+0x18/0x30 [ 2704.822015][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2704.822144][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2704.822268][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.822389][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2704.822511][ T44] ? register_pernet_subsys+0x19/0x40 [ 2704.822630][ T44] ? lock_acquire+0x10a/0x150 [ 2704.822746][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 2704.822863][ T44] down_write+0x1c6/0x1f0 [ 2704.822948][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2704.823090][ T44] register_pernet_subsys+0x19/0x40 [ 2704.823234][ T44] pg_init+0x29/0x1000 [pktgen] [ 2704.823348][ T44] do_one_initcall+0xa3/0x2e0 [ 2704.823467][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2704.823608][ T44] ? kasan_unpoison+0x44/0x70 [ 2704.823724][ T44] do_init_module+0x27b/0x820 [ 2704.823842][ T44] ? free_module+0x450/0x450 [ 2704.823953][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2704.824066][ T44] init_module_from_file+0x15d/0x180 [ 2704.824183][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 2704.824300][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.824411][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2704.824524][ T44] idempotent_init_module+0x226/0x760 [ 2704.824632][ T44] ? init_module_from_file+0x180/0x180 [ 2704.824746][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.824857][ T44] ? cap_capable+0x181/0x3f0 [ 2704.824970][ T44] __x64_sys_finit_module+0xcd/0x150 [ 2704.825080][ T44] do_syscall_64+0xbd/0xfc0 [ 2704.825198][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2704.825338][ T44] RIP: 0033:0x7f073f4a034d [ 2704.825459][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2704.825626][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 2704.825792][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 2704.825959][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 2704.826132][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 2704.826295][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 2704.826467][ T44] [ 2704.826553][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2704.826787][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2704.827056][ T44] Workqueue: netns cleanup_net [ 2704.827174][ T44] Call Trace: [ 2704.827257][ T44] [ 2704.827315][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2704.827462][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.827575][ T44] ? update_curr+0x411/0x670 [ 2704.827687][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.827797][ T44] ? trace_contention_end+0xd6/0x110 [ 2704.827908][ T44] ? __mutex_lock+0x285/0x1f20 [ 2704.828018][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2704.828162][ T44] ? mark_held_locks+0x40/0x70 [ 2704.828273][ T44] ? __schedule+0xd3e/0x1720 [ 2704.828385][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 2704.828526][ T44] ? __schedule+0xd41/0x1720 [ 2704.828639][ T44] ? asm_common_interrupt+0x26/0x40 [ 2704.828749][ T44] ? trace_contention_end+0x30/0x110 [ 2704.828859][ T44] ? nf_ct_iterate_cleanup+0x94/0x350 [nf_conntrack] [ 2704.828998][ T44] ? __might_sleep+0xd/0x100 [ 2704.829105][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2704.829248][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2704.829415][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2704.829580][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2704.829717][ T44] ? kfree+0x119/0x580 [ 2704.829800][ T44] ? lock_acquire+0x10a/0x150 [ 2704.829910][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2704.830051][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2704.830166][ T44] ? netns_install+0x280/0x280 [ 2704.830279][ T44] ? cleanup_net+0x3bc/0x890 [ 2704.830388][ T44] ? process_one_work+0xd16/0x1390 [ 2704.830502][ T44] ? net_passive_dec+0x190/0x190 [ 2704.830610][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.830721][ T44] ? process_one_work+0xd16/0x1390 [ 2704.830832][ T44] ? lock_acquire+0x10a/0x150 [ 2704.830941][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.831053][ T44] ? process_one_work+0xd57/0x1390 [ 2704.831170][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2704.831278][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2704.831390][ T44] ? assign_work+0x152/0x380 [ 2704.831502][ T44] ? worker_thread+0x4d6/0xd40 [ 2704.831617][ T44] ? process_one_work+0x1390/0x1390 [ 2704.831727][ T44] ? kthread+0x355/0x5b0 [ 2704.831813][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2704.831921][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2704.832030][ T44] ? rcu_is_watching+0x15/0xd0 [ 2704.832144][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2704.832252][ T44] ? ret_from_fork+0x3fb/0x510 [ 2704.832363][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2704.832509][ T44] ? __switch_to+0x602/0xd00 [ 2704.832618][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2704.832728][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2704.832846][ T44] [ 2704.832960][ T44] [ 2704.832960][ T44] Showing all locks held in the system: [ 2704.833203][ T44] 4 locks held by kworker/u16:0/12: [ 2704.833312][ T44] 1 lock held by khungtaskd/44: [ 2704.833422][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2704.833645][ T44] 1 lock held by modprobe/1442: [ 2704.833748][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2704.833966][ T44] [ 2704.834019][ T44] ============================================= [ 2704.834019][ T44] [ 2827.700193][ T44] INFO: task modprobe:1442 blocked for more than 860 seconds. [ 2827.700478][ T44] Tainted: G W 6.18.0-virtme #1 [ 2827.700651][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2827.700869][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2827.701204][ T44] Call Trace: [ 2827.701311][ T44] [ 2827.701384][ T44] __schedule+0xae2/0x1720 [ 2827.701527][ T44] ? io_schedule_timeout+0x130/0x130 [ 2827.701655][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.701778][ T44] schedule+0xd4/0x260 [ 2827.701870][ T44] schedule_preempt_disabled+0x18/0x30 [ 2827.701991][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2827.702124][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2827.702247][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.702369][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2827.702490][ T44] ? register_pernet_subsys+0x19/0x40 [ 2827.702616][ T44] ? lock_acquire+0x10a/0x150 [ 2827.702737][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 2827.702857][ T44] down_write+0x1c6/0x1f0 [ 2827.702941][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2827.703080][ T44] register_pernet_subsys+0x19/0x40 [ 2827.703200][ T44] pg_init+0x29/0x1000 [pktgen] [ 2827.703314][ T44] do_one_initcall+0xa3/0x2e0 [ 2827.703430][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2827.703570][ T44] ? kasan_unpoison+0x44/0x70 [ 2827.703683][ T44] do_init_module+0x27b/0x820 [ 2827.703800][ T44] ? free_module+0x450/0x450 [ 2827.703911][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2827.704024][ T44] init_module_from_file+0x15d/0x180 [ 2827.704143][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 2827.704260][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.704380][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2827.704492][ T44] idempotent_init_module+0x226/0x760 [ 2827.704603][ T44] ? init_module_from_file+0x180/0x180 [ 2827.704716][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.704827][ T44] ? cap_capable+0x181/0x3f0 [ 2827.704938][ T44] __x64_sys_finit_module+0xcd/0x150 [ 2827.705047][ T44] do_syscall_64+0xbd/0xfc0 [ 2827.705170][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2827.705306][ T44] RIP: 0033:0x7f073f4a034d [ 2827.705428][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2827.705593][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 2827.705758][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 2827.705921][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 2827.706082][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 2827.706253][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 2827.706430][ T44] [ 2827.706514][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2827.706748][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2827.707012][ T44] Workqueue: netns cleanup_net [ 2827.707130][ T44] Call Trace: [ 2827.707212][ T44] [ 2827.707268][ T44] ? rcu_read_lock_sched_held+0x3c/0x70 [ 2827.707381][ T44] ? update_se+0x3dd/0x600 [ 2827.707490][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.707596][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2827.707703][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2827.707844][ T44] ? lock_acquire+0x10a/0x150 [ 2827.707952][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.708063][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.708180][ T44] ? trace_contention_end+0xd6/0x110 [ 2827.708289][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.708398][ T44] ? __mutex_lock+0x285/0x1f20 [ 2827.708508][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2827.708645][ T44] ? mark_held_locks+0x40/0x70 [ 2827.708756][ T44] ? __schedule+0xd3e/0x1720 [ 2827.708863][ T44] ? irqentry_exit+0xf9/0x4d0 [ 2827.708975][ T44] ? common_interrupt+0xa6/0x100 [ 2827.709083][ T44] ? __schedule+0xd3e/0x1720 [ 2827.709208][ T44] ? lock_is_held_type+0xa/0x140 [ 2827.709325][ T44] ? trace_contention_end+0x30/0x110 [ 2827.709442][ T44] ? nf_ct_iterate_cleanup+0x8e/0x350 [nf_conntrack] [ 2827.709582][ T44] ? nf_conntrack_cleanup_net_list+0x1aa/0x550 [nf_conntrack] [ 2827.709746][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2827.709909][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2827.710073][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2827.710217][ T44] ? kfree+0x119/0x580 [ 2827.710300][ T44] ? lock_acquire+0x10a/0x150 [ 2827.710414][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2827.710553][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2827.710663][ T44] ? netns_install+0x280/0x280 [ 2827.710776][ T44] ? cleanup_net+0x3bc/0x890 [ 2827.710884][ T44] ? process_one_work+0xd16/0x1390 [ 2827.710994][ T44] ? net_passive_dec+0x190/0x190 [ 2827.711102][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.711216][ T44] ? process_one_work+0xd16/0x1390 [ 2827.711323][ T44] ? lock_acquire+0x10a/0x150 [ 2827.711436][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.711546][ T44] ? process_one_work+0xd57/0x1390 [ 2827.711658][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2827.711764][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2827.711879][ T44] ? assign_work+0x152/0x380 [ 2827.711994][ T44] ? worker_thread+0x4d6/0xd40 [ 2827.712106][ T44] ? process_one_work+0x1390/0x1390 [ 2827.712220][ T44] ? kthread+0x355/0x5b0 [ 2827.712304][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2827.712416][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2827.712526][ T44] ? rcu_is_watching+0x15/0xd0 [ 2827.712635][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2827.712742][ T44] ? ret_from_fork+0x3fb/0x510 [ 2827.712854][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2827.712994][ T44] ? __switch_to+0x602/0xd00 [ 2827.713104][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2827.713223][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2827.713347][ T44] [ 2827.713465][ T44] [ 2827.713465][ T44] Showing all locks held in the system: [ 2827.713626][ T44] 5 locks held by kworker/u16:0/12: [ 2827.713738][ T44] 1 lock held by khungtaskd/44: [ 2827.713840][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2827.714063][ T44] 1 lock held by modprobe/1442: [ 2827.714253][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2827.714475][ T44] [ 2827.714529][ T44] ============================================= [ 2827.714529][ T44] [ 2950.580193][ T44] INFO: task modprobe:1442 blocked for more than 983 seconds. [ 2950.580483][ T44] Tainted: G W 6.18.0-virtme #1 [ 2950.580656][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 2950.580890][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 2950.581212][ T44] Call Trace: [ 2950.581315][ T44] [ 2950.581387][ T44] __schedule+0xae2/0x1720 [ 2950.581525][ T44] ? io_schedule_timeout+0x130/0x130 [ 2950.581655][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.581791][ T44] schedule+0xd4/0x260 [ 2950.581879][ T44] schedule_preempt_disabled+0x18/0x30 [ 2950.581997][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 2950.582125][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 2950.582256][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.582381][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2950.582502][ T44] ? register_pernet_subsys+0x19/0x40 [ 2950.582626][ T44] ? lock_acquire+0x10a/0x150 [ 2950.582750][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 2950.582873][ T44] down_write+0x1c6/0x1f0 [ 2950.582961][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 2950.583103][ T44] register_pernet_subsys+0x19/0x40 [ 2950.583220][ T44] pg_init+0x29/0x1000 [pktgen] [ 2950.583336][ T44] do_one_initcall+0xa3/0x2e0 [ 2950.583452][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 2950.583590][ T44] ? kasan_unpoison+0x44/0x70 [ 2950.583705][ T44] do_init_module+0x27b/0x820 [ 2950.583829][ T44] ? free_module+0x450/0x450 [ 2950.583938][ T44] ? backing_file_read_iter+0x50e/0x730 [ 2950.584049][ T44] init_module_from_file+0x15d/0x180 [ 2950.584166][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 2950.584282][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.584393][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 2950.584502][ T44] idempotent_init_module+0x226/0x760 [ 2950.584611][ T44] ? init_module_from_file+0x180/0x180 [ 2950.584724][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.584832][ T44] ? cap_capable+0x181/0x3f0 [ 2950.584943][ T44] __x64_sys_finit_module+0xcd/0x150 [ 2950.585056][ T44] do_syscall_64+0xbd/0xfc0 [ 2950.585179][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 2950.585313][ T44] RIP: 0033:0x7f073f4a034d [ 2950.585432][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 2950.585597][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 2950.585761][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 2950.585925][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 2950.586087][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 2950.586255][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 2950.586425][ T44] [ 2950.586510][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 2950.586738][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 2950.587003][ T44] Workqueue: netns cleanup_net [ 2950.587121][ T44] Call Trace: [ 2950.587204][ T44] [ 2950.587263][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2950.587408][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.587520][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 2950.587627][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2950.587764][ T44] ? lock_acquire+0x10a/0x150 [ 2950.587879][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.587988][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.588097][ T44] ? trace_contention_end+0xd6/0x110 [ 2950.588212][ T44] ? __mutex_lock+0x285/0x1f20 [ 2950.588324][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 2950.588462][ T44] ? mark_held_locks+0x40/0x70 [ 2950.588575][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.588682][ T44] ? asm_common_interrupt+0x26/0x40 [ 2950.588794][ T44] ? common_interrupt+0xa6/0x100 [ 2950.588904][ T44] ? asm_common_interrupt+0x26/0x40 [ 2950.589012][ T44] ? trace_contention_end+0x30/0x110 [ 2950.589132][ T44] ? nf_ct_iterate_cleanup+0xa6/0x350 [nf_conntrack] [ 2950.589269][ T44] ? net_generic+0x1/0x1e0 [nf_conntrack] [ 2950.589383][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 2950.589547][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 2950.589715][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 2950.589852][ T44] ? kfree+0x119/0x580 [ 2950.589935][ T44] ? lock_acquire+0x10a/0x150 [ 2950.590048][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 2950.590197][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 2950.590309][ T44] ? netns_install+0x280/0x280 [ 2950.590427][ T44] ? cleanup_net+0x3bc/0x890 [ 2950.590535][ T44] ? process_one_work+0xd16/0x1390 [ 2950.590643][ T44] ? net_passive_dec+0x190/0x190 [ 2950.590750][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.590859][ T44] ? process_one_work+0xd16/0x1390 [ 2950.590965][ T44] ? lock_acquire+0x10a/0x150 [ 2950.591072][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.591185][ T44] ? process_one_work+0xd57/0x1390 [ 2950.591301][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 2950.591411][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 2950.591521][ T44] ? assign_work+0x152/0x380 [ 2950.591630][ T44] ? worker_thread+0x4d6/0xd40 [ 2950.591741][ T44] ? process_one_work+0x1390/0x1390 [ 2950.591848][ T44] ? kthread+0x355/0x5b0 [ 2950.591932][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2950.592045][ T44] ? __lock_release.isra.0+0x59/0x170 [ 2950.592158][ T44] ? rcu_is_watching+0x15/0xd0 [ 2950.592265][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2950.592376][ T44] ? ret_from_fork+0x3fb/0x510 [ 2950.592484][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 2950.592623][ T44] ? __switch_to+0x602/0xd00 [ 2950.592732][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 2950.592849][ T44] ? ret_from_fork_asm+0x11/0x20 [ 2950.592964][ T44] [ 2950.593088][ T44] [ 2950.593088][ T44] Showing all locks held in the system: [ 2950.593333][ T44] 4 locks held by kworker/u16:0/12: [ 2950.593442][ T44] 1 lock held by khungtaskd/44: [ 2950.593547][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 2950.593769][ T44] 1 lock held by modprobe/1442: [ 2950.593874][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 2950.594098][ T44] [ 2950.594155][ T44] ============================================= [ 2950.594155][ T44] [ 3073.460194][ T44] INFO: task modprobe:1442 blocked for more than 1105 seconds. [ 3073.460478][ T44] Tainted: G W 6.18.0-virtme #1 [ 3073.460645][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3073.460883][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 3073.461224][ T44] Call Trace: [ 3073.461318][ T44] [ 3073.461392][ T44] __schedule+0xae2/0x1720 [ 3073.461523][ T44] ? io_schedule_timeout+0x130/0x130 [ 3073.461649][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.461776][ T44] schedule+0xd4/0x260 [ 3073.461869][ T44] schedule_preempt_disabled+0x18/0x30 [ 3073.461999][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 3073.462128][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 3073.462244][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.462359][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3073.462472][ T44] ? register_pernet_subsys+0x19/0x40 [ 3073.462586][ T44] ? lock_acquire+0x10a/0x150 [ 3073.462702][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 3073.462820][ T44] down_write+0x1c6/0x1f0 [ 3073.462908][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 3073.463051][ T44] register_pernet_subsys+0x19/0x40 [ 3073.463170][ T44] pg_init+0x29/0x1000 [pktgen] [ 3073.463283][ T44] do_one_initcall+0xa3/0x2e0 [ 3073.463399][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 3073.463537][ T44] ? kasan_unpoison+0x44/0x70 [ 3073.463654][ T44] do_init_module+0x27b/0x820 [ 3073.463767][ T44] ? free_module+0x450/0x450 [ 3073.463875][ T44] ? backing_file_read_iter+0x50e/0x730 [ 3073.463989][ T44] init_module_from_file+0x15d/0x180 [ 3073.464099][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 3073.464222][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.464333][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 3073.464448][ T44] idempotent_init_module+0x226/0x760 [ 3073.464558][ T44] ? init_module_from_file+0x180/0x180 [ 3073.464671][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.464779][ T44] ? cap_capable+0x181/0x3f0 [ 3073.464890][ T44] __x64_sys_finit_module+0xcd/0x150 [ 3073.465001][ T44] do_syscall_64+0xbd/0xfc0 [ 3073.465112][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 3073.465263][ T44] RIP: 0033:0x7f073f4a034d [ 3073.465384][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 3073.465550][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 3073.465714][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 3073.465877][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 3073.466040][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 3073.466207][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 3073.466381][ T44] [ 3073.466467][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 3073.466699][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 3073.466963][ T44] Workqueue: netns cleanup_net [ 3073.467072][ T44] Call Trace: [ 3073.467161][ T44] [ 3073.467218][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3073.467363][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.467473][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3073.467582][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3073.467718][ T44] ? lock_acquire+0x10a/0x150 [ 3073.467829][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.467940][ T44] ? lockdep_hardirqs_on_prepare.part.0+0x9a/0x160 [ 3073.468074][ T44] ? trace_contention_end+0xd6/0x110 [ 3073.468193][ T44] ? __mutex_lock+0x285/0x1f20 [ 3073.468305][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3073.468445][ T44] ? mark_held_locks+0x40/0x70 [ 3073.468553][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 3073.468661][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 3073.468798][ T44] ? nf_conntrack_cleanup_net_list+0x3df/0x550 [nf_conntrack] [ 3073.468963][ T44] ? asm_common_interrupt+0x26/0x40 [ 3073.469072][ T44] ? trace_contention_end+0x30/0x110 [ 3073.469194][ T44] ? nf_conntrack_cleanup_net_list+0x1aa/0x550 [nf_conntrack] [ 3073.469359][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 3073.469531][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 3073.469700][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 3073.469838][ T44] ? kfree+0x119/0x580 [ 3073.469923][ T44] ? lock_acquire+0x10a/0x150 [ 3073.470034][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 3073.470183][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 3073.470295][ T44] ? netns_install+0x280/0x280 [ 3073.470415][ T44] ? cleanup_net+0x3bc/0x890 [ 3073.470526][ T44] ? process_one_work+0xd16/0x1390 [ 3073.470637][ T44] ? net_passive_dec+0x190/0x190 [ 3073.470746][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.470853][ T44] ? process_one_work+0xd16/0x1390 [ 3073.470961][ T44] ? lock_acquire+0x10a/0x150 [ 3073.471069][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.471186][ T44] ? process_one_work+0xd57/0x1390 [ 3073.471301][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 3073.471414][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 3073.471526][ T44] ? assign_work+0x152/0x380 [ 3073.471638][ T44] ? worker_thread+0x4d6/0xd40 [ 3073.471751][ T44] ? process_one_work+0x1390/0x1390 [ 3073.471857][ T44] ? kthread+0x355/0x5b0 [ 3073.471941][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3073.472048][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3073.472164][ T44] ? rcu_is_watching+0x15/0xd0 [ 3073.472273][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3073.472384][ T44] ? ret_from_fork+0x3fb/0x510 [ 3073.472493][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 3073.472628][ T44] ? __switch_to+0x602/0xd00 [ 3073.472737][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3073.472845][ T44] ? ret_from_fork_asm+0x11/0x20 [ 3073.472957][ T44] [ 3073.473086][ T44] [ 3073.473086][ T44] Showing all locks held in the system: [ 3073.473334][ T44] 4 locks held by kworker/u16:0/12: [ 3073.473443][ T44] 1 lock held by khungtaskd/44: [ 3073.473546][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 3073.473766][ T44] 1 lock held by modprobe/1442: [ 3073.473871][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 3073.474096][ T44] [ 3073.474153][ T44] ============================================= [ 3073.474153][ T44] [ 3196.340192][ T44] INFO: task modprobe:1442 blocked for more than 1228 seconds. [ 3196.340478][ T44] Tainted: G W 6.18.0-virtme #1 [ 3196.340639][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 3196.340863][ T44] task:modprobe state:D stack:25136 pid:1442 tgid:1442 ppid:1 task_flags:0x400100 flags:0x00080002 [ 3196.341183][ T44] Call Trace: [ 3196.341288][ T44] [ 3196.341357][ T44] __schedule+0xae2/0x1720 [ 3196.341487][ T44] ? io_schedule_timeout+0x130/0x130 [ 3196.341613][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3196.341736][ T44] schedule+0xd4/0x260 [ 3196.341828][ T44] schedule_preempt_disabled+0x18/0x30 [ 3196.341951][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 3196.342082][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 3196.342213][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.342340][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3196.342457][ T44] ? register_pernet_subsys+0x19/0x40 [ 3196.342570][ T44] ? lock_acquire+0x10a/0x150 [ 3196.342682][ T44] ? pg_cleanup+0x310/0x310 [pktgen] [ 3196.342801][ T44] down_write+0x1c6/0x1f0 [ 3196.342886][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 3196.343028][ T44] register_pernet_subsys+0x19/0x40 [ 3196.343178][ T44] pg_init+0x29/0x1000 [pktgen] [ 3196.343293][ T44] do_one_initcall+0xa3/0x2e0 [ 3196.343409][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 3196.343553][ T44] ? kasan_unpoison+0x44/0x70 [ 3196.343667][ T44] do_init_module+0x27b/0x820 [ 3196.343783][ T44] ? free_module+0x450/0x450 [ 3196.343892][ T44] ? backing_file_read_iter+0x50e/0x730 [ 3196.344005][ T44] init_module_from_file+0x15d/0x180 [ 3196.344123][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 3196.344247][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3196.344364][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 3196.344476][ T44] idempotent_init_module+0x226/0x760 [ 3196.344591][ T44] ? init_module_from_file+0x180/0x180 [ 3196.344704][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.344815][ T44] ? cap_capable+0x181/0x3f0 [ 3196.344928][ T44] __x64_sys_finit_module+0xcd/0x150 [ 3196.345040][ T44] do_syscall_64+0xbd/0xfc0 [ 3196.345157][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 3196.345292][ T44] RIP: 0033:0x7f073f4a034d [ 3196.345415][ T44] RSP: 002b:00007fff2a72ec98 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 3196.345582][ T44] RAX: ffffffffffffffda RBX: 000055c22c5c6a00 RCX: 00007f073f4a034d [ 3196.345747][ T44] RDX: 0000000000000000 RSI: 000055c2292f8f2e RDI: 0000000000000005 [ 3196.345910][ T44] RBP: 00007fff2a72ed30 R08: 0000000000000000 R09: 0000000000000000 [ 3196.346072][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 000055c2292f8f2e [ 3196.346242][ T44] R13: 0000000000040000 R14: 000055c22c5c6b20 R15: 000055c22c5c6460 [ 3196.346416][ T44] [ 3196.346500][ T44] INFO: task modprobe:1442 blocked on an rw-semaphore likely owned by task kworker/u16:0:12 [ 3196.346731][ T44] task:kworker/u16:0 state:R running task stack:25880 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 3196.347004][ T44] Workqueue: netns cleanup_net [ 3196.347121][ T44] Call Trace: [ 3196.347206][ T44] [ 3196.347262][ T44] ? rcu_read_lock_sched_held+0x3c/0x70 [ 3196.347383][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.347493][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 3196.347601][ T44] ? update_curr+0x383/0x670 [ 3196.347714][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.347822][ T44] ? trace_contention_end+0xd6/0x110 [ 3196.347932][ T44] ? irqentry_exit+0xf9/0x4d0 [ 3196.348046][ T44] ? __mutex_lock+0x285/0x1f20 [ 3196.348167][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3196.348311][ T44] ? asm_common_interrupt+0x26/0x40 [ 3196.348424][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.348534][ T44] ? trace_irq_disable.constprop.0+0xce/0x100 [ 3196.348672][ T44] ? common_interrupt+0xa6/0x100 [ 3196.348782][ T44] ? asm_common_interrupt+0x26/0x40 [ 3196.348891][ T44] ? trace_contention_end+0x30/0x110 [ 3196.349002][ T44] ? asm_exc_invalid_op+0x1a/0x20 [ 3196.349112][ T44] ? nf_ct_iterate_cleanup+0x77/0x350 [nf_conntrack] [ 3196.349257][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack] [ 3196.349397][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack] [ 3196.349565][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack] [ 3196.349730][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 3196.349868][ T44] ? kfree+0x119/0x580 [ 3196.349952][ T44] ? lock_acquire+0x10a/0x150 [ 3196.350060][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 3196.350202][ T44] ? ops_undo_list+0x2d7/0x8f0 [ 3196.350317][ T44] ? netns_install+0x280/0x280 [ 3196.350435][ T44] ? cleanup_net+0x3bc/0x890 [ 3196.350544][ T44] ? process_one_work+0xd16/0x1390 [ 3196.350656][ T44] ? net_passive_dec+0x190/0x190 [ 3196.350769][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.350876][ T44] ? process_one_work+0xd16/0x1390 [ 3196.350983][ T44] ? lock_acquire+0x10a/0x150 [ 3196.351089][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.351205][ T44] ? process_one_work+0xd57/0x1390 [ 3196.351321][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 3196.351433][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 3196.351544][ T44] ? assign_work+0x152/0x380 [ 3196.351653][ T44] ? worker_thread+0x4d6/0xd40 [ 3196.351765][ T44] ? process_one_work+0x1390/0x1390 [ 3196.351873][ T44] ? kthread+0x355/0x5b0 [ 3196.351957][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3196.352064][ T44] ? __lock_release.isra.0+0x59/0x170 [ 3196.352186][ T44] ? rcu_is_watching+0x15/0xd0 [ 3196.352298][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3196.352414][ T44] ? ret_from_fork+0x3fb/0x510 [ 3196.352522][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 3196.352658][ T44] ? __switch_to+0x602/0xd00 [ 3196.352768][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 3196.352878][ T44] ? ret_from_fork_asm+0x11/0x20 [ 3196.352993][ T44] [ 3196.353076][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings [ 3196.353300][ T44] [ 3196.353300][ T44] Showing all locks held in the system: [ 3196.353460][ T44] 4 locks held by kworker/u16:0/12: [ 3196.353568][ T44] 1 lock held by khungtaskd/44: [ 3196.353671][ T44] #0: ffffffffad184c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 3196.353895][ T44] 1 lock held by modprobe/1442: [ 3196.353998][ T44] #0: ffffffffad879190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 3196.354308][ T44] [ 3196.354364][ T44] ============================================= [ 3196.354364][ T44] WAIT TIMEOUT stderr Ctrl-C stderr Ctrl-C stderr