[ 788.762455][T14864] sctp: Hash tables configured (bind 32/56) [ 841.047731][ T12] ------------[ cut here ]------------ [ 841.048109][ T12] conntrack cleanup blocked for 60s [ 841.048111][ T12] WARNING: net/netfilter/nf_conntrack_core.c:2511 at 0x0, CPU#3: kworker/u16:0/12 [ 841.048512][ T12] Modules linked in: sctp(+) xt_conntrack cls_matchall ip_gre ip6_gre gre sch_fq sch_prio xt_mark cls_basic sch_fq_codel vxlan nft_chain_nat xt_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables cls_bpf netdevsim psample act_mirred act_tunnel_key bareudp mpls_gso mpls_iptunnel mpls_router act_gact cls_flower sch_ingress [ 841.049386][ T12] CPU: 3 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 6.18.0-virtme #1 PREEMPT(full) [ 841.049636][ T12] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011 [ 841.049821][ T12] Workqueue: netns cleanup_net [ 841.049973][ T12] RIP: 0010:nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack] [ 841.050182][ T12] Code: 00 48 8b 54 24 18 48 8b 05 c3 a8 0a f3 48 81 c2 60 ea 00 00 48 39 c2 78 0a e8 42 2e fa f1 e9 25 fd ff ff 48 8d 3d 56 26 14 00 <67> 48 0f b9 3a eb e8 48 8b 5c 24 28 48 b8 00 00 00 00 00 fc ff df [ 841.050652][ T12] RSP: 0018:ffa00000000c7980 EFLAGS: 00010297 [ 841.050835][ T12] RAX: 0000000100083fad RBX: ffa00000000c7af8 RCX: 0000000000000001 [ 841.051031][ T12] RDX: 0000000100083fac RSI: 0000000000000004 RDI: ffffffffc08a9030 [ 841.051223][ T12] RBP: 0000000000000001 R08: ffffffffc07667aa R09: ffe21c000175e70c [ 841.051415][ T12] R10: ffe21c000175e70d R11: 0000000000000001 R12: dffffc0000000000 [ 841.051613][ T12] R13: ff1100000baf3860 R14: fffffbfff8116c07 R15: ff1100000c0ffa00 [ 841.051840][ T12] FS: 0000000000000000(0000) GS:ff110000b758b000(0000) knlGS:0000000000000000 [ 841.052057][ T12] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 841.052566][ T12] CR2: 00007f113ace5a48 CR3: 000000000f19c001 CR4: 0000000000771ef0 [ 841.052763][ T12] PKRU: 55555554 [ 841.052859][ T12] Call Trace: [ 841.052955][ T12] [ 841.053028][ T12] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack] [ 841.053185][ T12] ? kfree+0x119/0x580 [ 841.053278][ T12] ? lock_acquire+0x10a/0x150 [ 841.053406][ T12] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack] [ 841.053562][ T12] ops_undo_list+0x2d7/0x8f0 [ 841.053694][ T12] ? netns_install+0x280/0x280 [ 841.053828][ T12] cleanup_net+0x3bc/0x890 [ 841.053951][ T12] ? process_one_work+0xd16/0x1390 [ 841.054076][ T12] ? net_passive_dec+0x190/0x190 [ 841.054199][ T12] ? rcu_is_watching+0x15/0xd0 [ 841.054321][ T12] ? process_one_work+0xd16/0x1390 [ 841.054443][ T12] ? lock_acquire+0x10a/0x150 [ 841.054562][ T12] ? rcu_is_watching+0x15/0xd0 [ 841.054694][ T12] process_one_work+0xd57/0x1390 [ 841.054826][ T12] ? pwq_dec_nr_in_flight+0x700/0x700 [ 841.054949][ T12] ? lock_acquire.part.0+0xbc/0x260 [ 841.055073][ T12] ? assign_work+0x152/0x380 [ 841.055197][ T12] worker_thread+0x4d6/0xd40 [ 841.055324][ T12] ? process_one_work+0x1390/0x1390 [ 841.055446][ T12] kthread+0x355/0x5b0 [ 841.055544][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 841.055666][ T12] ? __lock_release.isra.0+0x59/0x170 [ 841.055799][ T12] ? rcu_is_watching+0x15/0xd0 [ 841.055921][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 841.056046][ T12] ret_from_fork+0x3fb/0x510 [ 841.056168][ T12] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 841.056320][ T12] ? __switch_to+0x602/0xd00 [ 841.056446][ T12] ? kthread_is_per_cpu+0xe0/0xe0 [ 841.056569][ T12] ret_from_fork_asm+0x11/0x20 [ 841.056706][ T12] [ 841.056799][ T12] irq event stamp: 8777051 [ 841.056924][ T12] hardirqs last enabled at (8777059): [] __up_console_sem+0x5a/0x70 [ 841.057140][ T12] hardirqs last disabled at (8777066): [] __up_console_sem+0x3f/0x70 [ 841.057359][ T12] softirqs last enabled at (8776662): [] handle_softirqs+0x56f/0x840 [ 841.057574][ T12] softirqs last disabled at (8776657): [] irq_exit_rcu+0xa2/0xf0 [ 841.057799][ T12] ---[ end trace 0000000000000000 ]--- [ 984.491744][ T44] INFO: task modprobe:14864 blocked for more than 122 seconds. [ 984.492156][ T44] Tainted: G W 6.18.0-virtme #1 [ 984.492327][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 984.492551][ T44] task:modprobe state:D stack:25136 pid:14864 tgid:14864 ppid:37 task_flags:0x400100 flags:0x00080000 [ 984.492876][ T44] Call Trace: [ 984.492986][ T44] [ 984.493061][ T44] __schedule+0xae2/0x1720 [ 984.493201][ T44] ? io_schedule_timeout+0x130/0x130 [ 984.493325][ T44] ? __lock_release.isra.0+0x59/0x170 [ 984.493451][ T44] schedule+0xd4/0x260 [ 984.493540][ T44] schedule_preempt_disabled+0x18/0x30 [ 984.493659][ T44] rwsem_down_write_slowpath+0x40b/0xc10 [ 984.493788][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10 [ 984.493910][ T44] ? rcu_is_watching+0x15/0xd0 [ 984.494038][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 984.494162][ T44] ? register_pernet_subsys+0x19/0x40 [ 984.494280][ T44] ? lock_acquire+0x10a/0x150 [ 984.494392][ T44] down_write+0x1c6/0x1f0 [ 984.494477][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10 [ 984.494613][ T44] ? _raw_spin_unlock+0x2d/0x50 [ 984.494732][ T44] ? __register_sysctl_table+0x36a/0x4e0 [ 984.494851][ T44] register_pernet_subsys+0x19/0x40 [ 984.494970][ T44] sctp_init+0x48d/0x5f0 [sctp] [ 984.495104][ T44] ? gre_exit+0x450/0x450 [gre] [ 984.495215][ T44] do_one_initcall+0xa3/0x2e0 [ 984.495334][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0 [ 984.495475][ T44] ? kasan_unpoison+0x44/0x70 [ 984.495594][ T44] do_init_module+0x27b/0x820 [ 984.496004][ T44] ? free_module+0x450/0x450 [ 984.496115][ T44] ? backing_file_read_iter+0x50e/0x730 [ 984.496228][ T44] init_module_from_file+0x15d/0x180 [ 984.496338][ T44] ? __do_sys_init_module+0x1f0/0x1f0 [ 984.496454][ T44] ? __lock_release.isra.0+0x59/0x170 [ 984.496563][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 984.496675][ T44] idempotent_init_module+0x226/0x760 [ 984.496797][ T44] ? init_module_from_file+0x180/0x180 [ 984.496910][ T44] ? rcu_is_watching+0x15/0xd0 [ 984.497026][ T44] ? cap_capable+0x181/0x3f0 [ 984.497138][ T44] __x64_sys_finit_module+0xcd/0x150 [ 984.497248][ T44] do_syscall_64+0xbd/0xfc0 [ 984.497362][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 984.497505][ T44] RIP: 0033:0x7fba6d0de34d [ 984.497624][ T44] RSP: 002b:00007ffdf632c668 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 984.497799][ T44] RAX: ffffffffffffffda RBX: 0000560b81ef0b10 RCX: 00007fba6d0de34d [ 984.497965][ T44] RDX: 0000000000000000 RSI: 0000560b61e1af2e RDI: 0000000000000000 [ 984.498139][ T44] RBP: 00007ffdf632c700 R08: 0000000000000000 R09: 0000000000000000 [ 984.498306][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000560b61e1af2e [ 984.498472][ T44] R13: 0000000000040000 R14: 0000560b81ef0c60 R15: 0000560b81ef0570 [ 984.498641][ T44] [ 984.498731][ T44] INFO: task modprobe:14864 is blocked on an rw-semaphore, but the owner is not found. [ 984.498924][ T44] [ 984.498924][ T44] Showing all locks held in the system: [ 984.499090][ T44] 4 locks held by kworker/u16:0/12: [ 984.499201][ T44] 2 locks held by kworker/u19:0/37: [ 984.499305][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 984.499519][ T44] #1: ffa000000029fcc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 984.499846][ T44] 1 lock held by khungtaskd/44: [ 984.499951][ T44] #0: ffffffffb3b84c20 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 984.500166][ T44] 2 locks held by kworker/u19:2/244: [ 984.500270][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 984.500480][ T44] #1: ffa0000000af7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 984.500727][ T44] 2 locks held by kworker/u19:1/8985: [ 984.500831][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 984.501045][ T44] #1: ffa0000001237cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 984.501286][ T44] 1 lock held by modprobe/14864: [ 984.501388][ T44] #0: ffffffffb4279190 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40 [ 984.501598][ T44] 2 locks held by kworker/u19:3/14869: [ 984.501704][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 984.501914][ T44] #1: ffa0000000e97cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 984.502153][ T44] 2 locks held by kworker/u19:4/14874: [ 984.502256][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 984.502464][ T44] #1: ffa0000000e87cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 984.502711][ T44] 2 locks held by kworker/u19:5/14881: [ 984.502815][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 984.503026][ T44] #1: ffa00000005d7cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 984.503264][ T44] 2 locks held by kworker/u19:6/14888: [ 984.503371][ T44] #0: ff11000001151d48 ((wq_completion)events_unbound){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 984.503579][ T44] #1: ffa0000000537cc0 ((work_completion)(&sub_info->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 984.503822][ T44] [ 984.503877][ T44] ============================================= [ 984.503877][ T44]