[ 1852.541241][T11474] pktgen: Packet Generator for packet performance testing. Version: 2.75
[ 1884.502634][ T12] ------------[ cut here ]------------
[ 1884.502984][ T12] conntrack cleanup blocked for 60s
[ 1884.502986][ T12] WARNING: net/netfilter/nf_conntrack_core.c:2511 at nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack], CPU#0: kworker/u16:0/12
[ 1884.503515][ T12] Modules linked in: pktgen(+) xt_conntrack nf_conntrack nf_defrag_ipv4 nf_defrag_ipv6 cls_matchall act_gact cls_flower sch_ingress vxlan chacha libchacha chacha20poly1305 libpoly1305 tls ip6t_REJECT nf_reject_ipv6 nft_compat nf_tables
[ 1884.504176][ T12] CPU: 0 UID: 0 PID: 12 Comm: kworker/u16:0 Not tainted 6.19.0-rc1-virtme #1 PREEMPT(full)
[ 1884.504444][ T12] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011
[ 1884.504622][ T12] Workqueue: netns cleanup_net
[ 1884.504764][ T12] RIP: 0010:nf_conntrack_cleanup_net_list+0x3da/0x550 [nf_conntrack]
[ 1884.504970][ T12] Code: 00 48 8b 54 24 18 48 8b 05 b3 88 68 c4 48 81 c2 60 ea 00 00 48 39 c2 78 0a e8 42 5e 5d c3 e9 25 fd ff ff 48 8d 3d 46 06 e0 ff <67> 48 0f b9 3a eb e8 48 8b 5c 24 28 48 b8 00 00 00 00 00 fc ff df
[ 1884.505418][ T12] RSP: 0018:ffa00000000c7980 EFLAGS: 00010297
[ 1884.505577][ T12] RAX: 0000000100182b9c RBX: ffa00000000c7af8 RCX: 0000000000000001
[ 1884.505774][ T12] RDX: 0000000100182b9b RSI: 0000000000000004 RDI: ffffffffc0589030
[ 1884.505961][ T12] RBP: 0000000000000001 R08: ffffffffc07887ba R09: ffe21c0002be8a4c
[ 1884.506147][ T12] R10: ffe21c0002be8a4d R11: 0000000000000001 R12: dffffc0000000000
[ 1884.506334][ T12] R13: ff11000015f45260 R14: fffffbfff80b2c07 R15: ff1100000bf96e00
[ 1884.506522][ T12] FS: 0000000000000000(0000) GS:ff110000e5df3000(0000) knlGS:0000000000000000
[ 1884.506746][ T12] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1884.506903][ T12] CR2: 00007f35b4d22000 CR3: 0000000011edf001 CR4: 0000000000771ef0
[ 1884.507085][ T12] PKRU: 55555554
[ 1884.507473][ T12] Call Trace:
[ 1884.507567][ T12]
[ 1884.507642][ T12] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 1884.507798][ T12] ? kfree+0x119/0x580
[ 1884.507891][ T12] ? lock_acquire+0x10a/0x150
[ 1884.508016][ T12] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 1884.508175][ T12] ops_undo_list+0x2d7/0x8f0
[ 1884.508306][ T12] ? netns_install+0x280/0x280
[ 1884.508431][ T12] cleanup_net+0x3bc/0x890
[ 1884.508554][ T12] ? process_one_work+0xd16/0x1390
[ 1884.508684][ T12] ? net_passive_dec+0x190/0x190
[ 1884.508806][ T12] ? rcu_is_watching+0x15/0xd0
[ 1884.508928][ T12] ? process_one_work+0xd16/0x1390
[ 1884.509050][ T12] ? lock_acquire+0x10a/0x150
[ 1884.509170][ T12] ? rcu_is_watching+0x15/0xd0
[ 1884.509293][ T12] process_one_work+0xd57/0x1390
[ 1884.509417][ T12] ? pwq_dec_nr_in_flight+0x700/0x700
[ 1884.509539][ T12] ? lock_acquire.part.0+0xbc/0x260
[ 1884.509667][ T12] ? assign_work+0x152/0x380
[ 1884.509791][ T12] worker_thread+0x4d6/0xd40
[ 1884.509916][ T12] ? process_one_work+0x1390/0x1390
[ 1884.510035][ T12] kthread+0x355/0x5b0
[ 1884.510127][ T12] ? kthread_is_per_cpu+0xe0/0xe0
[ 1884.510254][ T12] ? __lock_release.isra.0+0x59/0x170
[ 1884.510378][ T12] ? rcu_is_watching+0x15/0xd0
[ 1884.510500][ T12] ? kthread_is_per_cpu+0xe0/0xe0
[ 1884.510627][ T12] ret_from_fork+0x3fb/0x510
[ 1884.510750][ T12] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 1884.510904][ T12] ? __switch_to+0x53c/0xd00
[ 1884.511026][ T12] ? kthread_is_per_cpu+0xe0/0xe0
[ 1884.511148][ T12] ret_from_fork_asm+0x11/0x20
[ 1884.511283][ T12]
[ 1884.511375][ T12] irq event stamp: 13161981
[ 1884.511500][ T12] hardirqs last enabled at (13161989): [] __up_console_sem+0x5a/0x70
[ 1884.511718][ T12] hardirqs last disabled at (13161998): [] __up_console_sem+0x3f/0x70
[ 1884.511930][ T12] softirqs last enabled at (13161594): [] handle_softirqs+0x56f/0x840
[ 1884.512147][ T12] softirqs last disabled at (13161589): [] irq_exit_rcu+0xa2/0xf0
[ 1884.512365][ T12] ---[ end trace 0000000000000000 ]---
[ 2090.427699][ T44] INFO: task modprobe:11474 blocked for more than 122 seconds.
[ 2090.428147][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2090.428397][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2090.428714][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 2090.429052][ T44] Call Trace:
[ 2090.429197][ T44]
[ 2090.429313][ T44] __schedule+0xae2/0x1720
[ 2090.429499][ T44] ? io_schedule_timeout+0x130/0x130
[ 2090.429696][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2090.429885][ T44] schedule+0xd4/0x260
[ 2090.430027][ T44] schedule_preempt_disabled+0x18/0x30
[ 2090.430199][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2090.430372][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2090.430543][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.430728][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2090.430907][ T44] ? register_pernet_subsys+0x19/0x40
[ 2090.431074][ T44] ? lock_acquire+0x10a/0x150
[ 2090.431245][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 2090.431418][ T44] down_write+0x1c6/0x1f0
[ 2090.431561][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2090.431783][ T44] register_pernet_subsys+0x19/0x40
[ 2090.431954][ T44] pg_init+0x29/0x1000 [pktgen]
[ 2090.432126][ T44] do_one_initcall+0xa3/0x2e0
[ 2090.432296][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2090.432498][ T44] ? kasan_unpoison+0x44/0x70
[ 2090.432684][ T44] do_init_module+0x27b/0x820
[ 2090.432867][ T44] ? free_module+0x450/0x450
[ 2090.433036][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2090.433208][ T44] init_module_from_file+0x15d/0x180
[ 2090.433377][ T44] ? __do_sys_init_module+0x200/0x200
[ 2090.433832][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2090.434001][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2090.434165][ T44] idempotent_init_module+0x226/0x760
[ 2090.434330][ T44] ? init_module_from_file+0x180/0x180
[ 2090.434497][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.434676][ T44] ? cap_capable+0x181/0x3f0
[ 2090.434845][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2090.435010][ T44] do_syscall_64+0xbd/0xfc0
[ 2090.435176][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2090.435364][ T44] RIP: 0033:0x7f04ee6ed34d
[ 2090.435534][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2090.435776][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 2090.435996][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 2090.436213][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 2090.436434][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 2090.436663][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 2090.436899][ T44]
[ 2090.437038][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2090.437319][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2090.437656][ T44] Workqueue: netns cleanup_net
[ 2090.437827][ T44] Call Trace:
[ 2090.437964][ T44]
[ 2090.438075][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2090.438275][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.438439][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2090.438601][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2090.438814][ T44] ? lock_acquire+0x10a/0x150
[ 2090.438976][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.439142][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.439308][ T44] ? trace_contention_end+0xd6/0x110
[ 2090.439474][ T44] ? __mutex_lock+0x285/0x1f20
[ 2090.439658][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2090.439858][ T44] ? ww_mutex_lock+0x160/0x160
[ 2090.440028][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2090.440196][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2090.440392][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2090.440587][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2090.440830][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2090.441051][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2090.441243][ T44] ? kfree+0x119/0x580
[ 2090.441381][ T44] ? lock_acquire+0x10a/0x150
[ 2090.441548][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2090.441763][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2090.441930][ T44] ? netns_install+0x280/0x280
[ 2090.442097][ T44] ? cleanup_net+0x3bc/0x890
[ 2090.442260][ T44] ? process_one_work+0xd16/0x1390
[ 2090.442426][ T44] ? net_passive_dec+0x190/0x190
[ 2090.442632][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.442801][ T44] ? process_one_work+0xd16/0x1390
[ 2090.442965][ T44] ? lock_acquire+0x10a/0x150
[ 2090.443130][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.443295][ T44] ? process_one_work+0xd57/0x1390
[ 2090.443464][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2090.443645][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2090.443820][ T44] ? assign_work+0x152/0x380
[ 2090.443985][ T44] ? worker_thread+0x4d6/0xd40
[ 2090.444153][ T44] ? process_one_work+0x1390/0x1390
[ 2090.444316][ T44] ? kthread+0x355/0x5b0
[ 2090.444453][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2090.444633][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2090.444801][ T44] ? rcu_is_watching+0x15/0xd0
[ 2090.444966][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2090.445132][ T44] ? ret_from_fork+0x3fb/0x510
[ 2090.445302][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2090.445498][ T44] ? __switch_to+0x53c/0xd00
[ 2090.445680][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2090.445846][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2090.446015][ T44]
[ 2090.446166][ T44]
[ 2090.446166][ T44] Showing all locks held in the system:
[ 2090.446384][ T44] 4 locks held by kworker/u16:0/12:
[ 2090.446546][ T44] 1 lock held by khungtaskd/44:
[ 2090.446788][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2090.447065][ T44] 1 lock held by modprobe/11474:
[ 2090.447220][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2090.447488][ T44]
[ 2090.447597][ T44] =============================================
[ 2090.447597][ T44]
[ 2213.306669][ T44] INFO: task modprobe:11474 blocked for more than 245 seconds.
[ 2213.306996][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2213.307250][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2213.307531][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 2213.307910][ T44] Call Trace:
[ 2213.308066][ T44]
[ 2213.308181][ T44] __schedule+0xae2/0x1720
[ 2213.308360][ T44] ? io_schedule_timeout+0x130/0x130
[ 2213.308542][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2213.308735][ T44] schedule+0xd4/0x260
[ 2213.308877][ T44] schedule_preempt_disabled+0x18/0x30
[ 2213.309049][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2213.309221][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2213.309391][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.309565][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2213.309779][ T44] ? register_pernet_subsys+0x19/0x40
[ 2213.309947][ T44] ? lock_acquire+0x10a/0x150
[ 2213.310117][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 2213.310289][ T44] down_write+0x1c6/0x1f0
[ 2213.310434][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2213.310651][ T44] register_pernet_subsys+0x19/0x40
[ 2213.310815][ T44] pg_init+0x29/0x1000 [pktgen]
[ 2213.310982][ T44] do_one_initcall+0xa3/0x2e0
[ 2213.311149][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2213.311343][ T44] ? kasan_unpoison+0x44/0x70
[ 2213.311521][ T44] do_init_module+0x27b/0x820
[ 2213.311707][ T44] ? free_module+0x450/0x450
[ 2213.311872][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2213.312042][ T44] init_module_from_file+0x15d/0x180
[ 2213.312209][ T44] ? __do_sys_init_module+0x200/0x200
[ 2213.312378][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2213.312551][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2213.312735][ T44] idempotent_init_module+0x226/0x760
[ 2213.312900][ T44] ? init_module_from_file+0x180/0x180
[ 2213.313068][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.313231][ T44] ? cap_capable+0x181/0x3f0
[ 2213.313397][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2213.313568][ T44] do_syscall_64+0xbd/0xfc0
[ 2213.313751][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2213.313942][ T44] RIP: 0033:0x7f04ee6ed34d
[ 2213.314113][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2213.314331][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 2213.314548][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 2213.314775][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 2213.314998][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 2213.315215][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 2213.315443][ T44]
[ 2213.315581][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2213.315880][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2213.316198][ T44] Workqueue: netns cleanup_net
[ 2213.316361][ T44] Call Trace:
[ 2213.316504][ T44]
[ 2213.316629][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.316827][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.316992][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2213.317154][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.317345][ T44] ? lock_acquire+0x10a/0x150
[ 2213.317513][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.317691][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.317854][ T44] ? trace_contention_end+0xd6/0x110
[ 2213.318018][ T44] ? __mutex_lock+0x285/0x1f20
[ 2213.318186][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.318380][ T44] ? ww_mutex_lock+0x160/0x160
[ 2213.318550][ T44] ? __lock_acquire+0x577/0xc10
[ 2213.318729][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2213.318895][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.319088][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2213.319284][ T44] ? net_generic+0xb8/0x1e0 [nf_conntrack]
[ 2213.319482][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2213.319720][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2213.319909][ T44] ? kfree+0x119/0x580
[ 2213.320044][ T44] ? lock_acquire+0x10a/0x150
[ 2213.320209][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2213.320407][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2213.320580][ T44] ? netns_install+0x280/0x280
[ 2213.320756][ T44] ? cleanup_net+0x3bc/0x890
[ 2213.320917][ T44] ? process_one_work+0xd16/0x1390
[ 2213.321083][ T44] ? net_passive_dec+0x190/0x190
[ 2213.321246][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.321409][ T44] ? process_one_work+0xd16/0x1390
[ 2213.321581][ T44] ? lock_acquire+0x10a/0x150
[ 2213.321754][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.321922][ T44] ? process_one_work+0xd57/0x1390
[ 2213.322089][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2213.322251][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2213.322417][ T44] ? assign_work+0x152/0x380
[ 2213.322590][ T44] ? worker_thread+0x4d6/0xd40
[ 2213.322773][ T44] ? process_one_work+0x1390/0x1390
[ 2213.322939][ T44] ? kthread+0x355/0x5b0
[ 2213.323077][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2213.323242][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2213.323405][ T44] ? rcu_is_watching+0x15/0xd0
[ 2213.323578][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2213.323754][ T44] ? ret_from_fork+0x3fb/0x510
[ 2213.323920][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2213.324116][ T44] ? __switch_to+0x53c/0xd00
[ 2213.324281][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2213.324450][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2213.324633][ T44]
[ 2213.324782][ T44]
[ 2213.324782][ T44] Showing all locks held in the system:
[ 2213.324995][ T44] 4 locks held by kworker/u16:0/12:
[ 2213.325155][ T44] 1 lock held by khungtaskd/44:
[ 2213.325315][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2213.325602][ T44] 1 lock held by modprobe/11474:
[ 2213.325874][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2213.326144][ T44]
[ 2213.326249][ T44] =============================================
[ 2213.326249][ T44]
[ 2336.186666][ T44] INFO: task modprobe:11474 blocked for more than 368 seconds.
[ 2336.186993][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2336.187248][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2336.187513][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 2336.187870][ T44] Call Trace:
[ 2336.188018][ T44]
[ 2336.188145][ T44] __schedule+0xae2/0x1720
[ 2336.188340][ T44] ? io_schedule_timeout+0x130/0x130
[ 2336.188512][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2336.188704][ T44] schedule+0xd4/0x260
[ 2336.188846][ T44] schedule_preempt_disabled+0x18/0x30
[ 2336.189013][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2336.189192][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2336.189368][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.189535][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2336.189749][ T44] ? register_pernet_subsys+0x19/0x40
[ 2336.189915][ T44] ? lock_acquire+0x10a/0x150
[ 2336.190085][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 2336.190263][ T44] down_write+0x1c6/0x1f0
[ 2336.190403][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2336.190601][ T44] register_pernet_subsys+0x19/0x40
[ 2336.190784][ T44] pg_init+0x29/0x1000 [pktgen]
[ 2336.190950][ T44] do_one_initcall+0xa3/0x2e0
[ 2336.191129][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2336.191323][ T44] ? kasan_unpoison+0x44/0x70
[ 2336.191492][ T44] do_init_module+0x27b/0x820
[ 2336.191674][ T44] ? free_module+0x450/0x450
[ 2336.191838][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2336.192008][ T44] init_module_from_file+0x15d/0x180
[ 2336.192180][ T44] ? __do_sys_init_module+0x200/0x200
[ 2336.192351][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2336.192517][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2336.192693][ T44] idempotent_init_module+0x226/0x760
[ 2336.192858][ T44] ? init_module_from_file+0x180/0x180
[ 2336.193028][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.193197][ T44] ? cap_capable+0x181/0x3f0
[ 2336.193362][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2336.193527][ T44] do_syscall_64+0xbd/0xfc0
[ 2336.193708][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2336.193897][ T44] RIP: 0033:0x7f04ee6ed34d
[ 2336.194068][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2336.194293][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 2336.194509][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 2336.194737][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 2336.194951][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 2336.195171][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 2336.195392][ T44]
[ 2336.195531][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2336.195832][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2336.196157][ T44] Workqueue: netns cleanup_net
[ 2336.196323][ T44] Call Trace:
[ 2336.196458][ T44]
[ 2336.196564][ T44] ? __lock_acquire+0x577/0xc10
[ 2336.196748][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2336.196914][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2336.197114][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2336.197311][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2336.197532][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2336.197768][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2336.197957][ T44] ? kfree+0x119/0x580
[ 2336.198092][ T44] ? lock_acquire+0x10a/0x150
[ 2336.198266][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2336.198464][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2336.198646][ T44] ? netns_install+0x280/0x280
[ 2336.198811][ T44] ? cleanup_net+0x3bc/0x890
[ 2336.198973][ T44] ? process_one_work+0xd16/0x1390
[ 2336.199142][ T44] ? net_passive_dec+0x190/0x190
[ 2336.199305][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.199465][ T44] ? process_one_work+0xd16/0x1390
[ 2336.199647][ T44] ? lock_acquire+0x10a/0x150
[ 2336.199810][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.199975][ T44] ? process_one_work+0xd57/0x1390
[ 2336.200146][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2336.200308][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2336.200474][ T44] ? assign_work+0x152/0x380
[ 2336.200654][ T44] ? worker_thread+0x4d6/0xd40
[ 2336.200824][ T44] ? process_one_work+0x1390/0x1390
[ 2336.200988][ T44] ? kthread+0x355/0x5b0
[ 2336.201130][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2336.201293][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2336.201456][ T44] ? rcu_is_watching+0x15/0xd0
[ 2336.201635][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2336.201798][ T44] ? ret_from_fork+0x3fb/0x510
[ 2336.201962][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2336.202162][ T44] ? __switch_to+0x53c/0xd00
[ 2336.202325][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2336.202491][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2336.202670][ T44]
[ 2336.202825][ T44]
[ 2336.202825][ T44] Showing all locks held in the system:
[ 2336.203035][ T44] 4 locks held by kworker/u16:0/12:
[ 2336.203198][ T44] 1 lock held by khungtaskd/44:
[ 2336.203360][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2336.203746][ T44] 1 lock held by modprobe/11474:
[ 2336.203899][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2336.204174][ T44]
[ 2336.204278][ T44] =============================================
[ 2336.204278][ T44]
[ 2459.067669][ T44] INFO: task modprobe:11474 blocked for more than 491 seconds.
[ 2459.068001][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2459.068248][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2459.068519][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 2459.068864][ T44] Call Trace:
[ 2459.069007][ T44]
[ 2459.069121][ T44] __schedule+0xae2/0x1720
[ 2459.069306][ T44] ? io_schedule_timeout+0x130/0x130
[ 2459.069479][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2459.069675][ T44] schedule+0xd4/0x260
[ 2459.069821][ T44] schedule_preempt_disabled+0x18/0x30
[ 2459.069990][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2459.070160][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2459.070332][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.070502][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2459.070718][ T44] ? register_pernet_subsys+0x19/0x40
[ 2459.070896][ T44] ? lock_acquire+0x10a/0x150
[ 2459.071065][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 2459.071237][ T44] down_write+0x1c6/0x1f0
[ 2459.071377][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2459.071574][ T44] register_pernet_subsys+0x19/0x40
[ 2459.071758][ T44] pg_init+0x29/0x1000 [pktgen]
[ 2459.071933][ T44] do_one_initcall+0xa3/0x2e0
[ 2459.072103][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2459.072298][ T44] ? kasan_unpoison+0x44/0x70
[ 2459.072467][ T44] do_init_module+0x27b/0x820
[ 2459.072652][ T44] ? free_module+0x450/0x450
[ 2459.072820][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2459.072986][ T44] init_module_from_file+0x15d/0x180
[ 2459.073149][ T44] ? __do_sys_init_module+0x200/0x200
[ 2459.073319][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2459.073483][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2459.073665][ T44] idempotent_init_module+0x226/0x760
[ 2459.073838][ T44] ? init_module_from_file+0x180/0x180
[ 2459.074006][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.074171][ T44] ? cap_capable+0x181/0x3f0
[ 2459.074336][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2459.074504][ T44] do_syscall_64+0xbd/0xfc0
[ 2459.074686][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2459.074882][ T44] RIP: 0033:0x7f04ee6ed34d
[ 2459.075052][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2459.075271][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 2459.075490][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 2459.075716][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 2459.075936][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 2459.076152][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 2459.076373][ T44]
[ 2459.076513][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2459.076818][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2459.077141][ T44] Workqueue: netns cleanup_net
[ 2459.077302][ T44] Call Trace:
[ 2459.077436][ T44]
[ 2459.077548][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.077760][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.077935][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2459.078098][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.078290][ T44] ? lock_acquire+0x10a/0x150
[ 2459.078452][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.078631][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.078801][ T44] ? trace_contention_end+0xd6/0x110
[ 2459.078966][ T44] ? __mutex_lock+0x285/0x1f20
[ 2459.079131][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.079324][ T44] ? ww_mutex_lock+0x160/0x160
[ 2459.079488][ T44] ? __lock_acquire+0x577/0xc10
[ 2459.079668][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2459.079841][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.080039][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2459.080230][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2459.080453][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2459.080692][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2459.080892][ T44] ? kfree+0x119/0x580
[ 2459.081034][ T44] ? lock_acquire+0x10a/0x150
[ 2459.081198][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2459.081392][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2459.081557][ T44] ? netns_install+0x280/0x280
[ 2459.081740][ T44] ? cleanup_net+0x3bc/0x890
[ 2459.081908][ T44] ? process_one_work+0xd16/0x1390
[ 2459.082074][ T44] ? net_passive_dec+0x190/0x190
[ 2459.082244][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.082406][ T44] ? process_one_work+0xd16/0x1390
[ 2459.082568][ T44] ? lock_acquire+0x10a/0x150
[ 2459.082747][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.082919][ T44] ? process_one_work+0xd57/0x1390
[ 2459.083087][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2459.083248][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2459.083418][ T44] ? assign_work+0x152/0x380
[ 2459.083583][ T44] ? worker_thread+0x4d6/0xd40
[ 2459.083764][ T44] ? process_one_work+0x1390/0x1390
[ 2459.083933][ T44] ? kthread+0x355/0x5b0
[ 2459.084070][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2459.084233][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2459.084396][ T44] ? rcu_is_watching+0x15/0xd0
[ 2459.084562][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2459.084746][ T44] ? ret_from_fork+0x3fb/0x510
[ 2459.084917][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2459.085108][ T44] ? __switch_to+0x53c/0xd00
[ 2459.085273][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2459.085438][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2459.085604][ T44]
[ 2459.085785][ T44]
[ 2459.085785][ T44] Showing all locks held in the system:
[ 2459.086003][ T44] 4 locks held by kworker/u16:0/12:
[ 2459.086167][ T44] 1 lock held by khungtaskd/44:
[ 2459.086326][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2459.086604][ T44] 1 lock held by modprobe/11474:
[ 2459.086887][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2459.087165][ T44]
[ 2459.087269][ T44] =============================================
[ 2459.087269][ T44]
[ 2581.946666][ T44] INFO: task modprobe:11474 blocked for more than 614 seconds.
[ 2581.947035][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2581.947319][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2581.947668][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 2581.948006][ T44] Call Trace:
[ 2581.948150][ T44]
[ 2581.948266][ T44] __schedule+0xae2/0x1720
[ 2581.948448][ T44] ? io_schedule_timeout+0x130/0x130
[ 2581.948639][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2581.948824][ T44] schedule+0xd4/0x260
[ 2581.948966][ T44] schedule_preempt_disabled+0x18/0x30
[ 2581.949132][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2581.949301][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2581.949473][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.949689][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2581.949859][ T44] ? register_pernet_subsys+0x19/0x40
[ 2581.950027][ T44] ? lock_acquire+0x10a/0x150
[ 2581.950192][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 2581.950360][ T44] down_write+0x1c6/0x1f0
[ 2581.950499][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2581.950714][ T44] register_pernet_subsys+0x19/0x40
[ 2581.950885][ T44] pg_init+0x29/0x1000 [pktgen]
[ 2581.951051][ T44] do_one_initcall+0xa3/0x2e0
[ 2581.951221][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2581.951412][ T44] ? kasan_unpoison+0x44/0x70
[ 2581.951581][ T44] do_init_module+0x27b/0x820
[ 2581.951761][ T44] ? free_module+0x450/0x450
[ 2581.951932][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2581.952099][ T44] init_module_from_file+0x15d/0x180
[ 2581.952263][ T44] ? __do_sys_init_module+0x200/0x200
[ 2581.952434][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2581.952598][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2581.952777][ T44] idempotent_init_module+0x226/0x760
[ 2581.952950][ T44] ? init_module_from_file+0x180/0x180
[ 2581.953117][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.953281][ T44] ? cap_capable+0x181/0x3f0
[ 2581.953450][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2581.953629][ T44] do_syscall_64+0xbd/0xfc0
[ 2581.953799][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2581.953988][ T44] RIP: 0033:0x7f04ee6ed34d
[ 2581.954156][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2581.954371][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 2581.954589][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 2581.954823][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 2581.955041][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 2581.955252][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 2581.955482][ T44]
[ 2581.955634][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2581.955919][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2581.956235][ T44] Workqueue: netns cleanup_net
[ 2581.956399][ T44] Call Trace:
[ 2581.956538][ T44]
[ 2581.956664][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.956869][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.957031][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2581.957195][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.957385][ T44] ? lock_acquire+0x10a/0x150
[ 2581.957546][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.957723][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.957890][ T44] ? trace_contention_end+0xd6/0x110
[ 2581.958056][ T44] ? __mutex_lock+0x285/0x1f20
[ 2581.958221][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.958414][ T44] ? ww_mutex_lock+0x160/0x160
[ 2581.958577][ T44] ? __lock_acquire+0x577/0xc10
[ 2581.958756][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2581.958927][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.959119][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2581.959311][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2581.959538][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2581.959774][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2581.959976][ T44] ? kfree+0x119/0x580
[ 2581.960113][ T44] ? lock_acquire+0x10a/0x150
[ 2581.960278][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2581.960472][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2581.960652][ T44] ? netns_install+0x280/0x280
[ 2581.960823][ T44] ? cleanup_net+0x3bc/0x890
[ 2581.960986][ T44] ? process_one_work+0xd16/0x1390
[ 2581.961151][ T44] ? net_passive_dec+0x190/0x190
[ 2581.961316][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.961476][ T44] ? process_one_work+0xd16/0x1390
[ 2581.961653][ T44] ? lock_acquire+0x10a/0x150
[ 2581.961820][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.961985][ T44] ? process_one_work+0xd57/0x1390
[ 2581.962151][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2581.962314][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2581.962482][ T44] ? assign_work+0x152/0x380
[ 2581.962662][ T44] ? worker_thread+0x4d6/0xd40
[ 2581.962836][ T44] ? process_one_work+0x1390/0x1390
[ 2581.962998][ T44] ? kthread+0x355/0x5b0
[ 2581.963136][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2581.963298][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2581.963462][ T44] ? rcu_is_watching+0x15/0xd0
[ 2581.963645][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2581.963817][ T44] ? ret_from_fork+0x3fb/0x510
[ 2581.963982][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2581.964174][ T44] ? __switch_to+0x53c/0xd00
[ 2581.964339][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2581.964507][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2581.964691][ T44]
[ 2581.964861][ T44]
[ 2581.964861][ T44] Showing all locks held in the system:
[ 2581.965076][ T44] 4 locks held by kworker/u16:0/12:
[ 2581.965238][ T44] 1 lock held by khungtaskd/44:
[ 2581.965392][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2581.965750][ T44] 1 lock held by modprobe/11474:
[ 2581.965911][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2581.966188][ T44]
[ 2581.966291][ T44] =============================================
[ 2581.966291][ T44]
[ 2704.826666][ T44] INFO: task modprobe:11474 blocked for more than 737 seconds.
[ 2704.826994][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2704.827255][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2704.827527][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 2704.827899][ T44] Call Trace:
[ 2704.828051][ T44]
[ 2704.828169][ T44] __schedule+0xae2/0x1720
[ 2704.828348][ T44] ? io_schedule_timeout+0x130/0x130
[ 2704.828520][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2704.828715][ T44] schedule+0xd4/0x260
[ 2704.828859][ T44] schedule_preempt_disabled+0x18/0x30
[ 2704.829037][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2704.829218][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2704.829393][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.829562][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2704.829780][ T44] ? register_pernet_subsys+0x19/0x40
[ 2704.829950][ T44] ? lock_acquire+0x10a/0x150
[ 2704.830133][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 2704.830305][ T44] down_write+0x1c6/0x1f0
[ 2704.830443][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2704.830663][ T44] register_pernet_subsys+0x19/0x40
[ 2704.830832][ T44] pg_init+0x29/0x1000 [pktgen]
[ 2704.831002][ T44] do_one_initcall+0xa3/0x2e0
[ 2704.831180][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2704.831381][ T44] ? kasan_unpoison+0x44/0x70
[ 2704.831551][ T44] do_init_module+0x27b/0x820
[ 2704.831738][ T44] ? free_module+0x450/0x450
[ 2704.831903][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2704.832069][ T44] init_module_from_file+0x15d/0x180
[ 2704.832242][ T44] ? __do_sys_init_module+0x200/0x200
[ 2704.832414][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2704.832587][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2704.832767][ T44] idempotent_init_module+0x226/0x760
[ 2704.832932][ T44] ? init_module_from_file+0x180/0x180
[ 2704.833105][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.833270][ T44] ? cap_capable+0x181/0x3f0
[ 2704.833436][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2704.833603][ T44] do_syscall_64+0xbd/0xfc0
[ 2704.833787][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2704.833977][ T44] RIP: 0033:0x7f04ee6ed34d
[ 2704.834152][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2704.834376][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 2704.834594][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 2704.834830][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 2704.835049][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 2704.835271][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 2704.835493][ T44]
[ 2704.835646][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2704.835932][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2704.836257][ T44] Workqueue: netns cleanup_net
[ 2704.836420][ T44] Call Trace:
[ 2704.836556][ T44]
[ 2704.836681][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.836884][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.837050][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2704.837216][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.837408][ T44] ? lock_acquire+0x10a/0x150
[ 2704.837570][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.837746][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.837910][ T44] ? trace_contention_end+0xd6/0x110
[ 2704.838077][ T44] ? __mutex_lock+0x285/0x1f20
[ 2704.838250][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.838443][ T44] ? ww_mutex_lock+0x160/0x160
[ 2704.838607][ T44] ? __lock_acquire+0x577/0xc10
[ 2704.838790][ T44] ? __mutex_unlock_slowpath+0x14b/0x540
[ 2704.838953][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2704.839122][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.839317][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2704.839512][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2704.839750][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2704.839974][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2704.840178][ T44] ? kfree+0x119/0x580
[ 2704.840318][ T44] ? lock_acquire+0x10a/0x150
[ 2704.840485][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2704.840694][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2704.840860][ T44] ? netns_install+0x280/0x280
[ 2704.841030][ T44] ? cleanup_net+0x3bc/0x890
[ 2704.841202][ T44] ? process_one_work+0xd16/0x1390
[ 2704.841365][ T44] ? net_passive_dec+0x190/0x190
[ 2704.841533][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.841711][ T44] ? process_one_work+0xd16/0x1390
[ 2704.841873][ T44] ? lock_acquire+0x10a/0x150
[ 2704.842038][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.842213][ T44] ? process_one_work+0xd57/0x1390
[ 2704.842383][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2704.842550][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2704.842732][ T44] ? assign_work+0x152/0x380
[ 2704.842899][ T44] ? worker_thread+0x4d6/0xd40
[ 2704.843071][ T44] ? process_one_work+0x1390/0x1390
[ 2704.843242][ T44] ? kthread+0x355/0x5b0
[ 2704.843382][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2704.843546][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2704.843722][ T44] ? rcu_is_watching+0x15/0xd0
[ 2704.843888][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2704.844056][ T44] ? ret_from_fork+0x3fb/0x510
[ 2704.844224][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2704.844415][ T44] ? __switch_to+0x53c/0xd00
[ 2704.844580][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2704.844758][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2704.844927][ T44]
[ 2704.845100][ T44]
[ 2704.845100][ T44] Showing all locks held in the system:
[ 2704.845320][ T44] 4 locks held by kworker/u16:0/12:
[ 2704.845480][ T44] 1 lock held by khungtaskd/44:
[ 2704.845745][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2704.846027][ T44] 1 lock held by modprobe/11474:
[ 2704.846184][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2704.846466][ T44]
[ 2704.846570][ T44] =============================================
[ 2704.846570][ T44]
[ 2827.706666][ T44] INFO: task modprobe:11474 blocked for more than 860 seconds.
[ 2827.706995][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2827.707243][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2827.707513][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 2827.707876][ T44] Call Trace:
[ 2827.708029][ T44]
[ 2827.708151][ T44] __schedule+0xae2/0x1720
[ 2827.708342][ T44] ? io_schedule_timeout+0x130/0x130
[ 2827.708520][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2827.708715][ T44] schedule+0xd4/0x260
[ 2827.708857][ T44] schedule_preempt_disabled+0x18/0x30
[ 2827.709026][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2827.709203][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2827.709376][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.709551][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2827.709764][ T44] ? register_pernet_subsys+0x19/0x40
[ 2827.709933][ T44] ? lock_acquire+0x10a/0x150
[ 2827.710104][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 2827.710280][ T44] down_write+0x1c6/0x1f0
[ 2827.710428][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2827.710645][ T44] register_pernet_subsys+0x19/0x40
[ 2827.710812][ T44] pg_init+0x29/0x1000 [pktgen]
[ 2827.710980][ T44] do_one_initcall+0xa3/0x2e0
[ 2827.711149][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2827.711346][ T44] ? kasan_unpoison+0x44/0x70
[ 2827.711522][ T44] do_init_module+0x27b/0x820
[ 2827.711711][ T44] ? free_module+0x450/0x450
[ 2827.711878][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2827.712049][ T44] init_module_from_file+0x15d/0x180
[ 2827.712213][ T44] ? __do_sys_init_module+0x200/0x200
[ 2827.712382][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2827.712553][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2827.712735][ T44] idempotent_init_module+0x226/0x760
[ 2827.712900][ T44] ? init_module_from_file+0x180/0x180
[ 2827.713071][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.713236][ T44] ? cap_capable+0x181/0x3f0
[ 2827.713410][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2827.713575][ T44] do_syscall_64+0xbd/0xfc0
[ 2827.713754][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2827.713944][ T44] RIP: 0033:0x7f04ee6ed34d
[ 2827.714114][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2827.714337][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 2827.714562][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 2827.714793][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 2827.715008][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 2827.715226][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 2827.715453][ T44]
[ 2827.715593][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2827.715891][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2827.716212][ T44] Workqueue: netns cleanup_net
[ 2827.716377][ T44] Call Trace:
[ 2827.716519][ T44]
[ 2827.716643][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.716839][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.717001][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2827.717161][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.717355][ T44] ? lock_acquire+0x10a/0x150
[ 2827.717519][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.717693][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.717855][ T44] ? trace_contention_end+0xd6/0x110
[ 2827.718019][ T44] ? __mutex_lock+0x285/0x1f20
[ 2827.718183][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.718374][ T44] ? ww_mutex_lock+0x160/0x160
[ 2827.718548][ T44] ? __lock_acquire+0x577/0xc10
[ 2827.718733][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2827.718899][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.719090][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2827.719282][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2827.719510][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 2827.719751][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2827.719942][ T44] ? kfree+0x119/0x580
[ 2827.720078][ T44] ? lock_acquire+0x10a/0x150
[ 2827.720242][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2827.720443][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2827.720625][ T44] ? netns_install+0x280/0x280
[ 2827.720796][ T44] ? cleanup_net+0x3bc/0x890
[ 2827.720962][ T44] ? process_one_work+0xd16/0x1390
[ 2827.721128][ T44] ? net_passive_dec+0x190/0x190
[ 2827.721293][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.721460][ T44] ? process_one_work+0xd16/0x1390
[ 2827.721640][ T44] ? lock_acquire+0x10a/0x150
[ 2827.721802][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.721967][ T44] ? process_one_work+0xd57/0x1390
[ 2827.722137][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2827.722301][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2827.722477][ T44] ? assign_work+0x152/0x380
[ 2827.722658][ T44] ? worker_thread+0x4d6/0xd40
[ 2827.722826][ T44] ? process_one_work+0x1390/0x1390
[ 2827.722988][ T44] ? kthread+0x355/0x5b0
[ 2827.723125][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2827.723290][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2827.723459][ T44] ? rcu_is_watching+0x15/0xd0
[ 2827.723639][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2827.723802][ T44] ? ret_from_fork+0x3fb/0x510
[ 2827.723967][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2827.724159][ T44] ? __switch_to+0x53c/0xd00
[ 2827.724324][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2827.724494][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2827.724675][ T44]
[ 2827.724848][ T44]
[ 2827.724848][ T44] Showing all locks held in the system:
[ 2827.725062][ T44] 4 locks held by kworker/u16:0/12:
[ 2827.725224][ T44] 1 lock held by khungtaskd/44:
[ 2827.725382][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2827.725772][ T44] 1 lock held by modprobe/11474:
[ 2827.725930][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2827.726209][ T44]
[ 2827.726312][ T44] =============================================
[ 2827.726312][ T44]
[ 2950.586668][ T44] INFO: task modprobe:11474 blocked for more than 983 seconds.
[ 2950.587037][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 2950.587316][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 2950.587599][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 2950.587951][ T44] Call Trace:
[ 2950.588095][ T44]
[ 2950.588212][ T44] __schedule+0xae2/0x1720
[ 2950.588406][ T44] ? io_schedule_timeout+0x130/0x130
[ 2950.588589][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2950.588827][ T44] schedule+0xd4/0x260
[ 2950.588972][ T44] schedule_preempt_disabled+0x18/0x30
[ 2950.589143][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 2950.589315][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 2950.589489][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.589675][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2950.589850][ T44] ? register_pernet_subsys+0x19/0x40
[ 2950.590017][ T44] ? lock_acquire+0x10a/0x150
[ 2950.590188][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 2950.590361][ T44] down_write+0x1c6/0x1f0
[ 2950.590502][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 2950.590722][ T44] register_pernet_subsys+0x19/0x40
[ 2950.590891][ T44] pg_init+0x29/0x1000 [pktgen]
[ 2950.591062][ T44] do_one_initcall+0xa3/0x2e0
[ 2950.591229][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 2950.591427][ T44] ? kasan_unpoison+0x44/0x70
[ 2950.591599][ T44] do_init_module+0x27b/0x820
[ 2950.591792][ T44] ? free_module+0x450/0x450
[ 2950.591958][ T44] ? backing_file_read_iter+0x50e/0x730
[ 2950.592123][ T44] init_module_from_file+0x15d/0x180
[ 2950.592287][ T44] ? __do_sys_init_module+0x200/0x200
[ 2950.592459][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2950.592639][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 2950.592813][ T44] idempotent_init_module+0x226/0x760
[ 2950.592981][ T44] ? init_module_from_file+0x180/0x180
[ 2950.593147][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.593312][ T44] ? cap_capable+0x181/0x3f0
[ 2950.593478][ T44] __x64_sys_finit_module+0xcd/0x160
[ 2950.593664][ T44] do_syscall_64+0xbd/0xfc0
[ 2950.593839][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 2950.594029][ T44] RIP: 0033:0x7f04ee6ed34d
[ 2950.594199][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 2950.594418][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 2950.594651][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 2950.594871][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 2950.595085][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 2950.595301][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 2950.595522][ T44]
[ 2950.595678][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 2950.595977][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 2950.596293][ T44] Workqueue: netns cleanup_net
[ 2950.596462][ T44] Call Trace:
[ 2950.596597][ T44]
[ 2950.596727][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.596932][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.597096][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2950.597259][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.597449][ T44] ? lock_acquire+0x10a/0x150
[ 2950.597626][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.597794][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.597960][ T44] ? trace_contention_end+0xd6/0x110
[ 2950.598126][ T44] ? __mutex_lock+0x285/0x1f20
[ 2950.598291][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.598484][ T44] ? ww_mutex_lock+0x160/0x160
[ 2950.598663][ T44] ? __pick_next_task+0xf5/0x6e0
[ 2950.598834][ T44] ? __lock_acquire+0x577/0xc10
[ 2950.599000][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 2950.599174][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.599371][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 2950.599566][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 2950.599811][ T44] ? nf_conntrack_cleanup_net_list+0x1aa/0x550 [nf_conntrack]
[ 2950.600035][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 2950.600232][ T44] ? kfree+0x119/0x580
[ 2950.600368][ T44] ? lock_acquire+0x10a/0x150
[ 2950.600533][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 2950.600744][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 2950.600909][ T44] ? netns_install+0x280/0x280
[ 2950.601076][ T44] ? cleanup_net+0x3bc/0x890
[ 2950.601239][ T44] ? process_one_work+0xd16/0x1390
[ 2950.601403][ T44] ? net_passive_dec+0x190/0x190
[ 2950.601573][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.601757][ T44] ? process_one_work+0xd16/0x1390
[ 2950.601918][ T44] ? lock_acquire+0x10a/0x150
[ 2950.602085][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.602251][ T44] ? process_one_work+0xd57/0x1390
[ 2950.602417][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 2950.602579][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 2950.602766][ T44] ? assign_work+0x152/0x380
[ 2950.602930][ T44] ? worker_thread+0x4d6/0xd40
[ 2950.603098][ T44] ? process_one_work+0x1390/0x1390
[ 2950.603262][ T44] ? kthread+0x355/0x5b0
[ 2950.603399][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2950.603561][ T44] ? __lock_release.isra.0+0x59/0x170
[ 2950.603743][ T44] ? rcu_is_watching+0x15/0xd0
[ 2950.603916][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2950.604081][ T44] ? ret_from_fork+0x3fb/0x510
[ 2950.604246][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 2950.604438][ T44] ? __switch_to+0x53c/0xd00
[ 2950.604602][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 2950.604790][ T44] ? ret_from_fork_asm+0x11/0x20
[ 2950.604958][ T44]
[ 2950.605134][ T44]
[ 2950.605134][ T44] Showing all locks held in the system:
[ 2950.605352][ T44] 4 locks held by kworker/u16:0/12:
[ 2950.605515][ T44] 1 lock held by khungtaskd/44:
[ 2950.605759][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 2950.606037][ T44] 1 lock held by modprobe/11474:
[ 2950.606190][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 2950.606473][ T44]
[ 2950.606577][ T44] =============================================
[ 2950.606577][ T44]
[ 3073.466668][ T44] INFO: task modprobe:11474 blocked for more than 1105 seconds.
[ 3073.467009][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 3073.467250][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3073.467518][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 3073.467866][ T44] Call Trace:
[ 3073.468013][ T44]
[ 3073.468127][ T44] __schedule+0xae2/0x1720
[ 3073.468307][ T44] ? io_schedule_timeout+0x130/0x130
[ 3073.468481][ T44] ? __lock_release.isra.0+0x59/0x170
[ 3073.468676][ T44] schedule+0xd4/0x260
[ 3073.468820][ T44] schedule_preempt_disabled+0x18/0x30
[ 3073.468996][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 3073.469168][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 3073.469341][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.469510][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 3073.469724][ T44] ? register_pernet_subsys+0x19/0x40
[ 3073.469896][ T44] ? lock_acquire+0x10a/0x150
[ 3073.470074][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 3073.470245][ T44] down_write+0x1c6/0x1f0
[ 3073.470384][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 3073.470581][ T44] register_pernet_subsys+0x19/0x40
[ 3073.470772][ T44] pg_init+0x29/0x1000 [pktgen]
[ 3073.470945][ T44] do_one_initcall+0xa3/0x2e0
[ 3073.471111][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 3073.471306][ T44] ? kasan_unpoison+0x44/0x70
[ 3073.471475][ T44] do_init_module+0x27b/0x820
[ 3073.471659][ T44] ? free_module+0x450/0x450
[ 3073.471828][ T44] ? backing_file_read_iter+0x50e/0x730
[ 3073.472004][ T44] init_module_from_file+0x15d/0x180
[ 3073.472169][ T44] ? __do_sys_init_module+0x200/0x200
[ 3073.472337][ T44] ? __lock_release.isra.0+0x59/0x170
[ 3073.472500][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 3073.472679][ T44] idempotent_init_module+0x226/0x760
[ 3073.472842][ T44] ? init_module_from_file+0x180/0x180
[ 3073.473021][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.473189][ T44] ? cap_capable+0x181/0x3f0
[ 3073.473356][ T44] __x64_sys_finit_module+0xcd/0x160
[ 3073.473522][ T44] do_syscall_64+0xbd/0xfc0
[ 3073.473703][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 3073.473892][ T44] RIP: 0033:0x7f04ee6ed34d
[ 3073.474069][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 3073.474294][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 3073.474510][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 3073.474740][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 3073.474965][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 3073.475184][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 3073.475405][ T44]
[ 3073.475542][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 3073.475841][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 3073.476166][ T44] Workqueue: netns cleanup_net
[ 3073.476330][ T44] Call Trace:
[ 3073.476466][ T44]
[ 3073.476575][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.476785][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 3073.476951][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.477144][ T44] ? lock_acquire+0x10a/0x150
[ 3073.477309][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.477472][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.477651][ T44] ? trace_contention_end+0xd6/0x110
[ 3073.477816][ T44] ? __mutex_lock+0x285/0x1f20
[ 3073.477989][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.478182][ T44] ? ww_mutex_lock+0x160/0x160
[ 3073.478345][ T44] ? __lock_acquire+0x577/0xc10
[ 3073.478513][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 3073.478694][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.478885][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3073.479085][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 3073.479308][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 3073.479531][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 3073.479740][ T44] ? kfree+0x119/0x580
[ 3073.479875][ T44] ? lock_acquire+0x10a/0x150
[ 3073.480040][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 3073.480233][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 3073.480398][ T44] ? netns_install+0x280/0x280
[ 3073.480563][ T44] ? cleanup_net+0x3bc/0x890
[ 3073.480741][ T44] ? process_one_work+0xd16/0x1390
[ 3073.480917][ T44] ? net_passive_dec+0x190/0x190
[ 3073.481081][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.481241][ T44] ? process_one_work+0xd16/0x1390
[ 3073.481402][ T44] ? lock_acquire+0x10a/0x150
[ 3073.481563][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.481743][ T44] ? process_one_work+0xd57/0x1390
[ 3073.481916][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 3073.482081][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 3073.482248][ T44] ? assign_work+0x152/0x380
[ 3073.482418][ T44] ? worker_thread+0x4d6/0xd40
[ 3073.482587][ T44] ? process_one_work+0x1390/0x1390
[ 3073.482766][ T44] ? kthread+0x355/0x5b0
[ 3073.482911][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 3073.483077][ T44] ? __lock_release.isra.0+0x59/0x170
[ 3073.483246][ T44] ? rcu_is_watching+0x15/0xd0
[ 3073.483410][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 3073.483575][ T44] ? ret_from_fork+0x3fb/0x510
[ 3073.483753][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 3073.483950][ T44] ? __switch_to+0x53c/0xd00
[ 3073.484114][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 3073.484279][ T44] ? ret_from_fork_asm+0x11/0x20
[ 3073.484450][ T44]
[ 3073.484648][ T44]
[ 3073.484648][ T44] Showing all locks held in the system:
[ 3073.484860][ T44] 4 locks held by kworker/u16:0/12:
[ 3073.485027][ T44] 1 lock held by khungtaskd/44:
[ 3073.485184][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 3073.485463][ T44] 1 lock held by modprobe/11474:
[ 3073.485728][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 3073.486016][ T44]
[ 3073.486120][ T44] =============================================
[ 3073.486120][ T44]
[ 3196.346666][ T44] INFO: task modprobe:11474 blocked for more than 1228 seconds.
[ 3196.347003][ T44] Tainted: G W 6.19.0-rc1-virtme #1
[ 3196.347260][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 3196.347532][ T44] task:modprobe state:D stack:25136 pid:11474 tgid:11474 ppid:1 task_flags:0x400100 flags:0x00080002
[ 3196.347904][ T44] Call Trace:
[ 3196.348054][ T44]
[ 3196.348167][ T44] __schedule+0xae2/0x1720
[ 3196.348349][ T44] ? io_schedule_timeout+0x130/0x130
[ 3196.348523][ T44] ? __lock_release.isra.0+0x59/0x170
[ 3196.348718][ T44] schedule+0xd4/0x260
[ 3196.348863][ T44] schedule_preempt_disabled+0x18/0x30
[ 3196.349042][ T44] rwsem_down_write_slowpath+0x40b/0xc10
[ 3196.349216][ T44] ? rwsem_down_read_slowpath+0xd10/0xd10
[ 3196.349393][ T44] ? rcu_is_watching+0x15/0xd0
[ 3196.349562][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 3196.349779][ T44] ? register_pernet_subsys+0x19/0x40
[ 3196.349957][ T44] ? lock_acquire+0x10a/0x150
[ 3196.350131][ T44] ? pg_cleanup+0x80/0x80 [pktgen]
[ 3196.350299][ T44] down_write+0x1c6/0x1f0
[ 3196.350438][ T44] ? rwsem_down_write_slowpath+0xc10/0xc10
[ 3196.350653][ T44] register_pernet_subsys+0x19/0x40
[ 3196.350818][ T44] pg_init+0x29/0x1000 [pktgen]
[ 3196.350984][ T44] do_one_initcall+0xa3/0x2e0
[ 3196.351157][ T44] ? trace_event_raw_event_initcall_level+0x1f0/0x1f0
[ 3196.351351][ T44] ? kasan_unpoison+0x44/0x70
[ 3196.351520][ T44] do_init_module+0x27b/0x820
[ 3196.351703][ T44] ? free_module+0x450/0x450
[ 3196.351869][ T44] ? backing_file_read_iter+0x50e/0x730
[ 3196.352042][ T44] init_module_from_file+0x15d/0x180
[ 3196.352207][ T44] ? __do_sys_init_module+0x200/0x200
[ 3196.352379][ T44] ? __lock_release.isra.0+0x59/0x170
[ 3196.352542][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 3196.352721][ T44] idempotent_init_module+0x226/0x760
[ 3196.352884][ T44] ? init_module_from_file+0x180/0x180
[ 3196.353056][ T44] ? rcu_is_watching+0x15/0xd0
[ 3196.353220][ T44] ? cap_capable+0x181/0x3f0
[ 3196.353385][ T44] __x64_sys_finit_module+0xcd/0x160
[ 3196.353554][ T44] do_syscall_64+0xbd/0xfc0
[ 3196.353737][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 3196.353927][ T44] RIP: 0033:0x7f04ee6ed34d
[ 3196.354100][ T44] RSP: 002b:00007ffe9845f698 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 3196.354320][ T44] RAX: ffffffffffffffda RBX: 0000563bb96c8a00 RCX: 00007f04ee6ed34d
[ 3196.354534][ T44] RDX: 0000000000000000 RSI: 0000563badf89f2e RDI: 0000000000000005
[ 3196.354767][ T44] RBP: 00007ffe9845f730 R08: 0000000000000000 R09: 0000000000000000
[ 3196.354980][ T44] R10: 0000000000000000 R11: 0000000000000246 R12: 0000563badf89f2e
[ 3196.355203][ T44] R13: 0000000000040000 R14: 0000563bb96c8b20 R15: 0000563bb96c8460
[ 3196.355424][ T44]
[ 3196.355563][ T44] INFO: task modprobe:11474 blocked on an rw-semaphore likely owned by task kworker/u16:0:12
[ 3196.355865][ T44] task:kworker/u16:0 state:R running task stack:25424 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 3196.356188][ T44] Workqueue: netns cleanup_net
[ 3196.356349][ T44] Call Trace:
[ 3196.356482][ T44]
[ 3196.356591][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3196.356811][ T44] ? rcu_is_watching+0x15/0xd0
[ 3196.356973][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 3196.357143][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3196.357333][ T44] ? lock_acquire+0x10a/0x150
[ 3196.357496][ T44] ? rcu_is_watching+0x15/0xd0
[ 3196.357673][ T44] ? rcu_is_watching+0x15/0xd0
[ 3196.357840][ T44] ? trace_contention_end+0xd6/0x110
[ 3196.358004][ T44] ? __mutex_lock+0x285/0x1f20
[ 3196.358176][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3196.358370][ T44] ? ww_mutex_lock+0x160/0x160
[ 3196.358533][ T44] ? __lock_acquire+0x577/0xc10
[ 3196.358714][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 3196.358879][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3196.359074][ T44] ? nf_ct_iterate_cleanup+0x4b/0x350 [nf_conntrack]
[ 3196.359265][ T44] ? nf_ct_port_nlattr_to_tuple+0x230/0x230 [nf_conntrack]
[ 3196.359486][ T44] ? nf_conntrack_cleanup_net_list+0x19d/0x550 [nf_conntrack]
[ 3196.359719][ T44] ? nf_conntrack_cleanup_end+0x70/0x70 [nf_conntrack]
[ 3196.359912][ T44] ? kfree+0x119/0x580
[ 3196.360056][ T44] ? lock_acquire+0x10a/0x150
[ 3196.360223][ T44] ? nf_conntrack_pernet_exit+0xa1/0x140 [nf_conntrack]
[ 3196.360417][ T44] ? ops_undo_list+0x2d7/0x8f0
[ 3196.360583][ T44] ? netns_install+0x280/0x280
[ 3196.360767][ T44] ? cleanup_net+0x3bc/0x890
[ 3196.360930][ T44] ? process_one_work+0xd16/0x1390
[ 3196.361100][ T44] ? net_passive_dec+0x190/0x190
[ 3196.361264][ T44] ? rcu_is_watching+0x15/0xd0
[ 3196.361430][ T44] ? process_one_work+0xd16/0x1390
[ 3196.361594][ T44] ? lock_acquire+0x10a/0x150
[ 3196.361773][ T44] ? rcu_is_watching+0x15/0xd0
[ 3196.361941][ T44] ? process_one_work+0xd57/0x1390
[ 3196.362115][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 3196.362278][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 3196.362443][ T44] ? assign_work+0x152/0x380
[ 3196.362628][ T44] ? worker_thread+0x4d6/0xd40
[ 3196.362796][ T44] ? process_one_work+0x1390/0x1390
[ 3196.362959][ T44] ? kthread+0x355/0x5b0
[ 3196.363101][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 3196.363271][ T44] ? __lock_release.isra.0+0x59/0x170
[ 3196.363434][ T44] ? rcu_is_watching+0x15/0xd0
[ 3196.363597][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 3196.363779][ T44] ? ret_from_fork+0x3fb/0x510
[ 3196.363944][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 3196.364140][ T44] ? __switch_to+0x53c/0xd00
[ 3196.364303][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 3196.364466][ T44] ? ret_from_fork_asm+0x11/0x20
[ 3196.364648][ T44]
[ 3196.364780][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
[ 3196.365067][ T44]
[ 3196.365067][ T44] Showing all locks held in the system:
[ 3196.365279][ T44] 4 locks held by kworker/u16:0/12:
[ 3196.365438][ T44] 1 lock held by khungtaskd/44:
[ 3196.365593][ T44] #0: ffffffff851850c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 3196.365979][ T44] 1 lock held by modprobe/11474:
[ 3196.366140][ T44] #0: ffffffff85885bd0 (pernet_ops_rwsem){++++}-{4:4}, at: register_pernet_subsys+0x19/0x40
[ 3196.366428][ T44]
[ 3196.366531][ T44] =============================================
[ 3196.366531][ T44]
WAIT TIMEOUT stderr
Ctrl-C stderr
Ctrl-C stderr