[ 247.103387][ T44] INFO: task ip:334 blocked for more than 122 seconds. [ 247.103734][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 247.103877][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 247.104133][ T44] task:ip state:D stack:24544 pid:334 tgid:334 ppid:1 task_flags:0x400100 flags:0x00080002 [ 247.104433][ T44] Call Trace: [ 247.104536][ T44] [ 247.104609][ T44] __schedule+0xae2/0x1720 [ 247.104770][ T44] ? io_schedule_timeout+0x130/0x130 [ 247.104907][ T44] ? __lock_release.isra.0+0x59/0x170 [ 247.105040][ T44] ? __lock_release.isra.0+0x59/0x170 [ 247.105172][ T44] schedule+0xd4/0x260 [ 247.105263][ T44] schedule_preempt_disabled+0x18/0x30 [ 247.105390][ T44] __mutex_lock+0xd76/0x1f20 [ 247.105511][ T44] ? __folio_put+0x1f5/0x320 [ 247.105633][ T44] ? napi_enable+0x3b/0x80 [ 247.105762][ T44] ? ww_mutex_lock+0x160/0x160 [ 247.105884][ T44] ? dma_map_phys+0x1b1/0x220 [ 247.106007][ T44] ? __asan_memset+0x27/0x50 [ 247.106134][ T44] ? sg_init_table+0x19/0x60 [ 247.106255][ T44] ? try_fill_recv+0x3ec/0x1750 [ 247.106385][ T44] ? napi_enable+0x3b/0x80 [ 247.106498][ T44] napi_enable+0x3b/0x80 [ 247.106589][ T44] __virtnet_rx_resume+0x156/0x290 [ 247.106708][ T44] ? queue_delayed_work_on+0x91/0x100 [ 247.106830][ T44] virtnet_rx_resume_all+0xe7/0x1a0 [ 247.106948][ T44] virtnet_xdp_set+0xef7/0x1aa0 [ 247.107068][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0 [ 247.107184][ T44] dev_xdp_install+0x3e4/0x890 [ 247.107299][ T44] ? netif_set_xps_queue+0x40/0x40 [ 247.107646][ T44] dev_xdp_attach+0x50d/0x1310 [ 247.107768][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0 [ 247.107887][ T44] ? __bpf_prog_get+0x320/0x320 [ 247.108005][ T44] dev_change_xdp_fd+0x176/0x210 [ 247.108122][ T44] do_setlink.isra.0+0x1c2d/0x2880 [ 247.108239][ T44] ? rtnl_newlink_create+0x900/0x900 [ 247.108358][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 247.108502][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 247.108618][ T44] ? validate_chain+0x37b/0xb70 [ 247.108738][ T44] ? rcu_read_lock_any_held+0x66/0x90 [ 247.108851][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 247.108967][ T44] ? __lock_acquire+0x577/0xc10 [ 247.109081][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 247.109196][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 247.109308][ T44] ? lock_acquire+0x10a/0x150 [ 247.109426][ T44] ? rcu_is_watching+0x15/0xd0 [ 247.109540][ T44] ? rcu_is_watching+0x15/0xd0 [ 247.109654][ T44] ? trace_contention_end+0xd6/0x110 [ 247.109771][ T44] ? __mutex_lock+0x285/0x1f20 [ 247.109886][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 247.110000][ T44] ? ww_mutex_lock+0x160/0x160 [ 247.110115][ T44] ? nla_get_range_signed+0x3d0/0x3d0 [ 247.110235][ T44] ? __rtnl_newlink+0x3fa/0xa40 [ 247.110358][ T44] rtnl_newlink+0x75c/0xe90 [ 247.110476][ T44] ? __lock_release.isra.0+0x59/0x170 [ 247.110591][ T44] ? rtnl_setlink+0x880/0x880 [ 247.110704][ T44] ? __lock_acquire+0x577/0xc10 [ 247.110820][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 247.110935][ T44] ? find_held_lock+0x2b/0x80 [ 247.111050][ T44] ? rtnl_setlink+0x880/0x880 [ 247.111163][ T44] ? __lock_release.isra.0+0x59/0x170 [ 247.111277][ T44] ? rtnl_setlink+0x880/0x880 [ 247.111394][ T44] rtnetlink_rcv_msg+0x6fe/0xb90 [ 247.111508][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 247.111620][ T44] ? __lock_acquire+0x577/0xc10 [ 247.111739][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 247.111853][ T44] ? find_held_lock+0x2b/0x80 [ 247.111968][ T44] netlink_rcv_skb+0x123/0x380 [ 247.112083][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 247.112198][ T44] ? netlink_ack+0xcc0/0xcc0 [ 247.112318][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 247.112436][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 247.112555][ T44] netlink_unicast+0x4a3/0x770 [ 247.112671][ T44] ? netlink_attachskb+0x810/0x810 [ 247.112788][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 247.112902][ T44] ? __lock_acquire+0x577/0xc10 [ 247.113017][ T44] netlink_sendmsg+0x735/0xc60 [ 247.113132][ T44] ? netlink_unicast+0x770/0x770 [ 247.113251][ T44] ____sys_sendmsg+0x419/0x850 [ 247.113372][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 247.113485][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 247.113630][ T44] ? move_addr_to_kernel+0x40/0x40 [ 247.113747][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 247.113858][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 247.113972][ T44] ___sys_sendmsg+0xfd/0x180 [ 247.114084][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00 [ 247.114230][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 247.114363][ T44] __sys_sendmsg+0x124/0x1c0 [ 247.114478][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 247.114593][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 247.114705][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00 [ 247.114852][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 247.114965][ T44] ? rcu_is_watching+0x15/0xd0 [ 247.115080][ T44] do_syscall_64+0xbd/0xfc0 [ 247.115195][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 247.115337][ T44] RIP: 0033:0x7fe5ac980c5e [ 247.115463][ T44] RSP: 002b:00007ffd4cfeaaf0 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 247.115636][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007fe5ac980c5e [ 247.115811][ T44] RDX: 0000000000000000 RSI: 00007ffd4cfeabb0 RDI: 0000000000000003 [ 247.115985][ T44] RBP: 00007ffd4cfeab00 R08: 0000000000000000 R09: 0000000000000000 [ 247.116155][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffd4cfeb270 [ 247.116325][ T44] R13: 00000000695b7092 R14: 0000000000000000 R15: 0000000000499600 [ 247.116504][ T44] [ 247.116599][ T44] INFO: task ip:334 is blocked on a mutex likely owned by task kworker/0:0:9. [ 247.116789][ T44] task:kworker/0:0 state:D stack:27144 pid:9 tgid:9 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 247.117037][ T44] Workqueue: events refill_work [ 247.117150][ T44] Call Trace: [ 247.117237][ T44] [ 247.117297][ T44] __schedule+0xae2/0x1720 [ 247.117419][ T44] ? usleep_range_state+0xba/0x1b0 [ 247.117534][ T44] ? io_schedule_timeout+0x130/0x130 [ 247.117648][ T44] ? __lock_release.isra.0+0x59/0x170 [ 247.117764][ T44] ? hrtimer_start_range_ns+0xe8/0x1d0 [ 247.117879][ T44] ? usleep_range_state+0xba/0x1b0 [ 247.117993][ T44] schedule+0xd4/0x260 [ 247.118080][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0 [ 247.118225][ T44] ? schedule_timeout_idle+0xe0/0xe0 [ 247.118340][ T44] ? __hrtimer_cb_get_time+0x50/0x50 [ 247.118459][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 247.118572][ T44] ? ktime_get+0x15e/0x220 [ 247.118684][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 247.118799][ T44] ? ktime_get+0xa9/0x220 [ 247.118886][ T44] usleep_range_state+0x133/0x1b0 [ 247.119002][ T44] ? schedule_hrtimeout+0x20/0x20 [ 247.119115][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 247.119228][ T44] ? refill_work+0x149/0x210 [ 247.119342][ T44] napi_disable_locked+0x379/0x5f0 [ 247.119460][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 247.119575][ T44] ? register_netdevice_notifier_net+0x40/0x40 [ 247.119718][ T44] napi_disable+0x43/0x80 [ 247.119809][ T44] refill_work+0x116/0x210 [ 247.119926][ T44] process_one_work+0xd57/0x1390 [ 247.120042][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 247.120156][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 247.120272][ T44] ? assign_work+0x152/0x380 [ 247.120391][ T44] worker_thread+0x4d6/0xd40 [ 247.120510][ T44] ? process_one_work+0x1390/0x1390 [ 247.120628][ T44] kthread+0x355/0x5b0 [ 247.120715][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 247.120831][ T44] ? __lock_release.isra.0+0x59/0x170 [ 247.120945][ T44] ? rcu_is_watching+0x15/0xd0 [ 247.121058][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 247.121173][ T44] ret_from_fork+0x3fb/0x510 [ 247.121287][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 247.121433][ T44] ? __switch_to+0x53c/0xd00 [ 247.121549][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 247.121663][ T44] ret_from_fork_asm+0x11/0x20 [ 247.121788][ T44] [ 247.121883][ T44] [ 247.121883][ T44] Showing all locks held in the system: [ 247.122049][ T44] 3 locks held by kworker/0:0/9: [ 247.122160][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 247.122464][ T44] #1: ffa0000000097cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 247.122710][ T44] #2: ff110000091b2c98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 247.122907][ T44] 3 locks held by kworker/u16:0/12: [ 247.123016][ T44] #0: ff1100000983f148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 247.123237][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 247.123514][ T44] #2: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 247.123712][ T44] 3 locks held by kworker/2:0/28: [ 247.123824][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 247.124045][ T44] #1: ffa00000001ffcc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 247.124264][ T44] #2: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0 [ 247.124465][ T44] 1 lock held by khungtaskd/44: [ 247.124574][ T44] #0: ffffffffb73710c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 247.124807][ T44] 2 locks held by ip/334: [ 247.124891][ T44] #0: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90 [ 247.125084][ T44] #1: ff110000091b2c98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80 [ 247.125277][ T44] 1 lock held by xdp_helper/342: [ 247.125389][ T44] #0: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: xsk_bind+0x17f/0x1b10 [ 247.125587][ T44] 1 lock held by ip/401: [ 247.125669][ T44] #0: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2ab/0xb90 [ 247.125866][ T44] [ 247.125924][ T44] ============================================= [ 247.125924][ T44] [ 369.983363][ T44] INFO: task kworker/u16:0:12 blocked for more than 122 seconds. [ 369.983638][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 369.983779][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.984016][ T44] task:kworker/u16:0 state:D stack:27952 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 369.984330][ T44] Workqueue: ipv6_addrconf addrconf_verify_work [ 369.984514][ T44] Call Trace: [ 369.984608][ T44] [ 369.984677][ T44] __schedule+0xae2/0x1720 [ 369.984804][ T44] ? io_schedule_timeout+0x130/0x130 [ 369.984935][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.985059][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.985185][ T44] schedule+0xd4/0x260 [ 369.985277][ T44] schedule_preempt_disabled+0x18/0x30 [ 369.985405][ T44] __mutex_lock+0xd76/0x1f20 [ 369.985529][ T44] ? addrconf_verify_work+0x12/0x30 [ 369.985648][ T44] ? ww_mutex_lock+0x160/0x160 [ 369.985767][ T44] ? process_one_work+0xd16/0x1390 [ 369.985885][ T44] ? rcu_is_watching+0x15/0xd0 [ 369.986006][ T44] ? rcu_is_watching+0x15/0xd0 [ 369.986122][ T44] ? addrconf_verify_work+0x12/0x30 [ 369.986238][ T44] addrconf_verify_work+0x12/0x30 [ 369.986359][ T44] process_one_work+0xd57/0x1390 [ 369.986478][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 369.986594][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 369.986710][ T44] ? assign_work+0x152/0x380 [ 369.986826][ T44] worker_thread+0x4d6/0xd40 [ 369.986946][ T44] ? process_one_work+0x1390/0x1390 [ 369.987060][ T44] kthread+0x355/0x5b0 [ 369.987147][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 369.987260][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.987379][ T44] ? rcu_is_watching+0x15/0xd0 [ 369.987492][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 369.987605][ T44] ret_from_fork+0x3fb/0x510 [ 369.987722][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 369.987863][ T44] ? __switch_to+0x53c/0xd00 [ 369.987980][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 369.988097][ T44] ret_from_fork_asm+0x11/0x20 [ 369.988218][ T44] [ 369.988312][ T44] INFO: task kworker/u16:0:12 is blocked on a mutex likely owned by task ip:334. [ 369.988507][ T44] INFO: task kworker/2:0:28 blocked for more than 122 seconds. [ 369.988666][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 369.988773][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.988959][ T44] task:kworker/2:0 state:D stack:27344 pid:28 tgid:28 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 369.989204][ T44] Workqueue: events xp_release_deferred [ 369.989321][ T44] Call Trace: [ 369.989411][ T44] [ 369.989471][ T44] __schedule+0xae2/0x1720 [ 369.989590][ T44] ? io_schedule_timeout+0x130/0x130 [ 369.989704][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.989819][ T44] schedule+0xd4/0x260 [ 369.989908][ T44] schedule_preempt_disabled+0x18/0x30 [ 369.990021][ T44] __mutex_lock+0xd76/0x1f20 [ 369.990140][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 369.990254][ T44] ? ww_mutex_lock+0x160/0x160 [ 369.990374][ T44] ? __lock_acquire+0x577/0xc10 [ 369.990489][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 369.990601][ T44] ? process_one_work+0xd16/0x1390 [ 369.990717][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 369.990829][ T44] xp_release_deferred+0x1b/0x1e0 [ 369.990948][ T44] process_one_work+0xd57/0x1390 [ 369.991063][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 369.991175][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 369.991291][ T44] ? assign_work+0x152/0x380 [ 369.991410][ T44] worker_thread+0x4d6/0xd40 [ 369.991527][ T44] ? process_one_work+0x1390/0x1390 [ 369.991639][ T44] kthread+0x355/0x5b0 [ 369.991727][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 369.991839][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.991954][ T44] ? rcu_is_watching+0x15/0xd0 [ 369.992067][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 369.992180][ T44] ret_from_fork+0x3fb/0x510 [ 369.992293][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 369.992437][ T44] ? __switch_to+0x53c/0xd00 [ 369.992550][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 369.992667][ T44] ret_from_fork_asm+0x11/0x20 [ 369.992785][ T44] [ 369.992871][ T44] INFO: task kworker/2:0:28 is blocked on a mutex likely owned by task ip:334. [ 369.993076][ T44] INFO: task ip:334 blocked for more than 245 seconds. [ 369.993207][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 369.993312][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.993501][ T44] task:ip state:D stack:24544 pid:334 tgid:334 ppid:1 task_flags:0x400100 flags:0x00080002 [ 369.993745][ T44] Call Trace: [ 369.993831][ T44] [ 369.993890][ T44] __schedule+0xae2/0x1720 [ 369.994008][ T44] ? io_schedule_timeout+0x130/0x130 [ 369.994122][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.994236][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.994357][ T44] schedule+0xd4/0x260 [ 369.994444][ T44] schedule_preempt_disabled+0x18/0x30 [ 369.994557][ T44] __mutex_lock+0xd76/0x1f20 [ 369.994672][ T44] ? __folio_put+0x1f5/0x320 [ 369.994785][ T44] ? napi_enable+0x3b/0x80 [ 369.994900][ T44] ? ww_mutex_lock+0x160/0x160 [ 369.995016][ T44] ? dma_map_phys+0x1b1/0x220 [ 369.995133][ T44] ? __asan_memset+0x27/0x50 [ 369.995247][ T44] ? sg_init_table+0x19/0x60 [ 369.995366][ T44] ? try_fill_recv+0x3ec/0x1750 [ 369.995483][ T44] ? napi_enable+0x3b/0x80 [ 369.995595][ T44] napi_enable+0x3b/0x80 [ 369.995679][ T44] __virtnet_rx_resume+0x156/0x290 [ 369.995791][ T44] ? queue_delayed_work_on+0x91/0x100 [ 369.995910][ T44] virtnet_rx_resume_all+0xe7/0x1a0 [ 369.996025][ T44] virtnet_xdp_set+0xef7/0x1aa0 [ 369.996142][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0 [ 369.996255][ T44] dev_xdp_install+0x3e4/0x890 [ 369.996373][ T44] ? netif_set_xps_queue+0x40/0x40 [ 369.996490][ T44] dev_xdp_attach+0x50d/0x1310 [ 369.996606][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0 [ 369.996721][ T44] ? __bpf_prog_get+0x320/0x320 [ 369.996838][ T44] dev_change_xdp_fd+0x176/0x210 [ 369.996954][ T44] do_setlink.isra.0+0x1c2d/0x2880 [ 369.997070][ T44] ? rtnl_newlink_create+0x900/0x900 [ 369.997183][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 369.997324][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 369.997447][ T44] ? validate_chain+0x37b/0xb70 [ 369.997562][ T44] ? rcu_read_lock_any_held+0x66/0x90 [ 369.997675][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 369.997789][ T44] ? __lock_acquire+0x577/0xc10 [ 369.997904][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 369.998020][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 369.998132][ T44] ? lock_acquire+0x10a/0x150 [ 369.998244][ T44] ? rcu_is_watching+0x15/0xd0 [ 369.998360][ T44] ? rcu_is_watching+0x15/0xd0 [ 369.998473][ T44] ? trace_contention_end+0xd6/0x110 [ 369.998588][ T44] ? __mutex_lock+0x285/0x1f20 [ 369.998702][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 369.998817][ T44] ? ww_mutex_lock+0x160/0x160 [ 369.998934][ T44] ? nla_get_range_signed+0x3d0/0x3d0 [ 369.999050][ T44] ? __rtnl_newlink+0x3fa/0xa40 [ 369.999165][ T44] rtnl_newlink+0x75c/0xe90 [ 369.999283][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.999399][ T44] ? rtnl_setlink+0x880/0x880 [ 369.999513][ T44] ? __lock_acquire+0x577/0xc10 [ 369.999627][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 369.999740][ T44] ? find_held_lock+0x2b/0x80 [ 369.999853][ T44] ? rtnl_setlink+0x880/0x880 [ 369.999968][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.000082][ T44] ? rtnl_setlink+0x880/0x880 [ 370.000195][ T44] rtnetlink_rcv_msg+0x6fe/0xb90 [ 370.000310][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 370.000427][ T44] ? __lock_acquire+0x577/0xc10 [ 370.000541][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.000653][ T44] ? find_held_lock+0x2b/0x80 [ 370.000767][ T44] netlink_rcv_skb+0x123/0x380 [ 370.000881][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 370.000999][ T44] ? netlink_ack+0xcc0/0xcc0 [ 370.001116][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 370.001228][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 370.001342][ T44] netlink_unicast+0x4a3/0x770 [ 370.001460][ T44] ? netlink_attachskb+0x810/0x810 [ 370.001577][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.001691][ T44] ? __lock_acquire+0x577/0xc10 [ 370.001805][ T44] netlink_sendmsg+0x735/0xc60 [ 370.001924][ T44] ? netlink_unicast+0x770/0x770 [ 370.002041][ T44] ____sys_sendmsg+0x419/0x850 [ 370.002155][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 370.002267][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 370.002412][ T44] ? move_addr_to_kernel+0x40/0x40 [ 370.002526][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 370.002638][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 370.002754][ T44] ___sys_sendmsg+0xfd/0x180 [ 370.002868][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00 [ 370.003018][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 370.003149][ T44] __sys_sendmsg+0x124/0x1c0 [ 370.003263][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 370.003380][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 370.003491][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00 [ 370.003636][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 370.003749][ T44] ? rcu_is_watching+0x15/0xd0 [ 370.003864][ T44] do_syscall_64+0xbd/0xfc0 [ 370.003981][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 370.004122][ T44] RIP: 0033:0x7fe5ac980c5e [ 370.004240][ T44] RSP: 002b:00007ffd4cfeaaf0 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 370.004426][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007fe5ac980c5e [ 370.004595][ T44] RDX: 0000000000000000 RSI: 00007ffd4cfeabb0 RDI: 0000000000000003 [ 370.004764][ T44] RBP: 00007ffd4cfeab00 R08: 0000000000000000 R09: 0000000000000000 [ 370.004934][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffd4cfeb270 [ 370.005105][ T44] R13: 00000000695b7092 R14: 0000000000000000 R15: 0000000000499600 [ 370.005280][ T44] [ 370.005368][ T44] INFO: task ip:334 is blocked on a mutex likely owned by task kworker/0:0:9. [ 370.005553][ T44] task:kworker/0:0 state:D stack:27144 pid:9 tgid:9 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 370.005797][ T44] Workqueue: events refill_work [ 370.005914][ T44] Call Trace: [ 370.006002][ T44] [ 370.006061][ T44] __schedule+0xae2/0x1720 [ 370.006178][ T44] ? usleep_range_state+0xba/0x1b0 [ 370.006291][ T44] ? io_schedule_timeout+0x130/0x130 [ 370.006410][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.006525][ T44] ? usleep_range_state+0xba/0x1b0 [ 370.006636][ T44] schedule+0xd4/0x260 [ 370.006726][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0 [ 370.006866][ T44] ? schedule_timeout_idle+0xe0/0xe0 [ 370.006982][ T44] ? __hrtimer_cb_get_time+0x50/0x50 [ 370.007096][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 370.007208][ T44] ? ktime_get+0x15e/0x220 [ 370.007320][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 370.007437][ T44] ? ktime_get+0xa9/0x220 [ 370.007525][ T44] usleep_range_state+0x133/0x1b0 [ 370.007639][ T44] ? schedule_hrtimeout+0x20/0x20 [ 370.007753][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 370.007867][ T44] ? refill_work+0x149/0x210 [ 370.007984][ T44] napi_disable_locked+0x379/0x5f0 [ 370.008098][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 370.008212][ T44] ? register_netdevice_notifier_net+0x40/0x40 [ 370.008361][ T44] napi_disable+0x43/0x80 [ 370.008448][ T44] refill_work+0x116/0x210 [ 370.008562][ T44] process_one_work+0xd57/0x1390 [ 370.008679][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 370.008794][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.008912][ T44] ? assign_work+0x152/0x380 [ 370.009027][ T44] worker_thread+0x4d6/0xd40 [ 370.009148][ T44] ? process_one_work+0x1390/0x1390 [ 370.009326][ T44] kthread+0x355/0x5b0 [ 370.009416][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 370.009528][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.009642][ T44] ? rcu_is_watching+0x15/0xd0 [ 370.009809][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 370.009925][ T44] ret_from_fork+0x3fb/0x510 [ 370.010039][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 370.010183][ T44] ? __switch_to+0x53c/0xd00 [ 370.010357][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 370.010472][ T44] ret_from_fork_asm+0x11/0x20 [ 370.010591][ T44] [ 370.010676][ T44] INFO: task xdp_helper:342 blocked for more than 122 seconds. [ 370.010833][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 370.010942][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 370.011126][ T44] task:xdp_helper state:D stack:26224 pid:342 tgid:342 ppid:1 task_flags:0x400100 flags:0x00080000 [ 370.011428][ T44] Call Trace: [ 370.011513][ T44] [ 370.011571][ T44] __schedule+0xae2/0x1720 [ 370.011687][ T44] ? io_schedule_timeout+0x130/0x130 [ 370.011858][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.011974][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.012089][ T44] schedule+0xd4/0x260 [ 370.012176][ T44] schedule_preempt_disabled+0x18/0x30 [ 370.012344][ T44] __mutex_lock+0xd76/0x1f20 [ 370.012465][ T44] ? find_held_lock+0x2b/0x80 [ 370.012578][ T44] ? xsk_bind+0x17f/0x1b10 [ 370.012696][ T44] ? ww_mutex_lock+0x160/0x160 [ 370.012867][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 370.013010][ T44] ? validate_chain+0x37b/0xb70 [ 370.013124][ T44] ? __lock_acquire+0x577/0xc10 [ 370.013240][ T44] ? xsk_bind+0x17f/0x1b10 [ 370.013414][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.013529][ T44] xsk_bind+0x17f/0x1b10 [ 370.013616][ T44] ? __might_fault+0x97/0x140 [ 370.013731][ T44] ? find_held_lock+0x2b/0x80 [ 370.013848][ T44] ? __xsk_rcv_zc+0x910/0x910 [ 370.013965][ T44] ? __might_fault+0x97/0x140 [ 370.014078][ T44] __sys_bind+0x174/0x230 [ 370.014165][ T44] ? __sys_bind_socket+0xb0/0xb0 [ 370.014336][ T44] ? __sys_setsockopt+0xe4/0x150 [ 370.014455][ T44] __x64_sys_bind+0x72/0xd0 [ 370.014569][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 370.014680][ T44] ? do_syscall_64+0x88/0xfc0 [ 370.014850][ T44] do_syscall_64+0xbd/0xfc0 [ 370.014965][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 370.015105][ T44] RIP: 0033:0x7f87aff3febb [ 370.015223][ T44] RSP: 002b:00007ffce6773748 EFLAGS: 00000206 ORIG_RAX: 0000000000000031 [ 370.015455][ T44] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f87aff3febb [ 370.015624][ T44] RDX: 0000000000000010 RSI: 00007ffce6773760 RDI: 0000000000000003 [ 370.015795][ T44] RBP: 0000000000000000 R08: 0000000000000004 R09: 0000000000000000 [ 370.015966][ T44] R10: 0000000000000000 R11: 0000000000000206 R12: 00007ffce67738d8 [ 370.016132][ T44] R13: 00007f87afe36000 R14: 0000000000000004 R15: 0000000000402e00 [ 370.016367][ T44] [ 370.016457][ T44] INFO: task xdp_helper:342 is blocked on a mutex likely owned by task ip:334. [ 370.016648][ T44] INFO: task ip:401 blocked for more than 122 seconds. [ 370.016827][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 370.016934][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 370.017122][ T44] task:ip state:D stack:25624 pid:401 tgid:401 ppid:400 task_flags:0x400100 flags:0x00080000 [ 370.017425][ T44] Call Trace: [ 370.017509][ T44] [ 370.017567][ T44] __schedule+0xae2/0x1720 [ 370.017681][ T44] ? io_schedule_timeout+0x130/0x130 [ 370.017850][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.017966][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.018080][ T44] schedule+0xd4/0x260 [ 370.018168][ T44] schedule_preempt_disabled+0x18/0x30 [ 370.018333][ T44] __mutex_lock+0xd76/0x1f20 [ 370.018453][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.018565][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 370.018676][ T44] ? write_profile+0xf0/0xf0 [ 370.018850][ T44] ? ww_mutex_lock+0x160/0x160 [ 370.018969][ T44] ? find_held_lock+0x2b/0x80 [ 370.019083][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.019213][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 370.019384][ T44] rtnetlink_rcv_msg+0x2ab/0xb90 [ 370.019499][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 370.019611][ T44] ? __lock_acquire+0x577/0xc10 [ 370.019725][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.019891][ T44] ? find_held_lock+0x2b/0x80 [ 370.020010][ T44] netlink_rcv_skb+0x123/0x380 [ 370.020126][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 370.020239][ T44] ? netlink_ack+0xcc0/0xcc0 [ 370.020418][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 370.020529][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 370.020641][ T44] netlink_unicast+0x4a3/0x770 [ 370.020754][ T44] ? netlink_attachskb+0x810/0x810 [ 370.020922][ T44] ? __lock_acquire+0x577/0xc10 [ 370.021037][ T44] netlink_sendmsg+0x735/0xc60 [ 370.021150][ T44] ? netlink_unicast+0x770/0x770 [ 370.021265][ T44] ____sys_sendmsg+0x419/0x850 [ 370.021437][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 370.021549][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 370.021688][ T44] ? move_addr_to_kernel+0x40/0x40 [ 370.021799][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 370.021972][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 370.022111][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 370.022224][ T44] ___sys_sendmsg+0xfd/0x180 [ 370.022336][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 370.022454][ T44] ? __lock_acquire+0x577/0xc10 [ 370.022568][ T44] ? find_held_lock+0x2b/0x80 [ 370.022684][ T44] ? move_addr_to_user+0x43/0x60 [ 370.022797][ T44] ? do_getsockname+0xcd/0x150 [ 370.022971][ T44] __sys_sendmsg+0x124/0x1c0 [ 370.023083][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 370.023200][ T44] ? __sys_getsockname+0x91/0xf0 [ 370.023312][ T44] ? rcu_is_watching+0x15/0xd0 [ 370.023484][ T44] do_syscall_64+0xbd/0xfc0 [ 370.023597][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 370.023735][ T44] RIP: 0033:0x7f320e7a9c5e [ 370.023915][ T44] RSP: 002b:00007ffcef9a6fb0 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 370.024085][ T44] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f320e7a9c5e [ 370.024254][ T44] RDX: 0000000000000000 RSI: 00007ffcef9a7070 RDI: 0000000000000004 [ 370.024425][ T44] RBP: 00007ffcef9a6fc0 R08: 0000000000000000 R09: 0000000000000000 [ 370.024591][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffcef9a868d [ 370.024758][ T44] R13: 00000000695b7164 R14: 000000000000000d R15: 00007ffcef9a7120 [ 370.024991][ T44] [ 370.025078][ T44] INFO: task ip:401 is blocked on a mutex likely owned by task ip:334. [ 370.025238][ T44] [ 370.025238][ T44] Showing all locks held in the system: [ 370.025455][ T44] 3 locks held by kworker/0:0/9: [ 370.025567][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 370.025787][ T44] #1: ffa0000000097cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 370.026087][ T44] #2: ff110000091b2c98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 370.026280][ T44] 3 locks held by kworker/u16:0/12: [ 370.026448][ T44] #0: ff1100000983f148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 370.026668][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 370.026992][ T44] #2: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 370.027188][ T44] 3 locks held by kworker/2:0/28: [ 370.027354][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 370.027571][ T44] #1: ffa00000001ffcc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 370.027790][ T44] #2: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0 [ 370.027988][ T44] 1 lock held by khungtaskd/44: [ 370.028096][ T44] #0: ffffffffb73710c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 370.028373][ T44] 2 locks held by ip/334: [ 370.028455][ T44] #0: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90 [ 370.028649][ T44] #1: ff110000091b2c98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80 [ 370.028895][ T44] 1 lock held by xdp_helper/342: [ 370.029005][ T44] #0: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: xsk_bind+0x17f/0x1b10 [ 370.029198][ T44] 1 lock held by ip/401: [ 370.029331][ T44] #0: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2ab/0xb90 [ 370.029528][ T44] [ 370.029584][ T44] ============================================= [ 370.029584][ T44] [ 492.863362][ T44] INFO: task kworker/u16:0:12 blocked for more than 245 seconds. [ 492.863625][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 492.863758][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.863981][ T44] task:kworker/u16:0 state:D stack:27952 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 492.864288][ T44] Workqueue: ipv6_addrconf addrconf_verify_work [ 492.864467][ T44] Call Trace: [ 492.864571][ T44] [ 492.864642][ T44] __schedule+0xae2/0x1720 [ 492.864792][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.864919][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.865044][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.865168][ T44] schedule+0xd4/0x260 [ 492.865261][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.865385][ T44] __mutex_lock+0xd76/0x1f20 [ 492.865505][ T44] ? addrconf_verify_work+0x12/0x30 [ 492.865626][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.865751][ T44] ? process_one_work+0xd16/0x1390 [ 492.865871][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.865989][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.866116][ T44] ? addrconf_verify_work+0x12/0x30 [ 492.866230][ T44] addrconf_verify_work+0x12/0x30 [ 492.866344][ T44] process_one_work+0xd57/0x1390 [ 492.866473][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 492.866587][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.866703][ T44] ? assign_work+0x152/0x380 [ 492.866819][ T44] worker_thread+0x4d6/0xd40 [ 492.866937][ T44] ? process_one_work+0x1390/0x1390 [ 492.867053][ T44] kthread+0x355/0x5b0 [ 492.867143][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.867260][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.867379][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.867492][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.867606][ T44] ret_from_fork+0x3fb/0x510 [ 492.867721][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 492.867867][ T44] ? __switch_to+0x53c/0xd00 [ 492.867981][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.868096][ T44] ret_from_fork_asm+0x11/0x20 [ 492.868217][ T44] [ 492.868307][ T44] INFO: task kworker/u16:0:12 is blocked on a mutex likely owned by task ip:334. [ 492.868501][ T44] INFO: task kworker/2:0:28 blocked for more than 245 seconds. [ 492.868660][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 492.868768][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.868953][ T44] task:kworker/2:0 state:D stack:27344 pid:28 tgid:28 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 492.869202][ T44] Workqueue: events xp_release_deferred [ 492.869313][ T44] Call Trace: [ 492.869404][ T44] [ 492.869465][ T44] __schedule+0xae2/0x1720 [ 492.869582][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.869697][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.869813][ T44] schedule+0xd4/0x260 [ 492.869900][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.870013][ T44] __mutex_lock+0xd76/0x1f20 [ 492.870128][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 492.870246][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.870363][ T44] ? __lock_acquire+0x577/0xc10 [ 492.870479][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.870592][ T44] ? process_one_work+0xd16/0x1390 [ 492.870708][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 492.870822][ T44] xp_release_deferred+0x1b/0x1e0 [ 492.870939][ T44] process_one_work+0xd57/0x1390 [ 492.871057][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 492.871175][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.871291][ T44] ? assign_work+0x152/0x380 [ 492.871411][ T44] worker_thread+0x4d6/0xd40 [ 492.871529][ T44] ? process_one_work+0x1390/0x1390 [ 492.871641][ T44] kthread+0x355/0x5b0 [ 492.871727][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.871842][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.871957][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.872070][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.872185][ T44] ret_from_fork+0x3fb/0x510 [ 492.872299][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 492.872444][ T44] ? __switch_to+0x53c/0xd00 [ 492.872556][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.872671][ T44] ret_from_fork_asm+0x11/0x20 [ 492.872791][ T44] [ 492.872878][ T44] INFO: task kworker/2:0:28 is blocked on a mutex likely owned by task ip:334. [ 492.873069][ T44] INFO: task ip:334 blocked for more than 368 seconds. [ 492.873206][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 492.873309][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.873493][ T44] task:ip state:D stack:24544 pid:334 tgid:334 ppid:1 task_flags:0x400100 flags:0x00080002 [ 492.873738][ T44] Call Trace: [ 492.873823][ T44] [ 492.873885][ T44] __schedule+0xae2/0x1720 [ 492.873999][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.874117][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.874229][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.874347][ T44] schedule+0xd4/0x260 [ 492.874444][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.874562][ T44] __mutex_lock+0xd76/0x1f20 [ 492.874676][ T44] ? __folio_put+0x1f5/0x320 [ 492.874790][ T44] ? napi_enable+0x3b/0x80 [ 492.874905][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.875020][ T44] ? dma_map_phys+0x1b1/0x220 [ 492.875138][ T44] ? __asan_memset+0x27/0x50 [ 492.875256][ T44] ? sg_init_table+0x19/0x60 [ 492.875377][ T44] ? try_fill_recv+0x3ec/0x1750 [ 492.875492][ T44] ? napi_enable+0x3b/0x80 [ 492.875606][ T44] napi_enable+0x3b/0x80 [ 492.875692][ T44] __virtnet_rx_resume+0x156/0x290 [ 492.875805][ T44] ? queue_delayed_work_on+0x91/0x100 [ 492.875920][ T44] virtnet_rx_resume_all+0xe7/0x1a0 [ 492.876036][ T44] virtnet_xdp_set+0xef7/0x1aa0 [ 492.876160][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0 [ 492.876274][ T44] dev_xdp_install+0x3e4/0x890 [ 492.876392][ T44] ? netif_set_xps_queue+0x40/0x40 [ 492.876511][ T44] dev_xdp_attach+0x50d/0x1310 [ 492.876626][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0 [ 492.876741][ T44] ? __bpf_prog_get+0x320/0x320 [ 492.876858][ T44] dev_change_xdp_fd+0x176/0x210 [ 492.876973][ T44] do_setlink.isra.0+0x1c2d/0x2880 [ 492.877096][ T44] ? rtnl_newlink_create+0x900/0x900 [ 492.877210][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 492.877356][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 492.877470][ T44] ? validate_chain+0x37b/0xb70 [ 492.877585][ T44] ? rcu_read_lock_any_held+0x66/0x90 [ 492.877699][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 492.877817][ T44] ? __lock_acquire+0x577/0xc10 [ 492.877932][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.878047][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 492.878164][ T44] ? lock_acquire+0x10a/0x150 [ 492.878277][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.878396][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.878510][ T44] ? trace_contention_end+0xd6/0x110 [ 492.878627][ T44] ? __mutex_lock+0x285/0x1f20 [ 492.878742][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 492.878856][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.878968][ T44] ? nla_get_range_signed+0x3d0/0x3d0 [ 492.879086][ T44] ? __rtnl_newlink+0x3fa/0xa40 [ 492.879207][ T44] rtnl_newlink+0x75c/0xe90 [ 492.879323][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.879442][ T44] ? rtnl_setlink+0x880/0x880 [ 492.879560][ T44] ? __lock_acquire+0x577/0xc10 [ 492.879675][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.879787][ T44] ? find_held_lock+0x2b/0x80 [ 492.879902][ T44] ? rtnl_setlink+0x880/0x880 [ 492.880014][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.880130][ T44] ? rtnl_setlink+0x880/0x880 [ 492.880243][ T44] rtnetlink_rcv_msg+0x6fe/0xb90 [ 492.880364][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.880479][ T44] ? __lock_acquire+0x577/0xc10 [ 492.880594][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.880706][ T44] ? find_held_lock+0x2b/0x80 [ 492.880822][ T44] netlink_rcv_skb+0x123/0x380 [ 492.880935][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.881048][ T44] ? netlink_ack+0xcc0/0xcc0 [ 492.881168][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 492.881283][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 492.881403][ T44] netlink_unicast+0x4a3/0x770 [ 492.881519][ T44] ? netlink_attachskb+0x810/0x810 [ 492.881632][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.881745][ T44] ? __lock_acquire+0x577/0xc10 [ 492.881860][ T44] netlink_sendmsg+0x735/0xc60 [ 492.881977][ T44] ? netlink_unicast+0x770/0x770 [ 492.882100][ T44] ____sys_sendmsg+0x419/0x850 [ 492.882213][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 492.882327][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 492.882472][ T44] ? move_addr_to_kernel+0x40/0x40 [ 492.882586][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 492.882698][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 492.882814][ T44] ___sys_sendmsg+0xfd/0x180 [ 492.882929][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00 [ 492.883069][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 492.883200][ T44] __sys_sendmsg+0x124/0x1c0 [ 492.883315][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 492.883433][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 492.883546][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00 [ 492.883690][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 492.883806][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.883920][ T44] do_syscall_64+0xbd/0xfc0 [ 492.884035][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 492.884178][ T44] RIP: 0033:0x7fe5ac980c5e [ 492.884297][ T44] RSP: 002b:00007ffd4cfeaaf0 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 492.884471][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007fe5ac980c5e [ 492.884642][ T44] RDX: 0000000000000000 RSI: 00007ffd4cfeabb0 RDI: 0000000000000003 [ 492.884811][ T44] RBP: 00007ffd4cfeab00 R08: 0000000000000000 R09: 0000000000000000 [ 492.884978][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffd4cfeb270 [ 492.885149][ T44] R13: 00000000695b7092 R14: 0000000000000000 R15: 0000000000499600 [ 492.885326][ T44] [ 492.885415][ T44] INFO: task ip:334 is blocked on a mutex likely owned by task kworker/0:0:9. [ 492.885602][ T44] task:kworker/0:0 state:D stack:27144 pid:9 tgid:9 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 492.885846][ T44] Workqueue: events refill_work [ 492.885958][ T44] Call Trace: [ 492.886042][ T44] [ 492.886104][ T44] __schedule+0xae2/0x1720 [ 492.886223][ T44] ? usleep_range_state+0xba/0x1b0 [ 492.886336][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.886455][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.886571][ T44] ? usleep_range_state+0xba/0x1b0 [ 492.886685][ T44] schedule+0xd4/0x260 [ 492.886771][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0 [ 492.886914][ T44] ? schedule_timeout_idle+0xe0/0xe0 [ 492.887029][ T44] ? __hrtimer_cb_get_time+0x50/0x50 [ 492.887145][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 492.887260][ T44] ? ktime_get+0x15e/0x220 [ 492.887385][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 492.887498][ T44] ? ktime_get+0xa9/0x220 [ 492.887586][ T44] usleep_range_state+0x133/0x1b0 [ 492.887704][ T44] ? schedule_hrtimeout+0x20/0x20 [ 492.887818][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 492.887932][ T44] ? refill_work+0x149/0x210 [ 492.888048][ T44] napi_disable_locked+0x379/0x5f0 [ 492.888164][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 492.888277][ T44] ? register_netdevice_notifier_net+0x40/0x40 [ 492.888425][ T44] napi_disable+0x43/0x80 [ 492.888512][ T44] refill_work+0x116/0x210 [ 492.888630][ T44] process_one_work+0xd57/0x1390 [ 492.888749][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 492.888862][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.888976][ T44] ? assign_work+0x152/0x380 [ 492.889092][ T44] worker_thread+0x4d6/0xd40 [ 492.889209][ T44] ? process_one_work+0x1390/0x1390 [ 492.889383][ T44] kthread+0x355/0x5b0 [ 492.889473][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.889585][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.889697][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.889865][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.889979][ T44] ret_from_fork+0x3fb/0x510 [ 492.890095][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 492.890237][ T44] ? __switch_to+0x53c/0xd00 [ 492.890413][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.890528][ T44] ret_from_fork_asm+0x11/0x20 [ 492.890647][ T44] [ 492.890732][ T44] INFO: task xdp_helper:342 blocked for more than 245 seconds. [ 492.890938][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 492.891044][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.891229][ T44] task:xdp_helper state:D stack:26224 pid:342 tgid:342 ppid:1 task_flags:0x400100 flags:0x00080000 [ 492.891533][ T44] Call Trace: [ 492.891618][ T44] [ 492.891676][ T44] __schedule+0xae2/0x1720 [ 492.891791][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.891961][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.892075][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.892196][ T44] schedule+0xd4/0x260 [ 492.892282][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.892457][ T44] __mutex_lock+0xd76/0x1f20 [ 492.892573][ T44] ? find_held_lock+0x2b/0x80 [ 492.892688][ T44] ? xsk_bind+0x17f/0x1b10 [ 492.892806][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.892975][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 492.893119][ T44] ? validate_chain+0x37b/0xb70 [ 492.893237][ T44] ? __lock_acquire+0x577/0xc10 [ 492.893357][ T44] ? xsk_bind+0x17f/0x1b10 [ 492.893470][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.893584][ T44] xsk_bind+0x17f/0x1b10 [ 492.893670][ T44] ? __might_fault+0x97/0x140 [ 492.893784][ T44] ? find_held_lock+0x2b/0x80 [ 492.893899][ T44] ? __xsk_rcv_zc+0x910/0x910 [ 492.894012][ T44] ? __might_fault+0x97/0x140 [ 492.894130][ T44] __sys_bind+0x174/0x230 [ 492.894216][ T44] ? __sys_bind_socket+0xb0/0xb0 [ 492.894394][ T44] ? __sys_setsockopt+0xe4/0x150 [ 492.894513][ T44] __x64_sys_bind+0x72/0xd0 [ 492.894627][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 492.894739][ T44] ? do_syscall_64+0x88/0xfc0 [ 492.894909][ T44] do_syscall_64+0xbd/0xfc0 [ 492.895024][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 492.895170][ T44] RIP: 0033:0x7f87aff3febb [ 492.895293][ T44] RSP: 002b:00007ffce6773748 EFLAGS: 00000206 ORIG_RAX: 0000000000000031 [ 492.895525][ T44] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f87aff3febb [ 492.895694][ T44] RDX: 0000000000000010 RSI: 00007ffce6773760 RDI: 0000000000000003 [ 492.895921][ T44] RBP: 0000000000000000 R08: 0000000000000004 R09: 0000000000000000 [ 492.896099][ T44] R10: 0000000000000000 R11: 0000000000000206 R12: 00007ffce67738d8 [ 492.896268][ T44] R13: 00007f87afe36000 R14: 0000000000000004 R15: 0000000000402e00 [ 492.896447][ T44] [ 492.896536][ T44] INFO: task xdp_helper:342 is blocked on a mutex likely owned by task ip:334. [ 492.896725][ T44] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings [ 492.896962][ T44] [ 492.896962][ T44] Showing all locks held in the system: [ 492.897132][ T44] 3 locks held by kworker/0:0/9: [ 492.897242][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 492.897616][ T44] #1: ffa0000000097cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 492.897865][ T44] #2: ff110000091b2c98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 492.898060][ T44] 3 locks held by kworker/u16:0/12: [ 492.898171][ T44] #0: ff1100000983f148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 492.898393][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 492.898669][ T44] #2: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 492.898866][ T44] 3 locks held by kworker/2:0/28: [ 492.898974][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 492.899197][ T44] #1: ffa00000001ffcc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 492.899476][ T44] #2: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0 [ 492.899670][ T44] 1 lock held by khungtaskd/44: [ 492.899832][ T44] #0: ffffffffb73710c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 492.900054][ T44] 2 locks held by ip/334: [ 492.900141][ T44] #0: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90 [ 492.900389][ T44] #1: ff110000091b2c98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80 [ 492.900581][ T44] 1 lock held by xdp_helper/342: [ 492.900690][ T44] #0: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: xsk_bind+0x17f/0x1b10 [ 492.900936][ T44] 1 lock held by ip/401: [ 492.901021][ T44] #0: ffffffffb7a859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2ab/0xb90 [ 492.901270][ T44] [ 492.901325][ T44] ============================================= [ 492.901325][ T44]