[ 369.982707][ T44] INFO: task ip:972 blocked for more than 122 seconds. [ 369.983177][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 369.983385][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.983708][ T44] task:ip state:D stack:24976 pid:972 tgid:972 ppid:1 task_flags:0x400100 flags:0x00080002 [ 369.984071][ T44] Call Trace: [ 369.984229][ T44] [ 369.984360][ T44] __schedule+0xae2/0x1720 [ 369.984559][ T44] ? io_schedule_timeout+0x130/0x130 [ 369.984772][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.984956][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.985141][ T44] schedule+0xd4/0x260 [ 369.985301][ T44] schedule_preempt_disabled+0x18/0x30 [ 369.985479][ T44] __mutex_lock+0xd76/0x1f20 [ 369.985678][ T44] ? napi_enable+0x3b/0x80 [ 369.985866][ T44] ? ww_mutex_lock+0x160/0x160 [ 369.986043][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.986209][ T44] ? __asan_memset+0x27/0x50 [ 369.986383][ T44] ? sg_init_table+0x19/0x60 [ 369.986550][ T44] ? try_fill_recv+0x3ec/0x1750 [ 369.986734][ T44] ? napi_enable+0x3b/0x80 [ 369.986901][ T44] napi_enable+0x3b/0x80 [ 369.987037][ T44] __virtnet_rx_resume+0x156/0x290 [ 369.987199][ T44] ? queue_delayed_work_on+0x91/0x100 [ 369.987374][ T44] virtnet_rx_resume_all+0xe7/0x1a0 [ 369.987542][ T44] virtnet_xdp_set+0xef7/0x1aa0 [ 369.987721][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0 [ 369.987887][ T44] dev_xdp_install+0x3e4/0x890 [ 369.988053][ T44] ? netif_set_xps_queue+0x40/0x40 [ 369.988221][ T44] dev_xdp_attach+0x50d/0x1310 [ 369.988409][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0 [ 369.988860][ T44] ? __bpf_prog_get+0x320/0x320 [ 369.989034][ T44] dev_change_xdp_fd+0x176/0x210 [ 369.989202][ T44] do_setlink.isra.0+0x1c2d/0x2880 [ 369.989380][ T44] ? rtnl_newlink_create+0x900/0x900 [ 369.989547][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 369.989756][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 369.989919][ T44] ? validate_chain+0x37b/0xb70 [ 369.990086][ T44] ? rcu_read_lock_any_held+0x66/0x90 [ 369.990252][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 369.990421][ T44] ? __lock_acquire+0x577/0xc10 [ 369.990588][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 369.990768][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 369.990934][ T44] ? lock_acquire+0x10a/0x150 [ 369.991097][ T44] ? rcu_is_watching+0x15/0xd0 [ 369.991259][ T44] ? rcu_is_watching+0x15/0xd0 [ 369.991427][ T44] ? trace_contention_end+0xd6/0x110 [ 369.991592][ T44] ? __mutex_lock+0x285/0x1f20 [ 369.991771][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 369.991937][ T44] ? ww_mutex_lock+0x160/0x160 [ 369.992101][ T44] ? nla_get_range_signed+0x3d0/0x3d0 [ 369.992273][ T44] ? __rtnl_newlink+0x3fa/0xa40 [ 369.992446][ T44] rtnl_newlink+0x75c/0xe90 [ 369.992614][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.992793][ T44] ? rtnl_setlink+0x880/0x880 [ 369.992959][ T44] ? __lock_acquire+0x577/0xc10 [ 369.993125][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 369.993297][ T44] ? find_held_lock+0x2b/0x80 [ 369.993463][ T44] ? rtnl_setlink+0x880/0x880 [ 369.993629][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.993805][ T44] ? rtnl_setlink+0x880/0x880 [ 369.993970][ T44] rtnetlink_rcv_msg+0x6fe/0xb90 [ 369.994137][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 369.994311][ T44] ? __lock_acquire+0x577/0xc10 [ 369.994476][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 369.994638][ T44] ? find_held_lock+0x2b/0x80 [ 369.994814][ T44] netlink_rcv_skb+0x123/0x380 [ 369.994979][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 369.995144][ T44] ? netlink_ack+0xcc0/0xcc0 [ 369.995318][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 369.995483][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 369.995649][ T44] netlink_unicast+0x4a3/0x770 [ 369.995829][ T44] ? netlink_attachskb+0x810/0x810 [ 369.995992][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 369.996159][ T44] ? __lock_acquire+0x577/0xc10 [ 369.996331][ T44] netlink_sendmsg+0x735/0xc60 [ 369.996496][ T44] ? netlink_unicast+0x770/0x770 [ 369.996677][ T44] ____sys_sendmsg+0x419/0x850 [ 369.996845][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 369.997010][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 369.997201][ T44] ? move_addr_to_kernel+0x40/0x40 [ 369.997373][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 369.997537][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 369.997715][ T44] ___sys_sendmsg+0xfd/0x180 [ 369.997882][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00 [ 369.998074][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 369.998252][ T44] __sys_sendmsg+0x124/0x1c0 [ 369.998420][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 369.998582][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 369.998760][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00 [ 369.998957][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 369.999125][ T44] ? rcu_is_watching+0x15/0xd0 [ 369.999295][ T44] do_syscall_64+0xbd/0xfc0 [ 369.999461][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 369.999654][ T44] RIP: 0033:0x7f0121de1c5e [ 369.999839][ T44] RSP: 002b:00007fffa41b4700 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 370.000061][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f0121de1c5e [ 370.000285][ T44] RDX: 0000000000000000 RSI: 00007fffa41b47c0 RDI: 0000000000000003 [ 370.000511][ T44] RBP: 00007fffa41b4710 R08: 0000000000000000 R09: 0000000000000000 [ 370.000740][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fffa41b4e80 [ 370.000959][ T44] R13: 0000000069569b71 R14: 0000000000000000 R15: 0000000000499600 [ 370.001189][ T44] [ 370.001330][ T44] INFO: task ip:972 is blocked on a mutex likely owned by task kworker/0:0:9. [ 370.001570][ T44] task:kworker/0:0 state:D stack:27416 pid:9 tgid:9 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 370.001883][ T44] Workqueue: events refill_work [ 370.002048][ T44] Call Trace: [ 370.002183][ T44] [ 370.002298][ T44] __schedule+0xae2/0x1720 [ 370.002467][ T44] ? usleep_range_state+0xba/0x1b0 [ 370.002632][ T44] ? io_schedule_timeout+0x130/0x130 [ 370.002809][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.002976][ T44] ? usleep_range_state+0xba/0x1b0 [ 370.003138][ T44] schedule+0xd4/0x260 [ 370.003274][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0 [ 370.003472][ T44] ? schedule_timeout_idle+0xe0/0xe0 [ 370.003639][ T44] ? __hrtimer_cb_get_time+0x50/0x50 [ 370.003819][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 370.003982][ T44] ? ktime_get+0x15e/0x220 [ 370.004144][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 370.004314][ T44] ? ktime_get+0xa9/0x220 [ 370.004450][ T44] usleep_range_state+0x133/0x1b0 [ 370.004614][ T44] ? schedule_hrtimeout+0x20/0x20 [ 370.004795][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 370.004961][ T44] ? refill_work+0x149/0x210 [ 370.005127][ T44] napi_disable_locked+0x379/0x5f0 [ 370.005303][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 370.005470][ T44] ? register_netdevice_notifier_net+0x40/0x40 [ 370.005680][ T44] napi_disable+0x43/0x80 [ 370.005815][ T44] refill_work+0x116/0x210 [ 370.005984][ T44] process_one_work+0xd57/0x1390 [ 370.006151][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 370.006319][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.006487][ T44] ? assign_work+0x152/0x380 [ 370.006650][ T44] worker_thread+0x4d6/0xd40 [ 370.006831][ T44] ? process_one_work+0x1390/0x1390 [ 370.006993][ T44] kthread+0x355/0x5b0 [ 370.007132][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 370.007300][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.007463][ T44] ? rcu_is_watching+0x15/0xd0 [ 370.007627][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 370.007801][ T44] ret_from_fork+0x3fb/0x510 [ 370.007966][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 370.008158][ T44] ? __switch_to+0x53c/0xd00 [ 370.008329][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 370.008492][ T44] ret_from_fork_asm+0x11/0x20 [ 370.008677][ T44] [ 370.008819][ T44] [ 370.008819][ T44] Showing all locks held in the system: [ 370.009037][ T44] 3 locks held by kworker/0:0/9: [ 370.009196][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 370.009478][ T44] #1: ffa0000000097cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 370.009790][ T44] #2: ff11000008ad2c98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 370.010033][ T44] 3 locks held by kworker/u16:0/12: [ 370.010190][ T44] #0: ff110000090bf148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 370.010462][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 370.010874][ T44] #2: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 370.011123][ T44] 3 locks held by kworker/2:0/28: [ 370.011281][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 370.011563][ T44] #1: ffa00000001ffcc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 370.011844][ T44] #2: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0 [ 370.012094][ T44] 1 lock held by khungtaskd/44: [ 370.012252][ T44] #0: ffffffff98b710c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 370.012538][ T44] 2 locks held by ip/972: [ 370.012683][ T44] #0: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90 [ 370.012927][ T44] #1: ff11000008ad2c98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80 [ 370.013174][ T44] 1 lock held by ip/1040: [ 370.013308][ T44] #0: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2ab/0xb90 [ 370.013558][ T44] [ 370.013677][ T44] ============================================= [ 370.013677][ T44] [ 492.862688][ T44] INFO: task kworker/u16:0:12 blocked for more than 122 seconds. [ 492.863069][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 492.863275][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.863570][ T44] task:kworker/u16:0 state:D stack:27696 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 492.863981][ T44] Workqueue: ipv6_addrconf addrconf_verify_work [ 492.864222][ T44] Call Trace: [ 492.864373][ T44] [ 492.864496][ T44] __schedule+0xae2/0x1720 [ 492.864714][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.864903][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.865094][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.865278][ T44] schedule+0xd4/0x260 [ 492.865425][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.865608][ T44] __mutex_lock+0xd76/0x1f20 [ 492.865806][ T44] ? addrconf_verify_work+0x12/0x30 [ 492.865993][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.866180][ T44] ? process_one_work+0xd16/0x1390 [ 492.866361][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.866525][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.866705][ T44] ? addrconf_verify_work+0x12/0x30 [ 492.866868][ T44] addrconf_verify_work+0x12/0x30 [ 492.867036][ T44] process_one_work+0xd57/0x1390 [ 492.867212][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 492.867375][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.867540][ T44] ? assign_work+0x152/0x380 [ 492.867725][ T44] worker_thread+0x4d6/0xd40 [ 492.867894][ T44] ? process_one_work+0x1390/0x1390 [ 492.868057][ T44] kthread+0x355/0x5b0 [ 492.868194][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.868365][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.868530][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.868709][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.868880][ T44] ret_from_fork+0x3fb/0x510 [ 492.869046][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 492.869243][ T44] ? __switch_to+0x53c/0xd00 [ 492.869414][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.869582][ T44] ret_from_fork_asm+0x11/0x20 [ 492.869770][ T44] [ 492.869921][ T44] INFO: task kworker/u16:0:12 is blocked on a mutex likely owned by task ip:972. [ 492.870168][ T44] INFO: task kworker/2:0:28 blocked for more than 122 seconds. [ 492.870377][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 492.870532][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.870780][ T44] task:kworker/2:0 state:D stack:27096 pid:28 tgid:28 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 492.871078][ T44] Workqueue: events xp_release_deferred [ 492.871254][ T44] Call Trace: [ 492.871387][ T44] [ 492.871495][ T44] __schedule+0xae2/0x1720 [ 492.871662][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.871842][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.872007][ T44] schedule+0xd4/0x260 [ 492.872143][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.872313][ T44] __mutex_lock+0xd76/0x1f20 [ 492.872484][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 492.872649][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.872829][ T44] ? __lock_acquire+0x577/0xc10 [ 492.872996][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.873159][ T44] ? process_one_work+0xd16/0x1390 [ 492.873332][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 492.873493][ T44] xp_release_deferred+0x1b/0x1e0 [ 492.873663][ T44] process_one_work+0xd57/0x1390 [ 492.873845][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 492.874008][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.874175][ T44] ? assign_work+0x152/0x380 [ 492.874346][ T44] worker_thread+0x4d6/0xd40 [ 492.874516][ T44] ? process_one_work+0x1390/0x1390 [ 492.874694][ T44] kthread+0x355/0x5b0 [ 492.874835][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.874999][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.875165][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.875332][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.875496][ T44] ret_from_fork+0x3fb/0x510 [ 492.875661][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 492.875869][ T44] ? __switch_to+0x53c/0xd00 [ 492.876037][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.876208][ T44] ret_from_fork_asm+0x11/0x20 [ 492.876378][ T44] [ 492.876515][ T44] INFO: task kworker/2:0:28 is blocked on a mutex likely owned by task ip:972. [ 492.876776][ T44] INFO: task ip:972 blocked for more than 245 seconds. [ 492.876958][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 492.877113][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.877355][ T44] task:ip state:D stack:24976 pid:972 tgid:972 ppid:1 task_flags:0x400100 flags:0x00080002 [ 492.877650][ T44] Call Trace: [ 492.877797][ T44] [ 492.877904][ T44] __schedule+0xae2/0x1720 [ 492.878071][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.878241][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.878408][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.878574][ T44] schedule+0xd4/0x260 [ 492.878723][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.878885][ T44] __mutex_lock+0xd76/0x1f20 [ 492.879051][ T44] ? napi_enable+0x3b/0x80 [ 492.879222][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.879387][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.879555][ T44] ? __asan_memset+0x27/0x50 [ 492.879736][ T44] ? sg_init_table+0x19/0x60 [ 492.879901][ T44] ? try_fill_recv+0x3ec/0x1750 [ 492.880070][ T44] ? napi_enable+0x3b/0x80 [ 492.880239][ T44] napi_enable+0x3b/0x80 [ 492.880377][ T44] __virtnet_rx_resume+0x156/0x290 [ 492.880541][ T44] ? queue_delayed_work_on+0x91/0x100 [ 492.880717][ T44] virtnet_rx_resume_all+0xe7/0x1a0 [ 492.880888][ T44] virtnet_xdp_set+0xef7/0x1aa0 [ 492.881057][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0 [ 492.881226][ T44] dev_xdp_install+0x3e4/0x890 [ 492.881392][ T44] ? netif_set_xps_queue+0x40/0x40 [ 492.881562][ T44] dev_xdp_attach+0x50d/0x1310 [ 492.881744][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0 [ 492.881908][ T44] ? __bpf_prog_get+0x320/0x320 [ 492.882082][ T44] dev_change_xdp_fd+0x176/0x210 [ 492.882254][ T44] do_setlink.isra.0+0x1c2d/0x2880 [ 492.882423][ T44] ? rtnl_newlink_create+0x900/0x900 [ 492.882586][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 492.882794][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 492.882959][ T44] ? validate_chain+0x37b/0xb70 [ 492.883124][ T44] ? rcu_read_lock_any_held+0x66/0x90 [ 492.883298][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 492.883462][ T44] ? __lock_acquire+0x577/0xc10 [ 492.883627][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.883807][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 492.883970][ T44] ? lock_acquire+0x10a/0x150 [ 492.884133][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.884299][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.884467][ T44] ? trace_contention_end+0xd6/0x110 [ 492.884633][ T44] ? __mutex_lock+0x285/0x1f20 [ 492.884808][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 492.884974][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.885139][ T44] ? nla_get_range_signed+0x3d0/0x3d0 [ 492.885313][ T44] ? __rtnl_newlink+0x3fa/0xa40 [ 492.885479][ T44] rtnl_newlink+0x75c/0xe90 [ 492.885646][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.885827][ T44] ? rtnl_setlink+0x880/0x880 [ 492.885996][ T44] ? __lock_acquire+0x577/0xc10 [ 492.886163][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.886331][ T44] ? find_held_lock+0x2b/0x80 [ 492.886496][ T44] ? rtnl_setlink+0x880/0x880 [ 492.886680][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.886845][ T44] ? rtnl_setlink+0x880/0x880 [ 492.887014][ T44] rtnetlink_rcv_msg+0x6fe/0xb90 [ 492.887184][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.887356][ T44] ? __lock_acquire+0x577/0xc10 [ 492.887526][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.887720][ T44] ? find_held_lock+0x2b/0x80 [ 492.887886][ T44] netlink_rcv_skb+0x123/0x380 [ 492.888054][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.888225][ T44] ? netlink_ack+0xcc0/0xcc0 [ 492.888398][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 492.888562][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 492.888742][ T44] netlink_unicast+0x4a3/0x770 [ 492.888907][ T44] ? netlink_attachskb+0x810/0x810 [ 492.889072][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.889242][ T44] ? __lock_acquire+0x577/0xc10 [ 492.889408][ T44] netlink_sendmsg+0x735/0xc60 [ 492.889578][ T44] ? netlink_unicast+0x770/0x770 [ 492.889759][ T44] ____sys_sendmsg+0x419/0x850 [ 492.889927][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 492.890092][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 492.890291][ T44] ? move_addr_to_kernel+0x40/0x40 [ 492.890458][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 492.890621][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 492.890815][ T44] ___sys_sendmsg+0xfd/0x180 [ 492.890980][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00 [ 492.891174][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 492.891357][ T44] __sys_sendmsg+0x124/0x1c0 [ 492.891522][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 492.891700][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 492.891863][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00 [ 492.892064][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 492.892237][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.892404][ T44] do_syscall_64+0xbd/0xfc0 [ 492.892571][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 492.892777][ T44] RIP: 0033:0x7f0121de1c5e [ 492.892947][ T44] RSP: 002b:00007fffa41b4700 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 492.893176][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f0121de1c5e [ 492.893401][ T44] RDX: 0000000000000000 RSI: 00007fffa41b47c0 RDI: 0000000000000003 [ 492.893622][ T44] RBP: 00007fffa41b4710 R08: 0000000000000000 R09: 0000000000000000 [ 492.893856][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fffa41b4e80 [ 492.894075][ T44] R13: 0000000069569b71 R14: 0000000000000000 R15: 0000000000499600 [ 492.894307][ T44] [ 492.894445][ T44] INFO: task ip:972 is blocked on a mutex likely owned by task kworker/0:0:9. [ 492.894696][ T44] task:kworker/0:0 state:D stack:27416 pid:9 tgid:9 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 492.894998][ T44] Workqueue: events refill_work [ 492.895164][ T44] Call Trace: [ 492.895307][ T44] [ 492.895414][ T44] __schedule+0xae2/0x1720 [ 492.895582][ T44] ? usleep_range_state+0xba/0x1b0 [ 492.895761][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.895929][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.896095][ T44] ? usleep_range_state+0xba/0x1b0 [ 492.896265][ T44] schedule+0xd4/0x260 [ 492.896406][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0 [ 492.896600][ T44] ? schedule_timeout_idle+0xe0/0xe0 [ 492.896778][ T44] ? __hrtimer_cb_get_time+0x50/0x50 [ 492.896945][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 492.897109][ T44] ? ktime_get+0x15e/0x220 [ 492.897279][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 492.897444][ T44] ? ktime_get+0xa9/0x220 [ 492.897585][ T44] usleep_range_state+0x133/0x1b0 [ 492.897766][ T44] ? schedule_hrtimeout+0x20/0x20 [ 492.897931][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 492.898095][ T44] ? refill_work+0x149/0x210 [ 492.898268][ T44] napi_disable_locked+0x379/0x5f0 [ 492.898431][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 492.898595][ T44] ? register_netdevice_notifier_net+0x40/0x40 [ 492.898808][ T44] napi_disable+0x43/0x80 [ 492.898946][ T44] refill_work+0x116/0x210 [ 492.899114][ T44] process_one_work+0xd57/0x1390 [ 492.899290][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 492.899453][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.899622][ T44] ? assign_work+0x152/0x380 [ 492.899801][ T44] worker_thread+0x4d6/0xd40 [ 492.899971][ T44] ? process_one_work+0x1390/0x1390 [ 492.900140][ T44] kthread+0x355/0x5b0 [ 492.900284][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.900448][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.900612][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.900794][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.900962][ T44] ret_from_fork+0x3fb/0x510 [ 492.901128][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 492.901408][ T44] ? __switch_to+0x53c/0xd00 [ 492.901575][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.901753][ T44] ret_from_fork_asm+0x11/0x20 [ 492.901926][ T44] [ 492.902066][ T44] INFO: task ip:1040 blocked for more than 122 seconds. [ 492.902255][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 492.902411][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.902712][ T44] task:ip state:D stack:25624 pid:1040 tgid:1040 ppid:1039 task_flags:0x400100 flags:0x00080000 [ 492.903010][ T44] Call Trace: [ 492.903145][ T44] [ 492.903316][ T44] __schedule+0xae2/0x1720 [ 492.903484][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.903650][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.903833][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.904056][ T44] schedule+0xd4/0x260 [ 492.904194][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.904365][ T44] __mutex_lock+0xd76/0x1f20 [ 492.904530][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.904766][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 492.904930][ T44] ? write_profile+0xf0/0xf0 [ 492.905101][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.905273][ T44] ? find_held_lock+0x2b/0x80 [ 492.905495][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.905660][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 492.905838][ T44] rtnetlink_rcv_msg+0x2ab/0xb90 [ 492.906003][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.906225][ T44] ? __lock_acquire+0x577/0xc10 [ 492.906392][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.906560][ T44] ? find_held_lock+0x2b/0x80 [ 492.906739][ T44] netlink_rcv_skb+0x123/0x380 [ 492.906961][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.907124][ T44] ? netlink_ack+0xcc0/0xcc0 [ 492.907299][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 492.907462][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 492.907698][ T44] netlink_unicast+0x4a3/0x770 [ 492.907867][ T44] ? netlink_attachskb+0x810/0x810 [ 492.908034][ T44] ? __lock_acquire+0x577/0xc10 [ 492.908210][ T44] netlink_sendmsg+0x735/0xc60 [ 492.908434][ T44] ? netlink_unicast+0x770/0x770 [ 492.908601][ T44] ____sys_sendmsg+0x419/0x850 [ 492.908778][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 492.908941][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 492.909193][ T44] ? move_addr_to_kernel+0x40/0x40 [ 492.909366][ T44] ? __lock_acquire+0x577/0xc10 [ 492.909532][ T44] ___sys_sendmsg+0xfd/0x180 [ 492.909710][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 492.909935][ T44] ? do_pte_missing+0x545/0x8f0 [ 492.910103][ T44] ? lock_vma_under_rcu+0x12a/0x3e0 [ 492.910273][ T44] __sys_sendmsg+0x124/0x1c0 [ 492.910438][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 492.910684][ T44] ? do_user_addr_fault+0x799/0xdc0 [ 492.910848][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.911011][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.911177][ T44] do_syscall_64+0xbd/0xfc0 [ 492.911349][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 492.911541][ T44] RIP: 0033:0x7f3f6f6f6c5e [ 492.911721][ T44] RSP: 002b:00007ffd7d5bbd30 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 492.912006][ T44] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f3f6f6f6c5e [ 492.912231][ T44] RDX: 0000000000000000 RSI: 00007ffd7d5bbdf0 RDI: 0000000000000004 [ 492.912452][ T44] RBP: 00007ffd7d5bbd40 R08: 0000000000000000 R09: 0000000000000000 [ 492.912744][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffd7d5be68b [ 492.912969][ T44] R13: 0000000069569c40 R14: 000000000000000d R15: 00007ffd7d5bbea0 [ 492.913196][ T44] [ 492.913398][ T44] INFO: task ip:1040 is blocked on a mutex likely owned by task ip:972. [ 492.913610][ T44] [ 492.913610][ T44] Showing all locks held in the system: [ 492.913840][ T44] 3 locks held by kworker/0:0/9: [ 492.914054][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 492.914339][ T44] #1: ffa0000000097cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 492.914705][ T44] #2: ff11000008ad2c98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 492.914949][ T44] 3 locks held by kworker/u16:0/12: [ 492.915107][ T44] #0: ff110000090bf148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 492.915437][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 492.915835][ T44] #2: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 492.916082][ T44] 3 locks held by kworker/2:0/28: [ 492.916252][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 492.916578][ T44] #1: ffa00000001ffcc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 492.916865][ T44] #2: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0 [ 492.917164][ T44] 1 lock held by khungtaskd/44: [ 492.917327][ T44] #0: ffffffff98b710c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 492.917604][ T44] 2 locks held by ip/972: [ 492.917795][ T44] #0: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90 [ 492.918039][ T44] #1: ff11000008ad2c98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80 [ 492.918293][ T44] 1 lock held by ip/1040: [ 492.918481][ T44] #0: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2ab/0xb90 [ 492.918737][ T44] [ 492.918840][ T44] ============================================= [ 492.918840][ T44] [ 615.742688][ T44] INFO: task kworker/u16:0:12 blocked for more than 245 seconds. [ 615.743073][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 615.743282][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.743569][ T44] task:kworker/u16:0 state:D stack:27696 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 615.743964][ T44] Workqueue: ipv6_addrconf addrconf_verify_work [ 615.744208][ T44] Call Trace: [ 615.744368][ T44] [ 615.744482][ T44] __schedule+0xae2/0x1720 [ 615.744695][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.744882][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.745067][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.745263][ T44] schedule+0xd4/0x260 [ 615.745415][ T44] schedule_preempt_disabled+0x18/0x30 [ 615.745580][ T44] __mutex_lock+0xd76/0x1f20 [ 615.745764][ T44] ? addrconf_verify_work+0x12/0x30 [ 615.745937][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.746110][ T44] ? process_one_work+0xd16/0x1390 [ 615.746285][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.746450][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.746615][ T44] ? addrconf_verify_work+0x12/0x30 [ 615.746793][ T44] addrconf_verify_work+0x12/0x30 [ 615.746959][ T44] process_one_work+0xd57/0x1390 [ 615.747136][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 615.747306][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.747473][ T44] ? assign_work+0x152/0x380 [ 615.747639][ T44] worker_thread+0x4d6/0xd40 [ 615.747822][ T44] ? process_one_work+0x1390/0x1390 [ 615.747986][ T44] kthread+0x355/0x5b0 [ 615.748122][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.748291][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.748460][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.748624][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.748804][ T44] ret_from_fork+0x3fb/0x510 [ 615.748970][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 615.749172][ T44] ? __switch_to+0x53c/0xd00 [ 615.749339][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.749503][ T44] ret_from_fork_asm+0x11/0x20 [ 615.749691][ T44] [ 615.749839][ T44] INFO: task kworker/u16:0:12 is blocked on a mutex likely owned by task ip:972. [ 615.750081][ T44] INFO: task kworker/2:0:28 blocked for more than 245 seconds. [ 615.750295][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 615.750448][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.750694][ T44] task:kworker/2:0 state:D stack:27096 pid:28 tgid:28 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 615.750996][ T44] Workqueue: events xp_release_deferred [ 615.751168][ T44] Call Trace: [ 615.751302][ T44] [ 615.751408][ T44] __schedule+0xae2/0x1720 [ 615.751576][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.751755][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.751927][ T44] schedule+0xd4/0x260 [ 615.752063][ T44] schedule_preempt_disabled+0x18/0x30 [ 615.752229][ T44] __mutex_lock+0xd76/0x1f20 [ 615.752394][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 615.752560][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.752740][ T44] ? __lock_acquire+0x577/0xc10 [ 615.752909][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.753072][ T44] ? process_one_work+0xd16/0x1390 [ 615.753252][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 615.753415][ T44] xp_release_deferred+0x1b/0x1e0 [ 615.753580][ T44] process_one_work+0xd57/0x1390 [ 615.753763][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 615.753927][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.754094][ T44] ? assign_work+0x152/0x380 [ 615.754263][ T44] worker_thread+0x4d6/0xd40 [ 615.754439][ T44] ? process_one_work+0x1390/0x1390 [ 615.754604][ T44] kthread+0x355/0x5b0 [ 615.754756][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.754922][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.755087][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.755260][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.755424][ T44] ret_from_fork+0x3fb/0x510 [ 615.755592][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 615.755800][ T44] ? __switch_to+0x53c/0xd00 [ 615.755965][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.756136][ T44] ret_from_fork_asm+0x11/0x20 [ 615.756304][ T44] [ 615.756443][ T44] INFO: task kworker/2:0:28 is blocked on a mutex likely owned by task ip:972. [ 615.756701][ T44] INFO: task ip:972 blocked for more than 368 seconds. [ 615.756885][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 615.757042][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.757278][ T44] task:ip state:D stack:24976 pid:972 tgid:972 ppid:1 task_flags:0x400100 flags:0x00080002 [ 615.757577][ T44] Call Trace: [ 615.757725][ T44] [ 615.757832][ T44] __schedule+0xae2/0x1720 [ 615.758003][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.758173][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.758337][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.758501][ T44] schedule+0xd4/0x260 [ 615.758636][ T44] schedule_preempt_disabled+0x18/0x30 [ 615.758811][ T44] __mutex_lock+0xd76/0x1f20 [ 615.758977][ T44] ? napi_enable+0x3b/0x80 [ 615.759152][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.759316][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.759480][ T44] ? __asan_memset+0x27/0x50 [ 615.759646][ T44] ? sg_init_table+0x19/0x60 [ 615.759828][ T44] ? try_fill_recv+0x3ec/0x1750 [ 615.759994][ T44] ? napi_enable+0x3b/0x80 [ 615.760163][ T44] napi_enable+0x3b/0x80 [ 615.760300][ T44] __virtnet_rx_resume+0x156/0x290 [ 615.760464][ T44] ? queue_delayed_work_on+0x91/0x100 [ 615.760630][ T44] virtnet_rx_resume_all+0xe7/0x1a0 [ 615.760807][ T44] virtnet_xdp_set+0xef7/0x1aa0 [ 615.760977][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0 [ 615.761145][ T44] dev_xdp_install+0x3e4/0x890 [ 615.761312][ T44] ? netif_set_xps_queue+0x40/0x40 [ 615.761485][ T44] dev_xdp_attach+0x50d/0x1310 [ 615.761653][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0 [ 615.761830][ T44] ? __bpf_prog_get+0x320/0x320 [ 615.761997][ T44] dev_change_xdp_fd+0x176/0x210 [ 615.762168][ T44] do_setlink.isra.0+0x1c2d/0x2880 [ 615.762337][ T44] ? rtnl_newlink_create+0x900/0x900 [ 615.762503][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 615.762713][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 615.762878][ T44] ? validate_chain+0x37b/0xb70 [ 615.763044][ T44] ? rcu_read_lock_any_held+0x66/0x90 [ 615.763216][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 615.763381][ T44] ? __lock_acquire+0x577/0xc10 [ 615.763550][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.763729][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 615.763898][ T44] ? lock_acquire+0x10a/0x150 [ 615.764062][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.764228][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.764393][ T44] ? trace_contention_end+0xd6/0x110 [ 615.764559][ T44] ? __mutex_lock+0x285/0x1f20 [ 615.764740][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 615.764907][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.765073][ T44] ? nla_get_range_signed+0x3d0/0x3d0 [ 615.765254][ T44] ? __rtnl_newlink+0x3fa/0xa40 [ 615.765422][ T44] rtnl_newlink+0x75c/0xe90 [ 615.765590][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.765769][ T44] ? rtnl_setlink+0x880/0x880 [ 615.765934][ T44] ? __lock_acquire+0x577/0xc10 [ 615.766101][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.766271][ T44] ? find_held_lock+0x2b/0x80 [ 615.766435][ T44] ? rtnl_setlink+0x880/0x880 [ 615.766603][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.766783][ T44] ? rtnl_setlink+0x880/0x880 [ 615.766948][ T44] rtnetlink_rcv_msg+0x6fe/0xb90 [ 615.767114][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 615.767288][ T44] ? __lock_acquire+0x577/0xc10 [ 615.767457][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.767620][ T44] ? find_held_lock+0x2b/0x80 [ 615.767804][ T44] netlink_rcv_skb+0x123/0x380 [ 615.767971][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 615.768145][ T44] ? netlink_ack+0xcc0/0xcc0 [ 615.768313][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 615.768477][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 615.768643][ T44] netlink_unicast+0x4a3/0x770 [ 615.768824][ T44] ? netlink_attachskb+0x810/0x810 [ 615.768988][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.769163][ T44] ? __lock_acquire+0x577/0xc10 [ 615.769330][ T44] netlink_sendmsg+0x735/0xc60 [ 615.769496][ T44] ? netlink_unicast+0x770/0x770 [ 615.769681][ T44] ____sys_sendmsg+0x419/0x850 [ 615.769848][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 615.770013][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 615.770210][ T44] ? move_addr_to_kernel+0x40/0x40 [ 615.770379][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 615.770543][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 615.770720][ T44] ___sys_sendmsg+0xfd/0x180 [ 615.770885][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00 [ 615.771078][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 615.771263][ T44] __sys_sendmsg+0x124/0x1c0 [ 615.771431][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 615.771596][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 615.771772][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00 [ 615.771969][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 615.772140][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.772305][ T44] do_syscall_64+0xbd/0xfc0 [ 615.772473][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 615.772679][ T44] RIP: 0033:0x7f0121de1c5e [ 615.772853][ T44] RSP: 002b:00007fffa41b4700 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 615.773075][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f0121de1c5e [ 615.773300][ T44] RDX: 0000000000000000 RSI: 00007fffa41b47c0 RDI: 0000000000000003 [ 615.773519][ T44] RBP: 00007fffa41b4710 R08: 0000000000000000 R09: 0000000000000000 [ 615.773761][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fffa41b4e80 [ 615.773981][ T44] R13: 0000000069569b71 R14: 0000000000000000 R15: 0000000000499600 [ 615.774213][ T44] [ 615.774348][ T44] INFO: task ip:972 is blocked on a mutex likely owned by task kworker/0:0:9. [ 615.774583][ T44] task:kworker/0:0 state:D stack:27416 pid:9 tgid:9 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 615.774911][ T44] Workqueue: events refill_work [ 615.775076][ T44] Call Trace: [ 615.775216][ T44] [ 615.775324][ T44] __schedule+0xae2/0x1720 [ 615.775492][ T44] ? usleep_range_state+0xba/0x1b0 [ 615.775656][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.775836][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.776008][ T44] ? usleep_range_state+0xba/0x1b0 [ 615.776172][ T44] schedule+0xd4/0x260 [ 615.776308][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0 [ 615.776500][ T44] ? schedule_timeout_idle+0xe0/0xe0 [ 615.776679][ T44] ? __hrtimer_cb_get_time+0x50/0x50 [ 615.776844][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 615.777008][ T44] ? ktime_get+0x15e/0x220 [ 615.777182][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 615.777345][ T44] ? ktime_get+0xa9/0x220 [ 615.777483][ T44] usleep_range_state+0x133/0x1b0 [ 615.777650][ T44] ? schedule_hrtimeout+0x20/0x20 [ 615.777829][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 615.777994][ T44] ? refill_work+0x149/0x210 [ 615.778165][ T44] napi_disable_locked+0x379/0x5f0 [ 615.778332][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 615.778497][ T44] ? register_netdevice_notifier_net+0x40/0x40 [ 615.778708][ T44] napi_disable+0x43/0x80 [ 615.778844][ T44] refill_work+0x116/0x210 [ 615.779011][ T44] process_one_work+0xd57/0x1390 [ 615.779180][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 615.779346][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.779516][ T44] ? assign_work+0x152/0x380 [ 615.779697][ T44] worker_thread+0x4d6/0xd40 [ 615.779870][ T44] ? process_one_work+0x1390/0x1390 [ 615.780034][ T44] kthread+0x355/0x5b0 [ 615.780174][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.780338][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.780563][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.780742][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.780912][ T44] ret_from_fork+0x3fb/0x510 [ 615.781077][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 615.781279][ T44] ? __switch_to+0x53c/0xd00 [ 615.781443][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.781608][ T44] ret_from_fork_asm+0x11/0x20 [ 615.781848][ T44] [ 615.781989][ T44] INFO: task ip:1040 blocked for more than 245 seconds. [ 615.782181][ T44] Not tainted 6.19.0-rc3-virtme #1 [ 615.782336][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.782621][ T44] task:ip state:D stack:25624 pid:1040 tgid:1040 ppid:1039 task_flags:0x400100 flags:0x00080000 [ 615.782930][ T44] Call Trace: [ 615.783064][ T44] [ 615.783173][ T44] __schedule+0xae2/0x1720 [ 615.783343][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.783510][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.783688][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.783913][ T44] schedule+0xd4/0x260 [ 615.784050][ T44] schedule_preempt_disabled+0x18/0x30 [ 615.784222][ T44] __mutex_lock+0xd76/0x1f20 [ 615.784388][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.784612][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 615.784789][ T44] ? write_profile+0xf0/0xf0 [ 615.784956][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.785124][ T44] ? find_held_lock+0x2b/0x80 [ 615.785352][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.785518][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 615.785698][ T44] rtnetlink_rcv_msg+0x2ab/0xb90 [ 615.785867][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 615.786087][ T44] ? __lock_acquire+0x577/0xc10 [ 615.786262][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.786426][ T44] ? find_held_lock+0x2b/0x80 [ 615.786591][ T44] netlink_rcv_skb+0x123/0x380 [ 615.786826][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 615.786992][ T44] ? netlink_ack+0xcc0/0xcc0 [ 615.787170][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 615.787337][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 615.787560][ T44] netlink_unicast+0x4a3/0x770 [ 615.787740][ T44] ? netlink_attachskb+0x810/0x810 [ 615.787905][ T44] ? __lock_acquire+0x577/0xc10 [ 615.788070][ T44] netlink_sendmsg+0x735/0xc60 [ 615.788297][ T44] ? netlink_unicast+0x770/0x770 [ 615.788466][ T44] ____sys_sendmsg+0x419/0x850 [ 615.788634][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 615.788807][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 615.789053][ T44] ? move_addr_to_kernel+0x40/0x40 [ 615.789227][ T44] ? __lock_acquire+0x577/0xc10 [ 615.789393][ T44] ___sys_sendmsg+0xfd/0x180 [ 615.789557][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 615.789796][ T44] ? do_pte_missing+0x545/0x8f0 [ 615.789971][ T44] ? lock_vma_under_rcu+0x12a/0x3e0 [ 615.790144][ T44] __sys_sendmsg+0x124/0x1c0 [ 615.790308][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 615.790534][ T44] ? do_user_addr_fault+0x799/0xdc0 [ 615.790713][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.790877][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.791041][ T44] do_syscall_64+0xbd/0xfc0 [ 615.791269][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 615.791466][ T44] RIP: 0033:0x7f3f6f6f6c5e [ 615.791633][ T44] RSP: 002b:00007ffd7d5bbd30 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 615.791925][ T44] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f3f6f6f6c5e [ 615.792149][ T44] RDX: 0000000000000000 RSI: 00007ffd7d5bbdf0 RDI: 0000000000000004 [ 615.792365][ T44] RBP: 00007ffd7d5bbd40 R08: 0000000000000000 R09: 0000000000000000 [ 615.792587][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffd7d5be68b [ 615.792820][ T44] R13: 0000000069569c40 R14: 000000000000000d R15: 00007ffd7d5bbea0 [ 615.793045][ T44] [ 615.793245][ T44] INFO: task ip:1040 is blocked on a mutex likely owned by task ip:972. [ 615.793457][ T44] [ 615.793457][ T44] Showing all locks held in the system: [ 615.793685][ T44] 3 locks held by kworker/0:0/9: [ 615.793899][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 615.794174][ T44] #1: ffa0000000097cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 615.794526][ T44] #2: ff11000008ad2c98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 615.794787][ T44] 3 locks held by kworker/u16:0/12: [ 615.794945][ T44] #0: ff110000090bf148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 615.795274][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 615.795602][ T44] #2: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 615.795918][ T44] 3 locks held by kworker/2:0/28: [ 615.796078][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 615.796410][ T44] #1: ffa00000001ffcc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 615.796698][ T44] #2: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0 [ 615.796948][ T44] 1 lock held by khungtaskd/44: [ 615.797104][ T44] #0: ffffffff98b710c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 615.797388][ T44] 2 locks held by ip/972: [ 615.797573][ T44] #0: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90 [ 615.797836][ T44] #1: ff11000008ad2c98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80 [ 615.798083][ T44] 1 lock held by ip/1040: [ 615.798274][ T44] #0: ffffffff992859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2ab/0xb90 [ 615.798523][ T44] [ 615.798625][ T44] ============================================= [ 615.798625][ T44]