[ 369.997793][ T44] INFO: task ip:971 blocked for more than 122 seconds. [ 369.998165][ T44] Not tainted 6.19.0-rc1-virtme #1 [ 369.998297][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 369.998524][ T44] task:ip state:D stack:24976 pid:971 tgid:971 ppid:1 task_flags:0x400100 flags:0x00080002 [ 369.998848][ T44] Call Trace: [ 369.998956][ T44] [ 369.999028][ T44] __schedule+0xae2/0x1720 [ 369.999169][ T44] ? io_schedule_timeout+0x130/0x130 [ 369.999297][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.999433][ T44] ? __lock_release.isra.0+0x59/0x170 [ 369.999558][ T44] schedule+0xd4/0x260 [ 369.999661][ T44] schedule_preempt_disabled+0x18/0x30 [ 369.999792][ T44] __mutex_lock+0xd76/0x1f20 [ 369.999922][ T44] ? napi_enable+0x3b/0x80 [ 370.000060][ T44] ? ww_mutex_lock+0x160/0x160 [ 370.000178][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.000307][ T44] ? __asan_memset+0x27/0x50 [ 370.000427][ T44] ? sg_init_table+0x19/0x60 [ 370.000555][ T44] ? try_fill_recv+0x3ec/0x1750 [ 370.000675][ T44] ? napi_enable+0x3b/0x80 [ 370.000802][ T44] napi_enable+0x3b/0x80 [ 370.000897][ T44] __virtnet_rx_resume+0x156/0x290 [ 370.001023][ T44] ? queue_delayed_work_on+0x91/0x100 [ 370.001141][ T44] virtnet_rx_resume_all+0xe7/0x1a0 [ 370.001266][ T44] virtnet_xdp_set+0xef7/0x1aa0 [ 370.001386][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0 [ 370.001506][ T44] dev_xdp_install+0x3e4/0x890 [ 370.001623][ T44] ? netif_set_xps_queue+0x40/0x40 [ 370.001747][ T44] dev_xdp_attach+0x50d/0x1310 [ 370.001878][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0 [ 370.001995][ T44] ? __bpf_prog_get+0x320/0x320 [ 370.002355][ T44] dev_change_xdp_fd+0x176/0x210 [ 370.002473][ T44] do_setlink.isra.0+0x1c2d/0x2880 [ 370.002601][ T44] ? rtnl_newlink_create+0x900/0x900 [ 370.002720][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 370.002879][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 370.002992][ T44] ? validate_chain+0x37b/0xb70 [ 370.003114][ T44] ? rcu_read_lock_any_held+0x66/0x90 [ 370.003226][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 370.003348][ T44] ? __lock_acquire+0x577/0xc10 [ 370.003463][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.003583][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 370.003697][ T44] ? lock_acquire+0x10a/0x150 [ 370.003818][ T44] ? rcu_is_watching+0x15/0xd0 [ 370.003934][ T44] ? rcu_is_watching+0x15/0xd0 [ 370.004056][ T44] ? trace_contention_end+0xd6/0x110 [ 370.004170][ T44] ? __mutex_lock+0x285/0x1f20 [ 370.004292][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 370.004405][ T44] ? ww_mutex_lock+0x160/0x160 [ 370.004526][ T44] ? nla_get_range_signed+0x3d0/0x3d0 [ 370.004646][ T44] ? __rtnl_newlink+0x3fa/0xa40 [ 370.004770][ T44] rtnl_newlink+0x75c/0xe90 [ 370.004889][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.005009][ T44] ? rtnl_setlink+0x880/0x880 [ 370.005122][ T44] ? __lock_acquire+0x577/0xc10 [ 370.005245][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.005356][ T44] ? find_held_lock+0x2b/0x80 [ 370.005480][ T44] ? rtnl_setlink+0x880/0x880 [ 370.005595][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.005715][ T44] ? rtnl_setlink+0x880/0x880 [ 370.005839][ T44] rtnetlink_rcv_msg+0x6fe/0xb90 [ 370.005958][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 370.006077][ T44] ? __lock_acquire+0x577/0xc10 [ 370.006191][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.006313][ T44] ? find_held_lock+0x2b/0x80 [ 370.006428][ T44] netlink_rcv_skb+0x123/0x380 [ 370.006549][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 370.006664][ T44] ? netlink_ack+0xcc0/0xcc0 [ 370.006791][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 370.006908][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 370.007029][ T44] netlink_unicast+0x4a3/0x770 [ 370.007145][ T44] ? netlink_attachskb+0x810/0x810 [ 370.007266][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.007382][ T44] ? __lock_acquire+0x577/0xc10 [ 370.007505][ T44] netlink_sendmsg+0x735/0xc60 [ 370.007619][ T44] ? netlink_unicast+0x770/0x770 [ 370.007744][ T44] ____sys_sendmsg+0x419/0x850 [ 370.007870][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 370.007983][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 370.008132][ T44] ? move_addr_to_kernel+0x40/0x40 [ 370.008249][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 370.008370][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 370.008486][ T44] ___sys_sendmsg+0xfd/0x180 [ 370.008607][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00 [ 370.008749][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 370.008885][ T44] __sys_sendmsg+0x124/0x1c0 [ 370.009000][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 370.009122][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 370.009235][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00 [ 370.009387][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 370.009500][ T44] ? rcu_is_watching+0x15/0xd0 [ 370.009621][ T44] do_syscall_64+0xbd/0xfc0 [ 370.009735][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 370.009888][ T44] RIP: 0033:0x7f76962cec5e [ 370.010013][ T44] RSP: 002b:00007ffc007e7160 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 370.010197][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f76962cec5e [ 370.010379][ T44] RDX: 0000000000000000 RSI: 00007ffc007e7220 RDI: 0000000000000003 [ 370.010558][ T44] RBP: 00007ffc007e7170 R08: 0000000000000000 R09: 0000000000000000 [ 370.010732][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffc007e78e0 [ 370.010921][ T44] R13: 0000000069538818 R14: 0000000000000000 R15: 0000000000499600 [ 370.011102][ T44] [ 370.011204][ T44] INFO: task ip:971 is blocked on a mutex likely owned by task kworker/3:1:69. [ 370.011394][ T44] task:kworker/3:1 state:D stack:28504 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 370.011659][ T44] Workqueue: events refill_work [ 370.011783][ T44] Call Trace: [ 370.011873][ T44] [ 370.011934][ T44] __schedule+0xae2/0x1720 [ 370.012057][ T44] ? usleep_range_state+0xba/0x1b0 [ 370.012170][ T44] ? io_schedule_timeout+0x130/0x130 [ 370.012293][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.012410][ T44] ? usleep_range_state+0xba/0x1b0 [ 370.012529][ T44] schedule+0xd4/0x260 [ 370.012615][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0 [ 370.012766][ T44] ? schedule_timeout_idle+0xe0/0xe0 [ 370.012883][ T44] ? __hrtimer_cb_get_time+0x50/0x50 [ 370.013004][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 370.013117][ T44] ? ktime_get+0x15e/0x220 [ 370.013241][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 370.013354][ T44] ? ktime_get+0xa9/0x220 [ 370.013448][ T44] usleep_range_state+0x133/0x1b0 [ 370.013562][ T44] ? schedule_hrtimeout+0x20/0x20 [ 370.013684][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 370.013805][ T44] ? refill_work+0x149/0x210 [ 370.013923][ T44] napi_disable_locked+0x379/0x5f0 [ 370.014043][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 370.014156][ T44] ? register_netdevice_notifier_net+0x40/0x40 [ 370.014307][ T44] napi_disable+0x43/0x80 [ 370.014393][ T44] refill_work+0x116/0x210 [ 370.014515][ T44] process_one_work+0xd57/0x1390 [ 370.014632][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 370.014751][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 370.014882][ T44] ? assign_work+0x152/0x380 [ 370.014997][ T44] worker_thread+0x4d6/0xd40 [ 370.015122][ T44] ? process_one_work+0x1390/0x1390 [ 370.015235][ T44] kthread+0x355/0x5b0 [ 370.015328][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 370.015441][ T44] ? __lock_release.isra.0+0x59/0x170 [ 370.015561][ T44] ? rcu_is_watching+0x15/0xd0 [ 370.015675][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 370.015798][ T44] ret_from_fork+0x3fb/0x510 [ 370.015914][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 370.016063][ T44] ? __switch_to+0x53c/0xd00 [ 370.016178][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 370.016299][ T44] ret_from_fork_asm+0x11/0x20 [ 370.016421][ T44] [ 370.016524][ T44] [ 370.016524][ T44] Showing all locks held in the system: [ 370.016694][ T44] 3 locks held by kworker/u16:0/12: [ 370.016886][ T44] #0: ff11000009117148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 370.017118][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 370.017398][ T44] #2: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 370.017604][ T44] 1 lock held by khungtaskd/44: [ 370.017713][ T44] #0: ffffffffbc571040 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 370.017948][ T44] 3 locks held by kworker/2:1/65: [ 370.018057][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 370.018283][ T44] #1: ffa0000000467cc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 370.018512][ T44] #2: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0 [ 370.018707][ T44] 3 locks held by kworker/3:1/69: [ 370.018831][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 370.019061][ T44] #1: ffa00000004a7cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 370.019315][ T44] #2: ff11000008a8ac98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 370.019509][ T44] 2 locks held by ip/971: [ 370.019598][ T44] #0: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90 [ 370.019801][ T44] #1: ff11000008a8ac98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80 [ 370.020008][ T44] 1 lock held by ip/1036: [ 370.020089][ T44] #0: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2ab/0xb90 [ 370.020291][ T44] [ 370.020348][ T44] ============================================= [ 370.020348][ T44] [ 492.877772][ T44] INFO: task kworker/u16:0:12 blocked for more than 122 seconds. [ 492.878061][ T44] Not tainted 6.19.0-rc1-virtme #1 [ 492.878186][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.878406][ T44] task:kworker/u16:0 state:D stack:27664 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 492.878696][ T44] Workqueue: ipv6_addrconf addrconf_verify_work [ 492.878872][ T44] Call Trace: [ 492.878981][ T44] [ 492.879049][ T44] __schedule+0xae2/0x1720 [ 492.879191][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.879318][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.879446][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.879568][ T44] schedule+0xd4/0x260 [ 492.879666][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.879797][ T44] __mutex_lock+0xd76/0x1f20 [ 492.879922][ T44] ? addrconf_verify_work+0x12/0x30 [ 492.880054][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.880179][ T44] ? process_one_work+0xd16/0x1390 [ 492.880306][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.880420][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.880544][ T44] ? addrconf_verify_work+0x12/0x30 [ 492.880659][ T44] addrconf_verify_work+0x12/0x30 [ 492.880785][ T44] process_one_work+0xd57/0x1390 [ 492.880903][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 492.881029][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.881146][ T44] ? assign_work+0x152/0x380 [ 492.881269][ T44] worker_thread+0x4d6/0xd40 [ 492.881387][ T44] ? process_one_work+0x1390/0x1390 [ 492.881508][ T44] kthread+0x355/0x5b0 [ 492.881597][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.881717][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.881844][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.881960][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.882089][ T44] ret_from_fork+0x3fb/0x510 [ 492.882205][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 492.882355][ T44] ? __switch_to+0x53c/0xd00 [ 492.882469][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.882592][ T44] ret_from_fork_asm+0x11/0x20 [ 492.882713][ T44] [ 492.882814][ T44] INFO: task kworker/u16:0:12 is blocked on a mutex likely owned by task ip:971. [ 492.883025][ T44] INFO: task kworker/2:1:65 blocked for more than 122 seconds. [ 492.883179][ T44] Not tainted 6.19.0-rc1-virtme #1 [ 492.883293][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.883477][ T44] task:kworker/2:1 state:D stack:26176 pid:65 tgid:65 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 492.883729][ T44] Workqueue: events xp_release_deferred [ 492.883851][ T44] Call Trace: [ 492.883937][ T44] [ 492.883999][ T44] __schedule+0xae2/0x1720 [ 492.884122][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.884240][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.884361][ T44] schedule+0xd4/0x260 [ 492.884447][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.884567][ T44] __mutex_lock+0xd76/0x1f20 [ 492.884682][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 492.884804][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.884918][ T44] ? __lock_acquire+0x577/0xc10 [ 492.885044][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.885158][ T44] ? process_one_work+0xd16/0x1390 [ 492.885282][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 492.885395][ T44] xp_release_deferred+0x1b/0x1e0 [ 492.885518][ T44] process_one_work+0xd57/0x1390 [ 492.885636][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 492.885759][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.885877][ T44] ? assign_work+0x152/0x380 [ 492.886000][ T44] worker_thread+0x4d6/0xd40 [ 492.886114][ T44] ? process_one_work+0x1390/0x1390 [ 492.886234][ T44] ? __kthread_parkme+0xb3/0x200 [ 492.886350][ T44] ? process_one_work+0x1390/0x1390 [ 492.886471][ T44] kthread+0x355/0x5b0 [ 492.886558][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.886678][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.886802][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.886915][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.887041][ T44] ret_from_fork+0x3fb/0x510 [ 492.887155][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 492.887303][ T44] ? __switch_to+0x53c/0xd00 [ 492.887417][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.887543][ T44] ret_from_fork_asm+0x11/0x20 [ 492.887665][ T44] [ 492.887765][ T44] INFO: task kworker/2:1:65 is blocked on a mutex likely owned by task ip:971. [ 492.887951][ T44] INFO: task ip:971 blocked for more than 245 seconds. [ 492.888092][ T44] Not tainted 6.19.0-rc1-virtme #1 [ 492.888196][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.888385][ T44] task:ip state:D stack:24976 pid:971 tgid:971 ppid:1 task_flags:0x400100 flags:0x00080002 [ 492.888650][ T44] Call Trace: [ 492.888737][ T44] [ 492.888805][ T44] __schedule+0xae2/0x1720 [ 492.888921][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.889045][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.889157][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.889283][ T44] schedule+0xd4/0x260 [ 492.889369][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.889489][ T44] __mutex_lock+0xd76/0x1f20 [ 492.889603][ T44] ? napi_enable+0x3b/0x80 [ 492.889727][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.889851][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.889965][ T44] ? __asan_memset+0x27/0x50 [ 492.890090][ T44] ? sg_init_table+0x19/0x60 [ 492.890207][ T44] ? try_fill_recv+0x3ec/0x1750 [ 492.890332][ T44] ? napi_enable+0x3b/0x80 [ 492.890445][ T44] napi_enable+0x3b/0x80 [ 492.890540][ T44] __virtnet_rx_resume+0x156/0x290 [ 492.890652][ T44] ? queue_delayed_work_on+0x91/0x100 [ 492.890777][ T44] virtnet_rx_resume_all+0xe7/0x1a0 [ 492.890892][ T44] virtnet_xdp_set+0xef7/0x1aa0 [ 492.891023][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0 [ 492.891136][ T44] dev_xdp_install+0x3e4/0x890 [ 492.891257][ T44] ? netif_set_xps_queue+0x40/0x40 [ 492.891373][ T44] dev_xdp_attach+0x50d/0x1310 [ 492.891496][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0 [ 492.891609][ T44] ? __bpf_prog_get+0x320/0x320 [ 492.891731][ T44] dev_change_xdp_fd+0x176/0x210 [ 492.891857][ T44] do_setlink.isra.0+0x1c2d/0x2880 [ 492.891972][ T44] ? rtnl_newlink_create+0x900/0x900 [ 492.892095][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 492.892234][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 492.892355][ T44] ? validate_chain+0x37b/0xb70 [ 492.892468][ T44] ? rcu_read_lock_any_held+0x66/0x90 [ 492.892587][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 492.892704][ T44] ? __lock_acquire+0x577/0xc10 [ 492.892827][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.892941][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 492.893064][ T44] ? lock_acquire+0x10a/0x150 [ 492.893177][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.893296][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.893410][ T44] ? trace_contention_end+0xd6/0x110 [ 492.893533][ T44] ? __mutex_lock+0x285/0x1f20 [ 492.893647][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 492.893769][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.893886][ T44] ? nla_get_range_signed+0x3d0/0x3d0 [ 492.894013][ T44] ? __rtnl_newlink+0x3fa/0xa40 [ 492.894129][ T44] rtnl_newlink+0x75c/0xe90 [ 492.894251][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.894364][ T44] ? rtnl_setlink+0x880/0x880 [ 492.894487][ T44] ? __lock_acquire+0x577/0xc10 [ 492.894602][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.894721][ T44] ? find_held_lock+0x2b/0x80 [ 492.894846][ T44] ? rtnl_setlink+0x880/0x880 [ 492.894961][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.895086][ T44] ? rtnl_setlink+0x880/0x880 [ 492.895201][ T44] rtnetlink_rcv_msg+0x6fe/0xb90 [ 492.895322][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.895438][ T44] ? __lock_acquire+0x577/0xc10 [ 492.895560][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.895673][ T44] ? find_held_lock+0x2b/0x80 [ 492.895797][ T44] netlink_rcv_skb+0x123/0x380 [ 492.895910][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.896035][ T44] ? netlink_ack+0xcc0/0xcc0 [ 492.896152][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 492.896274][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 492.896388][ T44] netlink_unicast+0x4a3/0x770 [ 492.896512][ T44] ? netlink_attachskb+0x810/0x810 [ 492.896624][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.896743][ T44] ? __lock_acquire+0x577/0xc10 [ 492.896867][ T44] netlink_sendmsg+0x735/0xc60 [ 492.896985][ T44] ? netlink_unicast+0x770/0x770 [ 492.897108][ T44] ____sys_sendmsg+0x419/0x850 [ 492.897225][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 492.897346][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 492.897489][ T44] ? move_addr_to_kernel+0x40/0x40 [ 492.897611][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 492.897722][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 492.897846][ T44] ___sys_sendmsg+0xfd/0x180 [ 492.897967][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00 [ 492.898121][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 492.898246][ T44] __sys_sendmsg+0x124/0x1c0 [ 492.898369][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 492.898483][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 492.898604][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00 [ 492.898750][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 492.898875][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.898992][ T44] do_syscall_64+0xbd/0xfc0 [ 492.899114][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 492.899256][ T44] RIP: 0033:0x7f76962cec5e [ 492.899384][ T44] RSP: 002b:00007ffc007e7160 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 492.899561][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f76962cec5e [ 492.899745][ T44] RDX: 0000000000000000 RSI: 00007ffc007e7220 RDI: 0000000000000003 [ 492.899928][ T44] RBP: 00007ffc007e7170 R08: 0000000000000000 R09: 0000000000000000 [ 492.900103][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffc007e78e0 [ 492.900280][ T44] R13: 0000000069538818 R14: 0000000000000000 R15: 0000000000499600 [ 492.900459][ T44] [ 492.900554][ T44] INFO: task ip:971 is blocked on a mutex likely owned by task kworker/3:1:69. [ 492.900740][ T44] task:kworker/3:1 state:D stack:28504 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 492.900995][ T44] Workqueue: events refill_work [ 492.901107][ T44] Call Trace: [ 492.901201][ T44] [ 492.901260][ T44] __schedule+0xae2/0x1720 [ 492.901377][ T44] ? usleep_range_state+0xba/0x1b0 [ 492.901496][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.901611][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.901732][ T44] ? usleep_range_state+0xba/0x1b0 [ 492.901851][ T44] schedule+0xd4/0x260 [ 492.901937][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0 [ 492.902089][ T44] ? schedule_timeout_idle+0xe0/0xe0 [ 492.902206][ T44] ? __hrtimer_cb_get_time+0x50/0x50 [ 492.902327][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 492.902439][ T44] ? ktime_get+0x15e/0x220 [ 492.902559][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 492.902672][ T44] ? ktime_get+0xa9/0x220 [ 492.902767][ T44] usleep_range_state+0x133/0x1b0 [ 492.902881][ T44] ? schedule_hrtimeout+0x20/0x20 [ 492.903008][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 492.903123][ T44] ? refill_work+0x149/0x210 [ 492.903246][ T44] napi_disable_locked+0x379/0x5f0 [ 492.903359][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 492.903481][ T44] ? register_netdevice_notifier_net+0x40/0x40 [ 492.903626][ T44] napi_disable+0x43/0x80 [ 492.903718][ T44] refill_work+0x116/0x210 [ 492.903845][ T44] process_one_work+0xd57/0x1390 [ 492.903961][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 492.904083][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.904199][ T44] ? assign_work+0x152/0x380 [ 492.904321][ T44] worker_thread+0x4d6/0xd40 [ 492.904438][ T44] ? process_one_work+0x1390/0x1390 [ 492.904556][ T44] kthread+0x355/0x5b0 [ 492.904714][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.904836][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.904950][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.905074][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.905243][ T44] ret_from_fork+0x3fb/0x510 [ 492.905364][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 492.905508][ T44] ? __switch_to+0x53c/0xd00 [ 492.905632][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 492.905812][ T44] ret_from_fork_asm+0x11/0x20 [ 492.905932][ T44] [ 492.906031][ T44] INFO: task ip:1036 blocked for more than 122 seconds. [ 492.906162][ T44] Not tainted 6.19.0-rc1-virtme #1 [ 492.906324][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 492.906513][ T44] task:ip state:D stack:25624 pid:1036 tgid:1036 ppid:1035 task_flags:0x400100 flags:0x00080000 [ 492.906822][ T44] Call Trace: [ 492.906907][ T44] [ 492.906965][ T44] __schedule+0xae2/0x1720 [ 492.907091][ T44] ? io_schedule_timeout+0x130/0x130 [ 492.907205][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.907380][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.907497][ T44] schedule+0xd4/0x260 [ 492.907589][ T44] schedule_preempt_disabled+0x18/0x30 [ 492.907702][ T44] __mutex_lock+0xd76/0x1f20 [ 492.907880][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.907996][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 492.908118][ T44] ? write_profile+0xf0/0xf0 [ 492.908233][ T44] ? ww_mutex_lock+0x160/0x160 [ 492.908417][ T44] ? find_held_lock+0x2b/0x80 [ 492.908535][ T44] ? __lock_release.isra.0+0x59/0x170 [ 492.908655][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 492.908776][ T44] rtnetlink_rcv_msg+0x2ab/0xb90 [ 492.908947][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.909073][ T44] ? __lock_acquire+0x577/0xc10 [ 492.909189][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 492.909309][ T44] ? find_held_lock+0x2b/0x80 [ 492.909485][ T44] netlink_rcv_skb+0x123/0x380 [ 492.909605][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 492.909719][ T44] ? netlink_ack+0xcc0/0xcc0 [ 492.909845][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 492.910015][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 492.910136][ T44] netlink_unicast+0x4a3/0x770 [ 492.910251][ T44] ? netlink_attachskb+0x810/0x810 [ 492.910375][ T44] ? __lock_acquire+0x577/0xc10 [ 492.910546][ T44] netlink_sendmsg+0x735/0xc60 [ 492.910668][ T44] ? netlink_unicast+0x770/0x770 [ 492.910793][ T44] ____sys_sendmsg+0x419/0x850 [ 492.910909][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 492.911091][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 492.911231][ T44] ? move_addr_to_kernel+0x40/0x40 [ 492.911352][ T44] ? __lock_acquire+0x577/0xc10 [ 492.911469][ T44] ___sys_sendmsg+0xfd/0x180 [ 492.911644][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 492.911771][ T44] ? do_pte_missing+0x545/0x8f0 [ 492.911889][ T44] ? lock_vma_under_rcu+0x12a/0x3e0 [ 492.912064][ T44] __sys_sendmsg+0x124/0x1c0 [ 492.912178][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 492.912303][ T44] ? do_user_addr_fault+0x799/0xdc0 [ 492.912421][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.912585][ T44] ? rcu_is_watching+0x15/0xd0 [ 492.912698][ T44] do_syscall_64+0xbd/0xfc0 [ 492.912824][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 492.912963][ T44] RIP: 0033:0x7fa6c6276c5e [ 492.913156][ T44] RSP: 002b:00007ffc07373ff0 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 492.913325][ T44] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fa6c6276c5e [ 492.913504][ T44] RDX: 0000000000000000 RSI: 00007ffc073740b0 RDI: 0000000000000004 [ 492.913740][ T44] RBP: 00007ffc07374000 R08: 0000000000000000 R09: 0000000000000000 [ 492.913917][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffc0737668b [ 492.914089][ T44] R13: 00000000695388e8 R14: 000000000000000d R15: 00007ffc07374160 [ 492.914332][ T44] [ 492.914422][ T44] INFO: task ip:1036 is blocked on a mutex likely owned by task ip:971. [ 492.914590][ T44] [ 492.914590][ T44] Showing all locks held in the system: [ 492.914903][ T44] 3 locks held by kworker/u16:0/12: [ 492.915015][ T44] #0: ff11000009117148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 492.915249][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 492.915530][ T44] #2: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 492.915724][ T44] 1 lock held by khungtaskd/44: [ 492.915846][ T44] #0: ffffffffbc571040 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 492.916080][ T44] 3 locks held by kworker/2:1/65: [ 492.916240][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 492.916467][ T44] #1: ffa0000000467cc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 492.916746][ T44] #2: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0 [ 492.916956][ T44] 3 locks held by kworker/3:1/69: [ 492.917066][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 492.917345][ T44] #1: ffa00000004a7cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 492.917595][ T44] #2: ff11000008a8ac98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 492.917852][ T44] 2 locks held by ip/971: [ 492.917933][ T44] #0: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90 [ 492.918136][ T44] #1: ff11000008a8ac98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80 [ 492.918391][ T44] 1 lock held by ip/1036: [ 492.918473][ T44] #0: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2ab/0xb90 [ 492.918720][ T44] [ 492.918786][ T44] ============================================= [ 492.918786][ T44] [ 615.757771][ T44] INFO: task kworker/u16:0:12 blocked for more than 245 seconds. [ 615.758059][ T44] Not tainted 6.19.0-rc1-virtme #1 [ 615.758207][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.758470][ T44] task:kworker/u16:0 state:D stack:27664 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 615.758765][ T44] Workqueue: ipv6_addrconf addrconf_verify_work [ 615.758921][ T44] Call Trace: [ 615.759025][ T44] [ 615.759091][ T44] __schedule+0xae2/0x1720 [ 615.759234][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.759362][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.759499][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.759619][ T44] schedule+0xd4/0x260 [ 615.759715][ T44] schedule_preempt_disabled+0x18/0x30 [ 615.759842][ T44] __mutex_lock+0xd76/0x1f20 [ 615.759962][ T44] ? addrconf_verify_work+0x12/0x30 [ 615.760086][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.760214][ T44] ? process_one_work+0xd16/0x1390 [ 615.760347][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.760465][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.760587][ T44] ? addrconf_verify_work+0x12/0x30 [ 615.760699][ T44] addrconf_verify_work+0x12/0x30 [ 615.760825][ T44] process_one_work+0xd57/0x1390 [ 615.760944][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 615.761063][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.761186][ T44] ? assign_work+0x152/0x380 [ 615.761308][ T44] worker_thread+0x4d6/0xd40 [ 615.761426][ T44] ? process_one_work+0x1390/0x1390 [ 615.761547][ T44] kthread+0x355/0x5b0 [ 615.761633][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.761753][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.761871][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.761995][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.762110][ T44] ret_from_fork+0x3fb/0x510 [ 615.762230][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 615.762372][ T44] ? __switch_to+0x53c/0xd00 [ 615.762493][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.762605][ T44] ret_from_fork_asm+0x11/0x20 [ 615.762730][ T44] [ 615.762828][ T44] INFO: task kworker/u16:0:12 is blocked on a mutex likely owned by task ip:971. [ 615.763032][ T44] INFO: task kworker/2:1:65 blocked for more than 245 seconds. [ 615.763190][ T44] Not tainted 6.19.0-rc1-virtme #1 [ 615.763303][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.763486][ T44] task:kworker/2:1 state:D stack:26176 pid:65 tgid:65 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 615.763741][ T44] Workqueue: events xp_release_deferred [ 615.763863][ T44] Call Trace: [ 615.763948][ T44] [ 615.764007][ T44] __schedule+0xae2/0x1720 [ 615.764133][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.764248][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.764370][ T44] schedule+0xd4/0x260 [ 615.764455][ T44] schedule_preempt_disabled+0x18/0x30 [ 615.764576][ T44] __mutex_lock+0xd76/0x1f20 [ 615.764689][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 615.764811][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.764925][ T44] ? __lock_acquire+0x577/0xc10 [ 615.765048][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.765163][ T44] ? process_one_work+0xd16/0x1390 [ 615.765284][ T44] ? xp_release_deferred+0x1b/0x1e0 [ 615.765398][ T44] xp_release_deferred+0x1b/0x1e0 [ 615.765520][ T44] process_one_work+0xd57/0x1390 [ 615.765638][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 615.765760][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.765874][ T44] ? assign_work+0x152/0x380 [ 615.765995][ T44] worker_thread+0x4d6/0xd40 [ 615.766111][ T44] ? process_one_work+0x1390/0x1390 [ 615.766232][ T44] ? __kthread_parkme+0xb3/0x200 [ 615.766346][ T44] ? process_one_work+0x1390/0x1390 [ 615.766468][ T44] kthread+0x355/0x5b0 [ 615.766555][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.766676][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.766798][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.766911][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.767033][ T44] ret_from_fork+0x3fb/0x510 [ 615.767152][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 615.767299][ T44] ? __switch_to+0x53c/0xd00 [ 615.767413][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.767535][ T44] ret_from_fork_asm+0x11/0x20 [ 615.767654][ T44] [ 615.767746][ T44] INFO: task kworker/2:1:65 is blocked on a mutex likely owned by task ip:971. [ 615.767943][ T44] INFO: task ip:971 blocked for more than 368 seconds. [ 615.768076][ T44] Not tainted 6.19.0-rc1-virtme #1 [ 615.768190][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.768373][ T44] task:ip state:D stack:24976 pid:971 tgid:971 ppid:1 task_flags:0x400100 flags:0x00080002 [ 615.768621][ T44] Call Trace: [ 615.768708][ T44] [ 615.768776][ T44] __schedule+0xae2/0x1720 [ 615.768897][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.769017][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.769131][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.769254][ T44] schedule+0xd4/0x260 [ 615.769340][ T44] schedule_preempt_disabled+0x18/0x30 [ 615.769461][ T44] __mutex_lock+0xd76/0x1f20 [ 615.769579][ T44] ? napi_enable+0x3b/0x80 [ 615.769706][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.769828][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.769943][ T44] ? __asan_memset+0x27/0x50 [ 615.770065][ T44] ? sg_init_table+0x19/0x60 [ 615.770184][ T44] ? try_fill_recv+0x3ec/0x1750 [ 615.770309][ T44] ? napi_enable+0x3b/0x80 [ 615.770421][ T44] napi_enable+0x3b/0x80 [ 615.770517][ T44] __virtnet_rx_resume+0x156/0x290 [ 615.770630][ T44] ? queue_delayed_work_on+0x91/0x100 [ 615.770751][ T44] virtnet_rx_resume_all+0xe7/0x1a0 [ 615.770877][ T44] virtnet_xdp_set+0xef7/0x1aa0 [ 615.770994][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0 [ 615.771117][ T44] dev_xdp_install+0x3e4/0x890 [ 615.771231][ T44] ? netif_set_xps_queue+0x40/0x40 [ 615.771359][ T44] dev_xdp_attach+0x50d/0x1310 [ 615.771475][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0 [ 615.771595][ T44] ? __bpf_prog_get+0x320/0x320 [ 615.771712][ T44] dev_change_xdp_fd+0x176/0x210 [ 615.771833][ T44] do_setlink.isra.0+0x1c2d/0x2880 [ 615.771949][ T44] ? rtnl_newlink_create+0x900/0x900 [ 615.772069][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0 [ 615.772274][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 615.772387][ T44] ? validate_chain+0x37b/0xb70 [ 615.772513][ T44] ? rcu_read_lock_any_held+0x66/0x90 [ 615.772626][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 615.772748][ T44] ? __lock_acquire+0x577/0xc10 [ 615.772873][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.772988][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 615.773111][ T44] ? lock_acquire+0x10a/0x150 [ 615.773223][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.773344][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.773457][ T44] ? trace_contention_end+0xd6/0x110 [ 615.773577][ T44] ? __mutex_lock+0x285/0x1f20 [ 615.773692][ T44] ? rtnl_newlink+0x6fc/0xe90 [ 615.773816][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.773930][ T44] ? nla_get_range_signed+0x3d0/0x3d0 [ 615.774056][ T44] ? __rtnl_newlink+0x3fa/0xa40 [ 615.774174][ T44] rtnl_newlink+0x75c/0xe90 [ 615.774296][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.774409][ T44] ? rtnl_setlink+0x880/0x880 [ 615.774530][ T44] ? __lock_acquire+0x577/0xc10 [ 615.774643][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.774765][ T44] ? find_held_lock+0x2b/0x80 [ 615.774880][ T44] ? rtnl_setlink+0x880/0x880 [ 615.775001][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.775118][ T44] ? rtnl_setlink+0x880/0x880 [ 615.775239][ T44] rtnetlink_rcv_msg+0x6fe/0xb90 [ 615.775353][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 615.775475][ T44] ? __lock_acquire+0x577/0xc10 [ 615.775589][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.775710][ T44] ? find_held_lock+0x2b/0x80 [ 615.775837][ T44] netlink_rcv_skb+0x123/0x380 [ 615.775951][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 615.776075][ T44] ? netlink_ack+0xcc0/0xcc0 [ 615.776194][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 615.776312][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 615.776427][ T44] netlink_unicast+0x4a3/0x770 [ 615.776550][ T44] ? netlink_attachskb+0x810/0x810 [ 615.776666][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.776790][ T44] ? __lock_acquire+0x577/0xc10 [ 615.776906][ T44] netlink_sendmsg+0x735/0xc60 [ 615.777029][ T44] ? netlink_unicast+0x770/0x770 [ 615.777150][ T44] ____sys_sendmsg+0x419/0x850 [ 615.777271][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 615.777384][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 615.777536][ T44] ? move_addr_to_kernel+0x40/0x40 [ 615.777652][ T44] ? stack_depot_save_flags+0x3f1/0x6f0 [ 615.777773][ T44] ? rcu_read_lock_any_held+0x3c/0x90 [ 615.777888][ T44] ___sys_sendmsg+0xfd/0x180 [ 615.778009][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00 [ 615.778156][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 615.778289][ T44] __sys_sendmsg+0x124/0x1c0 [ 615.778407][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 615.778528][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 615.778642][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00 [ 615.778797][ T44] ? do_raw_spin_unlock+0x59/0x250 [ 615.778911][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.779034][ T44] do_syscall_64+0xbd/0xfc0 [ 615.779153][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 615.779305][ T44] RIP: 0033:0x7f76962cec5e [ 615.779423][ T44] RSP: 002b:00007ffc007e7160 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 615.779603][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f76962cec5e [ 615.779783][ T44] RDX: 0000000000000000 RSI: 00007ffc007e7220 RDI: 0000000000000003 [ 615.779954][ T44] RBP: 00007ffc007e7170 R08: 0000000000000000 R09: 0000000000000000 [ 615.780138][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffc007e78e0 [ 615.780309][ T44] R13: 0000000069538818 R14: 0000000000000000 R15: 0000000000499600 [ 615.780494][ T44] [ 615.780582][ T44] INFO: task ip:971 is blocked on a mutex likely owned by task kworker/3:1:69. [ 615.780777][ T44] task:kworker/3:1 state:D stack:28504 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000 [ 615.781034][ T44] Workqueue: events refill_work [ 615.781150][ T44] Call Trace: [ 615.781242][ T44] [ 615.781302][ T44] __schedule+0xae2/0x1720 [ 615.781417][ T44] ? usleep_range_state+0xba/0x1b0 [ 615.781537][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.781653][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.781778][ T44] ? usleep_range_state+0xba/0x1b0 [ 615.781891][ T44] schedule+0xd4/0x260 [ 615.781985][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0 [ 615.782130][ T44] ? schedule_timeout_idle+0xe0/0xe0 [ 615.782251][ T44] ? __hrtimer_cb_get_time+0x50/0x50 [ 615.782367][ T44] ? lockdep_hardirqs_on+0x84/0x130 [ 615.782492][ T44] ? ktime_get+0x15e/0x220 [ 615.782604][ T44] ? kvm_clock_get_cycles+0x18/0x30 [ 615.782724][ T44] ? ktime_get+0xa9/0x220 [ 615.782820][ T44] usleep_range_state+0x133/0x1b0 [ 615.782933][ T44] ? schedule_hrtimeout+0x20/0x20 [ 615.783054][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0 [ 615.783169][ T44] ? refill_work+0x149/0x210 [ 615.783292][ T44] napi_disable_locked+0x379/0x5f0 [ 615.783408][ T44] ? _mutex_trylock_nest_lock+0x340/0x340 [ 615.783528][ T44] ? register_netdevice_notifier_net+0x40/0x40 [ 615.783672][ T44] napi_disable+0x43/0x80 [ 615.783766][ T44] refill_work+0x116/0x210 [ 615.783882][ T44] process_one_work+0xd57/0x1390 [ 615.784007][ T44] ? pwq_dec_nr_in_flight+0x700/0x700 [ 615.784125][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.784250][ T44] ? assign_work+0x152/0x380 [ 615.784364][ T44] worker_thread+0x4d6/0xd40 [ 615.784489][ T44] ? process_one_work+0x1390/0x1390 [ 615.784603][ T44] kthread+0x355/0x5b0 [ 615.784759][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.784873][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.784993][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.785111][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.785287][ T44] ret_from_fork+0x3fb/0x510 [ 615.785399][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140 [ 615.785549][ T44] ? __switch_to+0x53c/0xd00 [ 615.785663][ T44] ? kthread_is_per_cpu+0xe0/0xe0 [ 615.785843][ T44] ret_from_fork_asm+0x11/0x20 [ 615.785967][ T44] [ 615.786061][ T44] INFO: task ip:1036 blocked for more than 245 seconds. [ 615.786197][ T44] Not tainted 6.19.0-rc1-virtme #1 [ 615.786358][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 615.786538][ T44] task:ip state:D stack:25624 pid:1036 tgid:1036 ppid:1035 task_flags:0x400100 flags:0x00080000 [ 615.786841][ T44] Call Trace: [ 615.786925][ T44] [ 615.786985][ T44] __schedule+0xae2/0x1720 [ 615.787109][ T44] ? io_schedule_timeout+0x130/0x130 [ 615.787224][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.787400][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.787515][ T44] schedule+0xd4/0x260 [ 615.787606][ T44] schedule_preempt_disabled+0x18/0x30 [ 615.787721][ T44] __mutex_lock+0xd76/0x1f20 [ 615.787846][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.787957][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 615.788076][ T44] ? write_profile+0xf0/0xf0 [ 615.788193][ T44] ? ww_mutex_lock+0x160/0x160 [ 615.788375][ T44] ? find_held_lock+0x2b/0x80 [ 615.788489][ T44] ? __lock_release.isra.0+0x59/0x170 [ 615.788609][ T44] ? rtnetlink_rcv_msg+0x2ab/0xb90 [ 615.788722][ T44] rtnetlink_rcv_msg+0x2ab/0xb90 [ 615.788847][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 615.788958][ T44] ? __lock_acquire+0x577/0xc10 [ 615.789079][ T44] ? lock_acquire.part.0+0xbc/0x260 [ 615.789194][ T44] ? find_held_lock+0x2b/0x80 [ 615.789375][ T44] netlink_rcv_skb+0x123/0x380 [ 615.789488][ T44] ? rtnl_fdb_dump+0x620/0x620 [ 615.789609][ T44] ? netlink_ack+0xcc0/0xcc0 [ 615.789728][ T44] ? netlink_deliver_tap+0xc5/0x330 [ 615.789849][ T44] ? netlink_deliver_tap+0x13f/0x330 [ 615.789963][ T44] netlink_unicast+0x4a3/0x770 [ 615.790085][ T44] ? netlink_attachskb+0x810/0x810 [ 615.790202][ T44] ? __lock_acquire+0x577/0xc10 [ 615.790381][ T44] netlink_sendmsg+0x735/0xc60 [ 615.790495][ T44] ? netlink_unicast+0x770/0x770 [ 615.790619][ T44] ____sys_sendmsg+0x419/0x850 [ 615.790734][ T44] ? copy_msghdr_from_user+0x270/0x430 [ 615.790855][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0 [ 615.790994][ T44] ? move_addr_to_kernel+0x40/0x40 [ 615.791118][ T44] ? __lock_acquire+0x577/0xc10 [ 615.791233][ T44] ___sys_sendmsg+0xfd/0x180 [ 615.791411][ T44] ? copy_msghdr_from_user+0x430/0x430 [ 615.791528][ T44] ? do_pte_missing+0x545/0x8f0 [ 615.791653][ T44] ? lock_vma_under_rcu+0x12a/0x3e0 [ 615.791776][ T44] __sys_sendmsg+0x124/0x1c0 [ 615.791891][ T44] ? __sys_sendmsg_sock+0x20/0x20 [ 615.792014][ T44] ? do_user_addr_fault+0x799/0xdc0 [ 615.792131][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.792295][ T44] ? rcu_is_watching+0x15/0xd0 [ 615.792408][ T44] do_syscall_64+0xbd/0xfc0 [ 615.792533][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 615.792672][ T44] RIP: 0033:0x7fa6c6276c5e [ 615.792799][ T44] RSP: 002b:00007ffc07373ff0 EFLAGS: 00000202 ORIG_RAX: 000000000000002e [ 615.792975][ T44] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fa6c6276c5e [ 615.793154][ T44] RDX: 0000000000000000 RSI: 00007ffc073740b0 RDI: 0000000000000004 [ 615.793393][ T44] RBP: 00007ffc07374000 R08: 0000000000000000 R09: 0000000000000000 [ 615.793563][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffc0737668b [ 615.793739][ T44] R13: 00000000695388e8 R14: 000000000000000d R15: 00007ffc07374160 [ 615.793925][ T44] [ 615.794014][ T44] INFO: task ip:1036 is blocked on a mutex likely owned by task ip:971. [ 615.794189][ T44] [ 615.794189][ T44] Showing all locks held in the system: [ 615.794416][ T44] 3 locks held by kworker/u16:0/12: [ 615.794524][ T44] #0: ff11000009117148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 615.794896][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 615.795187][ T44] #2: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30 [ 615.795391][ T44] 1 lock held by khungtaskd/44: [ 615.795503][ T44] #0: ffffffffbc571040 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 615.795731][ T44] 3 locks held by kworker/2:1/65: [ 615.795849][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 615.796078][ T44] #1: ffa0000000467cc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 615.796361][ T44] #2: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0 [ 615.796555][ T44] 3 locks held by kworker/3:1/69: [ 615.796726][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390 [ 615.796955][ T44] #1: ffa00000004a7cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390 [ 615.797265][ T44] #2: ff11000008a8ac98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80 [ 615.797458][ T44] 2 locks held by ip/971: [ 615.797548][ T44] #0: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90 [ 615.797803][ T44] #1: ff11000008a8ac98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80 [ 615.797997][ T44] 1 lock held by ip/1036: [ 615.798086][ T44] #0: ffffffffbcc85768 (rtnl_mutex){+.+.}-{4:4}, at: rtnetlink_rcv_msg+0x2ab/0xb90 [ 615.798343][ T44] [ 615.798398][ T44] ============================================= [ 615.798398][ T44]