[ 859.468879][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 260095 ms
[ 859.469297][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 260095000 usecs ago
[ 861.517878][ T44] INFO: task kworker/u16:0:12 blocked for more than 245 seconds.
[ 861.518206][ T44] Not tainted 6.19.0-rc3-virtme #1
[ 861.518381][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 861.518672][ T44] task:kworker/u16:0 state:D stack:26272 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 861.519050][ T44] Workqueue: ipv6_addrconf addrconf_verify_work
[ 861.519282][ T44] Call Trace:
[ 861.519422][ T44]
[ 861.519560][ T44] __schedule+0xae2/0x1720
[ 861.519744][ T44] ? io_schedule_timeout+0x130/0x130
[ 861.519964][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.520136][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.520331][ T44] schedule+0xd4/0x260
[ 861.520470][ T44] schedule_preempt_disabled+0x18/0x30
[ 861.520661][ T44] __mutex_lock+0xd76/0x1f20
[ 861.520839][ T44] ? addrconf_verify_work+0x12/0x30
[ 861.521035][ T44] ? ww_mutex_lock+0x160/0x160
[ 861.521204][ T44] ? process_one_work+0xd16/0x1390
[ 861.521399][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.521563][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.521756][ T44] ? addrconf_verify_work+0x12/0x30
[ 861.521947][ T44] addrconf_verify_work+0x12/0x30
[ 861.522109][ T44] process_one_work+0xd57/0x1390
[ 861.522299][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 861.522462][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.522653][ T44] ? assign_work+0x152/0x380
[ 861.522824][ T44] worker_thread+0x4d6/0xd40
[ 861.523022][ T44] ? process_one_work+0x1390/0x1390
[ 861.523186][ T44] kthread+0x355/0x5b0
[ 861.523348][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.523512][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.523694][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.523893][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.524063][ T44] ret_from_fork+0x3fb/0x510
[ 861.524254][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 861.524469][ T44] ? __switch_to+0x53c/0xd00
[ 861.524632][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.524819][ T44] ret_from_fork_asm+0x11/0x20
[ 861.525018][ T44]
[ 861.525173][ T44] INFO: task kworker/u16:0:12 is blocked on a mutex likely owned by task ip:5309.
[ 861.525444][ T44] INFO: task ip:5309 blocked for more than 245 seconds.
[ 861.525649][ T44] Not tainted 6.19.0-rc3-virtme #1
[ 861.525805][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 861.526063][ T44] task:ip state:D stack:25024 pid:5309 tgid:5309 ppid:1 task_flags:0x400100 flags:0x00080002
[ 861.526386][ T44] Call Trace:
[ 861.526518][ T44]
[ 861.526647][ T44] __schedule+0xae2/0x1720
[ 861.526816][ T44] ? io_schedule_timeout+0x130/0x130
[ 861.527010][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.527172][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.527358][ T44] schedule+0xd4/0x260
[ 861.527494][ T44] schedule_preempt_disabled+0x18/0x30
[ 861.527682][ T44] __mutex_lock+0xd76/0x1f20
[ 861.527854][ T44] ? napi_enable+0x3b/0x80
[ 861.528048][ T44] ? ww_mutex_lock+0x160/0x160
[ 861.528212][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.528397][ T44] ? __asan_memset+0x27/0x50
[ 861.528562][ T44] ? sg_init_table+0x19/0x60
[ 861.528755][ T44] ? try_fill_recv+0x3ec/0x1750
[ 861.528954][ T44] ? napi_enable+0x3b/0x80
[ 861.529121][ T44] napi_enable+0x3b/0x80
[ 861.529283][ T44] __virtnet_rx_resume+0x156/0x290
[ 861.529445][ T44] ? queue_delayed_work_on+0x91/0x100
[ 861.529632][ T44] virtnet_rx_resume_all+0xe7/0x1a0
[ 861.529804][ T44] virtnet_xdp_set+0xef7/0x1aa0
[ 861.530002][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0
[ 861.530165][ T44] dev_xdp_install+0x3e4/0x890
[ 861.530358][ T44] ? netif_set_xps_queue+0x40/0x40
[ 861.530526][ T44] dev_xdp_attach+0x50d/0x1310
[ 861.530713][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0
[ 861.530909][ T44] ? __bpf_prog_get+0x320/0x320
[ 861.531078][ T44] dev_change_xdp_fd+0x176/0x210
[ 861.531270][ T44] do_setlink.isra.0+0x1c2d/0x2880
[ 861.531434][ T44] ? rtnl_newlink_create+0x900/0x900
[ 861.531624][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0
[ 861.531847][ T44] ? rcu_read_lock_any_held+0x3c/0x90
[ 861.532034][ T44] ? validate_chain+0x37b/0xb70
[ 861.532198][ T44] ? rcu_read_lock_any_held+0x66/0x90
[ 861.532381][ T44] ? stack_depot_save_flags+0x3f1/0x6f0
[ 861.532545][ T44] ? __lock_acquire+0x577/0xc10
[ 861.532729][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.532928][ T44] ? rtnl_newlink+0x6fc/0xe90
[ 861.533090][ T44] ? lock_acquire+0x10a/0x150
[ 861.533275][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.533439][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.533624][ T44] ? trace_contention_end+0xd6/0x110
[ 861.533795][ T44] ? __mutex_lock+0x285/0x1f20
[ 861.533989][ T44] ? rtnl_newlink+0x6fc/0xe90
[ 861.534154][ T44] ? ww_mutex_lock+0x160/0x160
[ 861.534347][ T44] ? nla_get_range_signed+0x3d0/0x3d0
[ 861.534514][ T44] ? __rtnl_newlink+0x3fa/0xa40
[ 861.534700][ T44] rtnl_newlink+0x75c/0xe90
[ 861.534901][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.535064][ T44] ? rtnl_setlink+0x880/0x880
[ 861.535252][ T44] ? __lock_acquire+0x577/0xc10
[ 861.535417][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.535606][ T44] ? find_held_lock+0x2b/0x80
[ 861.535778][ T44] ? rtnl_setlink+0x880/0x880
[ 861.535971][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.536135][ T44] ? rtnl_setlink+0x880/0x880
[ 861.536321][ T44] rtnetlink_rcv_msg+0x6fe/0xb90
[ 861.536485][ T44] ? rtnl_fdb_dump+0x620/0x620
[ 861.536669][ T44] ? __lock_acquire+0x577/0xc10
[ 861.536842][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.537039][ T44] ? find_held_lock+0x2b/0x80
[ 861.537204][ T44] netlink_rcv_skb+0x123/0x380
[ 861.537395][ T44] ? rtnl_fdb_dump+0x620/0x620
[ 861.537558][ T44] ? netlink_ack+0xcc0/0xcc0
[ 861.537750][ T44] ? netlink_deliver_tap+0xc5/0x330
[ 861.537942][ T44] ? netlink_deliver_tap+0x13f/0x330
[ 861.538107][ T44] netlink_unicast+0x4a3/0x770
[ 861.538304][ T44] ? netlink_attachskb+0x810/0x810
[ 861.538467][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.538655][ T44] ? __lock_acquire+0x577/0xc10
[ 861.538829][ T44] netlink_sendmsg+0x735/0xc60
[ 861.539025][ T44] ? netlink_unicast+0x770/0x770
[ 861.539192][ T44] ____sys_sendmsg+0x419/0x850
[ 861.539383][ T44] ? copy_msghdr_from_user+0x270/0x430
[ 861.539547][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0
[ 861.539770][ T44] ? move_addr_to_kernel+0x40/0x40
[ 861.539963][ T44] ? stack_depot_save_flags+0x3f1/0x6f0
[ 861.540126][ T44] ? rcu_read_lock_any_held+0x3c/0x90
[ 861.540316][ T44] ___sys_sendmsg+0xfd/0x180
[ 861.540479][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00
[ 861.540696][ T44] ? copy_msghdr_from_user+0x430/0x430
[ 861.540912][ T44] __sys_sendmsg+0x124/0x1c0
[ 861.541076][ T44] ? __sys_sendmsg_sock+0x20/0x20
[ 861.541264][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 861.541425][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00
[ 861.541640][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 861.541811][ T44] ? fpregs_restore_userregs+0x14a/0x2a0
[ 861.542003][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.542171][ T44] do_syscall_64+0xbd/0xfc0
[ 861.542360][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 861.542573][ T44] RIP: 0033:0x7fafa97c7c5e
[ 861.542749][ T44] RSP: 002b:00007ffd416b45d0 EFLAGS: 00000202 ORIG_RAX: 000000000000002e
[ 861.542998][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007fafa97c7c5e
[ 861.543240][ T44] RDX: 0000000000000000 RSI: 00007ffd416b4690 RDI: 0000000000000003
[ 861.543480][ T44] RBP: 00007ffd416b45e0 R08: 0000000000000000 R09: 0000000000000000
[ 861.543721][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffd416b4d50
[ 861.543971][ T44] R13: 0000000069562f2f R14: 0000000000000000 R15: 0000000000499600
[ 861.544220][ T44]
[ 861.544354][ T44] INFO: task ip:5309 is blocked on a mutex likely owned by task kworker/2:0:28.
[ 861.544618][ T44] task:kworker/2:0 state:D stack:28096 pid:28 tgid:28 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 861.544943][ T44] Workqueue: events refill_work
[ 861.545107][ T44] Call Trace:
[ 861.545262][ T44]
[ 861.545368][ T44] __schedule+0xae2/0x1720
[ 861.545593][ T44] ? usleep_range_state+0xba/0x1b0
[ 861.545765][ T44] ? io_schedule_timeout+0x130/0x130
[ 861.545961][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.546127][ T44] ? usleep_range_state+0xba/0x1b0
[ 861.546314][ T44] schedule+0xd4/0x260
[ 861.546451][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0
[ 861.546669][ T44] ? schedule_timeout_idle+0xe0/0xe0
[ 861.546843][ T44] ? __hrtimer_cb_get_time+0x50/0x50
[ 861.547037][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 861.547200][ T44] ? ktime_get+0x15e/0x220
[ 861.547384][ T44] ? kvm_clock_get_cycles+0x18/0x30
[ 861.547547][ T44] ? ktime_get+0xa9/0x220
[ 861.547702][ T44] usleep_range_state+0x133/0x1b0
[ 861.547900][ T44] ? schedule_hrtimeout+0x20/0x20
[ 861.548064][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 861.548256][ T44] ? refill_work+0x149/0x210
[ 861.548420][ T44] napi_disable_locked+0x379/0x5f0
[ 861.548604][ T44] ? _mutex_trylock_nest_lock+0x340/0x340
[ 861.548775][ T44] ? register_netdevice_notifier_net+0x40/0x40
[ 861.548999][ T44] napi_disable+0x43/0x80
[ 861.549134][ T44] refill_work+0x116/0x210
[ 861.549325][ T44] process_one_work+0xd57/0x1390
[ 861.549495][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 861.549679][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.549854][ T44] ? assign_work+0x152/0x380
[ 861.550050][ T44] worker_thread+0x4d6/0xd40
[ 861.550219][ T44] ? process_one_work+0x1390/0x1390
[ 861.550409][ T44] kthread+0x355/0x5b0
[ 861.550545][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.550735][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.550936][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.551100][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.551287][ T44] ret_from_fork+0x3fb/0x510
[ 861.551455][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 861.551671][ T44] ? __switch_to+0x53c/0xd00
[ 861.551840][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.552039][ T44] ret_from_fork_asm+0x11/0x20
[ 861.552210][ T44]
[ 861.552375][ T44]
[ 861.552375][ T44] Showing all locks held in the system:
[ 861.552612][ T44] 3 locks held by kworker/u16:0/12:
[ 861.552778][ T44] #0: ff110000091b7148 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 861.553085][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 861.553437][ T44] #2: ffffffff9ee859a8 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30
[ 861.553701][ T44] 3 locks held by kworker/2:0/28:
[ 861.553893][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 861.554186][ T44] #1: ffa00000001ffcc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 861.554509][ T44] #2: ff11000008ac2c98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80
[ 861.554782][ T44] 1 lock held by khungtaskd/44:
[ 861.554968][ T44] #0: ffffffff9e7710c0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 861.555265][ T44] 3 locks held by kworker/0:2/61:
[ 861.555426][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 861.555719][ T44] #1: ffa000000041fcc0 ((work_completion)(&pool->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 861.556030][ T44] #2: ffffffff9ee859a8 (rtnl_mutex){+.+.}-{4:4}, at: xp_release_deferred+0x1b/0x1e0
[ 861.556305][ T44] 2 locks held by ip/5309:
[ 861.556467][ T44] #0: ffffffff9ee859a8 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x6fc/0xe90
[ 861.556735][ T44] #1: ff11000008ac2c98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80
[ 861.557009][ T44] 1 lock held by xdp_helper/5317:
[ 861.557165][ T44] #0: ffffffff9ee859a8 (rtnl_mutex){+.+.}-{4:4}, at: xsk_bind+0x17f/0x1a10
[ 861.557433][ T44]
[ 861.557534][ T44] =============================================
[ 861.557534][ T44]
[ 865.101867][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 265728 ms
[ 865.102281][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 265728000 usecs ago
[ 870.221866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 270848 ms
[ 870.222296][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 270848000 usecs ago
[ 875.341866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 275968 ms
[ 875.342289][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 275968000 usecs ago
[ 881.485867][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 282112 ms
[ 881.486292][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 282112000 usecs ago
[ 887.117866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 287744 ms
[ 887.118285][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 287744000 usecs ago
[ 892.237866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 292864 ms
[ 892.238268][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 292864000 usecs ago
[ 897.357866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 297984 ms
[ 897.358276][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 297984000 usecs ago
[ 903.501866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 304128 ms
[ 903.502284][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 304128000 usecs ago
[ 909.133866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 309760 ms
[ 909.134361][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 309760000 usecs ago
[ 914.253867][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 314880 ms
[ 914.254295][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 314880000 usecs ago
[ 919.373866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 320000 ms
[ 919.374267][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 320000000 usecs ago
[ 925.517866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 326144 ms
[ 925.518288][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 326144000 usecs ago
[ 931.149866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 331776 ms
[ 931.150263][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 331776000 usecs ago
[ 936.269867][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 336896 ms
[ 936.270290][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 336896000 usecs ago
[ 941.389867][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 342016 ms
[ 941.390298][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 342016000 usecs ago
[ 947.533871][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 348160 ms
[ 947.534293][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 348160000 usecs ago
[ 953.165867][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 353792 ms
[ 953.166297][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 353792000 usecs ago
[ 958.285866][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 358912 ms
[ 958.286271][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 358912000 usecs ago
[ 963.405867][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 3 timed out 364032 ms
[ 963.406277][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 3, sq: output.3, vq: 0x7, name: output.3, 364032000 usecs ago