[ 756.542744][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 5631 ms
[ 756.543314][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 5631000 usecs ago
[ 762.175734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 11264 ms
[ 762.176120][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 11264000 usecs ago
[ 767.295740][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 16384 ms
[ 767.296153][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 16384000 usecs ago
[ 772.414740][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 21503 ms
[ 772.415134][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 21503000 usecs ago
[ 778.559739][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 27648 ms
[ 778.560139][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 27648000 usecs ago
[ 784.190739][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 33279 ms
[ 784.191134][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 33279000 usecs ago
[ 789.311742][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 38400 ms
[ 789.312139][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 38400000 usecs ago
[ 795.454742][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 44543 ms
[ 795.455182][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 44543000 usecs ago
[ 801.087734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 50176 ms
[ 801.088127][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 50176000 usecs ago
[ 806.207733][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 55296 ms
[ 806.208143][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 55296000 usecs ago
[ 811.327735][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 60416 ms
[ 811.328134][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 60416000 usecs ago
[ 817.471739][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 66560 ms
[ 817.472141][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 66560000 usecs ago
[ 823.102741][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 72191 ms
[ 823.103143][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 72191000 usecs ago
[ 828.222739][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 77311 ms
[ 828.223127][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 77311000 usecs ago
[ 833.342740][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 82431 ms
[ 833.343135][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 82431000 usecs ago
[ 839.487734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 88576 ms
[ 839.488122][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 88576000 usecs ago
[ 845.119741][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 94208 ms
[ 845.120141][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 94208000 usecs ago
[ 850.239739][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 99328 ms
[ 850.240141][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 99328000 usecs ago
[ 855.359734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 104448 ms
[ 855.360182][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 104448000 usecs ago
[ 861.502780][ T44] INFO: task ip:6159 blocked for more than 122 seconds.
[ 861.503087][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 861.503226][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 861.503469][ T44] task:ip state:D stack:25320 pid:6159 tgid:6159 ppid:6154 task_flags:0x400100 flags:0x00080001
[ 861.503793][ T44] Call Trace:
[ 861.503804][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 110592 ms
[ 861.503894][ T44]
[ 861.503899][ T44] __schedule+0x88d/0x1b90
[ 861.503910][ T44] ? io_schedule_timeout+0x130/0x130
[ 861.503916][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.503923][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.503925][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.503929][ T44] schedule+0xd4/0x260
[ 861.503932][ T44] schedule_preempt_disabled+0x18/0x30
[ 861.504297][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 110592000 usecs ago
[ 861.504352][ T44] __mutex_lock+0x9ae/0x1500
[ 861.504355][ T44] ? napi_enable+0x3b/0x80
[ 861.504361][ T44] ? virtqueue_add_inbuf_ctx+0x1850/0x1850
[ 861.504367][ T44] ? ww_mutex_lock+0x160/0x160
[ 861.504370][ T44] ? __asan_memset+0x27/0x50
[ 861.504374][ T44] ? sg_init_table+0x19/0x60
[ 861.504379][ T44] ? try_fill_recv+0x3ec/0x1750
[ 861.504386][ T44] ? napi_enable+0x3b/0x80
[ 861.504388][ T44] napi_enable+0x3b/0x80
[ 861.504391][ T44] __virtnet_rx_resume+0x156/0x290
[ 861.506696][ T44] ? queue_delayed_work_on+0x91/0x100
[ 861.506809][ T44] virtnet_rx_resume_all+0xe7/0x1a0
[ 861.506914][ T44] virtnet_xdp_set+0xef7/0x1aa0
[ 861.507021][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0
[ 861.507123][ T44] dev_xdp_install+0x3e4/0x8e0
[ 861.507394][ T44] ? netif_skb_check_for_xdp+0x600/0x600
[ 861.507504][ T44] dev_xdp_attach+0x50d/0x1330
[ 861.507610][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0
[ 861.507713][ T44] ? __bpf_prog_get+0x320/0x320
[ 861.507829][ T44] dev_change_xdp_fd+0x182/0x240
[ 861.507933][ T44] do_setlink.isra.0+0x1c2d/0x2880
[ 861.508039][ T44] ? rtnl_newlink_create+0x900/0x900
[ 861.508142][ T44] ? rcu_lockdep_current_cpu_online+0x3e/0x1b0
[ 861.508275][ T44] ? rcu_read_lock_any_held+0x41/0xa0
[ 861.508377][ T44] ? validate_chain+0x287/0x5f0
[ 861.508481][ T44] ? rcu_read_lock_any_held+0x6b/0xa0
[ 861.508584][ T44] ? stack_depot_save_flags+0x3e1/0x6e0
[ 861.508691][ T44] ? __lock_acquire+0x34c/0x740
[ 861.508807][ T44] ? trace_lock_acquire+0x88/0xd0
[ 861.508911][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 861.509016][ T44] ? lock_acquire+0x31/0x80
[ 861.509119][ T44] ? rtnl_newlink+0x5df/0xb20
[ 861.509222][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.509328][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.509432][ T44] ? trace_contention_end+0xd6/0x140
[ 861.509536][ T44] ? rtnl_newlink+0x5df/0xb20
[ 861.509637][ T44] ? __mutex_lock+0x32e/0x1500
[ 861.509765][ T44] ? ww_mutex_lock+0x160/0x160
[ 861.509868][ T44] ? nla_validate_int_range+0x5b0/0x5b0
[ 861.509975][ T44] ? __rtnl_newlink+0x3fa/0xa40
[ 861.510081][ T44] rtnl_newlink+0x637/0xb20
[ 861.510186][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 861.510295][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.510398][ T44] ? find_held_lock+0x2b/0x80
[ 861.510505][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 861.510607][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.510715][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 861.510823][ T44] rtnetlink_rcv_msg+0x6fe/0xb90
[ 861.510927][ T44] ? rtnl_port_fill+0x940/0x940
[ 861.511028][ T44] ? __lock_acquire+0x34c/0x740
[ 861.511133][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.511237][ T44] ? find_held_lock+0x2b/0x80
[ 861.511354][ T44] netlink_rcv_skb+0x123/0x380
[ 861.511458][ T44] ? rtnl_port_fill+0x940/0x940
[ 861.511570][ T44] ? netlink_ack+0xcc0/0xcc0
[ 861.511681][ T44] ? netlink_deliver_tap+0x13f/0x330
[ 861.511791][ T44] netlink_unicast+0x4a3/0x770
[ 861.511896][ T44] ? netlink_attachskb+0x810/0x810
[ 861.512000][ T44] ? __lock_acquire+0x34c/0x740
[ 861.512112][ T44] netlink_sendmsg+0x735/0xc60
[ 861.512219][ T44] ? netlink_unicast+0x770/0x770
[ 861.512327][ T44] ? __might_fault+0x97/0x140
[ 861.512434][ T44] ____sys_sendmsg+0x419/0x850
[ 861.512537][ T44] ? copy_msghdr_from_user+0x270/0x430
[ 861.512640][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0
[ 861.512777][ T44] ? move_addr_to_kernel+0x40/0x40
[ 861.512882][ T44] ? stack_depot_save_flags+0x3e1/0x6e0
[ 861.512988][ T44] ? validate_chain+0x287/0x5f0
[ 861.513095][ T44] ___sys_sendmsg+0xfd/0x180
[ 861.513197][ T44] ? __call_rcu_common.constprop.0+0xa5/0x680
[ 861.513328][ T44] ? copy_msghdr_from_user+0x430/0x430
[ 861.513446][ T44] __sys_sendmsg+0x124/0x1c0
[ 861.513550][ T44] ? __sys_sendmsg_sock+0x20/0x20
[ 861.513651][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 861.513768][ T44] ? __call_rcu_common.constprop.0+0x2fa/0x680
[ 861.513897][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 861.514001][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.514105][ T44] do_syscall_64+0xbb/0xfd0
[ 861.514209][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 861.514339][ T44] RIP: 0033:0x7f4f2b063c5e
[ 861.514452][ T44] RSP: 002b:00007ffc0b0b2d10 EFLAGS: 00000202 ORIG_RAX: 000000000000002e
[ 861.514615][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f4f2b063c5e
[ 861.514777][ T44] RDX: 0000000000000000 RSI: 00007ffc0b0b2dd0 RDI: 0000000000000003
[ 861.514933][ T44] RBP: 00007ffc0b0b2d20 R08: 0000000000000000 R09: 0000000000000000
[ 861.515090][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffc0b0b3490
[ 861.515251][ T44] R13: 000000006926b8f8 R14: 0000000000000000 R15: 0000000000499600
[ 861.515415][ T44]
[ 861.515499][ T44] INFO: task ip:6159 is blocked on a mutex likely owned by task kworker/3:1:71.
[ 861.515674][ T44] task:kworker/3:1 state:D stack:28200 pid:71 tgid:71 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 861.515911][ T44] Workqueue: events refill_work
[ 861.516019][ T44] Call Trace:
[ 861.516100][ T44]
[ 861.516157][ T44] __schedule+0x88d/0x1b90
[ 861.516269][ T44] ? io_schedule_timeout+0x130/0x130
[ 861.516377][ T44] ? usleep_range_state+0xba/0x1b0
[ 861.516482][ T44] ? find_held_lock+0x2b/0x80
[ 861.516590][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.516699][ T44] ? usleep_range_state+0xba/0x1b0
[ 861.516806][ T44] schedule+0xd4/0x260
[ 861.516885][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0
[ 861.517013][ T44] ? schedule_timeout_idle+0xe0/0xe0
[ 861.517116][ T44] ? hrtimer_cb_get_time+0x90/0x90
[ 861.517223][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 861.517330][ T44] ? ktime_get+0x15e/0x220
[ 861.517435][ T44] ? kvm_clock_get_cycles+0x18/0x30
[ 861.517538][ T44] ? ktime_get+0xa9/0x220
[ 861.517617][ T44] usleep_range_state+0x133/0x1b0
[ 861.517723][ T44] ? schedule_hrtimeout+0x20/0x20
[ 861.517843][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 861.517949][ T44] napi_disable_locked+0x379/0x5f0
[ 861.518052][ T44] ? _mutex_trylock_nest_lock+0x150/0x150
[ 861.518161][ T44] ? netdev_bind_sb_channel_queue+0x470/0x470
[ 861.518293][ T44] napi_disable+0x43/0x80
[ 861.518371][ T44] refill_work+0x116/0x210
[ 861.518475][ T44] process_one_work+0xd57/0x1390
[ 861.518583][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 861.518686][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.518800][ T44] ? assign_work+0x152/0x380
[ 861.518906][ T44] worker_thread+0x4d6/0xd40
[ 861.519014][ T44] ? process_one_work+0x1390/0x1390
[ 861.519117][ T44] ? __kthread_parkme+0xb3/0x200
[ 861.519223][ T44] ? process_one_work+0x1390/0x1390
[ 861.519328][ T44] kthread+0x355/0x5b0
[ 861.519408][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.519516][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.519621][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.519732][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.519837][ T44] ret_from_fork+0x1d4/0x270
[ 861.519940][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.520046][ T44] ret_from_fork_asm+0x11/0x20
[ 861.520156][ T44]
[ 861.520246][ T44]
[ 861.520246][ T44] Showing all locks held in the system:
[ 861.520396][ T44] 3 locks held by kworker/u16:0/12:
[ 861.520500][ T44] #0: ff1100000915cd48 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 861.520705][ T44] #1: ffa00000000c7d30 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 861.521177][ T44] #2: ffffffffa2a92a28 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30
[ 861.521358][ T44] 1 lock held by khungtaskd/44:
[ 861.521458][ T44] #0: ffffffffa2378fc0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x2b/0x1db
[ 861.521638][ T44] 3 locks held by kworker/3:1/71:
[ 861.521740][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 861.521939][ T44] #1: ffa00000004c7d30 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 861.522163][ T44] #2: ff11000008bcaca0 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80
[ 861.522348][ T44] 2 locks held by ip/6159:
[ 861.522445][ T44] #0: ffffffffa2a92a28 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x5df/0xb20
[ 861.522620][ T44] #1: ff11000008bcaca0 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80
[ 861.522799][ T44]
[ 861.522854][ T44] =============================================
[ 861.522854][ T44]
[ 867.135734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 116224 ms
[ 867.136158][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 116224000 usecs ago
[ 872.255734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 121344 ms
[ 872.256178][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 121344000 usecs ago
[ 877.375735][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 126464 ms
[ 877.376154][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 126464000 usecs ago
[ 883.518740][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 132607 ms
[ 883.519176][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 132607000 usecs ago
[ 889.151734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 138240 ms
[ 889.152157][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 138240000 usecs ago
[ 894.271739][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 143360 ms
[ 894.272157][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 143360000 usecs ago
[ 899.390734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 148479 ms
[ 899.391173][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 148479000 usecs ago
[ 905.535736][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 154624 ms
[ 905.536193][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 154624000 usecs ago
[ 911.167735][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 160256 ms
[ 911.168201][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 160256000 usecs ago
[ 916.287734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 165376 ms
[ 916.288164][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 165376000 usecs ago
[ 921.406741][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 170495 ms
[ 921.407172][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 170495000 usecs ago
[ 927.550740][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 176639 ms
[ 927.551176][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 176639000 usecs ago
[ 933.182735][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 182271 ms
[ 933.183177][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 182271000 usecs ago
[ 938.303735][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 187392 ms
[ 938.304176][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 187392000 usecs ago
[ 944.447736][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 193536 ms
[ 944.448155][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 193536000 usecs ago
[ 950.079740][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 199168 ms
[ 950.080179][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 199168000 usecs ago
[ 955.198740][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 204287 ms
[ 955.199184][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 204287000 usecs ago
[ 960.318734][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 209407 ms
[ 960.319152][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 209407000 usecs ago
[ 966.462740][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 215551 ms
[ 966.463171][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 215551000 usecs ago
[ 972.094741][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 221183 ms
[ 972.095172][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 221183000 usecs ago
[ 977.214735][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 226303 ms
[ 977.215169][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 226303000 usecs ago
[ 982.334740][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 231423 ms
[ 982.335168][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 231423000 usecs ago
[ 984.382757][ T44] INFO: task kworker/u16:0:12 blocked for more than 122 seconds.
[ 984.383101][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 984.383267][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 984.383558][ T44] task:kworker/u16:0 state:D stack:26112 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 984.383927][ T44] Workqueue: ipv6_addrconf addrconf_verify_work
[ 984.384113][ T44] Call Trace:
[ 984.384221][ T44]
[ 984.384295][ T44] __schedule+0x88d/0x1b90
[ 984.384445][ T44] ? io_schedule_timeout+0x130/0x130
[ 984.384583][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.384735][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.384869][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.385001][ T44] schedule+0xd4/0x260
[ 984.385100][ T44] schedule_preempt_disabled+0x18/0x30
[ 984.385231][ T44] __mutex_lock+0x9ae/0x1500
[ 984.385362][ T44] ? addrconf_verify_work+0x12/0x30
[ 984.385502][ T44] ? ww_mutex_lock+0x160/0x160
[ 984.385630][ T44] ? process_one_work+0xd16/0x1390
[ 984.385786][ T44] ? trace_lock_acquire+0x88/0xd0
[ 984.385908][ T44] ? trace_lock_acquire+0x88/0xd0
[ 984.386038][ T44] ? addrconf_verify_work+0x12/0x30
[ 984.386155][ T44] addrconf_verify_work+0x12/0x30
[ 984.386278][ T44] process_one_work+0xd57/0x1390
[ 984.386411][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 984.386541][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.386668][ T44] ? assign_work+0x152/0x380
[ 984.386802][ T44] worker_thread+0x4d6/0xd40
[ 984.386922][ T44] ? process_one_work+0x1390/0x1390
[ 984.387037][ T44] kthread+0x355/0x5b0
[ 984.387126][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.387240][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.387362][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.387481][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.387595][ T44] ret_from_fork+0x1d4/0x270
[ 984.387713][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.387836][ T44] ret_from_fork_asm+0x11/0x20
[ 984.387957][ T44]
[ 984.388083][ T44] INFO: task kworker/u16:0:12 is blocked on a mutex likely owned by task ip:6159.
[ 984.388296][ T44] INFO: task ip:6159 blocked for more than 245 seconds.
[ 984.388439][ T44] Not tainted 6.18.0-rc6-virtme #1
[ 984.388551][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 984.388752][ T44] task:ip state:D stack:25320 pid:6159 tgid:6159 ppid:6154 task_flags:0x400100 flags:0x00080001
[ 984.389010][ T44] Call Trace:
[ 984.389097][ T44]
[ 984.389162][ T44] __schedule+0x88d/0x1b90
[ 984.389280][ T44] ? io_schedule_timeout+0x130/0x130
[ 984.389396][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.389517][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.389634][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.389770][ T44] schedule+0xd4/0x260
[ 984.389859][ T44] schedule_preempt_disabled+0x18/0x30
[ 984.389976][ T44] __mutex_lock+0x9ae/0x1500
[ 984.390092][ T44] ? napi_enable+0x3b/0x80
[ 984.390211][ T44] ? virtqueue_add_inbuf_ctx+0x1850/0x1850
[ 984.390356][ T44] ? ww_mutex_lock+0x160/0x160
[ 984.390474][ T44] ? __asan_memset+0x27/0x50
[ 984.390592][ T44] ? sg_init_table+0x19/0x60
[ 984.390709][ T44] ? try_fill_recv+0x3ec/0x1750
[ 984.390832][ T44] ? napi_enable+0x3b/0x80
[ 984.390954][ T44] napi_enable+0x3b/0x80
[ 984.391041][ T44] __virtnet_rx_resume+0x156/0x290
[ 984.391157][ T44] ? queue_delayed_work_on+0x91/0x100
[ 984.391273][ T44] virtnet_rx_resume_all+0xe7/0x1a0
[ 984.391388][ T44] virtnet_xdp_set+0xef7/0x1aa0
[ 984.391512][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0
[ 984.391626][ T44] dev_xdp_install+0x3e4/0x8e0
[ 984.391762][ T44] ? netif_skb_check_for_xdp+0x600/0x600
[ 984.391880][ T44] dev_xdp_attach+0x50d/0x1330
[ 984.391998][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0
[ 984.392114][ T44] ? __bpf_prog_get+0x320/0x320
[ 984.392232][ T44] dev_change_xdp_fd+0x182/0x240
[ 984.392349][ T44] do_setlink.isra.0+0x1c2d/0x2880
[ 984.392469][ T44] ? rtnl_newlink_create+0x900/0x900
[ 984.392589][ T44] ? rcu_lockdep_current_cpu_online+0x3e/0x1b0
[ 984.392738][ T44] ? rcu_read_lock_any_held+0x41/0xa0
[ 984.392853][ T44] ? validate_chain+0x287/0x5f0
[ 984.392968][ T44] ? rcu_read_lock_any_held+0x6b/0xa0
[ 984.393082][ T44] ? stack_depot_save_flags+0x3e1/0x6e0
[ 984.393200][ T44] ? __lock_acquire+0x34c/0x740
[ 984.393320][ T44] ? trace_lock_acquire+0x88/0xd0
[ 984.393439][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 984.393554][ T44] ? lock_acquire+0x31/0x80
[ 984.393669][ T44] ? rtnl_newlink+0x5df/0xb20
[ 984.393803][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.393919][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.394036][ T44] ? trace_contention_end+0xd6/0x140
[ 984.394153][ T44] ? rtnl_newlink+0x5df/0xb20
[ 984.394271][ T44] ? __mutex_lock+0x32e/0x1500
[ 984.394389][ T44] ? ww_mutex_lock+0x160/0x160
[ 984.394506][ T44] ? nla_validate_int_range+0x5b0/0x5b0
[ 984.394624][ T44] ? __rtnl_newlink+0x3fa/0xa40
[ 984.394747][ T44] rtnl_newlink+0x637/0xb20
[ 984.394865][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 984.394983][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.395098][ T44] ? find_held_lock+0x2b/0x80
[ 984.395216][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 984.395330][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.395447][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 984.395561][ T44] rtnetlink_rcv_msg+0x6fe/0xb90
[ 984.395679][ T44] ? rtnl_port_fill+0x940/0x940
[ 984.395794][ T44] ? __lock_acquire+0x34c/0x740
[ 984.395914][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.396030][ T44] ? find_held_lock+0x2b/0x80
[ 984.396152][ T44] netlink_rcv_skb+0x123/0x380
[ 984.396271][ T44] ? rtnl_port_fill+0x940/0x940
[ 984.396388][ T44] ? netlink_ack+0xcc0/0xcc0
[ 984.396509][ T44] ? netlink_deliver_tap+0x13f/0x330
[ 984.396626][ T44] netlink_unicast+0x4a3/0x770
[ 984.396748][ T44] ? netlink_attachskb+0x810/0x810
[ 984.396865][ T44] ? __lock_acquire+0x34c/0x740
[ 984.396986][ T44] netlink_sendmsg+0x735/0xc60
[ 984.397102][ T44] ? netlink_unicast+0x770/0x770
[ 984.397216][ T44] ? __might_fault+0x97/0x140
[ 984.397333][ T44] ____sys_sendmsg+0x419/0x850
[ 984.397452][ T44] ? copy_msghdr_from_user+0x270/0x430
[ 984.397571][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0
[ 984.397715][ T44] ? move_addr_to_kernel+0x40/0x40
[ 984.397843][ T44] ? stack_depot_save_flags+0x3e1/0x6e0
[ 984.397959][ T44] ? validate_chain+0x287/0x5f0
[ 984.398075][ T44] ___sys_sendmsg+0xfd/0x180
[ 984.398190][ T44] ? __call_rcu_common.constprop.0+0xa5/0x680
[ 984.398334][ T44] ? copy_msghdr_from_user+0x430/0x430
[ 984.398466][ T44] __sys_sendmsg+0x124/0x1c0
[ 984.398583][ T44] ? __sys_sendmsg_sock+0x20/0x20
[ 984.398698][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 984.398831][ T44] ? __call_rcu_common.constprop.0+0x2fa/0x680
[ 984.398976][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 984.399092][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.399209][ T44] do_syscall_64+0xbb/0xfd0
[ 984.399326][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 984.399476][ T44] RIP: 0033:0x7f4f2b063c5e
[ 984.399599][ T44] RSP: 002b:00007ffc0b0b2d10 EFLAGS: 00000202 ORIG_RAX: 000000000000002e
[ 984.399787][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f4f2b063c5e
[ 984.399964][ T44] RDX: 0000000000000000 RSI: 00007ffc0b0b2dd0 RDI: 0000000000000003
[ 984.400136][ T44] RBP: 00007ffc0b0b2d20 R08: 0000000000000000 R09: 0000000000000000
[ 984.400311][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffc0b0b3490
[ 984.400498][ T44] R13: 000000006926b8f8 R14: 0000000000000000 R15: 0000000000499600
[ 984.400681][ T44]
[ 984.400779][ T44] INFO: task ip:6159 is blocked on a mutex likely owned by task kworker/3:1:71.
[ 984.400976][ T44] task:kworker/3:1 state:D stack:28200 pid:71 tgid:71 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 984.401245][ T44] Workqueue: events refill_work
[ 984.401363][ T44] Call Trace:
[ 984.401470][ T44]
[ 984.401533][ T44] __schedule+0x88d/0x1b90
[ 984.401655][ T44] ? io_schedule_timeout+0x130/0x130
[ 984.401788][ T44] ? usleep_range_state+0xba/0x1b0
[ 984.401903][ T44] ? find_held_lock+0x2b/0x80
[ 984.402021][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.402141][ T44] ? tick_program_event+0x20/0x100
[ 984.402260][ T44] ? usleep_range_state+0xba/0x1b0
[ 984.402377][ T44] schedule+0xd4/0x260
[ 984.402469][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0
[ 984.402611][ T44] ? schedule_timeout_idle+0xe0/0xe0
[ 984.402735][ T44] ? hrtimer_cb_get_time+0x90/0x90
[ 984.402851][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 984.402970][ T44] ? ktime_get+0x15e/0x220
[ 984.403085][ T44] ? kvm_clock_get_cycles+0x18/0x30
[ 984.403200][ T44] ? ktime_get+0xa9/0x220
[ 984.403288][ T44] usleep_range_state+0x133/0x1b0
[ 984.403407][ T44] ? schedule_hrtimeout+0x20/0x20
[ 984.403527][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 984.403643][ T44] napi_disable_locked+0x379/0x5f0
[ 984.403766][ T44] ? _mutex_trylock_nest_lock+0x150/0x150
[ 984.403882][ T44] ? netdev_bind_sb_channel_queue+0x470/0x470
[ 984.404030][ T44] napi_disable+0x43/0x80
[ 984.404118][ T44] refill_work+0x116/0x210
[ 984.404238][ T44] process_one_work+0xd57/0x1390
[ 984.404358][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 984.404480][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.404603][ T44] ? assign_work+0x152/0x380
[ 984.404720][ T44] worker_thread+0x4d6/0xd40
[ 984.404845][ T44] ? process_one_work+0x1390/0x1390
[ 984.404960][ T44] ? __kthread_parkme+0xb3/0x200
[ 984.405079][ T44] ? process_one_work+0x1390/0x1390
[ 984.405194][ T44] kthread+0x355/0x5b0
[ 984.405284][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.405398][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.405521][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.405639][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.405768][ T44] ret_from_fork+0x1d4/0x270
[ 984.405882][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.405998][ T44] ret_from_fork_asm+0x11/0x20
[ 984.406120][ T44]
[ 984.406217][ T44]
[ 984.406217][ T44] Showing all locks held in the system:
[ 984.406394][ T44] 3 locks held by kworker/u16:0/12:
[ 984.406512][ T44] #0: ff1100000915cd48 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 984.406994][ T44] #1: ffa00000000c7d30 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 984.407278][ T44] #2: ffffffffa2a92a28 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30
[ 984.407489][ T44] 1 lock held by khungtaskd/44:
[ 984.407598][ T44] #0: ffffffffa2378fc0 (rcu_read_lock){....}-{1:3}, at: debug_show_all_locks+0x2b/0x1db
[ 984.407808][ T44] 3 locks held by kworker/3:1/71:
[ 984.407919][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 984.408144][ T44] #1: ffa00000004c7d30 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 984.408400][ T44] #2: ff11000008bcaca0 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80
[ 984.408608][ T44] 2 locks held by ip/6159:
[ 984.408721][ T44] #0: ffffffffa2a92a28 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x5df/0xb20
[ 984.408923][ T44] #1: ff11000008bcaca0 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80
[ 984.409124][ T44]
[ 984.409180][ T44] =============================================
[ 984.409180][ T44]
[ 988.479754][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 237568 ms
[ 988.480199][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 237568000 usecs ago
[ 994.110741][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 243199 ms
[ 994.111174][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 243199000 usecs ago
[ 999.230736][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 248319 ms
[ 999.231173][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 248319000 usecs ago
[ 1004.350741][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 2 timed out 253439 ms
[ 1004.351173][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 2, sq: output.2, vq: 0x5, name: output.2, 253439000 usecs ago