[ 678.087349][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 5504 ms
[ 678.087919][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 5504000 usecs ago
[ 683.335346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 10752 ms
[ 683.335781][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 10752000 usecs ago
[ 689.479346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 16896 ms
[ 689.479743][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 16896000 usecs ago
[ 695.111346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 22528 ms
[ 695.111736][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 22528000 usecs ago
[ 700.231346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 27648 ms
[ 700.231746][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 27648000 usecs ago
[ 705.351346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 32768 ms
[ 705.351723][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 32768000 usecs ago
[ 711.495346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 38912 ms
[ 711.495727][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 38912000 usecs ago
[ 717.127346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 44544 ms
[ 717.127725][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 44544000 usecs ago
[ 722.247525][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 49664 ms
[ 722.247912][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 49664000 usecs ago
[ 727.367346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 54784 ms
[ 727.367720][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 54784000 usecs ago
[ 733.511346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 60928 ms
[ 733.511743][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 60928000 usecs ago
[ 739.143347][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 66560 ms
[ 739.143738][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 66560000 usecs ago
[ 744.263346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 71680 ms
[ 744.263742][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 71680000 usecs ago
[ 749.383346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 76800 ms
[ 749.383759][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 76800000 usecs ago
[ 755.527346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 82944 ms
[ 755.527759][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 82944000 usecs ago
[ 761.159347][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 88576 ms
[ 761.159755][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 88576000 usecs ago
[ 766.279346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 93696 ms
[ 766.279741][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 93696000 usecs ago
[ 771.399346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 98816 ms
[ 771.399759][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 98816000 usecs ago
[ 777.543346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 104960 ms
[ 777.543774][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 104960000 usecs ago
[ 783.175347][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 110592 ms
[ 783.175779][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 110592000 usecs ago
[ 788.295346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 115712 ms
[ 788.295809][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 115712000 usecs ago
[ 793.415346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 120832 ms
[ 793.415764][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 120832000 usecs ago
[ 799.559350][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 126976 ms
[ 799.559823][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 126976000 usecs ago
[ 805.191346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 132608 ms
[ 805.191776][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 132608000 usecs ago
[ 810.311346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 137728 ms
[ 810.311759][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 137728000 usecs ago
[ 816.455346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 143872 ms
[ 816.455771][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 143872000 usecs ago
[ 822.087346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 149504 ms
[ 822.087763][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 149504000 usecs ago
[ 827.207347][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 154624 ms
[ 827.207810][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 154624000 usecs ago
[ 832.327346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 159744 ms
[ 832.327765][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 159744000 usecs ago
[ 838.471350][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 165888 ms
[ 838.471781][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 165888000 usecs ago
[ 844.103346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 171520 ms
[ 844.103798][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 171520000 usecs ago
[ 849.223346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 176640 ms
[ 849.223772][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 176640000 usecs ago
[ 854.343347][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 181760 ms
[ 854.343774][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 181760000 usecs ago
[ 860.487346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 187904 ms
[ 860.487764][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 187904000 usecs ago
[ 861.511383][ T44] INFO: task ip:6036 blocked for more than 122 seconds.
[ 861.511724][ T44] Not tainted 6.18.0-virtme #1
[ 861.511956][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 861.512246][ T44] task:ip state:D stack:25520 pid:6036 tgid:6036 ppid:6031 task_flags:0x400100 flags:0x00080000
[ 861.512621][ T44] Call Trace:
[ 861.512769][ T44]
[ 861.512885][ T44] __schedule+0xae2/0x1720
[ 861.513071][ T44] ? io_schedule_timeout+0x130/0x130
[ 861.513252][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.513452][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.513624][ T44] schedule+0xd4/0x260
[ 861.513764][ T44] schedule_preempt_disabled+0x18/0x30
[ 861.513933][ T44] __mutex_lock+0xd76/0x1f20
[ 861.514103][ T44] ? napi_enable+0x3b/0x80
[ 861.514274][ T44] ? ww_mutex_lock+0x160/0x160
[ 861.514462][ T44] ? __asan_memset+0x27/0x50
[ 861.514632][ T44] ? sg_init_table+0x19/0x60
[ 861.514801][ T44] ? try_fill_recv+0x3ec/0x1750
[ 861.514974][ T44] ? napi_enable+0x3b/0x80
[ 861.515140][ T44] napi_enable+0x3b/0x80
[ 861.515277][ T44] __virtnet_rx_resume+0x156/0x290
[ 861.515463][ T44] ? queue_delayed_work_on+0x91/0x100
[ 861.515629][ T44] virtnet_rx_resume_all+0xe7/0x1a0
[ 861.515798][ T44] virtnet_xdp_set+0xef7/0x1aa0
[ 861.515968][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0
[ 861.516129][ T44] dev_xdp_install+0x3e4/0x890
[ 861.516296][ T44] ? alloc_netdev_dummy+0x30/0x30
[ 861.516482][ T44] dev_xdp_attach+0x50d/0x1310
[ 861.516650][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0
[ 861.516814][ T44] ? __bpf_prog_get+0x320/0x320
[ 861.517111][ T44] dev_change_xdp_fd+0x176/0x210
[ 861.517280][ T44] do_setlink.isra.0+0x1c2d/0x2880
[ 861.517469][ T44] ? rtnl_newlink_create+0x900/0x900
[ 861.517633][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0
[ 861.517826][ T44] ? rcu_read_lock_any_held+0x3c/0x90
[ 861.517992][ T44] ? validate_chain+0x30f/0x8e0
[ 861.518158][ T44] ? rcu_read_lock_any_held+0x66/0x90
[ 861.518329][ T44] ? stack_depot_save_flags+0x3e1/0x6e0
[ 861.518508][ T44] ? __lock_acquire+0x577/0xc10
[ 861.518680][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 861.518843][ T44] ? rtnl_newlink+0x5f9/0xb40
[ 861.519005][ T44] ? lock_acquire+0x10a/0x150
[ 861.519170][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.519357][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.519520][ T44] ? trace_contention_end+0xd6/0x110
[ 861.519688][ T44] ? __mutex_lock+0x285/0x1f20
[ 861.519854][ T44] ? rtnl_newlink+0x5f9/0xb40
[ 861.520020][ T44] ? ww_mutex_lock+0x160/0x160
[ 861.520184][ T44] ? nla_validate_int_range+0x5b0/0x5b0
[ 861.520369][ T44] ? __rtnl_newlink+0x3fa/0xa40
[ 861.520536][ T44] rtnl_newlink+0x649/0xb40
[ 861.520704][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.520871][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 861.521036][ T44] ? __lock_acquire+0x577/0xc10
[ 861.521201][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.521383][ T44] ? find_held_lock+0x2b/0x80
[ 861.521547][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 861.521711][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.521876][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 861.522039][ T44] rtnetlink_rcv_msg+0x6fe/0xb90
[ 861.522207][ T44] ? rtnl_port_fill+0x940/0x940
[ 861.522384][ T44] ? __lock_acquire+0x577/0xc10
[ 861.522549][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.522710][ T44] ? find_held_lock+0x2b/0x80
[ 861.522877][ T44] netlink_rcv_skb+0x123/0x380
[ 861.523043][ T44] ? rtnl_port_fill+0x940/0x940
[ 861.523207][ T44] ? netlink_ack+0xcc0/0xcc0
[ 861.523390][ T44] ? netlink_deliver_tap+0xc5/0x330
[ 861.523555][ T44] ? netlink_deliver_tap+0x13f/0x330
[ 861.523720][ T44] netlink_unicast+0x4a3/0x770
[ 861.523887][ T44] ? netlink_attachskb+0x810/0x810
[ 861.524051][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.524215][ T44] ? __lock_acquire+0x577/0xc10
[ 861.524401][ T44] netlink_sendmsg+0x735/0xc60
[ 861.524567][ T44] ? netlink_unicast+0x770/0x770
[ 861.524737][ T44] ____sys_sendmsg+0x419/0x850
[ 861.524902][ T44] ? copy_msghdr_from_user+0x270/0x430
[ 861.525064][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0
[ 861.525253][ T44] ? move_addr_to_kernel+0x40/0x40
[ 861.525437][ T44] ? stack_depot_save_flags+0x3e1/0x6e0
[ 861.525600][ T44] ? rcu_read_lock_any_held+0x3c/0x90
[ 861.525770][ T44] ___sys_sendmsg+0xfd/0x180
[ 861.525933][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00
[ 861.526125][ T44] ? copy_msghdr_from_user+0x430/0x430
[ 861.526300][ T44] __sys_sendmsg+0x124/0x1c0
[ 861.526484][ T44] ? __sys_sendmsg_sock+0x20/0x20
[ 861.526648][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 861.526812][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00
[ 861.527012][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 861.527175][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.527359][ T44] do_syscall_64+0xbd/0xfc0
[ 861.527523][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 861.527712][ T44] RIP: 0033:0x7f2a669b1c5e
[ 861.527884][ T44] RSP: 002b:00007ffe39ad88e0 EFLAGS: 00000202 ORIG_RAX: 000000000000002e
[ 861.528107][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f2a669b1c5e
[ 861.528335][ T44] RDX: 0000000000000000 RSI: 00007ffe39ad89a0 RDI: 0000000000000003
[ 861.528571][ T44] RBP: 00007ffe39ad88f0 R08: 0000000000000000 R09: 0000000000000000
[ 861.528790][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffe39ad9060
[ 861.529009][ T44] R13: 000000006937dc6d R14: 0000000000000000 R15: 0000000000499600
[ 861.529239][ T44]
[ 861.529393][ T44] INFO: task ip:6036 is blocked on a mutex likely owned by task kworker/3:1:69.
[ 861.529631][ T44] task:kworker/3:1 state:D stack:28032 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 861.529932][ T44] Workqueue: events refill_work
[ 861.530097][ T44] Call Trace:
[ 861.530234][ T44]
[ 861.530363][ T44] __schedule+0xae2/0x1720
[ 861.530550][ T44] ? clockevents_program_event+0xee/0x2f0
[ 861.530715][ T44] ? usleep_range_state+0xba/0x1b0
[ 861.530879][ T44] ? io_schedule_timeout+0x130/0x130
[ 861.531045][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.531211][ T44] ? usleep_range_state+0xba/0x1b0
[ 861.531389][ T44] schedule+0xd4/0x260
[ 861.531528][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0
[ 861.531722][ T44] ? schedule_timeout_idle+0xe0/0xe0
[ 861.531888][ T44] ? __hrtimer_cb_get_time+0x50/0x50
[ 861.532058][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 861.532218][ T44] ? ktime_get+0x15e/0x220
[ 861.532404][ T44] ? kvm_clock_get_cycles+0x18/0x30
[ 861.532571][ T44] ? ktime_get+0xa9/0x220
[ 861.532708][ T44] usleep_range_state+0x133/0x1b0
[ 861.532873][ T44] ? schedule_hrtimeout+0x20/0x20
[ 861.533037][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 861.533200][ T44] ? refill_work+0x149/0x210
[ 861.533387][ T44] napi_disable_locked+0x379/0x5f0
[ 861.533550][ T44] ? _mutex_trylock_nest_lock+0x340/0x340
[ 861.533714][ T44] ? netdev_bind_sb_channel_queue+0x470/0x470
[ 861.533911][ T44] napi_disable+0x43/0x80
[ 861.534048][ T44] refill_work+0x116/0x210
[ 861.534213][ T44] process_one_work+0xd57/0x1390
[ 861.534401][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 861.534563][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 861.534729][ T44] ? assign_work+0x152/0x380
[ 861.534893][ T44] worker_thread+0x4d6/0xd40
[ 861.535065][ T44] ? process_one_work+0x1390/0x1390
[ 861.535229][ T44] kthread+0x355/0x5b0
[ 861.535387][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.535551][ T44] ? __lock_release.isra.0+0x59/0x170
[ 861.535716][ T44] ? rcu_is_watching+0x15/0xd0
[ 861.535881][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.536045][ T44] ret_from_fork+0x3fb/0x510
[ 861.536211][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 861.536425][ T44] ? __switch_to+0x602/0xd00
[ 861.536590][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 861.536756][ T44] ret_from_fork_asm+0x11/0x20
[ 861.536926][ T44]
[ 861.537060][ T44]
[ 861.537060][ T44] Showing all locks held in the system:
[ 861.537275][ T44] 3 locks held by kworker/u16:0/12:
[ 861.537446][ T44] #0: ff11000008a76548 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 861.537719][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 861.538041][ T44] #2: ffffffff9bc78aa8 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30
[ 861.538286][ T44] 1 lock held by khungtaskd/44:
[ 861.538467][ T44] #0: ffffffff9b570ba0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 861.538736][ T44] 3 locks held by kworker/3:1/69:
[ 861.538892][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 861.539159][ T44] #1: ffa00000004a7cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 861.539475][ T44] #2: ff11000008492c98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80
[ 861.539727][ T44] 2 locks held by ip/6036:
[ 861.539881][ T44] #0: ffffffff9bc78aa8 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x5f9/0xb40
[ 861.540124][ T44] #1: ff11000008492c98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80
[ 861.540384][ T44]
[ 861.540487][ T44] =============================================
[ 861.540487][ T44]
[ 866.119353][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 193536 ms
[ 866.119765][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 193536000 usecs ago
[ 871.239351][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 198656 ms
[ 871.239821][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 198656000 usecs ago
[ 876.359350][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 203776 ms
[ 876.359767][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 203776000 usecs ago
[ 882.503346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 209920 ms
[ 882.503780][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 209920000 usecs ago
[ 888.135346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 215552 ms
[ 888.135771][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 215552000 usecs ago
[ 893.255346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 220672 ms
[ 893.255810][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 220672000 usecs ago
[ 898.376348][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 225793 ms
[ 898.376800][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 225793000 usecs ago
[ 904.519346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 231936 ms
[ 904.519777][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 231936000 usecs ago
[ 910.151346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 237568 ms
[ 910.151768][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 237568000 usecs ago
[ 915.271346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 242688 ms
[ 915.271763][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 242688000 usecs ago
[ 920.391346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 247808 ms
[ 920.391798][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 247808000 usecs ago
[ 926.535346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 253952 ms
[ 926.535764][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 253952000 usecs ago
[ 932.167346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 259584 ms
[ 932.167766][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 259584000 usecs ago
[ 937.288348][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 264705 ms
[ 937.288767][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 264705000 usecs ago
[ 942.407346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 269824 ms
[ 942.407765][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 269824000 usecs ago
[ 948.551347][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 275968 ms
[ 948.551774][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 275968000 usecs ago
[ 954.183346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 281600 ms
[ 954.183769][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 281600000 usecs ago
[ 959.303346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 286720 ms
[ 959.303775][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 286720000 usecs ago
[ 964.423346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 291840 ms
[ 964.423799][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 291840000 usecs ago
[ 970.568345][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 297985 ms
[ 970.568742][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 297985000 usecs ago
[ 976.199346][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 303616 ms
[ 976.199802][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 303616000 usecs ago
[ 981.320348][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 308737 ms
[ 981.320778][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 308737000 usecs ago
[ 984.392353][ T44] INFO: task kworker/u16:0:12 blocked for more than 122 seconds.
[ 984.392652][ T44] Not tainted 6.18.0-virtme #1
[ 984.392836][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 984.393121][ T44] task:kworker/u16:0 state:D stack:26240 pid:12 tgid:12 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 984.393513][ T44] Workqueue: ipv6_addrconf addrconf_verify_work
[ 984.393729][ T44] Call Trace:
[ 984.393870][ T44]
[ 984.393982][ T44] __schedule+0xae2/0x1720
[ 984.394158][ T44] ? io_schedule_timeout+0x130/0x130
[ 984.394331][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.394529][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.394703][ T44] schedule+0xd4/0x260
[ 984.394842][ T44] schedule_preempt_disabled+0x18/0x30
[ 984.395007][ T44] __mutex_lock+0xd76/0x1f20
[ 984.395178][ T44] ? addrconf_verify_work+0x12/0x30
[ 984.395360][ T44] ? ww_mutex_lock+0x160/0x160
[ 984.395537][ T44] ? process_one_work+0xd16/0x1390
[ 984.395709][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.395872][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.396039][ T44] ? addrconf_verify_work+0x12/0x30
[ 984.396202][ T44] addrconf_verify_work+0x12/0x30
[ 984.396384][ T44] process_one_work+0xd57/0x1390
[ 984.396560][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 984.396724][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.396892][ T44] ? assign_work+0x152/0x380
[ 984.397056][ T44] worker_thread+0x4d6/0xd40
[ 984.397221][ T44] ? process_one_work+0x1390/0x1390
[ 984.397398][ T44] kthread+0x355/0x5b0
[ 984.397542][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.397705][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.397869][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.398031][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.398198][ T44] ret_from_fork+0x3fb/0x510
[ 984.398378][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 984.398572][ T44] ? __switch_to+0x602/0xd00
[ 984.398736][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.398900][ T44] ret_from_fork_asm+0x11/0x20
[ 984.399068][ T44]
[ 984.399212][ T44] INFO: task kworker/u16:0:12 is blocked on a mutex likely owned by task ip:6036.
[ 984.399486][ T44] INFO: task ip:6036 blocked for more than 245 seconds.
[ 984.399667][ T44] Not tainted 6.18.0-virtme #1
[ 984.399823][ T44] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 984.400051][ T44] task:ip state:D stack:25520 pid:6036 tgid:6036 ppid:6031 task_flags:0x400100 flags:0x00080000
[ 984.400371][ T44] Call Trace:
[ 984.400514][ T44]
[ 984.400622][ T44] __schedule+0xae2/0x1720
[ 984.400789][ T44] ? io_schedule_timeout+0x130/0x130
[ 984.400952][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.401115][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.401279][ T44] schedule+0xd4/0x260
[ 984.401430][ T44] schedule_preempt_disabled+0x18/0x30
[ 984.401606][ T44] __mutex_lock+0xd76/0x1f20
[ 984.401770][ T44] ? napi_enable+0x3b/0x80
[ 984.401933][ T44] ? ww_mutex_lock+0x160/0x160
[ 984.402098][ T44] ? __asan_memset+0x27/0x50
[ 984.402262][ T44] ? sg_init_table+0x19/0x60
[ 984.402441][ T44] ? try_fill_recv+0x3ec/0x1750
[ 984.402610][ T44] ? napi_enable+0x3b/0x80
[ 984.402774][ T44] napi_enable+0x3b/0x80
[ 984.402915][ T44] __virtnet_rx_resume+0x156/0x290
[ 984.403081][ T44] ? queue_delayed_work_on+0x91/0x100
[ 984.403247][ T44] virtnet_rx_resume_all+0xe7/0x1a0
[ 984.403422][ T44] virtnet_xdp_set+0xef7/0x1aa0
[ 984.403596][ T44] ? virtnet_xdp_set+0x1aa0/0x1aa0
[ 984.403757][ T44] dev_xdp_install+0x3e4/0x890
[ 984.403923][ T44] ? alloc_netdev_dummy+0x30/0x30
[ 984.404094][ T44] dev_xdp_attach+0x50d/0x1310
[ 984.404262][ T44] ? bpf_xdp_link_update+0x6e0/0x6e0
[ 984.404438][ T44] ? __bpf_prog_get+0x320/0x320
[ 984.404607][ T44] dev_change_xdp_fd+0x176/0x210
[ 984.404772][ T44] do_setlink.isra.0+0x1c2d/0x2880
[ 984.404937][ T44] ? rtnl_newlink_create+0x900/0x900
[ 984.405100][ T44] ? rcu_lockdep_current_cpu_online+0x39/0x1b0
[ 984.405296][ T44] ? rcu_read_lock_any_held+0x3c/0x90
[ 984.405479][ T44] ? validate_chain+0x30f/0x8e0
[ 984.405641][ T44] ? rcu_read_lock_any_held+0x66/0x90
[ 984.405803][ T44] ? stack_depot_save_flags+0x3e1/0x6e0
[ 984.405965][ T44] ? __lock_acquire+0x577/0xc10
[ 984.406132][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 984.406295][ T44] ? rtnl_newlink+0x5f9/0xb40
[ 984.406480][ T44] ? lock_acquire+0x10a/0x150
[ 984.406642][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.406803][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.406965][ T44] ? trace_contention_end+0xd6/0x110
[ 984.407129][ T44] ? __mutex_lock+0x285/0x1f20
[ 984.407293][ T44] ? rtnl_newlink+0x5f9/0xb40
[ 984.407477][ T44] ? ww_mutex_lock+0x160/0x160
[ 984.407639][ T44] ? nla_validate_int_range+0x5b0/0x5b0
[ 984.407810][ T44] ? __rtnl_newlink+0x3fa/0xa40
[ 984.407976][ T44] rtnl_newlink+0x649/0xb40
[ 984.408143][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.408306][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 984.408486][ T44] ? __lock_acquire+0x577/0xc10
[ 984.408652][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.408814][ T44] ? find_held_lock+0x2b/0x80
[ 984.408976][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 984.409144][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.409307][ T44] ? __rtnl_newlink+0xa40/0xa40
[ 984.409487][ T44] rtnetlink_rcv_msg+0x6fe/0xb90
[ 984.409652][ T44] ? rtnl_port_fill+0x940/0x940
[ 984.409815][ T44] ? __lock_acquire+0x577/0xc10
[ 984.409979][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.410141][ T44] ? find_held_lock+0x2b/0x80
[ 984.410309][ T44] netlink_rcv_skb+0x123/0x380
[ 984.410488][ T44] ? rtnl_port_fill+0x940/0x940
[ 984.410651][ T44] ? netlink_ack+0xcc0/0xcc0
[ 984.410818][ T44] ? netlink_deliver_tap+0xc5/0x330
[ 984.410979][ T44] ? netlink_deliver_tap+0x13f/0x330
[ 984.411144][ T44] netlink_unicast+0x4a3/0x770
[ 984.411311][ T44] ? netlink_attachskb+0x810/0x810
[ 984.411490][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.411657][ T44] ? __lock_acquire+0x577/0xc10
[ 984.411821][ T44] netlink_sendmsg+0x735/0xc60
[ 984.411986][ T44] ? netlink_unicast+0x770/0x770
[ 984.412151][ T44] ____sys_sendmsg+0x419/0x850
[ 984.412314][ T44] ? copy_msghdr_from_user+0x270/0x430
[ 984.412496][ T44] ? get_timestamp.constprop.0+0x3a0/0x3a0
[ 984.412685][ T44] ? move_addr_to_kernel+0x40/0x40
[ 984.412854][ T44] ? stack_depot_save_flags+0x3e1/0x6e0
[ 984.413015][ T44] ? rcu_read_lock_any_held+0x3c/0x90
[ 984.413180][ T44] ___sys_sendmsg+0xfd/0x180
[ 984.413356][ T44] ? __call_rcu_common.constprop.0+0xa6/0xa00
[ 984.413551][ T44] ? copy_msghdr_from_user+0x430/0x430
[ 984.413725][ T44] __sys_sendmsg+0x124/0x1c0
[ 984.413890][ T44] ? __sys_sendmsg_sock+0x20/0x20
[ 984.414055][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 984.414216][ T44] ? __call_rcu_common.constprop.0+0x3bd/0xa00
[ 984.414421][ T44] ? do_raw_spin_unlock+0x59/0x250
[ 984.414592][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.414756][ T44] do_syscall_64+0xbd/0xfc0
[ 984.414921][ T44] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 984.415111][ T44] RIP: 0033:0x7f2a669b1c5e
[ 984.415282][ T44] RSP: 002b:00007ffe39ad88e0 EFLAGS: 00000202 ORIG_RAX: 000000000000002e
[ 984.415522][ T44] RAX: ffffffffffffffda RBX: 0000000000000007 RCX: 00007f2a669b1c5e
[ 984.415740][ T44] RDX: 0000000000000000 RSI: 00007ffe39ad89a0 RDI: 0000000000000003
[ 984.415957][ T44] RBP: 00007ffe39ad88f0 R08: 0000000000000000 R09: 0000000000000000
[ 984.416178][ T44] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ffe39ad9060
[ 984.416411][ T44] R13: 000000006937dc6d R14: 0000000000000000 R15: 0000000000499600
[ 984.416640][ T44]
[ 984.416777][ T44] INFO: task ip:6036 is blocked on a mutex likely owned by task kworker/3:1:69.
[ 984.417010][ T44] task:kworker/3:1 state:D stack:28032 pid:69 tgid:69 ppid:2 task_flags:0x4208060 flags:0x00080000
[ 984.417311][ T44] Workqueue: events refill_work
[ 984.417490][ T44] Call Trace:
[ 984.417625][ T44]
[ 984.417733][ T44] __schedule+0xae2/0x1720
[ 984.417897][ T44] ? usleep_range_state+0xba/0x1b0
[ 984.418062][ T44] ? io_schedule_timeout+0x130/0x130
[ 984.418226][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.418409][ T44] ? usleep_range_state+0xba/0x1b0
[ 984.418574][ T44] schedule+0xd4/0x260
[ 984.418709][ T44] schedule_hrtimeout_range_clock+0x1da/0x2f0
[ 984.418902][ T44] ? schedule_timeout_idle+0xe0/0xe0
[ 984.419066][ T44] ? __hrtimer_cb_get_time+0x50/0x50
[ 984.419229][ T44] ? lockdep_hardirqs_on+0x84/0x130
[ 984.419403][ T44] ? ktime_get+0x15e/0x220
[ 984.419572][ T44] ? kvm_clock_get_cycles+0x18/0x30
[ 984.419735][ T44] ? ktime_get+0xa9/0x220
[ 984.419872][ T44] usleep_range_state+0x133/0x1b0
[ 984.420035][ T44] ? schedule_hrtimeout+0x20/0x20
[ 984.420198][ T44] ? perf_trace_sched_switch+0x6d0/0x6d0
[ 984.420373][ T44] ? refill_work+0x149/0x210
[ 984.420542][ T44] napi_disable_locked+0x379/0x5f0
[ 984.420710][ T44] ? _mutex_trylock_nest_lock+0x340/0x340
[ 984.420875][ T44] ? netdev_bind_sb_channel_queue+0x470/0x470
[ 984.421072][ T44] napi_disable+0x43/0x80
[ 984.421209][ T44] refill_work+0x116/0x210
[ 984.421391][ T44] process_one_work+0xd57/0x1390
[ 984.421565][ T44] ? pwq_dec_nr_in_flight+0x700/0x700
[ 984.421729][ T44] ? lock_acquire.part.0+0xbc/0x260
[ 984.421899][ T44] ? assign_work+0x152/0x380
[ 984.422063][ T44] worker_thread+0x4d6/0xd40
[ 984.422231][ T44] ? process_one_work+0x1390/0x1390
[ 984.422407][ T44] kthread+0x355/0x5b0
[ 984.422545][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.422708][ T44] ? __lock_release.isra.0+0x59/0x170
[ 984.422872][ T44] ? rcu_is_watching+0x15/0xd0
[ 984.423035][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.423201][ T44] ret_from_fork+0x3fb/0x510
[ 984.423381][ T44] ? arch_exit_to_user_mode_prepare.isra.0+0x140/0x140
[ 984.423575][ T44] ? __switch_to+0x602/0xd00
[ 984.423739][ T44] ? kthread_is_per_cpu+0xe0/0xe0
[ 984.423901][ T44] ret_from_fork_asm+0x11/0x20
[ 984.424070][ T44]
[ 984.424202][ T44]
[ 984.424202][ T44] Showing all locks held in the system:
[ 984.424429][ T44] 3 locks held by kworker/u16:0/12:
[ 984.424592][ T44] #0: ff11000008a76548 ((wq_completion)ipv6_addrconf){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 984.424859][ T44] #1: ffa00000000c7cc0 ((work_completion)(&(&net->ipv6.addr_chk_work)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 984.425176][ T44] #2: ffffffff9bc78aa8 (rtnl_mutex){+.+.}-{4:4}, at: addrconf_verify_work+0x12/0x30
[ 984.425440][ T44] 1 lock held by khungtaskd/44:
[ 984.425600][ T44] #0: ffffffff9b570ba0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 984.425868][ T44] 3 locks held by kworker/3:1/69:
[ 984.426022][ T44] #0: ff11000001139b48 ((wq_completion)events){+.+.}-{0:0}, at: process_one_work+0xcb4/0x1390
[ 984.426288][ T44] #1: ffa00000004a7cc0 ((work_completion)(&(&vi->refill)->work)){+.+.}-{0:0}, at: process_one_work+0xd16/0x1390
[ 984.426602][ T44] #2: ff11000008492c98 (&dev->lock){+.+.}-{4:4}, at: napi_disable+0x3b/0x80
[ 984.426851][ T44] 2 locks held by ip/6036:
[ 984.427008][ T44] #0: ffffffff9bc78aa8 (rtnl_mutex){+.+.}-{4:4}, at: rtnl_newlink+0x5f9/0xb40
[ 984.427250][ T44] #1: ff11000008492c98 (&dev->lock){+.+.}-{4:4}, at: napi_enable+0x3b/0x80
[ 984.427518][ T44]
[ 984.427621][ T44] =============================================
[ 984.427621][ T44]
[ 987.463347][ C2] virtio_net virtio2 enp0s1: NETDEV WATCHDOG: CPU: 2: transmit queue 1 timed out 314880 ms
[ 987.463797][ C2] virtio_net virtio2 enp0s1: TX timeout on queue: 1, sq: output.1, vq: 0x3, name: output.1, 314880000 usecs ago