Oct 22 04:04:06 hw1 systemd[1]: Starting nipa-hw-worker.service - NIPA HW worker (one-shot test runner)...
Oct 22 04:04:06 hw1 python3[1167]: /srv/hw-worker/tests/5189 Starting tests
Oct 22 04:04:06 hw1 python3[1167]: Set ens1f0np0 combined channels to 32 (max=32, cpus=32)
Oct 22 04:04:06 hw1 python3[1167]: IRQ mapping: IRQ 186 -> CPU 0 ... IRQ 193 -> CPU 7
Oct 22 04:04:06 hw1 python3[1167]: Restored ens1f0np0 combined channels to 8
Oct 22 04:04:06 hw1 python3[1167]: Moved ens1f1np1 to netns nipa-peer
Oct 22 04:04:06 hw1 python3[1167]: Peer 10.88.0.2 reachable after 1s
Oct 22 04:04:09 hw1 python3[1167]: Peer fd00::2 reachable after 2s
Oct 22 04:04:09 hw1 python3[1167]: Wrote /srv/hw-worker/tests/5189/drivers/net/net.config
Oct 22 04:04:09 hw1 python3[1167]: Wrote /srv/hw-worker/tests/5189/drivers/net/hw/net.config
Oct 22 04:04:09 hw1 python3[1167]: Collected device info for ens1f0np0
Oct 22 04:04:09 hw1 python3[1167]: Found 53 tests, per-test timeout 1200s
Oct 22 04:04:09 hw1 python3[1167]: Saved 2892 lines of boot dmesg
Oct 22 04:04:09 hw1 python3[1167]: [1/53] Running drivers/net:gro_hw.py
Oct 22 04:04:09 hw1 python3[1167]: [1/53] drivers/net:gro_hw.py: rc=0 (0.6s)
Oct 22 04:04:09 hw1 python3[1167]: [2/53] Running drivers/net:gro_lro.py
Oct 22 04:04:10 hw1 python3[1167]: [2/53] drivers/net:gro_lro.py: rc=0 (0.6s)
Oct 22 04:04:10 hw1 python3[1167]: [3/53] Running drivers/net:gro_sw.py
Oct 22 04:04:23 hw1 python3[1167]: [3/53] drivers/net:gro_sw.py: all 3 failed cases match known bad, skipping retry
Oct 22 04:04:23 hw1 python3[1167]: [3/53] drivers/net:gro_sw.py: rc=1 (10.0s)
Oct 22 04:04:23 hw1 python3[1167]: [4/53] Running drivers/net:hds.py
Oct 22 04:04:24 hw1 python3[1167]: [4/53] drivers/net:hds.py: rc=0 (0.8s)
Oct 22 04:04:24 hw1 python3[1167]: [5/53] Running drivers/net:macsec.py
Oct 22 04:04:24 hw1 python3[1167]: [5/53] drivers/net:macsec.py: rc=0 (0.3s)
Oct 22 04:04:24 hw1 python3[1167]: [6/53] Running drivers/net:napi_id.py
Oct 22 04:04:25 hw1 python3[1167]: [6/53] drivers/net:napi_id.py: rc=0 (0.6s)
Oct 22 04:04:25 hw1 python3[1167]: [7/53] Running drivers/net:napi_threaded.py
Oct 22 04:04:25 hw1 python3[1167]: [7/53] drivers/net:napi_threaded.py: rc=0 (0.2s)
Oct 22 04:04:25 hw1 python3[1167]: [8/53] Running drivers/net:netpoll_basic.py
Oct 22 04:04:26 hw1 python3[1167]: [8/53] drivers/net:netpoll_basic.py: all 1 failed cases match known bad, skipping retry
Oct 22 04:04:26 hw1 python3[1167]: [8/53] drivers/net:netpoll_basic.py: rc=1 (1.0s)
Oct 22 04:04:26 hw1 python3[1167]: [9/53] Running drivers/net:ping.py
Oct 22 04:05:07 hw1 python3[1167]: [9/53] drivers/net:ping.py: rc=0 (41.4s)
Oct 22 04:05:07 hw1 python3[1167]: [10/53] Running drivers/net:pppoe_gro.py
Oct 22 04:05:08 hw1 python3[1167]: [10/53] drivers/net:pppoe_gro.py: rc=0 (0.6s)
Oct 22 04:05:08 hw1 python3[1167]: [11/53] Running drivers/net:psp.py
Oct 22 04:05:09 hw1 python3[1167]: [11/53] drivers/net:psp.py: rc=0 (0.4s)
Oct 22 04:05:09 hw1 python3[1167]: [12/53] Running drivers/net:queues.py
Oct 22 04:05:09 hw1 python3[1167]: [12/53] drivers/net:queues.py: rc=0 (0.1s)
Oct 22 04:05:09 hw1 python3[1167]: [13/53] Running drivers/net:ring_reconfig.py
Oct 22 04:05:13 hw1 python3[1167]: [13/53] drivers/net:ring_reconfig.py: rc=0 (4.8s)
Oct 22 04:05:13 hw1 python3[1167]: [14/53] Running drivers/net:rss_key.py
Oct 22 04:05:14 hw1 python3[1167]: [14/53] drivers/net:rss_key.py: rc=0 (0.1s)
Oct 22 04:05:14 hw1 python3[1167]: [15/53] Running drivers/net:shaper.py
Oct 22 04:05:14 hw1 python3[1167]: [15/53] drivers/net:shaper.py: rc=0 (0.1s)
Oct 22 04:05:14 hw1 python3[1167]: [16/53] Running drivers/net:so_txtime.py
Oct 22 04:05:24 hw1 python3[1167]: [16/53] drivers/net:so_txtime.py: rc=0 (10.7s)
Oct 22 04:05:24 hw1 python3[1167]: [17/53] Running drivers/net:stats.py
Oct 22 04:05:38 hw1 python3[1167]: [17/53] drivers/net:stats.py: rc=0 (13.2s)
Oct 22 04:05:38 hw1 python3[1167]: [18/53] Running drivers/net:xdp.py
Oct 22 04:10:03 hw1 python3[1167]: [18/53] drivers/net:xdp.py: all 6 failed cases match known bad, skipping retry
Oct 22 04:10:03 hw1 python3[1167]: [18/53] drivers/net:xdp.py: rc=1 (265.5s)
Oct 22 04:10:03 hw1 python3[1167]: [19/53] Running drivers/net/hw:csum.py
Oct 22 04:10:23 hw1 python3[1167]: [19/53] drivers/net/hw:csum.py: rc=0 (20.2s)
Oct 22 04:10:23 hw1 python3[1167]: [20/53] Running drivers/net/hw:devlink_rate_cross_esw.py
Oct 22 04:10:23 hw1 python3[1167]: [20/53] drivers/net/hw:devlink_rate_cross_esw.py: rc=0 (0.1s)
Oct 22 04:10:23 hw1 python3[1167]: [21/53] Running drivers/net/hw:devlink_rate_tc_bw.py
Oct 22 04:10:23 hw1 python3[1167]: [21/53] drivers/net/hw:devlink_rate_tc_bw.py: rc=0 (0.1s)
Oct 22 04:10:23 hw1 python3[1167]: [22/53] Running drivers/net/hw:devmem.py
Oct 22 04:10:24 hw1 python3[1167]: [22/53] drivers/net/hw:devmem.py: rc=0 (0.2s)
Oct 22 04:10:24 hw1 python3[1167]: [23/53] Running drivers/net/hw:ethtool.sh
Oct 22 04:10:24 hw1 python3[1167]: [23/53] drivers/net/hw:ethtool.sh: no failed subcases found, retrying
Oct 22 04:10:24 hw1 python3[1167]: [23/53] drivers/net/hw:ethtool.sh: retry rc=1
Oct 22 04:10:24 hw1 python3[1167]: [23/53] drivers/net/hw:ethtool.sh: rc=1 (0.0s)
Oct 22 04:10:24 hw1 python3[1167]: [24/53] Running drivers/net/hw:ethtool_extended_state.sh
Oct 22 04:10:24 hw1 python3[1167]: [24/53] drivers/net/hw:ethtool_extended_state.sh: no failed subcases found, retrying
Oct 22 04:10:24 hw1 python3[1167]: [24/53] drivers/net/hw:ethtool_extended_state.sh: retry rc=1
Oct 22 04:10:24 hw1 python3[1167]: [24/53] drivers/net/hw:ethtool_extended_state.sh: rc=1 (0.0s)
Oct 22 04:10:24 hw1 python3[1167]: [25/53] Running drivers/net/hw:ethtool_mm.sh
Oct 22 04:10:24 hw1 python3[1167]: [25/53] drivers/net/hw:ethtool_mm.sh: no failed subcases found, retrying
Oct 22 04:10:24 hw1 python3[1167]: [25/53] drivers/net/hw:ethtool_mm.sh: retry rc=1
Oct 22 04:10:24 hw1 python3[1167]: [25/53] drivers/net/hw:ethtool_mm.sh: rc=1 (0.0s)
Oct 22 04:10:24 hw1 python3[1167]: [26/53] Running drivers/net/hw:ethtool_rmon.sh
Oct 22 04:10:35 hw1 python3[1167]: [26/53] drivers/net/hw:ethtool_rmon.sh: all 2 failed cases match known bad, skipping retry
Oct 22 04:10:35 hw1 python3[1167]: [26/53] drivers/net/hw:ethtool_rmon.sh: rc=1 (10.9s)
Oct 22 04:10:35 hw1 python3[1167]: [27/53] Running drivers/net/hw:ethtool_std_stats.sh
Oct 22 04:10:44 hw1 python3[1167]: [27/53] drivers/net/hw:ethtool_std_stats.sh: all 9 failed cases match known bad, skipping retry
Oct 22 04:10:44 hw1 python3[1167]: [27/53] drivers/net/hw:ethtool_std_stats.sh: rc=1 (9.2s)
Oct 22 04:10:44 hw1 python3[1167]: [28/53] Running drivers/net/hw:gro_stats.py
Oct 22 04:10:44 hw1 python3[1167]: [28/53] drivers/net/hw:gro_stats.py: rc=0 (0.1s)
Oct 22 04:10:44 hw1 python3[1167]: [29/53] Running drivers/net/hw:hw_stats_l3.sh
Oct 22 04:10:44 hw1 python3[1167]: [29/53] drivers/net/hw:hw_stats_l3.sh: no failed subcases found, retrying
Oct 22 04:10:44 hw1 python3[1167]: [29/53] drivers/net/hw:hw_stats_l3.sh: retry rc=1
Oct 22 04:10:44 hw1 python3[1167]: [29/53] drivers/net/hw:hw_stats_l3.sh: rc=1 (0.0s)
Oct 22 04:10:44 hw1 python3[1167]: [30/53] Running drivers/net/hw:hw_stats_l3_gre.sh
Oct 22 04:10:44 hw1 python3[1167]: [30/53] drivers/net/hw:hw_stats_l3_gre.sh: no failed subcases found, retrying
Oct 22 04:10:44 hw1 python3[1167]: [30/53] drivers/net/hw:hw_stats_l3_gre.sh: retry rc=1
Oct 22 04:10:44 hw1 python3[1167]: [30/53] drivers/net/hw:hw_stats_l3_gre.sh: rc=1 (0.0s)
Oct 22 04:10:44 hw1 python3[1167]: [31/53] Running drivers/net/hw:iou-zcrx.py
Oct 22 04:10:44 hw1 python3[1167]: [31/53] drivers/net/hw:iou-zcrx.py: all 6 failed cases match known bad, skipping retry
Oct 22 04:10:44 hw1 python3[1167]: [31/53] drivers/net/hw:iou-zcrx.py: rc=1 (0.1s)
Oct 22 04:10:44 hw1 python3[1167]: [32/53] Running drivers/net/hw:ipsec_vxlan.py
Oct 22 04:10:44 hw1 python3[1167]: [32/53] drivers/net/hw:ipsec_vxlan.py: rc=0 (0.1s)
Oct 22 04:10:44 hw1 python3[1167]: [33/53] Running drivers/net/hw:irq.py
Oct 22 04:10:45 hw1 python3[1167]: [33/53] drivers/net/hw:irq.py: rc=0 (0.5s)
Oct 22 04:10:45 hw1 python3[1167]: [34/53] Running drivers/net/hw:loopback.sh
Oct 22 04:10:45 hw1 python3[1167]: [34/53] drivers/net/hw:loopback.sh: no failed subcases found, retrying
Oct 22 04:10:45 hw1 python3[1167]: [34/53] drivers/net/hw:loopback.sh: retry rc=1
Oct 22 04:10:45 hw1 python3[1167]: [34/53] drivers/net/hw:loopback.sh: rc=1 (0.0s)
Oct 22 04:10:45 hw1 python3[1167]: [35/53] Running drivers/net/hw:nic_timestamp.py
Oct 22 04:10:45 hw1 python3[1167]: [35/53] drivers/net/hw:nic_timestamp.py: rc=0 (0.1s)
Oct 22 04:10:45 hw1 python3[1167]: [36/53] Running drivers/net/hw:nk_devmem.py
Oct 22 04:10:45 hw1 python3[1167]: [36/53] drivers/net/hw:nk_devmem.py: rc=0 (0.5s)
Oct 22 04:10:45 hw1 python3[1167]: [37/53] Running drivers/net/hw:nk_netns.py
Oct 22 04:10:47 hw1 python3[1167]: [37/53] drivers/net/hw:nk_netns.py: rc=0 (1.3s)
Oct 22 04:10:47 hw1 python3[1167]: [38/53] Running drivers/net/hw:nk_qlease.py
Oct 22 04:10:48 hw1 python3[1167]: [38/53] drivers/net/hw:nk_qlease.py: all 5 failed cases match known bad, skipping retry
Oct 22 04:10:48 hw1 python3[1167]: [38/53] drivers/net/hw:nk_qlease.py: rc=1 (1.3s)
Oct 22 04:10:48 hw1 python3[1167]: [39/53] Running drivers/net/hw:ntuple.py
Oct 22 04:10:49 hw1 python3[1167]: [39/53] drivers/net/hw:ntuple.py: all 24 failed cases match known bad, skipping retry
Oct 22 04:10:49 hw1 python3[1167]: [39/53] drivers/net/hw:ntuple.py: rc=1 (0.8s)
Oct 22 04:10:49 hw1 python3[1167]: [40/53] Running drivers/net/hw:pause.py
Oct 22 04:11:12 hw1 python3[1167]: [40/53] drivers/net/hw:pause.py: all 12 failed cases match known bad, skipping retry
Oct 22 04:11:12 hw1 python3[1167]: [40/53] drivers/net/hw:pause.py: rc=1 (23.3s)
Oct 22 04:11:12 hw1 python3[1167]: [41/53] Running drivers/net/hw:pp_alloc_fail.py
Oct 22 04:11:13 hw1 python3[1167]: [41/53] drivers/net/hw:pp_alloc_fail.py: rc=0 (0.5s)
Oct 22 04:11:13 hw1 python3[1167]: [42/53] Running drivers/net/hw:rss_api.py
Oct 22 04:11:13 hw1 python3[1167]: [42/53] drivers/net/hw:rss_api.py: all 4 failed cases match known bad, skipping retry
Oct 22 04:11:13 hw1 python3[1167]: [42/53] drivers/net/hw:rss_api.py: rc=1 (0.4s)
Oct 22 04:11:13 hw1 python3[1167]: [43/53] Running drivers/net/hw:rss_ctx.py
Oct 22 04:11:34 hw1 python3[1167]: [43/53] drivers/net/hw:rss_ctx.py: case rss-ctx-test-hitless-key-update not found in known bad, retrying
Oct 22 04:11:35 hw1 python3[1167]: [43/53] drivers/net/hw:rss_ctx.py: retry rc=1
Oct 22 04:11:35 hw1 python3[1167]: [43/53] drivers/net/hw:rss_ctx.py: rc=1 (20.9s)
Oct 22 04:11:35 hw1 python3[1167]: [44/53] Running drivers/net/hw:rss_drv.py
Oct 22 04:11:36 hw1 python3[1167]: [44/53] drivers/net/hw:rss_drv.py: rc=0 (0.8s)
Oct 22 04:11:36 hw1 python3[1167]: [45/53] Running drivers/net/hw:rss_flow_label.py
Oct 22 04:11:37 hw1 python3[1167]: [45/53] drivers/net/hw:rss_flow_label.py: all 1 failed cases match known bad, skipping retry
Oct 22 04:11:37 hw1 python3[1167]: [45/53] drivers/net/hw:rss_flow_label.py: rc=1 (1.3s)
Oct 22 04:11:37 hw1 python3[1167]: [46/53] Running drivers/net/hw:rss_input_xfrm.py
Oct 22 04:11:37 hw1 python3[1167]: [46/53] drivers/net/hw:rss_input_xfrm.py: rc=0 (0.1s)
Oct 22 04:11:37 hw1 python3[1167]: [47/53] Running drivers/net/hw:toeplitz.py
Oct 22 04:12:03 hw1 python3[1167]: [47/53] drivers/net/hw:toeplitz.py: rc=0 (26.0s)
Oct 22 04:12:03 hw1 python3[1167]: [48/53] Running drivers/net/hw:tso.py
Oct 22 04:12:03 hw1 python3[19582]: detected unhandled Python exception in './tso.py'
Oct 22 04:12:03 hw1 python3[1167]: [48/53] drivers/net/hw:tso.py: no failed subcases found, retrying
Oct 22 04:12:03 hw1 python3[19647]: detected unhandled Python exception in './tso.py'
Oct 22 04:12:03 hw1 python3[1167]: [48/53] drivers/net/hw:tso.py: retry rc=1
Oct 22 04:12:03 hw1 python3[1167]: [48/53] drivers/net/hw:tso.py: rc=1 (0.1s)
Oct 22 04:12:03 hw1 python3[1167]: [49/53] Running drivers/net/hw:userns_devmem.py
Oct 22 04:12:04 hw1 python3[1167]: [49/53] drivers/net/hw:userns_devmem.py: rc=0 (0.6s)
Oct 22 04:12:04 hw1 python3[1167]: [50/53] Running drivers/net/hw:uso.py
Oct 22 04:12:12 hw1 python3[1167]: [50/53] drivers/net/hw:uso.py: rc=0 (8.3s)
Oct 22 04:12:12 hw1 python3[1167]: [51/53] Running drivers/net/hw:vlan.py
Oct 22 04:12:13 hw1 python3[1167]: [51/53] drivers/net/hw:vlan.py: rc=0 (0.3s)
Oct 22 04:12:13 hw1 python3[1167]: [52/53] Running drivers/net/hw:xdp_metadata.py
Oct 22 04:12:16 hw1 python3[1167]: [52/53] drivers/net/hw:xdp_metadata.py: rc=0 (3.6s)
Oct 22 04:12:16 hw1 python3[1167]: [53/53] Running drivers/net/hw:xsk_reconfig.py
Oct 22 04:12:18 hw1 python3[1167]: [53/53] drivers/net/hw:xsk_reconfig.py: all 1 failed cases match known bad, skipping retry
Oct 22 04:12:18 hw1 python3[1167]: [53/53] drivers/net/hw:xsk_reconfig.py: rc=1 (1.4s)
Oct 22 04:12:18 hw1 python3[1167]: Completed, results in /srv/hw-worker/results/5189
Oct 22 04:12:48 hw1 systemd[1]: nipa-hw-worker.service: State 'stop-sigterm' timed out. Aborting.
Oct 22 04:12:48 hw1 systemd[1]: nipa-hw-worker.service: Killing process 10258 (iperf3) with signal SIGABRT.
Oct 22 04:12:48 hw1 systemd-coredump[20365]: Process 10258 (iperf3) of user 0 dumped core.
                                             
                                             Module libz.so.1 from rpm zlib-ng-2.3.3-3.fc44.x86_64
                                             Module libsctp.so.1 from rpm lksctp-tools-1.0.21-3.fc44.x86_64
                                             Module libcrypto.so.3 from rpm openssl-3.5.5-2.fc44.x86_64
                                             Module libssl.so.3 from rpm openssl-3.5.5-2.fc44.x86_64
                                             Module libiperf.so.0 from rpm iperf3-3.20-2.fc44.x86_64
                                             Module libstdbuf.so from rpm coreutils-9.10-3.fc44.x86_64
                                             Module iperf3 from rpm iperf3-3.20-2.fc44.x86_64
                                             Stack trace of thread 10258:
                                             #0  0x00007f085a309b49 __lll_lock_wait_private (libc.so.6 + 0x6fb49)
                                             #1  0x00007f085a31c600 __libc_malloc2 (libc.so.6 + 0x82600)
                                             #2  0x00007f085a2ad85b __dcigettext (libc.so.6 + 0x1385b)
                                             #3  0x00007f085a4ac599 iperf_got_sigend (libiperf.so.0 + 0x19599)
                                             #4  0x000055ffc8eeaa6c run (iperf3 + 0xa6c)
                                             #5  0x000055ffc8eea6d6 main (iperf3 + 0x6d6)
                                             #6  0x00007f085a29d681 __libc_start_call_main (libc.so.6 + 0x3681)
                                             #7  0x00007f085a29d798 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x3798)
                                             #8  0x000055ffc8eea795 _start (iperf3 + 0x795)
                                             ELF object binary architecture: AMD x86-64
Oct 22 04:12:48 hw1 systemd[1]: nipa-hw-worker.service: Failed with result 'timeout'.
Oct 22 04:12:48 hw1 systemd[1]: Failed to start nipa-hw-worker.service - NIPA HW worker (one-shot test runner).
Oct 22 04:12:48 hw1 systemd[1]: nipa-hw-worker.service: Consumed 31.820s CPU time over 8min 38.801s wall clock time, 230M memory peak.