Jul 12 05:08:05 hw1 systemd[1]: Starting nipa-hw-worker.service - NIPA HW worker (one-shot test runner)...
Jul 12 05:08:05 hw1 python3[1151]: /srv/hw-worker/tests/2777 Starting tests
Jul 12 05:08:05 hw1 python3[1151]: Set ens1f0np0 combined channels to 32 (max=32, cpus=32)
Jul 12 05:08:05 hw1 python3[1151]: IRQ mapping: IRQ 186 -> CPU 0 ... IRQ 193 -> CPU 7
Jul 12 05:08:05 hw1 python3[1151]: Restored ens1f0np0 combined channels to 8
Jul 12 05:08:05 hw1 python3[1151]: Moved ens1f1np1 to netns nipa-peer
Jul 12 05:08:05 hw1 python3[1151]: Peer 10.88.0.2 reachable after 1s
Jul 12 05:08:07 hw1 python3[1151]: Peer fd00::2 reachable after 2s
Jul 12 05:08:07 hw1 python3[1151]: Wrote /srv/hw-worker/tests/2777/drivers/net/net.config
Jul 12 05:08:07 hw1 python3[1151]: Wrote /srv/hw-worker/tests/2777/drivers/net/hw/net.config
Jul 12 05:08:07 hw1 python3[1151]: Collected device info for ens1f0np0
Jul 12 05:08:07 hw1 python3[1151]: Found 48 tests
Jul 12 05:08:07 hw1 python3[1151]: Saved 2892 lines of boot dmesg
Jul 12 05:08:07 hw1 python3[1151]: [1/48] Running drivers/net:gro.py
Jul 12 05:08:23 hw1 python3[1151]: [1/48] Retrying drivers/net:gro.py
Jul 12 05:08:34 hw1 python3[1151]: [1/48] drivers/net:gro.py: retry rc=1
Jul 12 05:08:34 hw1 python3[1151]: [1/48] drivers/net:gro.py: rc=1 (10.7s)
Jul 12 05:08:34 hw1 python3[1151]: [2/48] Running drivers/net:hds.py
Jul 12 05:08:35 hw1 python3[1151]: [2/48] drivers/net:hds.py: rc=0 (0.8s)
Jul 12 05:08:35 hw1 python3[1151]: [3/48] Running drivers/net:macsec.py
Jul 12 05:08:35 hw1 python3[1151]: [3/48] drivers/net:macsec.py: rc=0 (0.2s)
Jul 12 05:08:35 hw1 python3[1151]: [4/48] Running drivers/net:napi_id.py
Jul 12 05:08:36 hw1 python3[1151]: [4/48] drivers/net:napi_id.py: rc=0 (0.6s)
Jul 12 05:08:36 hw1 python3[1151]: [5/48] Running drivers/net:napi_threaded.py
Jul 12 05:08:36 hw1 python3[1151]: [5/48] drivers/net:napi_threaded.py: rc=0 (0.2s)
Jul 12 05:08:36 hw1 python3[1151]: [6/48] Running drivers/net:netpoll_basic.py
Jul 12 05:08:37 hw1 python3[1151]: [6/48] Retrying drivers/net:netpoll_basic.py
Jul 12 05:08:38 hw1 python3[1151]: [6/48] drivers/net:netpoll_basic.py: retry rc=1
Jul 12 05:08:38 hw1 python3[1151]: [6/48] drivers/net:netpoll_basic.py: rc=1 (1.4s)
Jul 12 05:08:38 hw1 python3[1151]: [7/48] Running drivers/net:ping.py
Jul 12 05:09:20 hw1 python3[1151]: [7/48] drivers/net:ping.py: rc=0 (41.4s)
Jul 12 05:09:20 hw1 python3[1151]: [8/48] Running drivers/net:psp.py
Jul 12 05:09:20 hw1 python3[1151]: [8/48] drivers/net:psp.py: rc=0 (0.5s)
Jul 12 05:09:20 hw1 python3[1151]: [9/48] Running drivers/net:queues.py
Jul 12 05:09:20 hw1 python3[1151]: [9/48] drivers/net:queues.py: rc=0 (0.1s)
Jul 12 05:09:20 hw1 python3[1151]: [10/48] Running drivers/net:ring_reconfig.py
Jul 12 05:09:21 hw1 python3[1151]: [10/48] drivers/net:ring_reconfig.py: rc=0 (1.2s)
Jul 12 05:09:21 hw1 python3[1151]: [11/48] Running drivers/net:shaper.py
Jul 12 05:09:21 hw1 python3[1151]: [11/48] drivers/net:shaper.py: rc=0 (0.1s)
Jul 12 05:09:21 hw1 python3[1151]: [12/48] Running drivers/net:so_txtime.py
Jul 12 05:09:31 hw1 python3[1151]: [12/48] drivers/net:so_txtime.py: rc=0 (9.7s)
Jul 12 05:09:31 hw1 python3[1151]: [13/48] Running drivers/net:stats.py
Jul 12 05:09:34 hw1 python3[1151]: [13/48] drivers/net:stats.py: rc=0 (3.1s)
Jul 12 05:09:34 hw1 python3[1151]: [14/48] Running drivers/net:xdp.py
Jul 12 05:14:00 hw1 python3[1151]: [14/48] Retrying drivers/net:xdp.py
Jul 12 05:18:25 hw1 python3[1151]: [14/48] drivers/net:xdp.py: retry rc=1
Jul 12 05:18:25 hw1 python3[1151]: [14/48] drivers/net:xdp.py: rc=1 (265.5s)
Jul 12 05:18:25 hw1 python3[1151]: [15/48] Running drivers/net/hw:csum.py
Jul 12 05:18:45 hw1 python3[1151]: [15/48] drivers/net/hw:csum.py: rc=0 (20.2s)
Jul 12 05:18:45 hw1 python3[1151]: [16/48] Running drivers/net/hw:devlink_port_split.py
Jul 12 05:18:45 hw1 python3[1151]: [16/48] drivers/net/hw:devlink_port_split.py: rc=0 (0.1s)
Jul 12 05:18:45 hw1 python3[1151]: [17/48] Running drivers/net/hw:devlink_rate_cross_esw.py
Jul 12 05:18:45 hw1 python3[1151]: [17/48] drivers/net/hw:devlink_rate_cross_esw.py: rc=0 (0.1s)
Jul 12 05:18:45 hw1 python3[1151]: [18/48] Running drivers/net/hw:devlink_rate_tc_bw.py
Jul 12 05:18:45 hw1 python3[1151]: [18/48] drivers/net/hw:devlink_rate_tc_bw.py: rc=0 (0.1s)
Jul 12 05:18:45 hw1 python3[1151]: [19/48] Running drivers/net/hw:devmem.py
Jul 12 05:18:45 hw1 python3[1151]: [19/48] drivers/net/hw:devmem.py: rc=0 (0.1s)
Jul 12 05:18:45 hw1 python3[1151]: [20/48] Running drivers/net/hw:ethtool.sh
Jul 12 05:18:45 hw1 python3[1151]: [20/48] Retrying drivers/net/hw:ethtool.sh
Jul 12 05:18:46 hw1 python3[1151]: [20/48] drivers/net/hw:ethtool.sh: retry rc=1
Jul 12 05:18:46 hw1 python3[1151]: [20/48] drivers/net/hw:ethtool.sh: rc=1 (0.0s)
Jul 12 05:18:46 hw1 python3[1151]: [21/48] Running drivers/net/hw:ethtool_extended_state.sh
Jul 12 05:18:46 hw1 python3[1151]: [21/48] Retrying drivers/net/hw:ethtool_extended_state.sh
Jul 12 05:18:46 hw1 python3[1151]: [21/48] drivers/net/hw:ethtool_extended_state.sh: retry rc=1
Jul 12 05:18:46 hw1 python3[1151]: [21/48] drivers/net/hw:ethtool_extended_state.sh: rc=1 (0.0s)
Jul 12 05:18:46 hw1 python3[1151]: [22/48] Running drivers/net/hw:ethtool_mm.sh
Jul 12 05:18:46 hw1 python3[1151]: [22/48] Retrying drivers/net/hw:ethtool_mm.sh
Jul 12 05:18:46 hw1 python3[1151]: [22/48] drivers/net/hw:ethtool_mm.sh: retry rc=1
Jul 12 05:18:46 hw1 python3[1151]: [22/48] drivers/net/hw:ethtool_mm.sh: rc=1 (0.0s)
Jul 12 05:18:46 hw1 python3[1151]: [23/48] Running drivers/net/hw:ethtool_rmon.sh
Jul 12 05:18:57 hw1 python3[1151]: [23/48] Retrying drivers/net/hw:ethtool_rmon.sh
Jul 12 05:19:08 hw1 python3[1151]: [23/48] drivers/net/hw:ethtool_rmon.sh: retry rc=1
Jul 12 05:19:08 hw1 python3[1151]: [23/48] drivers/net/hw:ethtool_rmon.sh: rc=1 (10.9s)
Jul 12 05:19:08 hw1 python3[1151]: [24/48] Running drivers/net/hw:ethtool_std_stats.sh
Jul 12 05:19:17 hw1 python3[1151]: [24/48] Retrying drivers/net/hw:ethtool_std_stats.sh
Jul 12 05:19:26 hw1 python3[1151]: [24/48] drivers/net/hw:ethtool_std_stats.sh: retry rc=1
Jul 12 05:19:26 hw1 python3[1151]: [24/48] drivers/net/hw:ethtool_std_stats.sh: rc=1 (9.1s)
Jul 12 05:19:26 hw1 python3[1151]: [25/48] Running drivers/net/hw:gro_hw.py
Jul 12 05:19:26 hw1 python3[1151]: [25/48] drivers/net/hw:gro_hw.py: rc=0 (0.1s)
Jul 12 05:19:26 hw1 python3[1151]: [26/48] Running drivers/net/hw:hw_stats_l3.sh
Jul 12 05:19:26 hw1 python3[1151]: [26/48] Retrying drivers/net/hw:hw_stats_l3.sh
Jul 12 05:19:26 hw1 python3[1151]: [26/48] drivers/net/hw:hw_stats_l3.sh: retry rc=1
Jul 12 05:19:26 hw1 python3[1151]: [26/48] drivers/net/hw:hw_stats_l3.sh: rc=1 (0.0s)
Jul 12 05:19:26 hw1 python3[1151]: [27/48] Running drivers/net/hw:hw_stats_l3_gre.sh
Jul 12 05:19:26 hw1 python3[1151]: [27/48] Retrying drivers/net/hw:hw_stats_l3_gre.sh
Jul 12 05:19:26 hw1 python3[1151]: [27/48] drivers/net/hw:hw_stats_l3_gre.sh: retry rc=1
Jul 12 05:19:26 hw1 python3[1151]: [27/48] drivers/net/hw:hw_stats_l3_gre.sh: rc=1 (0.0s)
Jul 12 05:19:26 hw1 python3[1151]: [28/48] Running drivers/net/hw:iou-zcrx.py
Jul 12 05:19:26 hw1 python3[1151]: [28/48] Retrying drivers/net/hw:iou-zcrx.py
Jul 12 05:19:26 hw1 python3[1151]: [28/48] drivers/net/hw:iou-zcrx.py: retry rc=1
Jul 12 05:19:26 hw1 python3[1151]: [28/48] drivers/net/hw:iou-zcrx.py: rc=1 (0.1s)
Jul 12 05:19:26 hw1 python3[1151]: [29/48] Running drivers/net/hw:ipsec_vxlan.py
Jul 12 05:19:26 hw1 python3[1151]: [29/48] drivers/net/hw:ipsec_vxlan.py: rc=0 (0.1s)
Jul 12 05:19:26 hw1 python3[1151]: [30/48] Running drivers/net/hw:irq.py
Jul 12 05:19:27 hw1 python3[1151]: [30/48] drivers/net/hw:irq.py: rc=0 (0.5s)
Jul 12 05:19:27 hw1 python3[1151]: [31/48] Running drivers/net/hw:loopback.sh
Jul 12 05:19:27 hw1 python3[1151]: [31/48] Retrying drivers/net/hw:loopback.sh
Jul 12 05:19:27 hw1 python3[1151]: [31/48] drivers/net/hw:loopback.sh: retry rc=1
Jul 12 05:19:27 hw1 python3[1151]: [31/48] drivers/net/hw:loopback.sh: rc=1 (0.0s)
Jul 12 05:19:27 hw1 python3[1151]: [32/48] Running drivers/net/hw:nic_timestamp.py
Jul 12 05:19:27 hw1 python3[1151]: [32/48] drivers/net/hw:nic_timestamp.py: rc=0 (0.1s)
Jul 12 05:19:27 hw1 python3[1151]: [33/48] Running drivers/net/hw:nk_devmem.py
Jul 12 05:19:28 hw1 python3[1151]: [33/48] drivers/net/hw:nk_devmem.py: rc=0 (0.5s)
Jul 12 05:19:28 hw1 python3[1151]: [34/48] Running drivers/net/hw:nk_netns.py
Jul 12 05:19:28 hw1 python3[1151]: [34/48] drivers/net/hw:nk_netns.py: rc=0 (0.3s)
Jul 12 05:19:28 hw1 python3[1151]: [35/48] Running drivers/net/hw:nk_qlease.py
Jul 12 05:19:29 hw1 python3[1151]: [35/48] Retrying drivers/net/hw:nk_qlease.py
Jul 12 05:19:31 hw1 python3[1151]: [35/48] drivers/net/hw:nk_qlease.py: retry rc=1
Jul 12 05:19:31 hw1 python3[1151]: [35/48] drivers/net/hw:nk_qlease.py: rc=1 (1.5s)
Jul 12 05:19:31 hw1 python3[1151]: [36/48] Running drivers/net/hw:ntuple.py
Jul 12 05:19:32 hw1 python3[1151]: [36/48] Retrying drivers/net/hw:ntuple.py
Jul 12 05:19:32 hw1 python3[1151]: [36/48] drivers/net/hw:ntuple.py: retry rc=1
Jul 12 05:19:32 hw1 python3[1151]: [36/48] drivers/net/hw:ntuple.py: rc=1 (0.8s)
Jul 12 05:19:32 hw1 python3[1151]: [37/48] Running drivers/net/hw:pp_alloc_fail.py
Jul 12 05:19:32 hw1 python3[1151]: [37/48] drivers/net/hw:pp_alloc_fail.py: rc=0 (0.1s)
Jul 12 05:19:33 hw1 python3[1151]: [38/48] Running drivers/net/hw:rss_api.py
Jul 12 05:19:33 hw1 python3[1151]: [38/48] Retrying drivers/net/hw:rss_api.py
Jul 12 05:19:33 hw1 python3[1151]: [38/48] drivers/net/hw:rss_api.py: retry rc=1
Jul 12 05:19:33 hw1 python3[1151]: [38/48] drivers/net/hw:rss_api.py: rc=1 (0.4s)
Jul 12 05:19:33 hw1 python3[1151]: [39/48] Running drivers/net/hw:rss_ctx.py
Jul 12 05:19:54 hw1 python3[1151]: [39/48] Retrying drivers/net/hw:rss_ctx.py
Jul 12 05:19:55 hw1 python3[1151]: [39/48] drivers/net/hw:rss_ctx.py: retry rc=1
Jul 12 05:19:55 hw1 python3[1151]: [39/48] drivers/net/hw:rss_ctx.py: rc=1 (20.9s)
Jul 12 05:19:55 hw1 python3[1151]: [40/48] Running drivers/net/hw:rss_drv.py
Jul 12 05:19:56 hw1 python3[1151]: [40/48] drivers/net/hw:rss_drv.py: rc=0 (0.9s)
Jul 12 05:19:56 hw1 python3[1151]: [41/48] Running drivers/net/hw:rss_flow_label.py
Jul 12 05:19:57 hw1 python3[1151]: [41/48] Retrying drivers/net/hw:rss_flow_label.py
Jul 12 05:19:59 hw1 python3[1151]: [41/48] drivers/net/hw:rss_flow_label.py: retry rc=1
Jul 12 05:19:59 hw1 python3[1151]: [41/48] drivers/net/hw:rss_flow_label.py: rc=1 (1.4s)
Jul 12 05:19:59 hw1 python3[1151]: [42/48] Running drivers/net/hw:rss_input_xfrm.py
Jul 12 05:19:59 hw1 python3[1151]: [42/48] drivers/net/hw:rss_input_xfrm.py: rc=0 (0.1s)
Jul 12 05:19:59 hw1 python3[1151]: [43/48] Running drivers/net/hw:toeplitz.py
Jul 12 05:20:25 hw1 python3[1151]: [43/48] drivers/net/hw:toeplitz.py: rc=0 (26.6s)
Jul 12 05:20:26 hw1 python3[1151]: [44/48] Running drivers/net/hw:tso.py
Jul 12 05:20:26 hw1 python3[22448]: detected unhandled Python exception in './tso.py'
Jul 12 05:20:26 hw1 python3[1151]: [44/48] Retrying drivers/net/hw:tso.py
Jul 12 05:20:26 hw1 python3[22511]: detected unhandled Python exception in './tso.py'
Jul 12 05:20:26 hw1 python3[1151]: [44/48] drivers/net/hw:tso.py: retry rc=1
Jul 12 05:20:26 hw1 python3[1151]: [44/48] drivers/net/hw:tso.py: rc=1 (0.2s)
Jul 12 05:20:26 hw1 python3[1151]: [45/48] Running drivers/net/hw:userns_devmem.py
Jul 12 05:20:26 hw1 python3[1151]: [45/48] drivers/net/hw:userns_devmem.py: rc=0 (0.5s)
Jul 12 05:20:26 hw1 python3[1151]: [46/48] Running drivers/net/hw:uso.py
Jul 12 05:20:35 hw1 python3[1151]: [46/48] drivers/net/hw:uso.py: rc=0 (8.3s)
Jul 12 05:20:35 hw1 python3[1151]: [47/48] Running drivers/net/hw:xdp_metadata.py
Jul 12 05:20:38 hw1 python3[1151]: [47/48] drivers/net/hw:xdp_metadata.py: rc=0 (3.6s)
Jul 12 05:20:38 hw1 python3[1151]: [48/48] Running drivers/net/hw:xsk_reconfig.py
Jul 12 05:20:40 hw1 python3[1151]: [48/48] Retrying drivers/net/hw:xsk_reconfig.py
Jul 12 05:20:41 hw1 python3[1151]: [48/48] drivers/net/hw:xsk_reconfig.py: retry rc=1
Jul 12 05:20:41 hw1 python3[1151]: [48/48] drivers/net/hw:xsk_reconfig.py: rc=1 (1.4s)
Jul 12 05:20:41 hw1 python3[1151]: Completed, results in /srv/hw-worker/results/2777
Jul 12 05:21:11 hw1 systemd[1]: nipa-hw-worker.service: State 'stop-sigterm' timed out. Aborting.
Jul 12 05:21:11 hw1 systemd[1]: nipa-hw-worker.service: Killing process 12939 (iperf3) with signal SIGABRT.
Jul 12 05:21:11 hw1 systemd-coredump[23098]: Process 12939 (iperf3) of user 0 dumped core.
                                             
                                             Module libz.so.1 from rpm zlib-ng-2.3.3-3.fc44.x86_64
                                             Module libsctp.so.1 from rpm lksctp-tools-1.0.21-3.fc44.x86_64
                                             Module libcrypto.so.3 from rpm openssl-3.5.5-2.fc44.x86_64
                                             Module libssl.so.3 from rpm openssl-3.5.5-2.fc44.x86_64
                                             Module libiperf.so.0 from rpm iperf3-3.20-2.fc44.x86_64
                                             Module libstdbuf.so from rpm coreutils-9.10-3.fc44.x86_64
                                             Module iperf3 from rpm iperf3-3.20-2.fc44.x86_64
                                             Stack trace of thread 12939:
                                             #0  0x00007f1a7557bb49 __lll_lock_wait_private (libc.so.6 + 0x6fb49)
                                             #1  0x00007f1a7558e600 __libc_malloc2 (libc.so.6 + 0x82600)
                                             #2  0x00007f1a7551f85b __dcigettext (libc.so.6 + 0x1385b)
                                             #3  0x00007f1a7571e599 iperf_got_sigend (libiperf.so.0 + 0x19599)
                                             #4  0x000056180286aa6c run (iperf3 + 0xa6c)
                                             #5  0x000056180286a6d6 main (iperf3 + 0x6d6)
                                             #6  0x00007f1a7550f681 __libc_start_call_main (libc.so.6 + 0x3681)
                                             #7  0x00007f1a7550f798 __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x3798)
                                             #8  0x000056180286a795 _start (iperf3 + 0x795)
                                             ELF object binary architecture: AMD x86-64
Jul 12 05:21:11 hw1 systemd[1]: nipa-hw-worker.service: Failed with result 'timeout'.
Jul 12 05:21:11 hw1 systemd[1]: Failed to start nipa-hw-worker.service - NIPA HW worker (one-shot test runner).
Jul 12 05:21:11 hw1 systemd[1]: nipa-hw-worker.service: Consumed 46.526s CPU time over 13min 1.396s wall clock time, 231.8M memory peak.