====================================== | [ 758.510711] nipa-hw-worker: drivers/net:napi_threaded.py: timed out | [ 758.513077] nipa-hw-worker: [4/37] Retrying drivers/net:napi_threaded.py | [ 758.674061] kselftest: Running tests in drivers/net [ 989.726371] INFO: task python3:4989 blocked for more than 122 seconds. [ 989.734590] Not tainted 7.0.0-rc3-lfth-g1eba3b8e2cb7 #1 [ 989.741838] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 989.751418] task:python3 state:D stack:25312 pid:4989 tgid:4989 ppid:4988 task_flags:0x400100 flags:0x00080002 [ 989.764622] Call Trace: [ 989.768159] [ 989.771309] __schedule+0xad9/0x1490 [ 989.776121] ? io_schedule_timeout+0x130/0x130 [ 989.781904] ? find_held_lock+0x2b/0x80 [ 989.787007] ? __lock_release.isra.0+0x59/0x170 [ 989.792890] schedule+0xd4/0x260 [ 989.797307] schedule_preempt_disabled+0x18/0x30 [ 989.803285] __mutex_lock+0xd76/0x1f20 [ 989.808291] ? threaded_store+0x132/0x220 [ 989.813591] ? ww_mutex_lock+0x160/0x160 [ 989.818781] ? selinux_inode_free_security+0x310/0x310 [ 989.825348] ? cap_capable+0x1ed/0x3f0 [ 989.830355] ? sysfs_file_ops+0x140/0x140 [ 989.835652] ? threaded_store+0x132/0x220 [ 989.840945] threaded_store+0x132/0x220 [ 989.846044] ? tx_maxrate_store+0x4e0/0x4e0 [ 989.851539] ? sysfs_file_ops+0x140/0x140 [ 989.856836] ? sysfs_file_kobj+0xbd/0x1e0 [ 989.862129] ? sysfs_kf_write+0x65/0x160 [ 989.867327] kernfs_fop_write_iter+0x335/0x510 [ 989.873111] ? kernfs_ops+0xf0/0xf0 [ 989.877820] new_sync_write+0x308/0x6f0 [ 989.882924] ? new_sync_read+0x6f0/0x6f0 [ 989.888122] ? lock_acquire.part.0+0xbc/0x260 [ 989.893805] ? ksys_write+0xff/0x200 [ 989.898617] vfs_write+0x62a/0x980 [ 989.903230] ksys_write+0xff/0x200 [ 989.907844] ? __ia32_sys_read+0xc0/0xc0 [ 989.913041] ? kernfs_fop_llseek+0x160/0x1f0 [ 989.918630] ? rcu_is_watching+0x15/0xd0 [ 989.923828] do_syscall_64+0xf3/0x5e0 [ 989.928734] ? exc_page_fault+0x9e/0xb0 [ 989.933834] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 989.940302] RIP: 0033:0x7fba5444fc5e [ 989.945110] RSP: 002b:00007ffe5b3c10e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 989.954407] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fba5444fc5e [ 989.963215] RDX: 0000000000000001 RSI: 0000563422738110 RDI: 0000000000000007 [ 989.972024] RBP: 00007ffe5b3c10f0 R08: 0000000000000000 R09: 0000000000000000 [ 989.980830] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fba542eae80 [ 989.989641] R13: 0000563422738110 R14: 0000000000000007 R15: 00007fba54b50b98 | [ 990.371065] #2: ff11000120d4b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 | [ 990.382222] #3: ff11000120d89cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 | [ 990.392304] | [ 990.394767] =============================================\x0a | [ 1092.127283] kworker/u129:11 (306) used greatest stack depth: 21688 bytes left [ 1112.607366] INFO: task python3:4989 blocked for more than 245 seconds. [ 1112.615509] Not tainted 7.0.0-rc3-lfth-g1eba3b8e2cb7 #1 [ 1112.622760] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1112.632349] task:python3 state:D stack:25312 pid:4989 tgid:4989 ppid:4988 task_flags:0x400100 flags:0x00080002 [ 1112.645555] Call Trace: [ 1112.649093] [ 1112.652247] __schedule+0xad9/0x1490 [ 1112.657065] ? io_schedule_timeout+0x130/0x130 [ 1112.662850] ? find_held_lock+0x2b/0x80 [ 1112.667956] ? __lock_release.isra.0+0x59/0x170 [ 1112.673841] schedule+0xd4/0x260 [ 1112.678263] schedule_preempt_disabled+0x18/0x30 [ 1112.684243] __mutex_lock+0xd76/0x1f20 [ 1112.689251] ? threaded_store+0x132/0x220 [ 1112.694553] ? ww_mutex_lock+0x160/0x160 [ 1112.699750] ? selinux_inode_free_security+0x310/0x310 [ 1112.706319] ? cap_capable+0x1ed/0x3f0 [ 1112.711328] ? sysfs_file_ops+0x140/0x140 [ 1112.716628] ? threaded_store+0x132/0x220 [ 1112.721918] threaded_store+0x132/0x220 [ 1112.727018] ? tx_maxrate_store+0x4e0/0x4e0 [ 1112.732516] ? sysfs_file_ops+0x140/0x140 [ 1112.737814] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.743101] ? sysfs_kf_write+0x65/0x160 [ 1112.748300] kernfs_fop_write_iter+0x335/0x510 [ 1112.754086] ? kernfs_ops+0xf0/0xf0 [ 1112.758798] new_sync_write+0x308/0x6f0 [ 1112.763894] ? new_sync_read+0x6f0/0x6f0 [ 1112.769096] ? lock_acquire.part.0+0xbc/0x260 [ 1112.774783] ? ksys_write+0xff/0x200 [ 1112.779588] vfs_write+0x62a/0x980 [ 1112.784207] ksys_write+0xff/0x200 [ 1112.788820] ? __ia32_sys_read+0xc0/0xc0 [ 1112.794020] ? kernfs_fop_llseek+0x160/0x1f0 [ 1112.799610] ? rcu_is_watching+0x15/0xd0 [ 1112.804816] do_syscall_64+0xf3/0x5e0 [ 1112.809720] ? exc_page_fault+0x9e/0xb0 [ 1112.814822] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1112.821292] RIP: 0033:0x7fba5444fc5e [ 1112.826101] RSP: 002b:00007ffe5b3c10e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1112.835398] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fba5444fc5e [ 1112.844209] RDX: 0000000000000001 RSI: 0000563422738110 RDI: 0000000000000007 [ 1112.853018] RBP: 00007ffe5b3c10f0 R08: 0000000000000000 R09: 0000000000000000 [ 1112.861827] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fba542eae80 [ 1112.870636] R13: 0000563422738110 R14: 0000000000000007 R15: 00007fba54b50b98 | [ 1113.328467] #2: ffffffffa0536c48 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xd2/0x3b0 | [ 1113.339138] #3: ffffffffa0368dd0 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e7/0x640 | [ 1113.350783] #4: ffffffffa0536950 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f3/0x640 | [ 1113.363212] | [ 1113.365681] =============================================\x0a [ 1235.487556] INFO: task python3:4989 blocked for more than 368 seconds. [ 1235.495704] Not tainted 7.0.0-rc3-lfth-g1eba3b8e2cb7 #1 [ 1235.502953] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1235.512544] task:python3 state:D stack:25312 pid:4989 tgid:4989 ppid:4988 task_flags:0x400100 flags:0x00080002 [ 1235.525750] Call Trace: [ 1235.529291] [ 1235.532440] __schedule+0xad9/0x1490 [ 1235.537254] ? io_schedule_timeout+0x130/0x130 [ 1235.543029] ? find_held_lock+0x2b/0x80 [ 1235.548134] ? __lock_release.isra.0+0x59/0x170 [ 1235.554017] schedule+0xd4/0x260 [ 1235.558435] schedule_preempt_disabled+0x18/0x30 [ 1235.564414] __mutex_lock+0xd76/0x1f20 [ 1235.569420] ? threaded_store+0x132/0x220 [ 1235.574719] ? ww_mutex_lock+0x160/0x160 [ 1235.579918] ? selinux_inode_free_security+0x310/0x310 [ 1235.586486] ? cap_capable+0x1ed/0x3f0 [ 1235.591494] ? sysfs_file_ops+0x140/0x140 [ 1235.596791] ? threaded_store+0x132/0x220 [ 1235.602085] threaded_store+0x132/0x220 [ 1235.607188] ? tx_maxrate_store+0x4e0/0x4e0 [ 1235.612673] ? sysfs_file_ops+0x140/0x140 [ 1235.617968] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.623262] ? sysfs_kf_write+0x65/0x160 [ 1235.628462] kernfs_fop_write_iter+0x335/0x510 [ 1235.634246] ? kernfs_ops+0xf0/0xf0 [ 1235.638957] new_sync_write+0x308/0x6f0 [ 1235.644057] ? new_sync_read+0x6f0/0x6f0 [ 1235.649256] ? lock_acquire.part.0+0xbc/0x260 [ 1235.654942] ? ksys_write+0xff/0x200 [ 1235.659753] vfs_write+0x62a/0x980 [ 1235.664366] ksys_write+0xff/0x200 [ 1235.668977] ? __ia32_sys_read+0xc0/0xc0 [ 1235.674173] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.679760] ? rcu_is_watching+0x15/0xd0 [ 1235.684959] do_syscall_64+0xf3/0x5e0 [ 1235.689863] ? exc_page_fault+0x9e/0xb0 [ 1235.694965] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.701421] RIP: 0033:0x7fba5444fc5e [ 1235.706228] RSP: 002b:00007ffe5b3c10e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.715527] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fba5444fc5e [ 1235.724333] RDX: 0000000000000001 RSI: 0000563422738110 RDI: 0000000000000007 [ 1235.733138] RBP: 00007ffe5b3c10f0 R08: 0000000000000000 R09: 0000000000000000 [ 1235.741942] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fba542eae80 [ 1235.750748] R13: 0000563422738110 R14: 0000000000000007 R15: 00007fba54b50b98 | [ 1236.121151] #1: ff11000169105c88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 | [ 1236.132109] #2: ff11000120d4b0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 | [ 1236.143264] #3: ff11000120d89cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 | [ 1236.153357] | [ 1236.155824] =============================================\x0a [ 1358.368780] INFO: task python3:4989 blocked for more than 491 seconds. [ 1358.376924] Not tainted 7.0.0-rc3-lfth-g1eba3b8e2cb7 #1 [ 1358.384172] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1358.393763] task:python3 state:D stack:25312 pid:4989 tgid:4989 ppid:4988 task_flags:0x400100 flags:0x00080002 [ 1358.406969] Call Trace: [ 1358.410509] [ 1358.413661] __schedule+0xad9/0x1490 [ 1358.418476] ? io_schedule_timeout+0x130/0x130 [ 1358.424260] ? find_held_lock+0x2b/0x80 [ 1358.429365] ? __lock_release.isra.0+0x59/0x170 [ 1358.435240] schedule+0xd4/0x260 [ 1358.439648] schedule_preempt_disabled+0x18/0x30 [ 1358.445628] __mutex_lock+0xd76/0x1f20 [ 1358.450622] ? threaded_store+0x132/0x220 [ 1358.455918] ? ww_mutex_lock+0x160/0x160 [ 1358.461116] ? selinux_inode_free_security+0x310/0x310 [ 1358.467676] ? cap_capable+0x1ed/0x3f0 [ 1358.472690] ? sysfs_file_ops+0x140/0x140 [ 1358.478000] ? threaded_store+0x132/0x220 [ 1358.483295] threaded_store+0x132/0x220 [ 1358.488395] ? tx_maxrate_store+0x4e0/0x4e0 [ 1358.493883] ? sysfs_file_ops+0x140/0x140 [ 1358.499179] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.504473] ? sysfs_kf_write+0x65/0x160 [ 1358.509675] kernfs_fop_write_iter+0x335/0x510 [ 1358.515460] ? kernfs_ops+0xf0/0xf0 [ 1358.520164] new_sync_write+0x308/0x6f0 [ 1358.525264] ? new_sync_read+0x6f0/0x6f0 [ 1358.530464] ? lock_acquire.part.0+0xbc/0x260 [ 1358.536150] ? ksys_write+0xff/0x200 [ 1358.540961] vfs_write+0x62a/0x980 [ 1358.545564] ksys_write+0xff/0x200 [ 1358.550175] ? __ia32_sys_read+0xc0/0xc0 [ 1358.555370] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.560960] ? rcu_is_watching+0x15/0xd0 [ 1358.566160] do_syscall_64+0xf3/0x5e0 [ 1358.571056] ? exc_page_fault+0x9e/0xb0 [ 1358.576159] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.582624] RIP: 0033:0x7fba5444fc5e [ 1358.587435] RSP: 002b:00007ffe5b3c10e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.596723] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fba5444fc5e [ 1358.605521] RDX: 0000000000000001 RSI: 0000563422738110 RDI: 0000000000000007 [ 1358.614326] RBP: 00007ffe5b3c10f0 R08: 0000000000000000 R09: 0000000000000000 [ 1358.623134] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fba542eae80 [ 1358.631940] R13: 0000563422738110 R14: 0000000000000007 R15: 00007fba54b50b98 [ 1358.632978] nipa-hw-worker: drivers/net:napi_threaded.py: timed out Finger prints: __schedule:schedule:schedule_preempt_disabled:__mutex_lock:threaded_store