====================================== | [ 779.955597] nipa-hw-worker: drivers/net:napi_threaded.py: timed out | [ 779.958030] nipa-hw-worker: [4/40] Retrying drivers/net:napi_threaded.py | [ 780.109611] kselftest: Running tests in drivers/net [ 989.683953] INFO: task python3:5623 blocked for more than 122 seconds. [ 989.692195] Not tainted 7.0.0-rc6-yesk-ga6e6a7dcb853 #1 [ 989.699446] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 989.709034] task:python3 state:D stack:25336 pid:5623 tgid:5623 ppid:5622 task_flags:0x400100 flags:0x00080000 [ 989.722237] Call Trace: [ 989.725778] [ 989.728930] __schedule+0xae3/0x14d0 [ 989.733745] ? io_schedule_timeout+0x140/0x140 [ 989.739521] ? find_held_lock+0x2b/0x80 [ 989.744625] ? __lock_release.isra.0+0x6b/0x1a0 [ 989.750507] schedule+0xd4/0x260 [ 989.754925] schedule_preempt_disabled+0x18/0x30 [ 989.760904] __mutex_lock+0x6b2/0x1ee0 [ 989.765908] ? rcu_is_watching+0x15/0xd0 [ 989.771104] ? threaded_store+0x132/0x220 [ 989.776403] ? ww_mutex_lock+0x160/0x160 [ 989.781603] ? selinux_inode_free_security+0x330/0x330 [ 989.788175] ? cap_capable+0x1ed/0x3f0 [ 989.793181] ? threaded_store+0x132/0x220 [ 989.798478] threaded_store+0x132/0x220 [ 989.803582] ? xps_rxqs_store+0x2e0/0x2e0 [ 989.808878] ? sysfs_file_kobj+0xbd/0x1e0 [ 989.814173] ? sysfs_kf_write+0x65/0x160 [ 989.819373] ? sysfs_file_ops+0x140/0x140 [ 989.825140] kernfs_fop_write_iter+0x335/0x510 [ 989.830960] ? kernfs_ops+0xf0/0xf0 [ 989.835675] new_sync_write+0x333/0x750 [ 989.840786] ? __lock_acquire+0x508/0xc10 [ 989.846084] ? new_sync_read+0x740/0x740 [ 989.851294] ? lock_acquire.part.0+0xbc/0x260 [ 989.856984] ? ksys_write+0x116/0x250 [ 989.861895] vfs_write+0x657/0x9b0 [ 989.866512] ksys_write+0x116/0x250 [ 989.871227] ? __ia32_sys_read+0xc0/0xc0 [ 989.876429] ? kernfs_fop_llseek+0x160/0x1f0 [ 989.882019] ? rcu_is_watching+0x15/0xd0 [ 989.887283] do_syscall_64+0xf3/0x5e0 [ 989.892194] ? trace_hardirqs_off+0xd/0x30 [ 989.897588] ? exc_page_fault+0xda/0xf0 [ 989.902692] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 989.909164] RIP: 0033:0x7f4225baf22e [ 989.913975] RSP: 002b:00007ffc74f835c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 989.923278] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f4225baf22e [ 989.932090] RDX: 0000000000000001 RSI: 0000557332870f40 RDI: 0000000000000007 [ 989.940898] RBP: 00007ffc74f835d0 R08: 0000000000000000 R09: 0000000000000000 [ 989.949697] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f4225a27390 [ 989.958510] R13: 0000557332870f40 R14: 0000000000000007 R15: 0000000000000003 | [ 990.346516] #2: ff1100012199f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 | [ 990.357669] #3: ff11000121959d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 | [ 990.367759] 1 lock held by systemd/6309: | [ 990.372958] | [ 990.375489] =============================================\x0a [ 1112.564202] INFO: task python3:5623 blocked for more than 245 seconds. [ 1112.572349] Not tainted 7.0.0-rc6-yesk-ga6e6a7dcb853 #1 [ 1112.579598] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1112.589190] task:python3 state:D stack:25336 pid:5623 tgid:5623 ppid:5622 task_flags:0x400100 flags:0x00080000 [ 1112.602399] Call Trace: [ 1112.605940] [ 1112.609090] __schedule+0xae3/0x14d0 [ 1112.613911] ? io_schedule_timeout+0x140/0x140 [ 1112.619696] ? find_held_lock+0x2b/0x80 [ 1112.624803] ? __lock_release.isra.0+0x6b/0x1a0 [ 1112.630693] schedule+0xd4/0x260 [ 1112.635115] schedule_preempt_disabled+0x18/0x30 [ 1112.641095] __mutex_lock+0x6b2/0x1ee0 [ 1112.646103] ? rcu_is_watching+0x15/0xd0 [ 1112.651301] ? threaded_store+0x132/0x220 [ 1112.656603] ? ww_mutex_lock+0x160/0x160 [ 1112.661802] ? selinux_inode_free_security+0x330/0x330 [ 1112.668368] ? cap_capable+0x1ed/0x3f0 [ 1112.673368] ? threaded_store+0x132/0x220 [ 1112.678666] threaded_store+0x132/0x220 [ 1112.683763] ? xps_rxqs_store+0x2e0/0x2e0 [ 1112.689063] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.694361] ? sysfs_kf_write+0x65/0x160 [ 1112.699560] ? sysfs_file_ops+0x140/0x140 [ 1112.704855] kernfs_fop_write_iter+0x335/0x510 [ 1112.710645] ? kernfs_ops+0xf0/0xf0 [ 1112.715357] new_sync_write+0x333/0x750 [ 1112.720453] ? __lock_acquire+0x508/0xc10 [ 1112.725749] ? new_sync_read+0x740/0x740 [ 1112.730952] ? lock_acquire.part.0+0xbc/0x260 [ 1112.736637] ? ksys_write+0x116/0x250 [ 1112.741550] vfs_write+0x657/0x9b0 [ 1112.746167] ksys_write+0x116/0x250 [ 1112.750882] ? __ia32_sys_read+0xc0/0xc0 [ 1112.756081] ? kernfs_fop_llseek+0x160/0x1f0 [ 1112.761664] ? rcu_is_watching+0x15/0xd0 [ 1112.766862] do_syscall_64+0xf3/0x5e0 [ 1112.771771] ? trace_hardirqs_off+0xd/0x30 [ 1112.777169] ? exc_page_fault+0xda/0xf0 [ 1112.782272] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1112.788741] RIP: 0033:0x7f4225baf22e [ 1112.793551] RSP: 002b:00007ffc74f835c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1112.802840] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f4225baf22e [ 1112.811650] RDX: 0000000000000001 RSI: 0000557332870f40 RDI: 0000000000000007 [ 1112.820459] RBP: 00007ffc74f835d0 R08: 0000000000000000 R09: 0000000000000000 [ 1112.829268] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f4225a27390 [ 1112.838077] R13: 0000557332870f40 R14: 0000000000000007 R15: 0000000000000003 | [ 1113.243672] #1: ff110001e9762688 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510 | [ 1113.254633] #2: ff1100012199f0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 | [ 1113.265791] #3: ff11000121959d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 | [ 1113.275884] | [ 1113.278349] =============================================\x0a [ 1235.445461] INFO: task python3:5623 blocked for more than 368 seconds. [ 1235.453614] Not tainted 7.0.0-rc6-yesk-ga6e6a7dcb853 #1 [ 1235.460864] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1235.470559] task:python3 state:D stack:25336 pid:5623 tgid:5623 ppid:5622 task_flags:0x400100 flags:0x00080002 [ 1235.483769] Call Trace: [ 1235.487381] [ 1235.490523] __schedule+0xae3/0x14d0 [ 1235.495338] ? io_schedule_timeout+0x140/0x140 [ 1235.501121] ? find_held_lock+0x2b/0x80 [ 1235.506228] ? __lock_release.isra.0+0x6b/0x1a0 [ 1235.512111] schedule+0xd4/0x260 [ 1235.516530] schedule_preempt_disabled+0x18/0x30 [ 1235.522510] __mutex_lock+0x6b2/0x1ee0 [ 1235.527517] ? rcu_is_watching+0x15/0xd0 [ 1235.532714] ? threaded_store+0x132/0x220 [ 1235.538017] ? ww_mutex_lock+0x160/0x160 [ 1235.543218] ? selinux_inode_free_security+0x330/0x330 [ 1235.549789] ? cap_capable+0x1ed/0x3f0 [ 1235.554800] ? threaded_store+0x132/0x220 [ 1235.560098] threaded_store+0x132/0x220 [ 1235.565202] ? xps_rxqs_store+0x2e0/0x2e0 [ 1235.570498] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.575799] ? sysfs_kf_write+0x65/0x160 [ 1235.580999] ? sysfs_file_ops+0x140/0x140 [ 1235.586301] kernfs_fop_write_iter+0x335/0x510 [ 1235.592090] ? kernfs_ops+0xf0/0xf0 [ 1235.596802] new_sync_write+0x333/0x750 [ 1235.601897] ? __lock_acquire+0x508/0xc10 [ 1235.607195] ? new_sync_read+0x740/0x740 [ 1235.612397] ? lock_acquire.part.0+0xbc/0x260 [ 1235.618084] ? ksys_write+0x116/0x250 [ 1235.622991] vfs_write+0x657/0x9b0 [ 1235.627609] ksys_write+0x116/0x250 [ 1235.632325] ? __ia32_sys_read+0xc0/0xc0 [ 1235.637524] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.643115] ? rcu_is_watching+0x15/0xd0 [ 1235.648317] do_syscall_64+0xf3/0x5e0 [ 1235.653228] ? trace_hardirqs_off+0xd/0x30 [ 1235.658624] ? exc_page_fault+0xda/0xf0 [ 1235.663719] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.670186] RIP: 0033:0x7f4225baf22e [ 1235.674999] RSP: 002b:00007ffc74f835c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.684303] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f4225baf22e [ 1235.693132] RDX: 0000000000000001 RSI: 0000557332870f40 RDI: 0000000000000007 [ 1235.701959] RBP: 00007ffc74f835d0 R08: 0000000000000000 R09: 0000000000000000 [ 1235.710767] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f4225a27390 [ 1235.719563] R13: 0000557332870f40 R14: 0000000000000007 R15: 0000000000000003 | [ 1236.192995] #2: ffffffff8fb37108 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_lock_and_drain_offline+0xa6/0x560 | [ 1236.204638] #3: ffffffff8f968b50 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_update_dfl_csses+0x2a3/0x5c0 | [ 1236.216184] #4: ffffffff8fb36e10 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_update_dfl_csses+0x2af/0x5c0 | [ 1236.228514] | [ 1236.230977] =============================================\x0a [ 1358.326745] INFO: task python3:5623 blocked for more than 491 seconds. [ 1358.334895] Not tainted 7.0.0-rc6-yesk-ga6e6a7dcb853 #1 [ 1358.342144] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1358.351735] task:python3 state:D stack:25336 pid:5623 tgid:5623 ppid:5622 task_flags:0x400100 flags:0x00080002 [ 1358.364942] Call Trace: [ 1358.368482] [ 1358.371635] __schedule+0xae3/0x14d0 [ 1358.376452] ? io_schedule_timeout+0x140/0x140 [ 1358.382238] ? find_held_lock+0x2b/0x80 [ 1358.387343] ? __lock_release.isra.0+0x6b/0x1a0 [ 1358.393219] schedule+0xd4/0x260 [ 1358.397640] schedule_preempt_disabled+0x18/0x30 [ 1358.403620] __mutex_lock+0x6b2/0x1ee0 [ 1358.408628] ? rcu_is_watching+0x15/0xd0 [ 1358.413829] ? threaded_store+0x132/0x220 [ 1358.419129] ? ww_mutex_lock+0x160/0x160 [ 1358.424329] ? selinux_inode_free_security+0x330/0x330 [ 1358.430899] ? cap_capable+0x1ed/0x3f0 [ 1358.435908] ? threaded_store+0x132/0x220 [ 1358.441207] threaded_store+0x132/0x220 [ 1358.446308] ? xps_rxqs_store+0x2e0/0x2e0 [ 1358.451611] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.456910] ? sysfs_kf_write+0x65/0x160 [ 1358.462111] ? sysfs_file_ops+0x140/0x140 [ 1358.467407] kernfs_fop_write_iter+0x335/0x510 [ 1358.473191] ? kernfs_ops+0xf0/0xf0 [ 1358.477905] new_sync_write+0x333/0x750 [ 1358.483009] ? __lock_acquire+0x508/0xc10 [ 1358.488297] ? new_sync_read+0x740/0x740 [ 1358.493497] ? lock_acquire.part.0+0xbc/0x260 [ 1358.499175] ? ksys_write+0x116/0x250 [ 1358.504079] vfs_write+0x657/0x9b0 [ 1358.508697] ksys_write+0x116/0x250 [ 1358.513408] ? __ia32_sys_read+0xc0/0xc0 [ 1358.518608] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.524201] ? rcu_is_watching+0x15/0xd0 [ 1358.529403] do_syscall_64+0xf3/0x5e0 [ 1358.534311] ? trace_hardirqs_off+0xd/0x30 [ 1358.539706] ? exc_page_fault+0xda/0xf0 [ 1358.544807] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.551274] RIP: 0033:0x7f4225baf22e [ 1358.556084] RSP: 002b:00007ffc74f835c0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.565372] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f4225baf22e [ 1358.574172] RDX: 0000000000000001 RSI: 0000557332870f40 RDI: 0000000000000007 [ 1358.582969] RBP: 00007ffc74f835d0 R08: 0000000000000000 R09: 0000000000000000 [ 1358.591779] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f4225a27390 [ 1358.600589] R13: 0000557332870f40 R14: 0000000000000007 R15: 0000000000000003 Finger prints: __schedule:schedule:schedule_preempt_disabled:__mutex_lock:threaded_store