====================================== | [ 779.976722] nipa-hw-worker: drivers/net:napi_threaded.py: timed out | [ 779.979518] nipa-hw-worker: [4/40] Retrying drivers/net:napi_threaded.py | [ 780.111114] kselftest: Running tests in drivers/net [ 989.659622] INFO: task python3:5572 blocked for more than 122 seconds. [ 989.667855] Not tainted 7.0.0-rc6-ocqi-g88b19b5f0e96 #1 [ 989.675106] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 989.684693] task:python3 state:D stack:25336 pid:5572 tgid:5572 ppid:5571 task_flags:0x400100 flags:0x00080000 [ 989.697924] Call Trace: [ 989.701473] [ 989.704624] __schedule+0xae3/0x14d0 [ 989.709438] ? io_schedule_timeout+0x140/0x140 [ 989.715214] ? find_held_lock+0x2b/0x80 [ 989.720317] ? __lock_release.isra.0+0x6b/0x1a0 [ 989.726203] schedule+0xd4/0x260 [ 989.730620] schedule_preempt_disabled+0x18/0x30 [ 989.736598] __mutex_lock+0x6b2/0x1ee0 [ 989.741603] ? rcu_is_watching+0x15/0xd0 [ 989.746801] ? threaded_store+0x132/0x220 [ 989.752099] ? ww_mutex_lock+0x160/0x160 [ 989.757300] ? selinux_inode_free_security+0x330/0x330 [ 989.763872] ? cap_capable+0x1ed/0x3f0 [ 989.763880] ? threaded_store+0x132/0x220 [ 989.763884] threaded_store+0x132/0x220 [ 989.763888] ? xps_rxqs_store+0x2e0/0x2e0 [ 989.763891] ? sysfs_file_kobj+0xbd/0x1e0 [ 989.789939] ? sysfs_kf_write+0x65/0x160 [ 989.795140] ? sysfs_file_ops+0x140/0x140 [ 989.800434] kernfs_fop_write_iter+0x335/0x510 [ 989.806218] ? kernfs_ops+0xf0/0xf0 [ 989.810928] new_sync_write+0x333/0x750 [ 989.816031] ? __lock_acquire+0x508/0xc10 [ 989.821325] ? new_sync_read+0x740/0x740 [ 989.826526] ? lock_acquire.part.0+0xbc/0x260 [ 989.832212] ? ksys_write+0x116/0x250 [ 989.837119] vfs_write+0x657/0x9b0 [ 989.841731] ksys_write+0x116/0x250 [ 989.846442] ? __ia32_sys_read+0xc0/0xc0 [ 989.851639] ? kernfs_fop_llseek+0x160/0x1f0 [ 989.857226] ? rcu_is_watching+0x15/0xd0 [ 989.862424] do_syscall_64+0xf3/0x5e0 [ 989.867328] ? trace_hardirqs_off+0xd/0x30 [ 989.872723] ? exc_page_fault+0xda/0xf0 [ 989.877825] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 989.884304] RIP: 0033:0x7f518f65b22e [ 989.889113] RSP: 002b:00007ffc39b3f7e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 989.898412] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f518f65b22e [ 989.907217] RDX: 0000000000000001 RSI: 000055d6f6eaf340 RDI: 0000000000000007 [ 989.916025] RBP: 00007ffc39b3f7f0 R08: 0000000000000000 R09: 0000000000000000 [ 989.924836] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f518f4d3390 [ 989.933643] R13: 000055d6f6eaf340 R14: 0000000000000007 R15: 0000000000000003 | [ 990.414849] #2: ffffffffb9937108 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0 | [ 990.425517] #3: ffffffffb9768b50 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x6b0 | [ 990.437158] #4: ffffffffb9936e10 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x6b0 | [ 990.449586] | [ 990.452051] =============================================\x0a [ 1112.539871] INFO: task python3:5572 blocked for more than 245 seconds. [ 1112.548018] Not tainted 7.0.0-rc6-ocqi-g88b19b5f0e96 #1 [ 1112.555268] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1112.564859] task:python3 state:D stack:25336 pid:5572 tgid:5572 ppid:5571 task_flags:0x400100 flags:0x00080000 [ 1112.578065] Call Trace: [ 1112.581605] [ 1112.584759] __schedule+0xae3/0x14d0 [ 1112.589578] ? io_schedule_timeout+0x140/0x140 [ 1112.595363] ? find_held_lock+0x2b/0x80 [ 1112.600472] ? __lock_release.isra.0+0x6b/0x1a0 [ 1112.606355] schedule+0xd4/0x260 [ 1112.610768] schedule_preempt_disabled+0x18/0x30 [ 1112.616748] __mutex_lock+0x6b2/0x1ee0 [ 1112.621757] ? rcu_is_watching+0x15/0xd0 [ 1112.626958] ? threaded_store+0x132/0x220 [ 1112.632260] ? ww_mutex_lock+0x160/0x160 [ 1112.637460] ? selinux_inode_free_security+0x330/0x330 [ 1112.644029] ? cap_capable+0x1ed/0x3f0 [ 1112.649037] ? threaded_store+0x132/0x220 [ 1112.654334] threaded_store+0x132/0x220 [ 1112.659430] ? xps_rxqs_store+0x2e0/0x2e0 [ 1112.664731] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.670028] ? sysfs_kf_write+0x65/0x160 [ 1112.675229] ? sysfs_file_ops+0x140/0x140 [ 1112.680526] kernfs_fop_write_iter+0x335/0x510 [ 1112.686305] ? kernfs_ops+0xf0/0xf0 [ 1112.691016] new_sync_write+0x333/0x750 [ 1112.696120] ? __lock_acquire+0x508/0xc10 [ 1112.701410] ? new_sync_read+0x740/0x740 [ 1112.706612] ? lock_acquire.part.0+0xbc/0x260 [ 1112.712298] ? ksys_write+0x116/0x250 [ 1112.717208] vfs_write+0x657/0x9b0 [ 1112.721824] ksys_write+0x116/0x250 [ 1112.726537] ? __ia32_sys_read+0xc0/0xc0 [ 1112.731730] ? kernfs_fop_llseek+0x160/0x1f0 [ 1112.737322] ? rcu_is_watching+0x15/0xd0 [ 1112.742520] do_syscall_64+0xf3/0x5e0 [ 1112.747428] ? trace_hardirqs_off+0xd/0x30 [ 1112.752823] ? exc_page_fault+0xda/0xf0 [ 1112.757925] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1112.764385] RIP: 0033:0x7f518f65b22e [ 1112.769196] RSP: 002b:00007ffc39b3f7e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1112.778494] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f518f65b22e [ 1112.787304] RDX: 0000000000000001 RSI: 000055d6f6eaf340 RDI: 0000000000000007 [ 1112.796114] RBP: 00007ffc39b3f7f0 R08: 0000000000000000 R09: 0000000000000000 [ 1112.804923] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f518f4d3390 [ 1112.813731] R13: 000055d6f6eaf340 R14: 0000000000000007 R15: 0000000000000003 | [ 1113.217221] | [ 1113.219688] =============================================\x0a | [ 1212.057485] block nvme0n1: No UUID available providing old NGUID | SUBSYSTEM=block | DEVICE=b259:0 [ 1235.421137] INFO: task python3:5572 blocked for more than 368 seconds. [ 1235.429286] Not tainted 7.0.0-rc6-ocqi-g88b19b5f0e96 #1 [ 1235.436536] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1235.446126] task:python3 state:D stack:25336 pid:5572 tgid:5572 ppid:5571 task_flags:0x400100 flags:0x00080002 [ 1235.459336] Call Trace: [ 1235.462876] [ 1235.466024] __schedule+0xae3/0x14d0 [ 1235.470843] ? io_schedule_timeout+0x140/0x140 [ 1235.476628] ? find_held_lock+0x2b/0x80 [ 1235.481736] ? __lock_release.isra.0+0x6b/0x1a0 [ 1235.487622] schedule+0xd4/0x260 [ 1235.492039] schedule_preempt_disabled+0x18/0x30 [ 1235.498022] __mutex_lock+0x6b2/0x1ee0 [ 1235.503029] ? rcu_is_watching+0x15/0xd0 [ 1235.508228] ? threaded_store+0x132/0x220 [ 1235.513530] ? ww_mutex_lock+0x160/0x160 [ 1235.518728] ? selinux_inode_free_security+0x330/0x330 [ 1235.525291] ? cap_capable+0x1ed/0x3f0 [ 1235.530369] ? threaded_store+0x132/0x220 [ 1235.535667] threaded_store+0x132/0x220 [ 1235.540769] ? xps_rxqs_store+0x2e0/0x2e0 [ 1235.546066] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.551364] ? sysfs_kf_write+0x65/0x160 [ 1235.556565] ? sysfs_file_ops+0x140/0x140 [ 1235.561860] kernfs_fop_write_iter+0x335/0x510 [ 1235.567668] ? kernfs_ops+0xf0/0xf0 [ 1235.572411] new_sync_write+0x333/0x750 [ 1235.577517] ? __lock_acquire+0x508/0xc10 [ 1235.582884] ? new_sync_read+0x740/0x740 [ 1235.588086] ? lock_acquire.part.0+0xbc/0x260 [ 1235.593772] ? ksys_write+0x116/0x250 [ 1235.598682] vfs_write+0x657/0x9b0 [ 1235.603296] ksys_write+0x116/0x250 [ 1235.608006] ? __ia32_sys_read+0xc0/0xc0 [ 1235.613205] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.618793] ? rcu_is_watching+0x15/0xd0 [ 1235.623995] do_syscall_64+0xf3/0x5e0 [ 1235.628902] ? trace_hardirqs_off+0xd/0x30 [ 1235.634297] ? exc_page_fault+0xda/0xf0 [ 1235.639399] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.645867] RIP: 0033:0x7f518f65b22e [ 1235.650676] RSP: 002b:00007ffc39b3f7e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.659971] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f518f65b22e [ 1235.668780] RDX: 0000000000000001 RSI: 000055d6f6eaf340 RDI: 0000000000000007 [ 1235.677588] RBP: 00007ffc39b3f7f0 R08: 0000000000000000 R09: 0000000000000000 [ 1235.686397] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f518f4d3390 [ 1235.695205] R13: 000055d6f6eaf340 R14: 0000000000000007 R15: 0000000000000003 | [ 1236.077870] #2: ff11000122d230f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 | [ 1236.089029] #3: ff11000122d91d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 | [ 1236.099121] 1 lock held by systemd/7081: | [ 1236.104320] | [ 1236.106786] =============================================\x0a [ 1358.302377] INFO: task python3:5572 blocked for more than 491 seconds. [ 1358.310526] Not tainted 7.0.0-rc6-ocqi-g88b19b5f0e96 #1 [ 1358.317778] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1358.327370] task:python3 state:D stack:25336 pid:5572 tgid:5572 ppid:5571 task_flags:0x400100 flags:0x00080002 [ 1358.340580] Call Trace: [ 1358.344122] [ 1358.347276] __schedule+0xae3/0x14d0 [ 1358.352097] ? io_schedule_timeout+0x140/0x140 [ 1358.357884] ? find_held_lock+0x2b/0x80 [ 1358.362990] ? __lock_release.isra.0+0x6b/0x1a0 [ 1358.368915] schedule+0xd4/0x260 [ 1358.373551] schedule_preempt_disabled+0x18/0x30 [ 1358.379533] __mutex_lock+0x6b2/0x1ee0 [ 1358.384539] ? rcu_is_watching+0x15/0xd0 [ 1358.389740] ? threaded_store+0x132/0x220 [ 1358.395041] ? ww_mutex_lock+0x160/0x160 [ 1358.400239] ? selinux_inode_free_security+0x330/0x330 [ 1358.406811] ? cap_capable+0x1ed/0x3f0 [ 1358.411822] ? threaded_store+0x132/0x220 [ 1358.417117] threaded_store+0x132/0x220 [ 1358.422219] ? xps_rxqs_store+0x2e0/0x2e0 [ 1358.427520] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.432820] ? sysfs_kf_write+0x65/0x160 [ 1358.438022] ? sysfs_file_ops+0x140/0x140 [ 1358.443320] kernfs_fop_write_iter+0x335/0x510 [ 1358.449107] ? kernfs_ops+0xf0/0xf0 [ 1358.453821] new_sync_write+0x333/0x750 [ 1358.458926] ? __lock_acquire+0x508/0xc10 [ 1358.464229] ? new_sync_read+0x740/0x740 [ 1358.469429] ? lock_acquire.part.0+0xbc/0x260 [ 1358.475108] ? ksys_write+0x116/0x250 [ 1358.480019] vfs_write+0x657/0x9b0 [ 1358.484632] ksys_write+0x116/0x250 [ 1358.489344] ? __ia32_sys_read+0xc0/0xc0 [ 1358.494544] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.500125] ? rcu_is_watching+0x15/0xd0 [ 1358.505319] do_syscall_64+0xf3/0x5e0 [ 1358.510232] ? trace_hardirqs_off+0xd/0x30 [ 1358.515628] ? exc_page_fault+0xda/0xf0 [ 1358.520730] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.527187] RIP: 0033:0x7f518f65b22e [ 1358.531988] RSP: 002b:00007ffc39b3f7e0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.541289] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f518f65b22e [ 1358.550098] RDX: 0000000000000001 RSI: 000055d6f6eaf340 RDI: 0000000000000007 [ 1358.558906] RBP: 00007ffc39b3f7f0 R08: 0000000000000000 R09: 0000000000000000 [ 1358.567715] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f518f4d3390 [ 1358.576526] R13: 000055d6f6eaf340 R14: 0000000000000007 R15: 0000000000000003 Finger prints: __schedule:schedule:schedule_preempt_disabled:__mutex_lock:threaded_store