[ 789.069996] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 789.072483] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 789.198062] kselftest: Running tests in drivers/net
[ 989.707910] INFO: task python3:5836 blocked for more than 122 seconds.
[ 989.716147] Not tainted 7.1.0-rc6-jfew-ga29b13223e9d #1
[ 989.723993] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.733581] task:python3 state:D stack:25672 pid:5836 tgid:5836 ppid:5835 task_flags:0x400100 flags:0x00080000
[ 989.746788] Call Trace:
[ 989.750325]
[ 989.753474] __schedule+0xcc3/0x19c0
[ 989.758291] ? io_schedule_timeout+0x140/0x140
[ 989.764073] ? find_held_lock+0x2b/0x80
[ 989.769176] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.775062] schedule+0xd8/0x260
[ 989.779480] schedule_preempt_disabled+0x18/0x30
[ 989.785460] __mutex_lock+0x6af/0x2010
[ 989.790457] ? threaded_store+0x132/0x220
[ 989.795756] ? ww_mutex_lock+0x160/0x160
[ 989.800952] ? selinux_inode_free_security+0x330/0x330
[ 989.807523] ? cap_capable+0x1d6/0x3e0
[ 989.812529] ? threaded_store+0x132/0x220
[ 989.817816] threaded_store+0x132/0x220
[ 989.822919] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.828213] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.833512] ? sysfs_kf_write+0x65/0x160
[ 989.838700] ? sysfs_file_ops+0x140/0x140
[ 989.843994] kernfs_fop_write_iter+0x335/0x510
[ 989.849778] ? kernfs_ops+0xf0/0xf0
[ 989.854490] new_sync_write+0x333/0x750
[ 989.859594] ? __lock_acquire+0x508/0xc10
[ 989.864881] ? new_sync_read+0x740/0x740
[ 989.870081] ? lock_acquire.part.0+0xbc/0x260
[ 989.875765] ? ksys_write+0x116/0x250
[ 989.880676] vfs_write+0x657/0x9b0
[ 989.885289] ksys_write+0x116/0x250
[ 989.890001] ? __ia32_sys_read+0xc0/0xc0
[ 989.895199] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.900788] ? rcu_is_watching+0x15/0xd0
[ 989.905990] do_syscall_64+0xf3/0x650
[ 989.910896] ? trace_hardirqs_off+0xd/0x30
[ 989.916291] ? exc_page_fault+0xda/0xf0
[ 989.921391] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.927857] RIP: 0033:0x7febb0f7922e
[ 989.932655] RSP: 002b:00007ffdcccabf40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.941952] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007febb0f7922e
[ 989.950759] RDX: 0000000000000001 RSI: 0000556dd5445e90 RDI: 0000000000000007
[ 989.959568] RBP: 00007ffdcccabf50 R08: 0000000000000000 R09: 0000000000000000
[ 989.968376] R10: 0000000000000000 R11: 0000000000000202 R12: 00007febb0df1390
[ 989.977173] R13: 0000556dd5445e90 R14: 0000000000000007 R15: 0000000000000003
[ 989.985992]
[ 989.989282] INFO: task python3:5836 is blocked on a mutex likely owned by task python3:3788.
[ 989.999552] task:python3 state:D stack:25672 pid:3788 tgid:3788 ppid:3787 task_flags:0x400100 flags:0x00080802
[ 990.012751] Call Trace:
[ 990.016288]
[ 990.019436] __schedule+0xcc3/0x19c0
[ 990.024244] ? io_schedule_timeout+0x140/0x140
[ 990.030028] ? find_held_lock+0x2b/0x80
[ 990.035131] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.041306] schedule+0xd8/0x260
[ 990.045729] schedule_timeout+0x13f/0x290
[ 990.051027] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.057396] ? trace_contention_end+0xb3/0x1a0
[ 990.063180] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.068870] ? ww_mutex_lock+0x160/0x160
[ 990.074070] msleep+0x1b/0x30
[ 990.078193] napi_set_threaded+0x3a5/0x440
[ 990.083586] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.089080] netif_set_threaded+0x257/0x400
[ 990.094573] threaded_store+0x1b8/0x220
[ 990.099677] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.104973] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.110270] ? sysfs_kf_write+0x65/0x160
[ 990.115467] ? sysfs_file_ops+0x140/0x140
[ 990.120760] kernfs_fop_write_iter+0x335/0x510
[ 990.126546] ? kernfs_ops+0xf0/0xf0
[ 990.131257] new_sync_write+0x333/0x750
[ 990.136357] ? __lock_acquire+0x508/0xc10
[ 990.141650] ? new_sync_read+0x740/0x740
[ 990.146849] ? lock_acquire.part.0+0xbc/0x260
[ 990.152532] ? ksys_write+0x116/0x250
[ 990.157444] vfs_write+0x657/0x9b0
[ 990.162060] ksys_write+0x116/0x250
[ 990.166772] ? __ia32_sys_read+0xc0/0xc0
[ 990.171971] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.177558] ? rcu_is_watching+0x15/0xd0
[ 990.182757] do_syscall_64+0xf3/0x650
[ 990.187652] ? trace_hardirqs_off+0xd/0x30
[ 990.193043] ? exc_page_fault+0xda/0xf0
[ 990.198145] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.204612] RIP: 0033:0x7fc98fe5b22e
[ 990.209420] RSP: 002b:00007ffdef52f8f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.218715] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc98fe5b22e
[ 990.227523] RDX: 0000000000000001 RSI: 000055fccdfbfe90 RDI: 0000000000000007
[ 990.236331] RBP: 00007ffdef52f900 R08: 0000000000000000 R09: 0000000000000000
[ 990.245139] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc98fcd3390
[ 990.253937] R13: 000055fccdfbfe90 R14: 0000000000000007 R15: 0000000000000003
[ 990.262748]
[ 990.266006] \x0aShowing all locks held in the system:
[ 990.274542] 1 lock held by khungtaskd/184:
[ 990.279936] #0: ffffffff950fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.291409] 2 locks held by systemd-journal/802:
[ 990.297386] #0: ff1100180e1374e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.307274] #1: ff1100019c841b78 (&mm->mmap_lock){++++}-{4:4}, at: fs_reclaim_acquire+0x7b/0x120
[ 990.318048] 4 locks held by python3/3788:
[ 990.323343] #0: ff110001304943e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.333232] #1: ff1100017bb66e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.344192] #2: ff110001243f4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.355347] #3: ff11000124409d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.365443] 4 locks held by python3/5836:
[ 990.370739] #0: ff110001304943e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.380630] #1: ff110001f3385680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.391588] #2: ff110001243f4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.402746] #3: ff11000124409d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.412831]
[ 990.415296] =============================================\x0a
[ 1112.589121] INFO: task python3:5836 blocked for more than 245 seconds.
[ 1112.597267] Not tainted 7.1.0-rc6-jfew-ga29b13223e9d #1
[ 1112.604517] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.614110] task:python3 state:D stack:25672 pid:5836 tgid:5836 ppid:5835 task_flags:0x400100 flags:0x00080000
[ 1112.627318] Call Trace:
[ 1112.630858]
[ 1112.634011] __schedule+0xcc3/0x19c0
[ 1112.638828] ? io_schedule_timeout+0x140/0x140
[ 1112.644615] ? find_held_lock+0x2b/0x80
[ 1112.649722] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.655609] schedule+0xd8/0x260
[ 1112.660029] schedule_preempt_disabled+0x18/0x30
[ 1112.666010] __mutex_lock+0x6af/0x2010
[ 1112.671017] ? threaded_store+0x132/0x220
[ 1112.676319] ? ww_mutex_lock+0x160/0x160
[ 1112.681517] ? selinux_inode_free_security+0x330/0x330
[ 1112.688090] ? cap_capable+0x1d6/0x3e0
[ 1112.693099] ? threaded_store+0x132/0x220
[ 1112.698396] threaded_store+0x132/0x220
[ 1112.703493] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.708793] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.714096] ? sysfs_kf_write+0x65/0x160
[ 1112.719294] ? sysfs_file_ops+0x140/0x140
[ 1112.724590] kernfs_fop_write_iter+0x335/0x510
[ 1112.730376] ? kernfs_ops+0xf0/0xf0
[ 1112.735090] new_sync_write+0x333/0x750
[ 1112.740193] ? __lock_acquire+0x508/0xc10
[ 1112.745489] ? new_sync_read+0x740/0x740
[ 1112.750690] ? lock_acquire.part.0+0xbc/0x260
[ 1112.756366] ? ksys_write+0x116/0x250
[ 1112.761275] vfs_write+0x657/0x9b0
[ 1112.765891] ksys_write+0x116/0x250
[ 1112.770604] ? __ia32_sys_read+0xc0/0xc0
[ 1112.775796] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.781385] ? rcu_is_watching+0x15/0xd0
[ 1112.786588] do_syscall_64+0xf3/0x650
[ 1112.791497] ? trace_hardirqs_off+0xd/0x30
[ 1112.796891] ? exc_page_fault+0xda/0xf0
[ 1112.801994] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.808464] RIP: 0033:0x7febb0f7922e
[ 1112.813274] RSP: 002b:00007ffdcccabf40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.822572] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007febb0f7922e
[ 1112.831379] RDX: 0000000000000001 RSI: 0000556dd5445e90 RDI: 0000000000000007
[ 1112.840188] RBP: 00007ffdcccabf50 R08: 0000000000000000 R09: 0000000000000000
[ 1112.848999] R10: 0000000000000000 R11: 0000000000000202 R12: 00007febb0df1390
[ 1112.857810] R13: 0000556dd5445e90 R14: 0000000000000007 R15: 0000000000000003
[ 1112.866622]
[ 1112.869895] INFO: task python3:5836 is blocked on a mutex likely owned by task python3:3788.
[ 1112.880169] task:python3 state:D stack:25672 pid:3788 tgid:3788 ppid:3787 task_flags:0x400100 flags:0x00080802
[ 1112.893371] Call Trace:
[ 1112.896911]
[ 1112.900063] __schedule+0xcc3/0x19c0
[ 1112.904876] ? io_schedule_timeout+0x140/0x140
[ 1112.910659] ? find_held_lock+0x2b/0x80
[ 1112.915762] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.921943] schedule+0xd8/0x260
[ 1112.926361] schedule_timeout+0x13f/0x290
[ 1112.931658] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.938027] ? trace_contention_end+0xb3/0x1a0
[ 1112.943813] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.949505] ? ww_mutex_lock+0x160/0x160
[ 1112.954705] msleep+0x1b/0x30
[ 1112.958825] napi_set_threaded+0x3a5/0x440
[ 1112.964223] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.969718] netif_set_threaded+0x257/0x400
[ 1112.975211] threaded_store+0x1b8/0x220
[ 1112.980312] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.985610] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.990900] ? sysfs_kf_write+0x65/0x160
[ 1112.996098] ? sysfs_file_ops+0x140/0x140
[ 1113.001394] kernfs_fop_write_iter+0x335/0x510
[ 1113.007180] ? kernfs_ops+0xf0/0xf0
[ 1113.011890] new_sync_write+0x333/0x750
[ 1113.016994] ? __lock_acquire+0x508/0xc10
[ 1113.022288] ? new_sync_read+0x740/0x740
[ 1113.027488] ? lock_acquire.part.0+0xbc/0x260
[ 1113.033175] ? ksys_write+0x116/0x250
[ 1113.038078] vfs_write+0x657/0x9b0
[ 1113.042695] ksys_write+0x116/0x250
[ 1113.047408] ? __ia32_sys_read+0xc0/0xc0
[ 1113.052608] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.058199] ? rcu_is_watching+0x15/0xd0
[ 1113.063401] do_syscall_64+0xf3/0x650
[ 1113.068309] ? trace_hardirqs_off+0xd/0x30
[ 1113.073702] ? exc_page_fault+0xda/0xf0
[ 1113.078804] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.085271] RIP: 0033:0x7fc98fe5b22e
[ 1113.090070] RSP: 002b:00007ffdef52f8f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.099369] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc98fe5b22e
[ 1113.108177] RDX: 0000000000000001 RSI: 000055fccdfbfe90 RDI: 0000000000000007
[ 1113.116987] RBP: 00007ffdef52f900 R08: 0000000000000000 R09: 0000000000000000
[ 1113.125796] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc98fcd3390
[ 1113.134604] R13: 000055fccdfbfe90 R14: 0000000000000007 R15: 0000000000000003
[ 1113.143420]
[ 1113.146685] \x0aShowing all locks held in the system:
[ 1113.155224] 1 lock held by khungtaskd/184:
[ 1113.160618] #0: ffffffff950fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.172099] 4 locks held by python3/3788:
[ 1113.177395] #0: ff110001304943e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.187288] #1: ff1100017bb66e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.198250] #2: ff110001243f4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.209472] #3: ff11000124409d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.219571] 4 locks held by python3/5836:
[ 1113.224870] #0: ff110001304943e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.224879] #1: ff110001f3385680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.224887] #2: ff110001243f4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.224893] #3: ff11000124409d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.267005] 5 locks held by (systemd-user-)/6881:
[ 1113.273085] #0: ff11000130cf43e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.282973] #1: ff1100012e0cf480 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.293935] #2: ffffffff95169c20 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_kn_lock_live+0xe2/0x3e0
[ 1113.304605] #3: ffffffff94f687b0 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_procs_write_start+0x3e5/0x650
[ 1113.316249] #4: ffffffff95169990 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_procs_write_start+0x3f1/0x650
[ 1113.328680]
[ 1113.331138] =============================================\x0a
[ 1235.471378] INFO: task python3:5836 blocked for more than 368 seconds.
[ 1235.479525] Not tainted 7.1.0-rc6-jfew-ga29b13223e9d #1
[ 1235.486774] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.496363] task:python3 state:D stack:25672 pid:5836 tgid:5836 ppid:5835 task_flags:0x400100 flags:0x00080002
[ 1235.509569] Call Trace:
[ 1235.513104]
[ 1235.516257] __schedule+0xcc3/0x19c0
[ 1235.521070] ? io_schedule_timeout+0x140/0x140
[ 1235.526856] ? find_held_lock+0x2b/0x80
[ 1235.531961] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.537835] schedule+0xd8/0x260
[ 1235.542254] schedule_preempt_disabled+0x18/0x30
[ 1235.548233] __mutex_lock+0x6af/0x2010
[ 1235.553230] ? threaded_store+0x132/0x220
[ 1235.558527] ? ww_mutex_lock+0x160/0x160
[ 1235.563726] ? selinux_inode_free_security+0x330/0x330
[ 1235.570295] ? cap_capable+0x1d6/0x3e0
[ 1235.575301] ? threaded_store+0x132/0x220
[ 1235.580598] threaded_store+0x132/0x220
[ 1235.585697] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.590994] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.596286] ? sysfs_kf_write+0x65/0x160
[ 1235.601487] ? sysfs_file_ops+0x140/0x140
[ 1235.606782] kernfs_fop_write_iter+0x335/0x510
[ 1235.612564] ? kernfs_ops+0xf0/0xf0
[ 1235.617274] new_sync_write+0x333/0x750
[ 1235.622374] ? __lock_acquire+0x508/0xc10
[ 1235.627669] ? new_sync_read+0x740/0x740
[ 1235.632871] ? lock_acquire.part.0+0xbc/0x260
[ 1235.638555] ? ksys_write+0x116/0x250
[ 1235.643465] vfs_write+0x657/0x9b0
[ 1235.648081] ksys_write+0x116/0x250
[ 1235.652794] ? __ia32_sys_read+0xc0/0xc0
[ 1235.657981] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.663571] ? rcu_is_watching+0x15/0xd0
[ 1235.668772] do_syscall_64+0xf3/0x650
[ 1235.673676] ? trace_hardirqs_off+0xd/0x30
[ 1235.679071] ? exc_page_fault+0xda/0xf0
[ 1235.684172] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.690640] RIP: 0033:0x7febb0f7922e
[ 1235.695451] RSP: 002b:00007ffdcccabf40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.704749] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007febb0f7922e
[ 1235.713556] RDX: 0000000000000001 RSI: 0000556dd5445e90 RDI: 0000000000000007
[ 1235.722360] RBP: 00007ffdcccabf50 R08: 0000000000000000 R09: 0000000000000000
[ 1235.731166] R10: 0000000000000000 R11: 0000000000000202 R12: 00007febb0df1390
[ 1235.739976] R13: 0000556dd5445e90 R14: 0000000000000007 R15: 0000000000000003
[ 1235.748787]
[ 1235.752066] INFO: task python3:5836 is blocked on a mutex likely owned by task python3:3788.
[ 1235.762336] task:python3 state:D stack:25672 pid:3788 tgid:3788 ppid:3787 task_flags:0x400100 flags:0x00080802
[ 1235.775540] Call Trace:
[ 1235.779076]
[ 1235.782227] __schedule+0xcc3/0x19c0
[ 1235.787027] ? io_schedule_timeout+0x140/0x140
[ 1235.792812] ? find_held_lock+0x2b/0x80
[ 1235.797916] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.804094] schedule+0xd8/0x260
[ 1235.808511] schedule_timeout+0x13f/0x290
[ 1235.813809] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.820170] ? trace_contention_end+0xb3/0x1a0
[ 1235.825956] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.831647] ? ww_mutex_lock+0x160/0x160
[ 1235.836847] msleep+0x1b/0x30
[ 1235.840965] napi_set_threaded+0x3a5/0x440
[ 1235.846359] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.851853] netif_set_threaded+0x257/0x400
[ 1235.857344] threaded_store+0x1b8/0x220
[ 1235.862445] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.867744] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.873040] ? sysfs_kf_write+0x65/0x160
[ 1235.878240] ? sysfs_file_ops+0x140/0x140
[ 1235.883525] kernfs_fop_write_iter+0x335/0x510
[ 1235.889309] ? kernfs_ops+0xf0/0xf0
[ 1235.894021] new_sync_write+0x333/0x750
[ 1235.899123] ? __lock_acquire+0x508/0xc10
[ 1235.904418] ? new_sync_read+0x740/0x740
[ 1235.909615] ? lock_acquire.part.0+0xbc/0x260
[ 1235.915301] ? ksys_write+0x116/0x250
[ 1235.920211] vfs_write+0x657/0x9b0
[ 1235.924825] ksys_write+0x116/0x250
[ 1235.929538] ? __ia32_sys_read+0xc0/0xc0
[ 1235.934728] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.940315] ? rcu_is_watching+0x15/0xd0
[ 1235.945515] do_syscall_64+0xf3/0x650
[ 1235.950411] ? trace_hardirqs_off+0xd/0x30
[ 1235.955803] ? exc_page_fault+0xda/0xf0
[ 1235.960902] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.967368] RIP: 0033:0x7fc98fe5b22e
[ 1235.972174] RSP: 002b:00007ffdef52f8f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.981472] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc98fe5b22e
[ 1235.990280] RDX: 0000000000000001 RSI: 000055fccdfbfe90 RDI: 0000000000000007
[ 1235.999088] RBP: 00007ffdef52f900 R08: 0000000000000000 R09: 0000000000000000
[ 1236.007895] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc98fcd3390
[ 1236.016701] R13: 000055fccdfbfe90 R14: 0000000000000007 R15: 0000000000000003
[ 1236.025512]
[ 1236.028778] \x0aShowing all locks held in the system:
[ 1236.037306] 1 lock held by khungtaskd/184:
[ 1236.042700] #0: ffffffff950fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.054179] 4 locks held by python3/3788:
[ 1236.059473] #0: ff110001304943e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.069361] #1: ff1100017bb66e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.080321] #2: ff110001243f4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.091474] #3: ff11000124409d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.101568] 4 locks held by python3/5836:
[ 1236.106861] #0: ff110001304943e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.116748] #1: ff110001f3385680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.127708] #2: ff110001243f4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.138867] #3: ff11000124409d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.148959]
[ 1236.151424] =============================================\x0a
[ 1358.351580] INFO: task python3:5836 blocked for more than 491 seconds.
[ 1358.359730] Not tainted 7.1.0-rc6-jfew-ga29b13223e9d #1
[ 1358.366972] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.376563] task:python3 state:D stack:25672 pid:5836 tgid:5836 ppid:5835 task_flags:0x400100 flags:0x00080002
[ 1358.389769] Call Trace:
[ 1358.393309]
[ 1358.396458] __schedule+0xcc3/0x19c0
[ 1358.401277] ? io_schedule_timeout+0x140/0x140
[ 1358.407062] ? find_held_lock+0x2b/0x80
[ 1358.412167] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.418071] schedule+0xd8/0x260
[ 1358.422497] schedule_preempt_disabled+0x18/0x30
[ 1358.428479] __mutex_lock+0x6af/0x2010
[ 1358.433483] ? threaded_store+0x132/0x220
[ 1358.438781] ? ww_mutex_lock+0x160/0x160
[ 1358.443981] ? selinux_inode_free_security+0x330/0x330
[ 1358.450617] ? cap_capable+0x1d6/0x3e0
[ 1358.455615] ? threaded_store+0x132/0x220
[ 1358.460912] threaded_store+0x132/0x220
[ 1358.466016] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.471314] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.476614] ? sysfs_kf_write+0x65/0x160
[ 1358.481808] ? sysfs_file_ops+0x140/0x140
[ 1358.487105] kernfs_fop_write_iter+0x335/0x510
[ 1358.492881] ? kernfs_ops+0xf0/0xf0
[ 1358.497594] new_sync_write+0x333/0x750
[ 1358.502700] ? __lock_acquire+0x508/0xc10
[ 1358.507998] ? new_sync_read+0x740/0x740
[ 1358.513200] ? lock_acquire.part.0+0xbc/0x260
[ 1358.518887] ? ksys_write+0x116/0x250
[ 1358.523800] vfs_write+0x657/0x9b0
[ 1358.528421] ksys_write+0x116/0x250
[ 1358.533133] ? __ia32_sys_read+0xc0/0xc0
[ 1358.538334] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.543924] ? rcu_is_watching+0x15/0xd0
[ 1358.549124] do_syscall_64+0xf3/0x650
[ 1358.554023] ? trace_hardirqs_off+0xd/0x30
[ 1358.559422] ? exc_page_fault+0xda/0xf0
[ 1358.564524] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.570983] RIP: 0033:0x7febb0f7922e
[ 1358.575793] RSP: 002b:00007ffdcccabf40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.585085] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007febb0f7922e
[ 1358.593893] RDX: 0000000000000001 RSI: 0000556dd5445e90 RDI: 0000000000000007
[ 1358.602702] RBP: 00007ffdcccabf50 R08: 0000000000000000 R09: 0000000000000000
[ 1358.611510] R10: 0000000000000000 R11: 0000000000000202 R12: 00007febb0df1390
[ 1358.620317] R13: 0000556dd5445e90 R14: 0000000000000007 R15: 0000000000000003
[ 1358.629131]
[ 1358.632460] INFO: task python3:5836 is blocked on a mutex likely owned by task python3:3788.
[ 1358.642731] task:python3 state:D stack:25672 pid:3788 tgid:3788 ppid:3787 task_flags:0x400100 flags:0x00080802
[ 1358.655933] Call Trace:
[ 1358.659473]
[ 1358.662623] __schedule+0xcc3/0x19c0
[ 1358.667434] ? io_schedule_timeout+0x140/0x140
[ 1358.673217] ? find_held_lock+0x2b/0x80
[ 1358.678318] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.684497] schedule+0xd8/0x260
[ 1358.688979] schedule_timeout+0x13f/0x290
[ 1358.694275] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.700646] ? trace_contention_end+0xb3/0x1a0
[ 1358.706433] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.712122] ? ww_mutex_lock+0x160/0x160
[ 1358.717323] msleep+0x1b/0x30
[ 1358.721450] napi_set_threaded+0x3a5/0x440
[ 1358.726846] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.732340] netif_set_threaded+0x257/0x400
[ 1358.737835] threaded_store+0x1b8/0x220
[ 1358.742937] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.748227] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.753522] ? sysfs_kf_write+0x65/0x160
[ 1358.758722] ? sysfs_file_ops+0x140/0x140
[ 1358.764011] kernfs_fop_write_iter+0x335/0x510
[ 1358.769796] ? kernfs_ops+0xf0/0xf0
[ 1358.774508] new_sync_write+0x333/0x750
[ 1358.779605] ? __lock_acquire+0x508/0xc10
[ 1358.784902] ? new_sync_read+0x740/0x740
[ 1358.790095] ? lock_acquire.part.0+0xbc/0x260
[ 1358.795784] ? ksys_write+0x116/0x250
[ 1358.800697] vfs_write+0x657/0x9b0
[ 1358.805314] ksys_write+0x116/0x250
[ 1358.810025] ? __ia32_sys_read+0xc0/0xc0
[ 1358.815227] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.820818] ? rcu_is_watching+0x15/0xd0
[ 1358.826019] do_syscall_64+0xf3/0x650
[ 1358.830925] ? trace_hardirqs_off+0xd/0x30
[ 1358.836319] ? exc_page_fault+0xda/0xf0
[ 1358.841427] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.847897] RIP: 0033:0x7fc98fe5b22e
[ 1358.852707] RSP: 002b:00007ffdef52f8f0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.861996] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc98fe5b22e
[ 1358.870807] RDX: 0000000000000001 RSI: 000055fccdfbfe90 RDI: 0000000000000007
[ 1358.879613] RBP: 00007ffdef52f900 R08: 0000000000000000 R09: 0000000000000000
[ 1358.888428] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc98fcd3390
[ 1358.897234] R13: 000055fccdfbfe90 R14: 0000000000000007 R15: 0000000000000003
[ 1358.906048]
[ 1358.909318] \x0aShowing all locks held in the system:
[ 1358.917860] 1 lock held by khungtaskd/184:
[ 1358.923252] #0: ffffffff950fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.934723] 2 locks held by systemd-journal/802:
[ 1358.940701] #0: ff1100180dcb74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.950590] #1: ffffffff95302ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.963416] 4 locks held by python3/3788:
[ 1358.968718] #0: ff110001304943e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.978606] #1: ff1100017bb66e80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.989570] #2: ff110001243f4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.000728] #3: ff11000124409d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.010827] 4 locks held by python3/5836:
[ 1359.016121] #0: ff110001304943e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.026010] #1: ff110001f3385680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.036973] #2: ff110001243f4b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.048128] #3: ff11000124409d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.058222]
[ 1359.060689] =============================================\x0a
[ 1372.292185] block nvme0n1: No UUID available providing old NGUID
SUBSYSTEM=block
DEVICE=b259:0