[ 767.792044] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 767.794856] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 767.946355] kselftest: Running tests in drivers/net
[ 961.027606] kworker/u130:2 (348) used greatest stack depth: 21936 bytes left
[ 989.699930] INFO: task python3:5487 blocked for more than 122 seconds.
[ 989.708167] Not tainted 7.1.0-rc7-vnda-gb51d13b87305 #1
[ 989.716126] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.725705] task:python3 state:D stack:25424 pid:5487 tgid:5487 ppid:5486 task_flags:0x400100 flags:0x00080000
[ 989.738911] Call Trace:
[ 989.742451]
[ 989.745603] __schedule+0xcc3/0x19c0
[ 989.750421] ? io_schedule_timeout+0x140/0x140
[ 989.756206] ? find_held_lock+0x2b/0x80
[ 989.761311] schedule+0xd8/0x260
[ 989.765730] schedule_preempt_disabled+0x18/0x30
[ 989.771708] __mutex_lock+0x6af/0x2010
[ 989.776713] ? threaded_store+0x132/0x220
[ 989.782013] ? ww_mutex_lock+0x160/0x160
[ 989.787210] ? selinux_inode_free_security+0x330/0x330
[ 989.793776] ? cap_capable+0x1d6/0x3e0
[ 989.798783] ? threaded_store+0x132/0x220
[ 989.804079] threaded_store+0x132/0x220
[ 989.809182] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.814476] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.819773] ? sysfs_kf_write+0x65/0x160
[ 989.824974] ? sysfs_file_ops+0x140/0x140
[ 989.830270] kernfs_fop_write_iter+0x335/0x510
[ 989.836055] ? kernfs_ops+0xf0/0xf0
[ 989.840769] new_sync_write+0x333/0x750
[ 989.845871] ? __lock_acquire+0x508/0xc10
[ 989.851165] ? new_sync_read+0x740/0x740
[ 989.856365] ? lock_acquire.part.0+0xbc/0x260
[ 989.862051] ? ksys_write+0x116/0x250
[ 989.866958] vfs_write+0x657/0x9b0
[ 989.871572] ksys_write+0x116/0x250
[ 989.876282] ? __ia32_sys_read+0xc0/0xc0
[ 989.881481] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.887070] ? rcu_is_watching+0x15/0xd0
[ 989.892261] do_syscall_64+0xf3/0x650
[ 989.897168] ? trace_hardirqs_off+0xd/0x30
[ 989.902563] ? exc_page_fault+0xda/0xf0
[ 989.907664] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.914129] RIP: 0033:0x7fed9cc2064e
[ 989.918940] RSP: 002b:00007ffcdda64080 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.928239] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fed9cc2064e
[ 989.937045] RDX: 0000000000000001 RSI: 000055b5b5858bf0 RDI: 0000000000000007
[ 989.945853] RBP: 00007ffcdda64090 R08: 0000000000000000 R09: 0000000000000000
[ 989.954662] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fed9ca99390
[ 989.963470] R13: 000055b5b5858bf0 R14: 0000000000000007 R15: 00007fed9ca2d120
[ 989.972283]
[ 989.975583] INFO: task python3:5487 is blocked on a mutex likely owned by task python3:3498.
[ 989.985860] task:python3 state:D stack:25672 pid:3498 tgid:3498 ppid:3497 task_flags:0x400100 flags:0x00080802
[ 989.999062] Call Trace:
[ 990.002599]
[ 990.005747] __schedule+0xcc3/0x19c0
[ 990.010557] ? io_schedule_timeout+0x140/0x140
[ 990.016341] ? find_held_lock+0x2b/0x80
[ 990.021443] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.027613] schedule+0xd8/0x260
[ 990.032030] schedule_timeout+0x13f/0x290
[ 990.037325] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.043693] ? trace_contention_end+0xb3/0x1a0
[ 990.049478] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.055167] ? ww_mutex_lock+0x160/0x160
[ 990.060365] msleep+0x1b/0x30
[ 990.064488] napi_set_threaded+0x3a5/0x440
[ 990.069885] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.075378] netif_set_threaded+0x257/0x400
[ 990.080872] threaded_store+0x1b8/0x220
[ 990.085971] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.091267] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.096556] ? sysfs_kf_write+0x65/0x160
[ 990.101755] ? sysfs_file_ops+0x140/0x140
[ 990.107051] kernfs_fop_write_iter+0x335/0x510
[ 990.112837] ? kernfs_ops+0xf0/0xf0
[ 990.117546] new_sync_write+0x333/0x750
[ 990.122647] ? __lock_acquire+0x508/0xc10
[ 990.127940] ? new_sync_read+0x740/0x740
[ 990.133141] ? lock_acquire.part.0+0xbc/0x260
[ 990.138816] ? ksys_write+0x116/0x250
[ 990.143724] vfs_write+0x657/0x9b0
[ 990.148339] ksys_write+0x116/0x250
[ 990.153041] ? __ia32_sys_read+0xc0/0xc0
[ 990.158238] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.163828] ? rcu_is_watching+0x15/0xd0
[ 990.169027] do_syscall_64+0xf3/0x650
[ 990.173931] ? irq_exit_rcu+0x1a/0x30
[ 990.178838] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.185302] RIP: 0033:0x7fd86e98a64e
[ 990.190109] RSP: 002b:00007ffc1f7c4fb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.199407] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd86e98a64e
[ 990.208215] RDX: 0000000000000001 RSI: 000055b83f05cbf0 RDI: 0000000000000007
[ 990.217011] RBP: 00007ffc1f7c4fc0 R08: 0000000000000000 R09: 0000000000000000
[ 990.225818] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd86e803390
[ 990.234625] R13: 000055b83f05cbf0 R14: 0000000000000007 R15: 00007fd86e799120
[ 990.243439]
[ 990.246698] \x0aShowing all locks held in the system:
[ 990.255238] 1 lock held by khungtaskd/185:
[ 990.260629] #0: ffffffff9a6fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.272115] 4 locks held by python3/3498:
[ 990.277410] #0: ff11000130c793e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.287296] #1: ff11000144140880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.298259] #2: ff11000111012b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.309415] #3: ff11000111069e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.319514] 4 locks held by python3/5487:
[ 990.324809] #0: ff11000130c793e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.334695] #1: ff1100018806a080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.345658] #2: ff11000111012b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.356814] #3: ff11000111069e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.366902]
[ 990.369370] =============================================\x0a
[ 1112.581208] INFO: task python3:5487 blocked for more than 245 seconds.
[ 1112.589349] Not tainted 7.1.0-rc7-vnda-gb51d13b87305 #1
[ 1112.596607] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.606200] task:python3 state:D stack:25424 pid:5487 tgid:5487 ppid:5486 task_flags:0x400100 flags:0x00080000
[ 1112.619416] Call Trace:
[ 1112.622959]
[ 1112.626115] __schedule+0xcc3/0x19c0
[ 1112.630933] ? io_schedule_timeout+0x140/0x140
[ 1112.636724] ? find_held_lock+0x2b/0x80
[ 1112.641828] schedule+0xd8/0x260
[ 1112.646251] schedule_preempt_disabled+0x18/0x30
[ 1112.652233] __mutex_lock+0x6af/0x2010
[ 1112.657234] ? threaded_store+0x132/0x220
[ 1112.662537] ? ww_mutex_lock+0x160/0x160
[ 1112.667742] ? selinux_inode_free_security+0x330/0x330
[ 1112.674376] ? cap_capable+0x1d6/0x3e0
[ 1112.679391] ? threaded_store+0x132/0x220
[ 1112.684689] threaded_store+0x132/0x220
[ 1112.689798] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.695100] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.700406] ? sysfs_kf_write+0x65/0x160
[ 1112.705611] ? sysfs_file_ops+0x140/0x140
[ 1112.710912] kernfs_fop_write_iter+0x335/0x510
[ 1112.716704] ? kernfs_ops+0xf0/0xf0
[ 1112.721419] new_sync_write+0x333/0x750
[ 1112.726594] ? __lock_acquire+0x508/0xc10
[ 1112.731887] ? new_sync_read+0x740/0x740
[ 1112.737090] ? lock_acquire.part.0+0xbc/0x260
[ 1112.742783] ? ksys_write+0x116/0x250
[ 1112.747695] vfs_write+0x657/0x9b0
[ 1112.752318] ksys_write+0x116/0x250
[ 1112.757092] ? __ia32_sys_read+0xc0/0xc0
[ 1112.762287] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.767878] ? rcu_is_watching+0x15/0xd0
[ 1112.773085] do_syscall_64+0xf3/0x650
[ 1112.777995] ? trace_hardirqs_off+0xd/0x30
[ 1112.783397] ? exc_page_fault+0xda/0xf0
[ 1112.788503] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.794978] RIP: 0033:0x7fed9cc2064e
[ 1112.799794] RSP: 002b:00007ffcdda64080 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.809101] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fed9cc2064e
[ 1112.817913] RDX: 0000000000000001 RSI: 000055b5b5858bf0 RDI: 0000000000000007
[ 1112.826728] RBP: 00007ffcdda64090 R08: 0000000000000000 R09: 0000000000000000
[ 1112.835531] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fed9ca99390
[ 1112.844345] R13: 000055b5b5858bf0 R14: 0000000000000007 R15: 00007fed9ca2d120
[ 1112.853161]
[ 1112.856469] INFO: task python3:5487 is blocked on a mutex likely owned by task python3:3498.
[ 1112.866741] task:python3 state:D stack:25672 pid:3498 tgid:3498 ppid:3497 task_flags:0x400100 flags:0x00080802
[ 1112.879952] Call Trace:
[ 1112.883494]
[ 1112.886648] __schedule+0xcc3/0x19c0
[ 1112.891463] ? io_schedule_timeout+0x140/0x140
[ 1112.897250] ? find_held_lock+0x2b/0x80
[ 1112.902419] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.908608] schedule+0xd8/0x260
[ 1112.913030] schedule_timeout+0x13f/0x290
[ 1112.918333] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.924707] ? trace_contention_end+0xb3/0x1a0
[ 1112.930498] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.936193] ? ww_mutex_lock+0x160/0x160
[ 1112.941397] msleep+0x1b/0x30
[ 1112.945526] napi_set_threaded+0x3a5/0x440
[ 1112.950928] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.956491] netif_set_threaded+0x257/0x400
[ 1112.961994] threaded_store+0x1b8/0x220
[ 1112.967102] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.972406] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.977705] ? sysfs_kf_write+0x65/0x160
[ 1112.982902] ? sysfs_file_ops+0x140/0x140
[ 1112.988203] kernfs_fop_write_iter+0x335/0x510
[ 1112.993997] ? kernfs_ops+0xf0/0xf0
[ 1112.998703] new_sync_write+0x333/0x750
[ 1113.003813] ? __lock_acquire+0x508/0xc10
[ 1113.009114] ? new_sync_read+0x740/0x740
[ 1113.014322] ? lock_acquire.part.0+0xbc/0x260
[ 1113.020015] ? ksys_write+0x116/0x250
[ 1113.025008] vfs_write+0x657/0x9b0
[ 1113.029627] ksys_write+0x116/0x250
[ 1113.034345] ? __ia32_sys_read+0xc0/0xc0
[ 1113.039552] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.045134] ? rcu_is_watching+0x15/0xd0
[ 1113.050331] do_syscall_64+0xf3/0x650
[ 1113.055306] ? irq_exit_rcu+0x1a/0x30
[ 1113.060218] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.066690] RIP: 0033:0x7fd86e98a64e
[ 1113.071502] RSP: 002b:00007ffc1f7c4fb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.080806] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd86e98a64e
[ 1113.089623] RDX: 0000000000000001 RSI: 000055b83f05cbf0 RDI: 0000000000000007
[ 1113.098434] RBP: 00007ffc1f7c4fc0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.107250] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd86e803390
[ 1113.116062] R13: 000055b83f05cbf0 R14: 0000000000000007 R15: 00007fd86e799120
[ 1113.124886]
[ 1113.128155] \x0aShowing all locks held in the system:
[ 1113.136707] 1 lock held by khungtaskd/185:
[ 1113.142107] #0: ffffffff9a6fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.153608] 4 locks held by python3/3498:
[ 1113.158910] #0: ff11000130c793e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.168807] #1: ff11000144140880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.179781] #2: ff11000111012b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.190947] #3: ff11000111069e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.201059] 4 locks held by python3/5487:
[ 1113.206354] #0: ff11000130c793e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.216250] #1: ff1100018806a080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.227221] #2: ff11000111012b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.238388] #3: ff11000111069e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.248486] 1 lock held by systemd/6558:
[ 1113.253692]
[ 1113.256162] =============================================\x0a
[ 1235.462470] INFO: task python3:5487 blocked for more than 368 seconds.
[ 1235.470620] Not tainted 7.1.0-rc7-vnda-gb51d13b87305 #1
[ 1235.477860] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.487451] task:python3 state:D stack:25424 pid:5487 tgid:5487 ppid:5486 task_flags:0x400100 flags:0x00080002
[ 1235.500658] Call Trace:
[ 1235.504195]
[ 1235.507346] __schedule+0xcc3/0x19c0
[ 1235.512162] ? io_schedule_timeout+0x140/0x140
[ 1235.517947] ? find_held_lock+0x2b/0x80
[ 1235.523056] schedule+0xd8/0x260
[ 1235.527474] schedule_preempt_disabled+0x18/0x30
[ 1235.533451] __mutex_lock+0x6af/0x2010
[ 1235.538456] ? threaded_store+0x132/0x220
[ 1235.543754] ? ww_mutex_lock+0x160/0x160
[ 1235.548951] ? selinux_inode_free_security+0x330/0x330
[ 1235.555520] ? cap_capable+0x1d6/0x3e0
[ 1235.560528] ? threaded_store+0x132/0x220
[ 1235.565822] threaded_store+0x132/0x220
[ 1235.570925] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.576214] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.581515] ? sysfs_kf_write+0x65/0x160
[ 1235.586713] ? sysfs_file_ops+0x140/0x140
[ 1235.592008] kernfs_fop_write_iter+0x335/0x510
[ 1235.597793] ? kernfs_ops+0xf0/0xf0
[ 1235.602503] new_sync_write+0x333/0x750
[ 1235.607597] ? __lock_acquire+0x508/0xc10
[ 1235.612894] ? new_sync_read+0x740/0x740
[ 1235.618096] ? lock_acquire.part.0+0xbc/0x260
[ 1235.623781] ? ksys_write+0x116/0x250
[ 1235.628689] vfs_write+0x657/0x9b0
[ 1235.633304] ksys_write+0x116/0x250
[ 1235.638018] ? __ia32_sys_read+0xc0/0xc0
[ 1235.643216] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.648803] ? rcu_is_watching+0x15/0xd0
[ 1235.654003] do_syscall_64+0xf3/0x650
[ 1235.658909] ? trace_hardirqs_off+0xd/0x30
[ 1235.664301] ? exc_page_fault+0xda/0xf0
[ 1235.669402] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.675871] RIP: 0033:0x7fed9cc2064e
[ 1235.680682] RSP: 002b:00007ffcdda64080 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.689978] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fed9cc2064e
[ 1235.698785] RDX: 0000000000000001 RSI: 000055b5b5858bf0 RDI: 0000000000000007
[ 1235.707593] RBP: 00007ffcdda64090 R08: 0000000000000000 R09: 0000000000000000
[ 1235.716400] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fed9ca99390
[ 1235.725207] R13: 000055b5b5858bf0 R14: 0000000000000007 R15: 00007fed9ca2d120
[ 1235.734020]
[ 1235.737298] INFO: task python3:5487 is blocked on a mutex likely owned by task python3:3498.
[ 1235.747569] task:python3 state:D stack:25672 pid:3498 tgid:3498 ppid:3497 task_flags:0x400100 flags:0x00080802
[ 1235.760771] Call Trace:
[ 1235.764313]
[ 1235.767452] __schedule+0xcc3/0x19c0
[ 1235.772261] ? io_schedule_timeout+0x140/0x140
[ 1235.778044] ? find_held_lock+0x2b/0x80
[ 1235.783145] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.789325] schedule+0xd8/0x260
[ 1235.793742] schedule_timeout+0x13f/0x290
[ 1235.799039] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.805407] ? trace_contention_end+0xb3/0x1a0
[ 1235.811191] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.816881] ? ww_mutex_lock+0x160/0x160
[ 1235.822082] msleep+0x1b/0x30
[ 1235.826208] napi_set_threaded+0x3a5/0x440
[ 1235.831594] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.837088] netif_set_threaded+0x257/0x400
[ 1235.842581] threaded_store+0x1b8/0x220
[ 1235.847682] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.852979] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.858276] ? sysfs_kf_write+0x65/0x160
[ 1235.863465] ? sysfs_file_ops+0x140/0x140
[ 1235.868760] kernfs_fop_write_iter+0x335/0x510
[ 1235.874545] ? kernfs_ops+0xf0/0xf0
[ 1235.879254] new_sync_write+0x333/0x750
[ 1235.884354] ? __lock_acquire+0x508/0xc10
[ 1235.889648] ? new_sync_read+0x740/0x740
[ 1235.894851] ? lock_acquire.part.0+0xbc/0x260
[ 1235.900536] ? ksys_write+0x116/0x250
[ 1235.905447] vfs_write+0x657/0x9b0
[ 1235.910060] ksys_write+0x116/0x250
[ 1235.914770] ? __ia32_sys_read+0xc0/0xc0
[ 1235.919971] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.925559] ? rcu_is_watching+0x15/0xd0
[ 1235.930756] do_syscall_64+0xf3/0x650
[ 1235.935661] ? irq_exit_rcu+0x1a/0x30
[ 1235.940570] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.947026] RIP: 0033:0x7fd86e98a64e
[ 1235.951831] RSP: 002b:00007ffc1f7c4fb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.961128] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd86e98a64e
[ 1235.969927] RDX: 0000000000000001 RSI: 000055b83f05cbf0 RDI: 0000000000000007
[ 1235.978733] RBP: 00007ffc1f7c4fc0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.987542] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd86e803390
[ 1235.996350] R13: 000055b83f05cbf0 R14: 0000000000000007 R15: 00007fd86e799120
[ 1236.005161]
[ 1236.008428] \x0aShowing all locks held in the system:
[ 1236.016959] 1 lock held by khungtaskd/185:
[ 1236.022354] #0: ffffffff9a6fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.033837] 4 locks held by python3/3498:
[ 1236.039129] #0: ff11000130c793e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.049019] #1: ff11000144140880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.059980] #2: ff11000111012b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.071138] #3: ff11000111069e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.081224] 4 locks held by python3/5487:
[ 1236.086517] #0: ff11000130c793e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.096402] #1: ff1100018806a080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.107363] #2: ff11000111012b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.118519] #3: ff11000111069e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.128613]
[ 1236.131077] =============================================\x0a
[ 1358.342726] INFO: task python3:5487 blocked for more than 491 seconds.
[ 1358.350893] Not tainted 7.1.0-rc7-vnda-gb51d13b87305 #1
[ 1358.358149] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.367739] task:python3 state:D stack:25424 pid:5487 tgid:5487 ppid:5486 task_flags:0x400100 flags:0x00080002
[ 1358.380946] Call Trace:
[ 1358.384483]
[ 1358.387631] __schedule+0xcc3/0x19c0
[ 1358.392446] ? io_schedule_timeout+0x140/0x140
[ 1358.398318] ? find_held_lock+0x2b/0x80
[ 1358.403426] schedule+0xd8/0x260
[ 1358.407844] schedule_preempt_disabled+0x18/0x30
[ 1358.413821] __mutex_lock+0x6af/0x2010
[ 1358.418825] ? threaded_store+0x132/0x220
[ 1358.424115] ? ww_mutex_lock+0x160/0x160
[ 1358.429305] ? selinux_inode_free_security+0x330/0x330
[ 1358.435864] ? cap_capable+0x1d6/0x3e0
[ 1358.440872] ? threaded_store+0x132/0x220
[ 1358.446168] threaded_store+0x132/0x220
[ 1358.451271] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.456568] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.461930] ? sysfs_kf_write+0x65/0x160
[ 1358.467131] ? sysfs_file_ops+0x140/0x140
[ 1358.472416] kernfs_fop_write_iter+0x335/0x510
[ 1358.478203] ? kernfs_ops+0xf0/0xf0
[ 1358.482915] new_sync_write+0x333/0x750
[ 1358.488018] ? __lock_acquire+0x508/0xc10
[ 1358.493316] ? new_sync_read+0x740/0x740
[ 1358.498515] ? lock_acquire.part.0+0xbc/0x260
[ 1358.504202] ? ksys_write+0x116/0x250
[ 1358.509113] vfs_write+0x657/0x9b0
[ 1358.513725] ksys_write+0x116/0x250
[ 1358.518434] ? __ia32_sys_read+0xc0/0xc0
[ 1358.523629] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.529219] ? rcu_is_watching+0x15/0xd0
[ 1358.534418] do_syscall_64+0xf3/0x650
[ 1358.539326] ? trace_hardirqs_off+0xd/0x30
[ 1358.544722] ? exc_page_fault+0xda/0xf0
[ 1358.549883] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.556350] RIP: 0033:0x7fed9cc2064e
[ 1358.561157] RSP: 002b:00007ffcdda64080 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.570456] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fed9cc2064e
[ 1358.579269] RDX: 0000000000000001 RSI: 000055b5b5858bf0 RDI: 0000000000000007
[ 1358.588076] RBP: 00007ffcdda64090 R08: 0000000000000000 R09: 0000000000000000
[ 1358.596884] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fed9ca99390
[ 1358.605690] R13: 000055b5b5858bf0 R14: 0000000000000007 R15: 00007fed9ca2d120
[ 1358.614504]
[ 1358.617822] INFO: task python3:5487 is blocked on a mutex likely owned by task python3:3498.
[ 1358.628098] task:python3 state:D stack:25672 pid:3498 tgid:3498 ppid:3497 task_flags:0x400100 flags:0x00080802
[ 1358.641304] Call Trace:
[ 1358.644845]
[ 1358.647995] __schedule+0xcc3/0x19c0
[ 1358.652806] ? io_schedule_timeout+0x140/0x140
[ 1358.658581] ? find_held_lock+0x2b/0x80
[ 1358.663677] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.669855] schedule+0xd8/0x260
[ 1358.674275] schedule_timeout+0x13f/0x290
[ 1358.679574] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.685946] ? trace_contention_end+0xb3/0x1a0
[ 1358.691732] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.697421] ? ww_mutex_lock+0x160/0x160
[ 1358.702620] msleep+0x1b/0x30
[ 1358.706748] napi_set_threaded+0x3a5/0x440
[ 1358.712147] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.717642] netif_set_threaded+0x257/0x400
[ 1358.723138] threaded_store+0x1b8/0x220
[ 1358.728240] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.733539] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.738838] ? sysfs_kf_write+0x65/0x160
[ 1358.744038] ? sysfs_file_ops+0x140/0x140
[ 1358.749327] kernfs_fop_write_iter+0x335/0x510
[ 1358.755102] ? kernfs_ops+0xf0/0xf0
[ 1358.759815] new_sync_write+0x333/0x750
[ 1358.764908] ? __lock_acquire+0x508/0xc10
[ 1358.770205] ? new_sync_read+0x740/0x740
[ 1358.775408] ? lock_acquire.part.0+0xbc/0x260
[ 1358.781096] ? ksys_write+0x116/0x250
[ 1358.785997] vfs_write+0x657/0x9b0
[ 1358.790614] ksys_write+0x116/0x250
[ 1358.795325] ? __ia32_sys_read+0xc0/0xc0
[ 1358.800523] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.806116] ? rcu_is_watching+0x15/0xd0
[ 1358.811319] do_syscall_64+0xf3/0x650
[ 1358.816228] ? irq_exit_rcu+0x1a/0x30
[ 1358.821135] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.827605] RIP: 0033:0x7fd86e98a64e
[ 1358.832415] RSP: 002b:00007ffc1f7c4fb0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.841715] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd86e98a64e
[ 1358.850523] RDX: 0000000000000001 RSI: 000055b83f05cbf0 RDI: 0000000000000007
[ 1358.859324] RBP: 00007ffc1f7c4fc0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.868135] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd86e803390
[ 1358.877015] R13: 000055b83f05cbf0 R14: 0000000000000007 R15: 00007fd86e799120
[ 1358.885830]
[ 1358.889100] \x0aShowing all locks held in the system:
[ 1358.897646] 1 lock held by khungtaskd/185:
[ 1358.903038] #0: ffffffff9a6fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.914518] 1 lock held by systemd-journal/803:
[ 1358.920408] 4 locks held by python3/3498:
[ 1358.925706] #0: ff11000130c793e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.935586] #1: ff11000144140880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.946542] #2: ff11000111012b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.957701] #3: ff11000111069e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.967802] 4 locks held by python3/5487:
[ 1358.973097] #0: ff11000130c793e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.982986] #1: ff1100018806a080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.993952] #2: ff11000111012b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.005112] #3: ff11000111069e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.015205] 5 locks held by systemd/7331:
[ 1359.020503] #0: ff11000130f213e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.030391] #1: ff110001236d2680 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.041352] #2: ffffffff9a769c20 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_lock_and_drain_offline+0xa6/0x640
[ 1359.052998] #3: ffffffff9a5687b0 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_update_dfl_csses+0x2a3/0x5c0
[ 1359.064544] #4: ffffffff9a769990 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_update_dfl_csses+0x2af/0x5c0
[ 1359.076876]
[ 1359.079344] =============================================\x0a