[ 761.160048] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 761.162744] nipa-hw-worker: [4/38] Retrying drivers/net:napi_threaded.py
[ 761.325888] kselftest: Running tests in drivers/net
[ 989.748101] INFO: task python3:5002 blocked for more than 122 seconds.
[ 989.756323] Not tainted 7.0.0-rc4-txcn-g6f46748b781a #1
[ 989.763572] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.773151] task:python3 state:D stack:25312 pid:5002 tgid:5002 ppid:5001 task_flags:0x400100 flags:0x00080000
[ 989.786356] Call Trace:
[ 989.789893]
[ 989.793043] __schedule+0xad9/0x1490
[ 989.797860] ? io_schedule_timeout+0x130/0x130
[ 989.803642] ? find_held_lock+0x2b/0x80
[ 989.808747] ? __lock_release.isra.0+0x59/0x170
[ 989.814629] schedule+0xd4/0x260
[ 989.819047] schedule_preempt_disabled+0x18/0x30
[ 989.825025] __mutex_lock+0xd76/0x1f20
[ 989.830030] ? threaded_store+0x132/0x220
[ 989.835329] ? ww_mutex_lock+0x160/0x160
[ 989.840531] ? selinux_inode_free_security+0x310/0x310
[ 989.847098] ? cap_capable+0x1ed/0x3f0
[ 989.852093] ? sysfs_file_ops+0x140/0x140
[ 989.857389] ? threaded_store+0x132/0x220
[ 989.862686] threaded_store+0x132/0x220
[ 989.867790] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.873282] ? sysfs_file_ops+0x140/0x140
[ 989.878577] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.883874] ? sysfs_kf_write+0x65/0x160
[ 989.889064] kernfs_fop_write_iter+0x335/0x510
[ 989.894849] ? kernfs_ops+0xf0/0xf0
[ 989.899550] new_sync_write+0x308/0x6f0
[ 989.904652] ? new_sync_read+0x6f0/0x6f0
[ 989.909850] ? lock_acquire.part.0+0xbc/0x260
[ 989.915535] ? ksys_write+0xff/0x200
[ 989.920345] vfs_write+0x62a/0x980
[ 989.924959] ksys_write+0xff/0x200
[ 989.929571] ? __ia32_sys_read+0xc0/0xc0
[ 989.934769] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.940359] ? rcu_is_watching+0x15/0xd0
[ 989.945557] do_syscall_64+0xf3/0x5e0
[ 989.950465] ? irq_exit_rcu+0x39/0xf0
[ 989.955372] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.961840] RIP: 0033:0x7ff060855c5e
[ 989.966648] RSP: 002b:00007fff14bee430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.975946] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff060855c5e
[ 989.984755] RDX: 0000000000000001 RSI: 000055d997270110 RDI: 0000000000000007
[ 989.993562] RBP: 00007fff14bee440 R08: 0000000000000000 R09: 0000000000000000
[ 990.002369] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff0606f0e80
[ 990.011176] R13: 000055d997270110 R14: 0000000000000007 R15: 00007ff060f56b98
[ 990.019987]
[ 990.023263] INFO: task python3:5002 is blocked on a mutex likely owned by task python3:3047.
[ 990.033541] task:python3 state:D stack:25312 pid:3047 tgid:3047 ppid:3046 task_flags:0x400100 flags:0x00080802
[ 990.046742] Call Trace:
[ 990.050279]
[ 990.053429] __schedule+0xad9/0x1490
[ 990.058231] ? io_schedule_timeout+0x130/0x130
[ 990.064011] ? find_held_lock+0x2b/0x80
[ 990.069111] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.075286] schedule+0xd4/0x260
[ 990.079703] schedule_timeout+0x103/0x260
[ 990.084999] ? rcu_is_watching+0x15/0xd0
[ 990.090194] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.096563] ? hrtimers_cpu_dying+0x250/0x250
[ 990.102250] ? ww_mutex_lock+0x160/0x160
[ 990.107450] msleep+0x1b/0x30
[ 990.111565] napi_set_threaded+0x376/0x410
[ 990.116958] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.122453] netif_set_threaded+0x254/0x400
[ 990.127947] ? sysfs_file_ops+0x140/0x140
[ 990.133242] threaded_store+0x1bb/0x220
[ 990.138344] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.143833] ? sysfs_file_ops+0x140/0x140
[ 990.149129] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.154424] ? sysfs_kf_write+0x65/0x160
[ 990.159625] kernfs_fop_write_iter+0x335/0x510
[ 990.165410] ? kernfs_ops+0xf0/0xf0
[ 990.170112] new_sync_write+0x308/0x6f0
[ 990.175213] ? new_sync_read+0x6f0/0x6f0
[ 990.180409] ? lock_acquire.part.0+0xbc/0x260
[ 990.186094] ? ksys_write+0xff/0x200
[ 990.190906] vfs_write+0x62a/0x980
[ 990.195521] ksys_write+0xff/0x200
[ 990.200124] ? __ia32_sys_read+0xc0/0xc0
[ 990.205322] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.210914] ? rcu_is_watching+0x15/0xd0
[ 990.216112] do_syscall_64+0xf3/0x5e0
[ 990.221016] ? irq_exit_rcu+0x39/0xf0
[ 990.225923] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.232386] RIP: 0033:0x7fc8c6b4ac5e
[ 990.237193] RSP: 002b:00007ffe41ce7ff0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.246487] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc8c6b4ac5e
[ 990.255292] RDX: 0000000000000001 RSI: 0000561d80e73110 RDI: 0000000000000007
[ 990.264102] RBP: 00007ffe41ce8000 R08: 0000000000000000 R09: 0000000000000000
[ 990.272908] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc8c69e5e80
[ 990.281713] R13: 0000561d80e73110 R14: 0000000000000007 R15: 00007fc8c6976480
[ 990.290524]
[ 990.293780] \x0aShowing all locks held in the system:
[ 990.302316] 1 lock held by khungtaskd/184:
[ 990.307707] #0: ffffffffa32ce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.319187] 4 locks held by python3/3047:
[ 990.324482] #0: ff1100012d9413f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.334274] #1: ff1100013c622088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.345234] #2: ff110001203bb0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.356390] #3: ff110001203a1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.366484] 4 locks held by python3/5002:
[ 990.371780] #0: ff1100012d9413f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.381568] #1: ff110001bd37ac88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.392528] #2: ff110001203bb0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.403685] #3: ff110001203a1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.413763]
[ 990.416227] =============================================\x0a
[ 1075.765017] kworker/u129:15 (309) used greatest stack depth: 21192 bytes left
[ 1112.629421] INFO: task python3:5002 blocked for more than 245 seconds.
[ 1112.637568] Not tainted 7.0.0-rc4-txcn-g6f46748b781a #1
[ 1112.644816] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.654405] task:python3 state:D stack:25312 pid:5002 tgid:5002 ppid:5001 task_flags:0x400100 flags:0x00080000
[ 1112.667611] Call Trace:
[ 1112.671149]
[ 1112.674297] __schedule+0xad9/0x1490
[ 1112.679110] ? io_schedule_timeout+0x130/0x130
[ 1112.684894] ? find_held_lock+0x2b/0x80
[ 1112.689996] ? __lock_release.isra.0+0x59/0x170
[ 1112.695880] schedule+0xd4/0x260
[ 1112.700300] schedule_preempt_disabled+0x18/0x30
[ 1112.706280] __mutex_lock+0xd76/0x1f20
[ 1112.711274] ? threaded_store+0x132/0x220
[ 1112.716570] ? ww_mutex_lock+0x160/0x160
[ 1112.721770] ? selinux_inode_free_security+0x310/0x310
[ 1112.728339] ? cap_capable+0x1ed/0x3f0
[ 1112.733346] ? sysfs_file_ops+0x140/0x140
[ 1112.738644] ? threaded_store+0x132/0x220
[ 1112.743938] threaded_store+0x132/0x220
[ 1112.749043] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.754535] ? sysfs_file_ops+0x140/0x140
[ 1112.759833] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.765117] ? sysfs_kf_write+0x65/0x160
[ 1112.770317] kernfs_fop_write_iter+0x335/0x510
[ 1112.776093] ? kernfs_ops+0xf0/0xf0
[ 1112.780801] new_sync_write+0x308/0x6f0
[ 1112.785903] ? new_sync_read+0x6f0/0x6f0
[ 1112.791103] ? lock_acquire.part.0+0xbc/0x260
[ 1112.796789] ? ksys_write+0xff/0x200
[ 1112.801600] vfs_write+0x62a/0x980
[ 1112.806207] ksys_write+0xff/0x200
[ 1112.810821] ? __ia32_sys_read+0xc0/0xc0
[ 1112.816018] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.821605] ? rcu_is_watching+0x15/0xd0
[ 1112.826806] do_syscall_64+0xf3/0x5e0
[ 1112.831711] ? irq_exit_rcu+0x39/0xf0
[ 1112.836619] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.843088] RIP: 0033:0x7ff060855c5e
[ 1112.847898] RSP: 002b:00007fff14bee430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.857194] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff060855c5e
[ 1112.865992] RDX: 0000000000000001 RSI: 000055d997270110 RDI: 0000000000000007
[ 1112.874797] RBP: 00007fff14bee440 R08: 0000000000000000 R09: 0000000000000000
[ 1112.883605] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff0606f0e80
[ 1112.892414] R13: 000055d997270110 R14: 0000000000000007 R15: 00007ff060f56b98
[ 1112.901225]
[ 1112.904492] INFO: task python3:5002 is blocked on a mutex likely owned by task python3:3047.
[ 1112.914766] task:python3 state:D stack:25312 pid:3047 tgid:3047 ppid:3046 task_flags:0x400100 flags:0x00080802
[ 1112.927967] Call Trace:
[ 1112.931506]
[ 1112.934655] __schedule+0xad9/0x1490
[ 1112.939463] ? io_schedule_timeout+0x130/0x130
[ 1112.945244] ? find_held_lock+0x2b/0x80
[ 1112.950344] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.956524] schedule+0xd4/0x260
[ 1112.960943] schedule_timeout+0x103/0x260
[ 1112.966240] ? rcu_is_watching+0x15/0xd0
[ 1112.971437] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.977808] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.983489] ? ww_mutex_lock+0x160/0x160
[ 1112.988690] msleep+0x1b/0x30
[ 1112.992813] napi_set_threaded+0x376/0x410
[ 1112.998206] ? napi_resume_irqs+0x1a0/0x1a0
[ 1113.003698] netif_set_threaded+0x254/0x400
[ 1113.009188] ? sysfs_file_ops+0x140/0x140
[ 1113.014482] threaded_store+0x1bb/0x220
[ 1113.019583] ? tx_maxrate_store+0x4e0/0x4e0
[ 1113.025073] ? sysfs_file_ops+0x140/0x140
[ 1113.030367] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.035662] ? sysfs_kf_write+0x65/0x160
[ 1113.040861] kernfs_fop_write_iter+0x335/0x510
[ 1113.046644] ? kernfs_ops+0xf0/0xf0
[ 1113.051353] new_sync_write+0x308/0x6f0
[ 1113.056456] ? new_sync_read+0x6f0/0x6f0
[ 1113.061654] ? lock_acquire.part.0+0xbc/0x260
[ 1113.067337] ? ksys_write+0xff/0x200
[ 1113.072149] vfs_write+0x62a/0x980
[ 1113.076762] ksys_write+0xff/0x200
[ 1113.081373] ? __ia32_sys_read+0xc0/0xc0
[ 1113.086570] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.092160] ? rcu_is_watching+0x15/0xd0
[ 1113.097359] do_syscall_64+0xf3/0x5e0
[ 1113.102269] ? irq_exit_rcu+0x39/0xf0
[ 1113.107174] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.113639] RIP: 0033:0x7fc8c6b4ac5e
[ 1113.118445] RSP: 002b:00007ffe41ce7ff0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.127739] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc8c6b4ac5e
[ 1113.136544] RDX: 0000000000000001 RSI: 0000561d80e73110 RDI: 0000000000000007
[ 1113.145352] RBP: 00007ffe41ce8000 R08: 0000000000000000 R09: 0000000000000000
[ 1113.154159] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc8c69e5e80
[ 1113.162966] R13: 0000561d80e73110 R14: 0000000000000007 R15: 00007fc8c6976480
[ 1113.171769]
[ 1113.175032] \x0aShowing all locks held in the system:
[ 1113.183571] 1 lock held by khungtaskd/184:
[ 1113.188965] #0: ffffffffa32ce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.200445] 4 locks held by python3/3047:
[ 1113.205739] #0: ff1100012d9413f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.215531] #1: ff1100013c622088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.226492] #2: ff110001203bb0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.237648] #3: ff110001203a1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.247740] 4 locks held by python3/5002:
[ 1113.253036] #0: ff1100012d9413f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.262825] #1: ff110001bd37ac88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.273787] #2: ff110001203bb0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.284935] #3: ff110001203a1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.295026]
[ 1113.297490] =============================================\x0a
[ 1235.509734] INFO: task python3:5002 blocked for more than 368 seconds.
[ 1235.517880] Not tainted 7.0.0-rc4-txcn-g6f46748b781a #1
[ 1235.525130] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.534719] task:python3 state:D stack:25312 pid:5002 tgid:5002 ppid:5001 task_flags:0x400100 flags:0x00080002
[ 1235.547926] Call Trace:
[ 1235.551468]
[ 1235.554617] __schedule+0xad9/0x1490
[ 1235.559433] ? io_schedule_timeout+0x130/0x130
[ 1235.565217] ? find_held_lock+0x2b/0x80
[ 1235.570324] ? __lock_release.isra.0+0x59/0x170
[ 1235.576209] schedule+0xd4/0x260
[ 1235.580626] schedule_preempt_disabled+0x18/0x30
[ 1235.586607] __mutex_lock+0xd76/0x1f20
[ 1235.591615] ? threaded_store+0x132/0x220
[ 1235.596917] ? ww_mutex_lock+0x160/0x160
[ 1235.602118] ? selinux_inode_free_security+0x310/0x310
[ 1235.608689] ? cap_capable+0x1ed/0x3f0
[ 1235.613699] ? sysfs_file_ops+0x140/0x140
[ 1235.619000] ? threaded_store+0x132/0x220
[ 1235.624296] threaded_store+0x132/0x220
[ 1235.629398] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.634884] ? sysfs_file_ops+0x140/0x140
[ 1235.640182] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.645477] ? sysfs_kf_write+0x65/0x160
[ 1235.650678] kernfs_fop_write_iter+0x335/0x510
[ 1235.656465] ? kernfs_ops+0xf0/0xf0
[ 1235.661177] new_sync_write+0x308/0x6f0
[ 1235.666280] ? new_sync_read+0x6f0/0x6f0
[ 1235.671480] ? lock_acquire.part.0+0xbc/0x260
[ 1235.677159] ? ksys_write+0xff/0x200
[ 1235.681970] vfs_write+0x62a/0x980
[ 1235.686587] ksys_write+0xff/0x200
[ 1235.691200] ? __ia32_sys_read+0xc0/0xc0
[ 1235.696397] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.701988] ? rcu_is_watching+0x15/0xd0
[ 1235.707194] do_syscall_64+0xf3/0x5e0
[ 1235.712099] ? irq_exit_rcu+0x39/0xf0
[ 1235.717008] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.723476] RIP: 0033:0x7ff060855c5e
[ 1235.728287] RSP: 002b:00007fff14bee430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.737578] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff060855c5e
[ 1235.746386] RDX: 0000000000000001 RSI: 000055d997270110 RDI: 0000000000000007
[ 1235.755195] RBP: 00007fff14bee440 R08: 0000000000000000 R09: 0000000000000000
[ 1235.763994] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff0606f0e80
[ 1235.772804] R13: 000055d997270110 R14: 0000000000000007 R15: 00007ff060f56b98
[ 1235.781619]
[ 1235.784889] INFO: task python3:5002 is blocked on a mutex likely owned by task python3:3047.
[ 1235.795162] task:python3 state:D stack:25312 pid:3047 tgid:3047 ppid:3046 task_flags:0x400100 flags:0x00080802
[ 1235.808365] Call Trace:
[ 1235.811904]
[ 1235.815057] __schedule+0xad9/0x1490
[ 1235.819870] ? io_schedule_timeout+0x130/0x130
[ 1235.825654] ? find_held_lock+0x2b/0x80
[ 1235.830758] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.836938] schedule+0xd4/0x260
[ 1235.841357] schedule_timeout+0x103/0x260
[ 1235.846652] ? rcu_is_watching+0x15/0xd0
[ 1235.851852] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.858222] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.863912] ? ww_mutex_lock+0x160/0x160
[ 1235.869112] msleep+0x1b/0x30
[ 1235.873229] napi_set_threaded+0x376/0x410
[ 1235.878624] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.884117] netif_set_threaded+0x254/0x400
[ 1235.889610] ? sysfs_file_ops+0x140/0x140
[ 1235.894908] threaded_store+0x1bb/0x220
[ 1235.900011] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.905501] ? sysfs_file_ops+0x140/0x140
[ 1235.910798] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.916094] ? sysfs_kf_write+0x65/0x160
[ 1235.921295] kernfs_fop_write_iter+0x335/0x510
[ 1235.927082] ? kernfs_ops+0xf0/0xf0
[ 1235.931795] new_sync_write+0x308/0x6f0
[ 1235.936888] ? new_sync_read+0x6f0/0x6f0
[ 1235.942092] ? lock_acquire.part.0+0xbc/0x260
[ 1235.947783] ? ksys_write+0xff/0x200
[ 1235.952598] vfs_write+0x62a/0x980
[ 1235.957214] ksys_write+0xff/0x200
[ 1235.961827] ? __ia32_sys_read+0xc0/0xc0
[ 1235.967028] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.972619] ? rcu_is_watching+0x15/0xd0
[ 1235.977819] do_syscall_64+0xf3/0x5e0
[ 1235.982717] ? irq_exit_rcu+0x39/0xf0
[ 1235.987623] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.994090] RIP: 0033:0x7fc8c6b4ac5e
[ 1235.998889] RSP: 002b:00007ffe41ce7ff0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1236.008186] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc8c6b4ac5e
[ 1236.016984] RDX: 0000000000000001 RSI: 0000561d80e73110 RDI: 0000000000000007
[ 1236.025793] RBP: 00007ffe41ce8000 R08: 0000000000000000 R09: 0000000000000000
[ 1236.034601] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc8c69e5e80
[ 1236.043409] R13: 0000561d80e73110 R14: 0000000000000007 R15: 00007fc8c6976480
[ 1236.052222]
[ 1236.055492] \x0aShowing all locks held in the system:
[ 1236.064031] 1 lock held by khungtaskd/184:
[ 1236.069426] #0: ffffffffa32ce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.080905] 4 locks held by python3/3047:
[ 1236.086202] #0: ff1100012d9413f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.095996] #1: ff1100013c622088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.106961] #2: ff110001203bb0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.118117] #3: ff110001203a1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.128209] 4 locks held by python3/5002:
[ 1236.133507] #0: ff1100012d9413f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.143298] #1: ff110001bd37ac88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.154262] #2: ff110001203bb0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.165421] #3: ff110001203a1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.175514]
[ 1236.177980] =============================================\x0a
[ 1358.392038] INFO: task python3:5002 blocked for more than 491 seconds.
[ 1358.400182] Not tainted 7.0.0-rc4-txcn-g6f46748b781a #1
[ 1358.407430] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.417016] task:python3 state:D stack:25312 pid:5002 tgid:5002 ppid:5001 task_flags:0x400100 flags:0x00080002
[ 1358.430221] Call Trace:
[ 1358.433760]
[ 1358.436908] __schedule+0xad9/0x1490
[ 1358.441722] ? io_schedule_timeout+0x130/0x130
[ 1358.447508] ? find_held_lock+0x2b/0x80
[ 1358.452615] ? __lock_release.isra.0+0x59/0x170
[ 1358.458510] schedule+0xd4/0x260
[ 1358.462928] schedule_preempt_disabled+0x18/0x30
[ 1358.468907] __mutex_lock+0xd76/0x1f20
[ 1358.473913] ? threaded_store+0x132/0x220
[ 1358.479211] ? ww_mutex_lock+0x160/0x160
[ 1358.484410] ? selinux_inode_free_security+0x310/0x310
[ 1358.490978] ? cap_capable+0x1ed/0x3f0
[ 1358.495984] ? sysfs_file_ops+0x140/0x140
[ 1358.501283] ? threaded_store+0x132/0x220
[ 1358.506576] threaded_store+0x132/0x220
[ 1358.511677] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.517168] ? sysfs_file_ops+0x140/0x140
[ 1358.522464] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.527759] ? sysfs_kf_write+0x65/0x160
[ 1358.532959] kernfs_fop_write_iter+0x335/0x510
[ 1358.538745] ? kernfs_ops+0xf0/0xf0
[ 1358.543456] new_sync_write+0x308/0x6f0
[ 1358.548559] ? new_sync_read+0x6f0/0x6f0
[ 1358.553758] ? lock_acquire.part.0+0xbc/0x260
[ 1358.559443] ? ksys_write+0xff/0x200
[ 1358.564254] vfs_write+0x62a/0x980
[ 1358.568858] ksys_write+0xff/0x200
[ 1358.573472] ? __ia32_sys_read+0xc0/0xc0
[ 1358.578668] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.584259] ? rcu_is_watching+0x15/0xd0
[ 1358.589459] do_syscall_64+0xf3/0x5e0
[ 1358.594363] ? irq_exit_rcu+0x39/0xf0
[ 1358.599270] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.605738] RIP: 0033:0x7ff060855c5e
[ 1358.610547] RSP: 002b:00007fff14bee430 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.619843] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff060855c5e
[ 1358.628651] RDX: 0000000000000001 RSI: 000055d997270110 RDI: 0000000000000007
[ 1358.637449] RBP: 00007fff14bee440 R08: 0000000000000000 R09: 0000000000000000
[ 1358.646256] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff0606f0e80
[ 1358.655062] R13: 000055d997270110 R14: 0000000000000007 R15: 00007ff060f56b98
[ 1358.663872]
[ 1358.667157] INFO: task python3:5002 is blocked on a mutex likely owned by task python3:3047.
[ 1358.677426] task:python3 state:D stack:25312 pid:3047 tgid:3047 ppid:3046 task_flags:0x400100 flags:0x00080802
[ 1358.690626] Call Trace:
[ 1358.694156]
[ 1358.697304] __schedule+0xad9/0x1490
[ 1358.702111] ? io_schedule_timeout+0x130/0x130
[ 1358.707896] ? find_held_lock+0x2b/0x80
[ 1358.712998] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.719176] schedule+0xd4/0x260
[ 1358.723591] schedule_timeout+0x103/0x260
[ 1358.728891] ? rcu_is_watching+0x15/0xd0
[ 1358.734088] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.740459] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.746147] ? ww_mutex_lock+0x160/0x160
[ 1358.751347] msleep+0x1b/0x30
[ 1358.755472] napi_set_threaded+0x376/0x410
[ 1358.760864] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.766356] netif_set_threaded+0x254/0x400
[ 1358.771848] ? sysfs_file_ops+0x140/0x140
[ 1358.777144] threaded_store+0x1bb/0x220
[ 1358.782248] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.787740] ? sysfs_file_ops+0x140/0x140
[ 1358.793036] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.798331] ? sysfs_kf_write+0x65/0x160
[ 1358.803532] kernfs_fop_write_iter+0x335/0x510
[ 1358.809317] ? kernfs_ops+0xf0/0xf0
[ 1358.814026] new_sync_write+0x308/0x6f0
[ 1358.819128] ? new_sync_read+0x6f0/0x6f0
[ 1358.824320] ? lock_acquire.part.0+0xbc/0x260
[ 1358.829997] ? ksys_write+0xff/0x200
[ 1358.834809] vfs_write+0x62a/0x980
[ 1358.839421] ksys_write+0xff/0x200
[ 1358.844034] ? __ia32_sys_read+0xc0/0xc0
[ 1358.849233] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.854823] ? rcu_is_watching+0x15/0xd0
[ 1358.860021] do_syscall_64+0xf3/0x5e0
[ 1358.864927] ? irq_exit_rcu+0x39/0xf0
[ 1358.869834] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.876290] RIP: 0033:0x7fc8c6b4ac5e
[ 1358.881095] RSP: 002b:00007ffe41ce7ff0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.890391] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fc8c6b4ac5e
[ 1358.899190] RDX: 0000000000000001 RSI: 0000561d80e73110 RDI: 0000000000000007
[ 1358.907990] RBP: 00007ffe41ce8000 R08: 0000000000000000 R09: 0000000000000000
[ 1358.916795] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fc8c69e5e80
[ 1358.925599] R13: 0000561d80e73110 R14: 0000000000000007 R15: 00007fc8c6976480
[ 1358.934411]
[ 1358.937679] \x0aShowing all locks held in the system:
[ 1358.946215] 1 lock held by khungtaskd/184:
[ 1358.951607] #0: ffffffffa32ce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.963077] 2 locks held by systemd-journal/805:
[ 1358.969054] #0: ff1100180e1b7220 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18f/0x1490
[ 1358.978942] #1: ffffffffa34c8ca0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: dput.part.0+0x2fd/0x410
[ 1358.991182] 4 locks held by python3/3047:
[ 1358.996474] #0: ff1100012d9413f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.006262] #1: ff1100013c622088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.017223] #2: ff110001203bb0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.028378] #3: ff110001203a1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.038469] 4 locks held by python3/5002:
[ 1359.043763] #0: ff1100012d9413f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.053555] #1: ff110001bd37ac88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.064515] #2: ff110001203bb0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.075671] #3: ff110001203a1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.085760]
[ 1359.088227] =============================================\x0a