[ 763.729980] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 763.732551] nipa-hw-worker: [4/38] Retrying drivers/net:napi_threaded.py
[ 763.870155] kselftest: Running tests in drivers/net
[ 989.745906] INFO: task python3:5072 blocked for more than 122 seconds.
[ 989.754139] Not tainted 7.0.0-rc4-gekm-g8587293b3de7 #1
[ 989.761390] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.770977] task:python3 state:D stack:25312 pid:5072 tgid:5072 ppid:5071 task_flags:0x400100 flags:0x00080000
[ 989.784180] Call Trace:
[ 989.787718]
[ 989.790868] __schedule+0xad9/0x1490
[ 989.795681] ? io_schedule_timeout+0x130/0x130
[ 989.801464] ? find_held_lock+0x2b/0x80
[ 989.806568] ? __lock_release.isra.0+0x59/0x170
[ 989.812453] schedule+0xd4/0x260
[ 989.816868] schedule_preempt_disabled+0x18/0x30
[ 989.822846] __mutex_lock+0xd76/0x1f20
[ 989.827850] ? threaded_store+0x132/0x220
[ 989.833148] ? ww_mutex_lock+0x160/0x160
[ 989.838347] ? selinux_inode_free_security+0x310/0x310
[ 989.844917] ? cap_capable+0x1ed/0x3f0
[ 989.849922] ? sysfs_file_ops+0x140/0x140
[ 989.855220] ? threaded_store+0x132/0x220
[ 989.860516] threaded_store+0x132/0x220
[ 989.865618] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.871101] ? sysfs_file_ops+0x140/0x140
[ 989.876396] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.881691] ? sysfs_kf_write+0x65/0x160
[ 989.886889] kernfs_fop_write_iter+0x335/0x510
[ 989.892671] ? kernfs_ops+0xf0/0xf0
[ 989.897382] new_sync_write+0x308/0x6f0
[ 989.902485] ? new_sync_read+0x6f0/0x6f0
[ 989.907687] ? lock_acquire.part.0+0xbc/0x260
[ 989.913371] ? ksys_write+0xff/0x200
[ 989.918185] vfs_write+0x62a/0x980
[ 989.922800] ksys_write+0xff/0x200
[ 989.927405] ? __ia32_sys_read+0xc0/0xc0
[ 989.932603] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.938192] ? rcu_is_watching+0x15/0xd0
[ 989.943384] do_syscall_64+0xf3/0x5e0
[ 989.948281] ? exc_page_fault+0x9e/0xb0
[ 989.953384] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.959851] RIP: 0033:0x7f74da0eec5e
[ 989.964649] RSP: 002b:00007ffd96066290 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.973946] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f74da0eec5e
[ 989.982755] RDX: 0000000000000001 RSI: 0000564381f4b110 RDI: 0000000000000007
[ 989.991560] RBP: 00007ffd960662a0 R08: 0000000000000000 R09: 0000000000000000
[ 990.000367] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f74d9f89e80
[ 990.009176] R13: 0000564381f4b110 R14: 0000000000000007 R15: 00007f74da7efb98
[ 990.017989]
[ 990.021271] INFO: task python3:5072 is blocked on a mutex likely owned by task python3:3124.
[ 990.031547] task:python3 state:D stack:25312 pid:3124 tgid:3124 ppid:3123 task_flags:0x400100 flags:0x00080802
[ 990.044748] Call Trace:
[ 990.048285]
[ 990.051432] __schedule+0xad9/0x1490
[ 990.056240] ? io_schedule_timeout+0x130/0x130
[ 990.062021] ? find_held_lock+0x2b/0x80
[ 990.067123] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.073293] schedule+0xd4/0x260
[ 990.077712] schedule_timeout+0x103/0x260
[ 990.083001] ? rcu_is_watching+0x15/0xd0
[ 990.088200] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.094569] ? hrtimers_cpu_dying+0x250/0x250
[ 990.100258] ? ww_mutex_lock+0x160/0x160
[ 990.105455] msleep+0x1b/0x30
[ 990.109580] napi_set_threaded+0x376/0x410
[ 990.114976] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.120460] netif_set_threaded+0x254/0x400
[ 990.125945] ? sysfs_file_ops+0x140/0x140
[ 990.131239] threaded_store+0x1bb/0x220
[ 990.136340] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.141830] ? sysfs_file_ops+0x140/0x140
[ 990.147125] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.152419] ? sysfs_kf_write+0x65/0x160
[ 990.157619] kernfs_fop_write_iter+0x335/0x510
[ 990.163394] ? kernfs_ops+0xf0/0xf0
[ 990.168103] new_sync_write+0x308/0x6f0
[ 990.173202] ? new_sync_read+0x6f0/0x6f0
[ 990.178402] ? lock_acquire.part.0+0xbc/0x260
[ 990.184086] ? ksys_write+0xff/0x200
[ 990.188900] vfs_write+0x62a/0x980
[ 990.193514] ksys_write+0xff/0x200
[ 990.198129] ? __ia32_sys_read+0xc0/0xc0
[ 990.203329] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.208910] ? rcu_is_watching+0x15/0xd0
[ 990.214107] do_syscall_64+0xf3/0x5e0
[ 990.219010] ? exc_page_fault+0x9e/0xb0
[ 990.224111] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.230576] RIP: 0033:0x7fe3f3e87c5e
[ 990.235384] RSP: 002b:00007fff435bf770 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.244679] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe3f3e87c5e
[ 990.253484] RDX: 0000000000000001 RSI: 000055942d44e110 RDI: 0000000000000007
[ 990.262289] RBP: 00007fff435bf780 R08: 0000000000000000 R09: 0000000000000000
[ 990.271095] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe3f3d22e80
[ 990.279900] R13: 000055942d44e110 R14: 0000000000000007 R15: 00007fe3f3cb2480
[ 990.288711]
[ 990.291972] \x0aShowing all locks held in the system:
[ 990.300506] 1 lock held by khungtaskd/185:
[ 990.305896] #0: ffffffff86cce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.317379] 4 locks held by python3/3124:
[ 990.322673] #0: ff1100012e3893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.332465] #1: ff1100013ea28888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.343425] #2: ff1100012255d0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.354580] #3: ff110001225e1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.364676] 4 locks held by python3/5072:
[ 990.369969] #0: ff1100012e3893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.379758] #1: ff110001be644488 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.390718] #2: ff1100012255d0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.401874] #3: ff110001225e1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.411965]
[ 990.414430] =============================================\x0a
[ 1066.455693] block nvme0n1: No UUID available providing old NGUID
SUBSYSTEM=block
DEVICE=b259:0
[ 1112.627152] INFO: task python3:5072 blocked for more than 245 seconds.
[ 1112.635299] Not tainted 7.0.0-rc4-gekm-g8587293b3de7 #1
[ 1112.642547] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.652130] task:python3 state:D stack:25312 pid:5072 tgid:5072 ppid:5071 task_flags:0x400100 flags:0x00080000
[ 1112.665337] Call Trace:
[ 1112.668875]
[ 1112.672026] __schedule+0xad9/0x1490
[ 1112.676843] ? io_schedule_timeout+0x130/0x130
[ 1112.682631] ? find_held_lock+0x2b/0x80
[ 1112.687739] ? __lock_release.isra.0+0x59/0x170
[ 1112.693627] schedule+0xd4/0x260
[ 1112.698044] schedule_preempt_disabled+0x18/0x30
[ 1112.704025] __mutex_lock+0xd76/0x1f20
[ 1112.709030] ? threaded_store+0x132/0x220
[ 1112.714328] ? ww_mutex_lock+0x160/0x160
[ 1112.719528] ? selinux_inode_free_security+0x310/0x310
[ 1112.726097] ? cap_capable+0x1ed/0x3f0
[ 1112.731105] ? sysfs_file_ops+0x140/0x140
[ 1112.736405] ? threaded_store+0x132/0x220
[ 1112.741695] threaded_store+0x132/0x220
[ 1112.746800] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.752294] ? sysfs_file_ops+0x140/0x140
[ 1112.757590] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.762890] ? sysfs_kf_write+0x65/0x160
[ 1112.768091] kernfs_fop_write_iter+0x335/0x510
[ 1112.773877] ? kernfs_ops+0xf0/0xf0
[ 1112.778589] new_sync_write+0x308/0x6f0
[ 1112.783686] ? new_sync_read+0x6f0/0x6f0
[ 1112.788888] ? lock_acquire.part.0+0xbc/0x260
[ 1112.794574] ? ksys_write+0xff/0x200
[ 1112.799388] vfs_write+0x62a/0x980
[ 1112.804002] ksys_write+0xff/0x200
[ 1112.808616] ? __ia32_sys_read+0xc0/0xc0
[ 1112.813816] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.819399] ? rcu_is_watching+0x15/0xd0
[ 1112.824603] do_syscall_64+0xf3/0x5e0
[ 1112.829511] ? exc_page_fault+0x9e/0xb0
[ 1112.834613] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.841082] RIP: 0033:0x7f74da0eec5e
[ 1112.845891] RSP: 002b:00007ffd96066290 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.855189] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f74da0eec5e
[ 1112.863999] RDX: 0000000000000001 RSI: 0000564381f4b110 RDI: 0000000000000007
[ 1112.872805] RBP: 00007ffd960662a0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.881615] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f74d9f89e80
[ 1112.890424] R13: 0000564381f4b110 R14: 0000000000000007 R15: 00007f74da7efb98
[ 1112.899239]
[ 1112.902516] INFO: task python3:5072 is blocked on a mutex likely owned by task python3:3124.
[ 1112.912789] task:python3 state:D stack:25312 pid:3124 tgid:3124 ppid:3123 task_flags:0x400100 flags:0x00080802
[ 1112.925994] Call Trace:
[ 1112.929533]
[ 1112.932682] __schedule+0xad9/0x1490
[ 1112.937493] ? io_schedule_timeout+0x130/0x130
[ 1112.943277] ? find_held_lock+0x2b/0x80
[ 1112.948380] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.954559] schedule+0xd4/0x260
[ 1112.958977] schedule_timeout+0x103/0x260
[ 1112.964272] ? rcu_is_watching+0x15/0xd0
[ 1112.969472] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.975843] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.981535] ? ww_mutex_lock+0x160/0x160
[ 1112.986733] msleep+0x1b/0x30
[ 1112.990857] napi_set_threaded+0x376/0x410
[ 1112.996254] ? napi_resume_irqs+0x1a0/0x1a0
[ 1113.001750] netif_set_threaded+0x254/0x400
[ 1113.007245] ? sysfs_file_ops+0x140/0x140
[ 1113.012542] threaded_store+0x1bb/0x220
[ 1113.017646] ? tx_maxrate_store+0x4e0/0x4e0
[ 1113.023130] ? sysfs_file_ops+0x140/0x140
[ 1113.028427] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.033725] ? sysfs_kf_write+0x65/0x160
[ 1113.038925] kernfs_fop_write_iter+0x335/0x510
[ 1113.044705] ? kernfs_ops+0xf0/0xf0
[ 1113.049416] new_sync_write+0x308/0x6f0
[ 1113.054520] ? new_sync_read+0x6f0/0x6f0
[ 1113.059719] ? lock_acquire.part.0+0xbc/0x260
[ 1113.065406] ? ksys_write+0xff/0x200
[ 1113.070219] vfs_write+0x62a/0x980
[ 1113.074834] ksys_write+0xff/0x200
[ 1113.079448] ? __ia32_sys_read+0xc0/0xc0
[ 1113.084641] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.090230] ? rcu_is_watching+0x15/0xd0
[ 1113.095429] do_syscall_64+0xf3/0x5e0
[ 1113.100335] ? exc_page_fault+0x9e/0xb0
[ 1113.105438] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.111897] RIP: 0033:0x7fe3f3e87c5e
[ 1113.116699] RSP: 002b:00007fff435bf770 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.125997] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe3f3e87c5e
[ 1113.134808] RDX: 0000000000000001 RSI: 000055942d44e110 RDI: 0000000000000007
[ 1113.143616] RBP: 00007fff435bf780 R08: 0000000000000000 R09: 0000000000000000
[ 1113.152426] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe3f3d22e80
[ 1113.161234] R13: 000055942d44e110 R14: 0000000000000007 R15: 00007fe3f3cb2480
[ 1113.170050]
[ 1113.173307] \x0aShowing all locks held in the system:
[ 1113.181844] 1 lock held by khungtaskd/185:
[ 1113.187238] #0: ffffffff86cce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.198714] 4 locks held by python3/3124:
[ 1113.203999] #0: ff1100012e3893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.213794] #1: ff1100013ea28888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.224760] #2: ff1100012255d0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.235918] #3: ff110001225e1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.246015] 4 locks held by python3/5072:
[ 1113.251313] #0: ff1100012e3893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.261102] #1: ff110001be644488 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.272062] #2: ff1100012255d0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.283221] #3: ff110001225e1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.293316]
[ 1113.295785] =============================================\x0a
[ 1235.508427] INFO: task python3:5072 blocked for more than 368 seconds.
[ 1235.516587] Not tainted 7.0.0-rc4-gekm-g8587293b3de7 #1
[ 1235.523840] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.533427] task:python3 state:D stack:25312 pid:5072 tgid:5072 ppid:5071 task_flags:0x400100 flags:0x00080002
[ 1235.546630] Call Trace:
[ 1235.550172]
[ 1235.553320] __schedule+0xad9/0x1490
[ 1235.558136] ? io_schedule_timeout+0x130/0x130
[ 1235.563920] ? find_held_lock+0x2b/0x80
[ 1235.569014] ? __lock_release.isra.0+0x59/0x170
[ 1235.574895] schedule+0xd4/0x260
[ 1235.579314] schedule_preempt_disabled+0x18/0x30
[ 1235.585292] __mutex_lock+0xd76/0x1f20
[ 1235.590298] ? threaded_store+0x132/0x220
[ 1235.595598] ? ww_mutex_lock+0x160/0x160
[ 1235.600794] ? selinux_inode_free_security+0x310/0x310
[ 1235.607360] ? cap_capable+0x1ed/0x3f0
[ 1235.612366] ? sysfs_file_ops+0x140/0x140
[ 1235.617663] ? threaded_store+0x132/0x220
[ 1235.622957] threaded_store+0x132/0x220
[ 1235.628058] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.633547] ? sysfs_file_ops+0x140/0x140
[ 1235.638841] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.644135] ? sysfs_kf_write+0x65/0x160
[ 1235.649335] kernfs_fop_write_iter+0x335/0x510
[ 1235.655120] ? kernfs_ops+0xf0/0xf0
[ 1235.659831] new_sync_write+0x308/0x6f0
[ 1235.664934] ? new_sync_read+0x6f0/0x6f0
[ 1235.670124] ? lock_acquire.part.0+0xbc/0x260
[ 1235.675810] ? ksys_write+0xff/0x200
[ 1235.680620] vfs_write+0x62a/0x980
[ 1235.685224] ksys_write+0xff/0x200
[ 1235.689837] ? __ia32_sys_read+0xc0/0xc0
[ 1235.695035] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.700625] ? rcu_is_watching+0x15/0xd0
[ 1235.705815] do_syscall_64+0xf3/0x5e0
[ 1235.710719] ? exc_page_fault+0x9e/0xb0
[ 1235.715819] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.722284] RIP: 0033:0x7f74da0eec5e
[ 1235.727093] RSP: 002b:00007ffd96066290 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.736389] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f74da0eec5e
[ 1235.745196] RDX: 0000000000000001 RSI: 0000564381f4b110 RDI: 0000000000000007
[ 1235.754000] RBP: 00007ffd960662a0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.762807] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f74d9f89e80
[ 1235.771614] R13: 0000564381f4b110 R14: 0000000000000007 R15: 00007f74da7efb98
[ 1235.780429]
[ 1235.783717] INFO: task python3:5072 is blocked on a mutex likely owned by task python3:3124.
[ 1235.793988] task:python3 state:D stack:25312 pid:3124 tgid:3124 ppid:3123 task_flags:0x400100 flags:0x00080802
[ 1235.807188] Call Trace:
[ 1235.810719]
[ 1235.813868] __schedule+0xad9/0x1490
[ 1235.818679] ? io_schedule_timeout+0x130/0x130
[ 1235.824462] ? find_held_lock+0x2b/0x80
[ 1235.829565] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.835742] schedule+0xd4/0x260
[ 1235.840151] schedule_timeout+0x103/0x260
[ 1235.845446] ? rcu_is_watching+0x15/0xd0
[ 1235.850641] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.857012] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.862700] ? ww_mutex_lock+0x160/0x160
[ 1235.867896] msleep+0x1b/0x30
[ 1235.872020] napi_set_threaded+0x376/0x410
[ 1235.877414] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.882907] netif_set_threaded+0x254/0x400
[ 1235.888402] ? sysfs_file_ops+0x140/0x140
[ 1235.893697] threaded_store+0x1bb/0x220
[ 1235.898796] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.904286] ? sysfs_file_ops+0x140/0x140
[ 1235.909582] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.914877] ? sysfs_kf_write+0x65/0x160
[ 1235.920078] kernfs_fop_write_iter+0x335/0x510
[ 1235.925864] ? kernfs_ops+0xf0/0xf0
[ 1235.930576] new_sync_write+0x308/0x6f0
[ 1235.935675] ? new_sync_read+0x6f0/0x6f0
[ 1235.940873] ? lock_acquire.part.0+0xbc/0x260
[ 1235.946559] ? ksys_write+0xff/0x200
[ 1235.951371] vfs_write+0x62a/0x980
[ 1235.955986] ksys_write+0xff/0x200
[ 1235.960599] ? __ia32_sys_read+0xc0/0xc0
[ 1235.965794] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.971382] ? rcu_is_watching+0x15/0xd0
[ 1235.976582] do_syscall_64+0xf3/0x5e0
[ 1235.981487] ? exc_page_fault+0x9e/0xb0
[ 1235.986590] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.993049] RIP: 0033:0x7fe3f3e87c5e
[ 1235.997857] RSP: 002b:00007fff435bf770 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1236.007153] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe3f3e87c5e
[ 1236.015958] RDX: 0000000000000001 RSI: 000055942d44e110 RDI: 0000000000000007
[ 1236.024763] RBP: 00007fff435bf780 R08: 0000000000000000 R09: 0000000000000000
[ 1236.033569] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe3f3d22e80
[ 1236.042375] R13: 000055942d44e110 R14: 0000000000000007 R15: 00007fe3f3cb2480
[ 1236.051186]
[ 1236.054453] \x0aShowing all locks held in the system:
[ 1236.062990] 1 lock held by khungtaskd/185:
[ 1236.068383] #0: ffffffff86cce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.079862] 4 locks held by python3/3124:
[ 1236.085154] #0: ff1100012e3893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.094943] #1: ff1100013ea28888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.105904] #2: ff1100012255d0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.117061] #3: ff110001225e1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.127157] 4 locks held by python3/5072:
[ 1236.132449] #0: ff1100012e3893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.142239] #1: ff110001be644488 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.153200] #2: ff1100012255d0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.164359] #3: ff110001225e1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.174450]
[ 1236.176915] =============================================\x0a
[ 1358.390703] INFO: task python3:5072 blocked for more than 491 seconds.
[ 1358.398849] Not tainted 7.0.0-rc4-gekm-g8587293b3de7 #1
[ 1358.406098] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.415691] task:python3 state:D stack:25312 pid:5072 tgid:5072 ppid:5071 task_flags:0x400100 flags:0x00080002
[ 1358.428897] Call Trace:
[ 1358.432440]
[ 1358.435581] __schedule+0xad9/0x1490
[ 1358.440399] ? io_schedule_timeout+0x130/0x130
[ 1358.446186] ? find_held_lock+0x2b/0x80
[ 1358.451291] ? __lock_release.isra.0+0x59/0x170
[ 1358.457180] schedule+0xd4/0x260
[ 1358.461601] schedule_preempt_disabled+0x18/0x30
[ 1358.467581] __mutex_lock+0xd76/0x1f20
[ 1358.472580] ? threaded_store+0x132/0x220
[ 1358.477882] ? ww_mutex_lock+0x160/0x160
[ 1358.483081] ? selinux_inode_free_security+0x310/0x310
[ 1358.489651] ? cap_capable+0x1ed/0x3f0
[ 1358.494657] ? sysfs_file_ops+0x140/0x140
[ 1358.499955] ? threaded_store+0x132/0x220
[ 1358.505250] threaded_store+0x132/0x220
[ 1358.510352] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.515846] ? sysfs_file_ops+0x140/0x140
[ 1358.521146] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.526444] ? sysfs_kf_write+0x65/0x160
[ 1358.531637] kernfs_fop_write_iter+0x335/0x510
[ 1358.537423] ? kernfs_ops+0xf0/0xf0
[ 1358.542137] new_sync_write+0x308/0x6f0
[ 1358.547243] ? new_sync_read+0x6f0/0x6f0
[ 1358.552444] ? lock_acquire.part.0+0xbc/0x260
[ 1358.558131] ? ksys_write+0xff/0x200
[ 1358.562945] vfs_write+0x62a/0x980
[ 1358.567560] ksys_write+0xff/0x200
[ 1358.572174] ? __ia32_sys_read+0xc0/0xc0
[ 1358.577374] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.582964] ? rcu_is_watching+0x15/0xd0
[ 1358.588170] do_syscall_64+0xf3/0x5e0
[ 1358.593078] ? exc_page_fault+0x9e/0xb0
[ 1358.598170] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.604640] RIP: 0033:0x7f74da0eec5e
[ 1358.609450] RSP: 002b:00007ffd96066290 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.618749] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f74da0eec5e
[ 1358.627560] RDX: 0000000000000001 RSI: 0000564381f4b110 RDI: 0000000000000007
[ 1358.636369] RBP: 00007ffd960662a0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.645178] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f74d9f89e80
[ 1358.653987] R13: 0000564381f4b110 R14: 0000000000000007 R15: 00007f74da7efb98
[ 1358.662803]
[ 1358.666066] INFO: task python3:5072 is blocked on a mutex likely owned by task python3:3124.
[ 1358.676341] task:python3 state:D stack:25312 pid:3124 tgid:3124 ppid:3123 task_flags:0x400100 flags:0x00080802
[ 1358.689545] Call Trace:
[ 1358.693084]
[ 1358.696232] __schedule+0xad9/0x1490
[ 1358.701042] ? io_schedule_timeout+0x130/0x130
[ 1358.706825] ? find_held_lock+0x2b/0x80
[ 1358.711928] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.718109] schedule+0xd4/0x260
[ 1358.722528] schedule_timeout+0x103/0x260
[ 1358.727825] ? rcu_is_watching+0x15/0xd0
[ 1358.733024] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.739394] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.745077] ? ww_mutex_lock+0x160/0x160
[ 1358.750275] msleep+0x1b/0x30
[ 1358.754404] napi_set_threaded+0x376/0x410
[ 1358.759801] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.765298] netif_set_threaded+0x254/0x400
[ 1358.770791] ? sysfs_file_ops+0x140/0x140
[ 1358.776079] threaded_store+0x1bb/0x220
[ 1358.781182] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.786677] ? sysfs_file_ops+0x140/0x140
[ 1358.791972] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.797272] ? sysfs_kf_write+0x65/0x160
[ 1358.802471] kernfs_fop_write_iter+0x335/0x510
[ 1358.808255] ? kernfs_ops+0xf0/0xf0
[ 1358.812966] new_sync_write+0x308/0x6f0
[ 1358.818069] ? new_sync_read+0x6f0/0x6f0
[ 1358.823269] ? lock_acquire.part.0+0xbc/0x260
[ 1358.828954] ? ksys_write+0xff/0x200
[ 1358.833765] vfs_write+0x62a/0x980
[ 1358.838382] ksys_write+0xff/0x200
[ 1358.842997] ? __ia32_sys_read+0xc0/0xc0
[ 1358.848195] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.853786] ? rcu_is_watching+0x15/0xd0
[ 1358.858979] do_syscall_64+0xf3/0x5e0
[ 1358.863885] ? exc_page_fault+0x9e/0xb0
[ 1358.868985] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.875454] RIP: 0033:0x7fe3f3e87c5e
[ 1358.880263] RSP: 002b:00007fff435bf770 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.889560] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fe3f3e87c5e
[ 1358.898368] RDX: 0000000000000001 RSI: 000055942d44e110 RDI: 0000000000000007
[ 1358.907176] RBP: 00007fff435bf780 R08: 0000000000000000 R09: 0000000000000000
[ 1358.915986] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fe3f3d22e80
[ 1358.924794] R13: 000055942d44e110 R14: 0000000000000007 R15: 00007fe3f3cb2480
[ 1358.933599]
[ 1358.936869] \x0aShowing all locks held in the system:
[ 1358.945408] 1 lock held by khungtaskd/185:
[ 1358.950801] #0: ffffffff86cce020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.962284] 4 locks held by python3/3124:
[ 1358.967579] #0: ff1100012e3893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.977373] #1: ff1100013ea28888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1358.988336] #2: ff1100012255d0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.999492] #3: ff110001225e1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.009583] 4 locks held by python3/5072:
[ 1359.014881] #0: ff1100012e3893f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.024671] #1: ff110001be644488 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.035636] #2: ff1100012255d0f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.046794] #3: ff110001225e1cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.056891] 1 lock held by systemd/6898:
[ 1359.062092]
[ 1359.064560] =============================================\x0a