[ 762.393937] nipa-hw-worker: drivers/net:napi_threaded.py: timed out [ 762.400047] nipa-hw-worker: [4/39] Retrying drivers/net:napi_threaded.py [ 762.536838] kselftest: Running tests in drivers/net [ 989.752962] INFO: task python3:5078 blocked for more than 122 seconds. [ 989.761182] Not tainted 7.0.0-rc4-zviy-g253dc35e0749 #1 [ 989.768430] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 989.778019] task:python3 state:D stack:25312 pid:5078 tgid:5078 ppid:5077 task_flags:0x400100 flags:0x00080000 [ 989.791224] Call Trace: [ 989.794762] [ 989.797913] __schedule+0xad9/0x1490 [ 989.802727] ? io_schedule_timeout+0x130/0x130 [ 989.808500] ? find_held_lock+0x2b/0x80 [ 989.813605] ? __lock_release.isra.0+0x59/0x170 [ 989.819490] schedule+0xd4/0x260 [ 989.823908] schedule_preempt_disabled+0x18/0x30 [ 989.829887] __mutex_lock+0xd76/0x1f20 [ 989.834893] ? threaded_store+0x132/0x220 [ 989.840193] ? ww_mutex_lock+0x160/0x160 [ 989.845389] ? selinux_inode_free_security+0x310/0x310 [ 989.851960] ? cap_capable+0x1ed/0x3f0 [ 989.856965] ? sysfs_file_ops+0x140/0x140 [ 989.862262] ? threaded_store+0x132/0x220 [ 989.867558] threaded_store+0x132/0x220 [ 989.872660] ? tx_maxrate_store+0x4e0/0x4e0 [ 989.878145] ? sysfs_file_ops+0x140/0x140 [ 989.883439] ? sysfs_file_kobj+0xbd/0x1e0 [ 989.888736] ? sysfs_kf_write+0x65/0x160 [ 989.893935] kernfs_fop_write_iter+0x335/0x510 [ 989.899719] ? kernfs_ops+0xf0/0xf0 [ 989.904429] new_sync_write+0x308/0x6f0 [ 989.909529] ? new_sync_read+0x6f0/0x6f0 [ 989.914728] ? lock_acquire.part.0+0xbc/0x260 [ 989.920403] ? ksys_write+0xff/0x200 [ 989.925217] vfs_write+0x62a/0x980 [ 989.929829] ksys_write+0xff/0x200 [ 989.934440] ? __ia32_sys_read+0xc0/0xc0 [ 989.939639] ? kernfs_fop_llseek+0x160/0x1f0 [ 989.945229] ? rcu_is_watching+0x15/0xd0 [ 989.950431] do_syscall_64+0xf3/0x5e0 [ 989.955337] ? exc_page_fault+0x9e/0xb0 [ 989.960437] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 989.966905] RIP: 0033:0x7f11edf0fc5e [ 989.971705] RSP: 002b:00007ffcceaefbe0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 989.981001] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f11edf0fc5e [ 989.989809] RDX: 0000000000000001 RSI: 0000560033a5f110 RDI: 0000000000000007 [ 989.998609] RBP: 00007ffcceaefbf0 R08: 0000000000000000 R09: 0000000000000000 [ 990.007406] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f11eddaae80 [ 990.016213] R13: 0000560033a5f110 R14: 0000000000000007 R15: 00007f11ee610b98 [ 990.025028] [ 990.028316] INFO: task python3:5078 is blocked on a mutex likely owned by task python3:3145. [ 990.038589] task:python3 state:D stack:25312 pid:3145 tgid:3145 ppid:3144 task_flags:0x400100 flags:0x00080802 [ 990.051795] Call Trace: [ 990.055332] [ 990.058483] __schedule+0xad9/0x1490 [ 990.063293] ? io_schedule_timeout+0x130/0x130 [ 990.069066] ? find_held_lock+0x2b/0x80 [ 990.074170] ? __try_to_del_timer_sync+0x154/0x1f0 [ 990.080348] schedule+0xd4/0x260 [ 990.084764] schedule_timeout+0x103/0x260 [ 990.090059] ? rcu_is_watching+0x15/0xd0 [ 990.095254] ? hrtimer_nanosleep_restart+0x180/0x180 [ 990.101624] ? hrtimers_cpu_dying+0x250/0x250 [ 990.107311] ? ww_mutex_lock+0x160/0x160 [ 990.112510] msleep+0x1b/0x30 [ 990.116634] napi_set_threaded+0x376/0x410 [ 990.122030] ? napi_resume_irqs+0x1a0/0x1a0 [ 990.127518] netif_set_threaded+0x254/0x400 [ 990.133011] ? sysfs_file_ops+0x140/0x140 [ 990.138305] threaded_store+0x1bb/0x220 [ 990.143408] ? tx_maxrate_store+0x4e0/0x4e0 [ 990.148888] ? sysfs_file_ops+0x140/0x140 [ 990.154185] ? sysfs_file_kobj+0xbd/0x1e0 [ 990.159479] ? sysfs_kf_write+0x65/0x160 [ 990.164679] kernfs_fop_write_iter+0x335/0x510 [ 990.170463] ? kernfs_ops+0xf0/0xf0 [ 990.175175] new_sync_write+0x308/0x6f0 [ 990.180276] ? new_sync_read+0x6f0/0x6f0 [ 990.185476] ? lock_acquire.part.0+0xbc/0x260 [ 990.191162] ? ksys_write+0xff/0x200 [ 990.195972] vfs_write+0x62a/0x980 [ 990.200586] ksys_write+0xff/0x200 [ 990.205197] ? __ia32_sys_read+0xc0/0xc0 [ 990.210394] ? kernfs_fop_llseek+0x160/0x1f0 [ 990.215981] ? rcu_is_watching+0x15/0xd0 [ 990.221179] do_syscall_64+0xf3/0x5e0 [ 990.226082] ? exc_page_fault+0x9e/0xb0 [ 990.231185] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 990.237645] RIP: 0033:0x7ff2f3035c5e [ 990.242452] RSP: 002b:00007ffc1e5b8ab0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 990.251749] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff2f3035c5e [ 990.260557] RDX: 0000000000000001 RSI: 0000563380564110 RDI: 0000000000000007 [ 990.269356] RBP: 00007ffc1e5b8ac0 R08: 0000000000000000 R09: 0000000000000000 [ 990.278156] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff2f2ed0e80 [ 990.286960] R13: 0000563380564110 R14: 0000000000000007 R15: 00007ff2f2e5e480 [ 990.295775] [ 990.299033] \x0aShowing all locks held in the system: [ 990.307571] 1 lock held by khungtaskd/185: [ 990.312962] #0: ffffffffadace020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 990.324444] 4 locks held by python3/3145: [ 990.329740] #0: ff1100012eb243f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 990.339528] #1: ff11000149410888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 990.350488] #2: ff11000122ba70f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.361646] #3: ff11000122b59cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.371740] 4 locks held by python3/5078: [ 990.377033] #0: ff1100012eb243f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 990.386823] #1: ff110001a4e7e288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 990.397787] #2: ff11000122ba70f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 990.408957] #3: ff11000122b59cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 990.419046] [ 990.421510] =============================================\x0a [ 1112.634260] INFO: task python3:5078 blocked for more than 245 seconds. [ 1112.642404] Not tainted 7.0.0-rc4-zviy-g253dc35e0749 #1 [ 1112.649653] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1112.659242] task:python3 state:D stack:25312 pid:5078 tgid:5078 ppid:5077 task_flags:0x400100 flags:0x00080000 [ 1112.672441] Call Trace: [ 1112.675982] [ 1112.679131] __schedule+0xad9/0x1490 [ 1112.683946] ? io_schedule_timeout+0x130/0x130 [ 1112.689731] ? find_held_lock+0x2b/0x80 [ 1112.694841] ? __lock_release.isra.0+0x59/0x170 [ 1112.700725] schedule+0xd4/0x260 [ 1112.705142] schedule_preempt_disabled+0x18/0x30 [ 1112.711120] __mutex_lock+0xd76/0x1f20 [ 1112.716125] ? threaded_store+0x132/0x220 [ 1112.721428] ? ww_mutex_lock+0x160/0x160 [ 1112.726628] ? selinux_inode_free_security+0x310/0x310 [ 1112.733197] ? cap_capable+0x1ed/0x3f0 [ 1112.738207] ? sysfs_file_ops+0x140/0x140 [ 1112.743508] ? threaded_store+0x132/0x220 [ 1112.748803] threaded_store+0x132/0x220 [ 1112.753909] ? tx_maxrate_store+0x4e0/0x4e0 [ 1112.759402] ? sysfs_file_ops+0x140/0x140 [ 1112.764700] ? sysfs_file_kobj+0xbd/0x1e0 [ 1112.769987] ? sysfs_kf_write+0x65/0x160 [ 1112.775188] kernfs_fop_write_iter+0x335/0x510 [ 1112.780975] ? kernfs_ops+0xf0/0xf0 [ 1112.785688] new_sync_write+0x308/0x6f0 [ 1112.790791] ? new_sync_read+0x6f0/0x6f0 [ 1112.795993] ? lock_acquire.part.0+0xbc/0x260 [ 1112.801678] ? ksys_write+0xff/0x200 [ 1112.806490] vfs_write+0x62a/0x980 [ 1112.811105] ksys_write+0xff/0x200 [ 1112.815718] ? __ia32_sys_read+0xc0/0xc0 [ 1112.820915] ? kernfs_fop_llseek+0x160/0x1f0 [ 1112.826508] ? rcu_is_watching+0x15/0xd0 [ 1112.831712] do_syscall_64+0xf3/0x5e0 [ 1112.836620] ? exc_page_fault+0x9e/0xb0 [ 1112.841723] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1112.848183] RIP: 0033:0x7f11edf0fc5e [ 1112.852994] RSP: 002b:00007ffcceaefbe0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1112.862293] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f11edf0fc5e [ 1112.871104] RDX: 0000000000000001 RSI: 0000560033a5f110 RDI: 0000000000000007 [ 1112.879913] RBP: 00007ffcceaefbf0 R08: 0000000000000000 R09: 0000000000000000 [ 1112.888721] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f11eddaae80 [ 1112.897521] R13: 0000560033a5f110 R14: 0000000000000007 R15: 00007f11ee610b98 [ 1112.906336] [ 1112.909607] INFO: task python3:5078 is blocked on a mutex likely owned by task python3:3145. [ 1112.919879] task:python3 state:D stack:25312 pid:3145 tgid:3145 ppid:3144 task_flags:0x400100 flags:0x00080802 [ 1112.933082] Call Trace: [ 1112.936621] [ 1112.939773] __schedule+0xad9/0x1490 [ 1112.944585] ? io_schedule_timeout+0x130/0x130 [ 1112.950369] ? find_held_lock+0x2b/0x80 [ 1112.955474] ? __try_to_del_timer_sync+0x154/0x1f0 [ 1112.961653] schedule+0xd4/0x260 [ 1112.966072] schedule_timeout+0x103/0x260 [ 1112.971369] ? rcu_is_watching+0x15/0xd0 [ 1112.976567] ? hrtimer_nanosleep_restart+0x180/0x180 [ 1112.982937] ? hrtimers_cpu_dying+0x250/0x250 [ 1112.988630] ? ww_mutex_lock+0x160/0x160 [ 1112.993830] msleep+0x1b/0x30 [ 1112.997948] napi_set_threaded+0x376/0x410 [ 1113.003346] ? napi_resume_irqs+0x1a0/0x1a0 [ 1113.008840] netif_set_threaded+0x254/0x400 [ 1113.014335] ? sysfs_file_ops+0x140/0x140 [ 1113.019634] threaded_store+0x1bb/0x220 [ 1113.024737] ? tx_maxrate_store+0x4e0/0x4e0 [ 1113.030228] ? sysfs_file_ops+0x140/0x140 [ 1113.035525] ? sysfs_file_kobj+0xbd/0x1e0 [ 1113.040824] ? sysfs_kf_write+0x65/0x160 [ 1113.046024] kernfs_fop_write_iter+0x335/0x510 [ 1113.051808] ? kernfs_ops+0xf0/0xf0 [ 1113.056521] new_sync_write+0x308/0x6f0 [ 1113.061614] ? new_sync_read+0x6f0/0x6f0 [ 1113.066816] ? lock_acquire.part.0+0xbc/0x260 [ 1113.072503] ? ksys_write+0xff/0x200 [ 1113.077314] vfs_write+0x62a/0x980 [ 1113.081929] ksys_write+0xff/0x200 [ 1113.086533] ? __ia32_sys_read+0xc0/0xc0 [ 1113.091731] ? kernfs_fop_llseek+0x160/0x1f0 [ 1113.097322] ? rcu_is_watching+0x15/0xd0 [ 1113.102521] do_syscall_64+0xf3/0x5e0 [ 1113.107425] ? exc_page_fault+0x9e/0xb0 [ 1113.112529] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1113.118998] RIP: 0033:0x7ff2f3035c5e [ 1113.123806] RSP: 002b:00007ffc1e5b8ab0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1113.133103] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff2f3035c5e [ 1113.141912] RDX: 0000000000000001 RSI: 0000563380564110 RDI: 0000000000000007 [ 1113.150722] RBP: 00007ffc1e5b8ac0 R08: 0000000000000000 R09: 0000000000000000 [ 1113.159532] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff2f2ed0e80 [ 1113.168339] R13: 0000563380564110 R14: 0000000000000007 R15: 00007ff2f2e5e480 [ 1113.177147] [ 1113.180402] \x0aShowing all locks held in the system: [ 1113.188941] 1 lock held by khungtaskd/185: [ 1113.194334] #0: ffffffffadace020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1113.205817] 4 locks held by python3/3145: [ 1113.211113] #0: ff1100012eb243f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1113.220907] #1: ff11000149410888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1113.231872] #2: ff11000122ba70f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.243030] #3: ff11000122b59cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.253123] 4 locks held by python3/5078: [ 1113.258417] #0: ff1100012eb243f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1113.268208] #1: ff110001a4e7e288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1113.279169] #2: ff11000122ba70f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1113.290327] #3: ff11000122b59cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1113.300418] [ 1113.302886] =============================================\x0a [ 1235.516548] INFO: task python3:5078 blocked for more than 368 seconds. [ 1235.524696] Not tainted 7.0.0-rc4-zviy-g253dc35e0749 #1 [ 1235.531945] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1235.541537] task:python3 state:D stack:25312 pid:5078 tgid:5078 ppid:5077 task_flags:0x400100 flags:0x00080002 [ 1235.554745] Call Trace: [ 1235.558285] [ 1235.561439] __schedule+0xad9/0x1490 [ 1235.566258] ? io_schedule_timeout+0x130/0x130 [ 1235.572044] ? find_held_lock+0x2b/0x80 [ 1235.577152] ? __lock_release.isra.0+0x59/0x170 [ 1235.583036] schedule+0xd4/0x260 [ 1235.587445] schedule_preempt_disabled+0x18/0x30 [ 1235.593422] __mutex_lock+0xd76/0x1f20 [ 1235.598431] ? threaded_store+0x132/0x220 [ 1235.603729] ? ww_mutex_lock+0x160/0x160 [ 1235.608929] ? selinux_inode_free_security+0x310/0x310 [ 1235.615499] ? cap_capable+0x1ed/0x3f0 [ 1235.620507] ? sysfs_file_ops+0x140/0x140 [ 1235.625807] ? threaded_store+0x132/0x220 [ 1235.631103] threaded_store+0x132/0x220 [ 1235.636206] ? tx_maxrate_store+0x4e0/0x4e0 [ 1235.641701] ? sysfs_file_ops+0x140/0x140 [ 1235.646999] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.652295] ? sysfs_kf_write+0x65/0x160 [ 1235.657495] kernfs_fop_write_iter+0x335/0x510 [ 1235.663273] ? kernfs_ops+0xf0/0xf0 [ 1235.667986] new_sync_write+0x308/0x6f0 [ 1235.673093] ? new_sync_read+0x6f0/0x6f0 [ 1235.678294] ? lock_acquire.part.0+0xbc/0x260 [ 1235.683981] ? ksys_write+0xff/0x200 [ 1235.688793] vfs_write+0x62a/0x980 [ 1235.693408] ksys_write+0xff/0x200 [ 1235.698021] ? __ia32_sys_read+0xc0/0xc0 [ 1235.703221] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.708813] ? rcu_is_watching+0x15/0xd0 [ 1235.714013] do_syscall_64+0xf3/0x5e0 [ 1235.718913] ? exc_page_fault+0x9e/0xb0 [ 1235.724016] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1235.730485] RIP: 0033:0x7f11edf0fc5e [ 1235.735295] RSP: 002b:00007ffcceaefbe0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1235.744584] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f11edf0fc5e [ 1235.753395] RDX: 0000000000000001 RSI: 0000560033a5f110 RDI: 0000000000000007 [ 1235.762205] RBP: 00007ffcceaefbf0 R08: 0000000000000000 R09: 0000000000000000 [ 1235.771013] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f11eddaae80 [ 1235.779822] R13: 0000560033a5f110 R14: 0000000000000007 R15: 00007f11ee610b98 [ 1235.788638] [ 1235.791907] INFO: task python3:5078 is blocked on a mutex likely owned by task python3:3145. [ 1235.802182] task:python3 state:D stack:25312 pid:3145 tgid:3145 ppid:3144 task_flags:0x400100 flags:0x00080802 [ 1235.815374] Call Trace: [ 1235.818913] [ 1235.822061] __schedule+0xad9/0x1490 [ 1235.826874] ? io_schedule_timeout+0x130/0x130 [ 1235.832659] ? find_held_lock+0x2b/0x80 [ 1235.837764] ? __try_to_del_timer_sync+0x154/0x1f0 [ 1235.843943] schedule+0xd4/0x260 [ 1235.848361] schedule_timeout+0x103/0x260 [ 1235.853649] ? rcu_is_watching+0x15/0xd0 [ 1235.858846] ? hrtimer_nanosleep_restart+0x180/0x180 [ 1235.865218] ? hrtimers_cpu_dying+0x250/0x250 [ 1235.870911] ? ww_mutex_lock+0x160/0x160 [ 1235.876110] msleep+0x1b/0x30 [ 1235.880237] napi_set_threaded+0x376/0x410 [ 1235.885626] ? napi_resume_irqs+0x1a0/0x1a0 [ 1235.891123] netif_set_threaded+0x254/0x400 [ 1235.896619] ? sysfs_file_ops+0x140/0x140 [ 1235.901914] threaded_store+0x1bb/0x220 [ 1235.907019] ? tx_maxrate_store+0x4e0/0x4e0 [ 1235.912510] ? sysfs_file_ops+0x140/0x140 [ 1235.917807] ? sysfs_file_kobj+0xbd/0x1e0 [ 1235.923103] ? sysfs_kf_write+0x65/0x160 [ 1235.928304] kernfs_fop_write_iter+0x335/0x510 [ 1235.934088] ? kernfs_ops+0xf0/0xf0 [ 1235.938798] new_sync_write+0x308/0x6f0 [ 1235.943900] ? new_sync_read+0x6f0/0x6f0 [ 1235.949102] ? lock_acquire.part.0+0xbc/0x260 [ 1235.954788] ? ksys_write+0xff/0x200 [ 1235.959598] vfs_write+0x62a/0x980 [ 1235.964213] ksys_write+0xff/0x200 [ 1235.968826] ? __ia32_sys_read+0xc0/0xc0 [ 1235.974023] ? kernfs_fop_llseek+0x160/0x1f0 [ 1235.979614] ? rcu_is_watching+0x15/0xd0 [ 1235.984817] do_syscall_64+0xf3/0x5e0 [ 1235.989726] ? exc_page_fault+0x9e/0xb0 [ 1235.994828] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1236.001296] RIP: 0033:0x7ff2f3035c5e [ 1236.006095] RSP: 002b:00007ffc1e5b8ab0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1236.015396] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff2f3035c5e [ 1236.024205] RDX: 0000000000000001 RSI: 0000563380564110 RDI: 0000000000000007 [ 1236.033012] RBP: 00007ffc1e5b8ac0 R08: 0000000000000000 R09: 0000000000000000 [ 1236.041821] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff2f2ed0e80 [ 1236.050630] R13: 0000563380564110 R14: 0000000000000007 R15: 00007ff2f2e5e480 [ 1236.059447] [ 1236.062710] \x0aShowing all locks held in the system: [ 1236.071250] 1 lock held by khungtaskd/185: [ 1236.076644] #0: ffffffffadace020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1236.088125] 4 locks held by python3/3145: [ 1236.093422] #0: ff1100012eb243f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1236.103215] #1: ff11000149410888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1236.114179] #2: ff11000122ba70f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.125336] #3: ff11000122b59cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.135420] 4 locks held by python3/5078: [ 1236.140713] #0: ff1100012eb243f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1236.150502] #1: ff110001a4e7e288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1236.161465] #2: ff11000122ba70f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1236.172622] #3: ff11000122b59cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1236.182707] [ 1236.185175] =============================================\x0a [ 1358.397842] INFO: task python3:5078 blocked for more than 491 seconds. [ 1358.405990] Not tainted 7.0.0-rc4-zviy-g253dc35e0749 #1 [ 1358.413241] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 1358.422833] task:python3 state:D stack:25312 pid:5078 tgid:5078 ppid:5077 task_flags:0x400100 flags:0x00080002 [ 1358.436041] Call Trace: [ 1358.439582] [ 1358.442736] __schedule+0xad9/0x1490 [ 1358.447554] ? io_schedule_timeout+0x130/0x130 [ 1358.453341] ? find_held_lock+0x2b/0x80 [ 1358.458449] ? __lock_release.isra.0+0x59/0x170 [ 1358.464333] schedule+0xd4/0x260 [ 1358.468750] schedule_preempt_disabled+0x18/0x30 [ 1358.474731] __mutex_lock+0xd76/0x1f20 [ 1358.479740] ? threaded_store+0x132/0x220 [ 1358.485038] ? ww_mutex_lock+0x160/0x160 [ 1358.490237] ? selinux_inode_free_security+0x310/0x310 [ 1358.496806] ? cap_capable+0x1ed/0x3f0 [ 1358.501814] ? sysfs_file_ops+0x140/0x140 [ 1358.507114] ? threaded_store+0x132/0x220 [ 1358.512409] threaded_store+0x132/0x220 [ 1358.517514] ? tx_maxrate_store+0x4e0/0x4e0 [ 1358.523008] ? sysfs_file_ops+0x140/0x140 [ 1358.528305] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.533601] ? sysfs_kf_write+0x65/0x160 [ 1358.538802] kernfs_fop_write_iter+0x335/0x510 [ 1358.544589] ? kernfs_ops+0xf0/0xf0 [ 1358.549301] new_sync_write+0x308/0x6f0 [ 1358.554408] ? new_sync_read+0x6f0/0x6f0 [ 1358.559609] ? lock_acquire.part.0+0xbc/0x260 [ 1358.565295] ? ksys_write+0xff/0x200 [ 1358.570106] vfs_write+0x62a/0x980 [ 1358.574721] ksys_write+0xff/0x200 [ 1358.579334] ? __ia32_sys_read+0xc0/0xc0 [ 1358.584534] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.590126] ? rcu_is_watching+0x15/0xd0 [ 1358.595326] do_syscall_64+0xf3/0x5e0 [ 1358.600234] ? exc_page_fault+0x9e/0xb0 [ 1358.605337] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.611806] RIP: 0033:0x7f11edf0fc5e [ 1358.616617] RSP: 002b:00007ffcceaefbe0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.625917] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f11edf0fc5e [ 1358.634726] RDX: 0000000000000001 RSI: 0000560033a5f110 RDI: 0000000000000007 [ 1358.643535] RBP: 00007ffcceaefbf0 R08: 0000000000000000 R09: 0000000000000000 [ 1358.652343] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f11eddaae80 [ 1358.661151] R13: 0000560033a5f110 R14: 0000000000000007 R15: 00007f11ee610b98 [ 1358.669966] [ 1358.673231] INFO: task python3:5078 is blocked on a mutex likely owned by task python3:3145. [ 1358.683506] task:python3 state:D stack:25312 pid:3145 tgid:3145 ppid:3144 task_flags:0x400100 flags:0x00080802 [ 1358.696707] Call Trace: [ 1358.700245] [ 1358.703396] __schedule+0xad9/0x1490 [ 1358.708207] ? io_schedule_timeout+0x130/0x130 [ 1358.713991] ? find_held_lock+0x2b/0x80 [ 1358.719095] ? __try_to_del_timer_sync+0x154/0x1f0 [ 1358.725274] schedule+0xd4/0x260 [ 1358.729693] schedule_timeout+0x103/0x260 [ 1358.734991] ? rcu_is_watching+0x15/0xd0 [ 1358.740192] ? hrtimer_nanosleep_restart+0x180/0x180 [ 1358.746562] ? hrtimers_cpu_dying+0x250/0x250 [ 1358.752244] ? ww_mutex_lock+0x160/0x160 [ 1358.757443] msleep+0x1b/0x30 [ 1358.761571] napi_set_threaded+0x376/0x410 [ 1358.766968] ? napi_resume_irqs+0x1a0/0x1a0 [ 1358.772465] netif_set_threaded+0x254/0x400 [ 1358.777960] ? sysfs_file_ops+0x140/0x140 [ 1358.783255] threaded_store+0x1bb/0x220 [ 1358.788357] ? tx_maxrate_store+0x4e0/0x4e0 [ 1358.793848] ? sysfs_file_ops+0x140/0x140 [ 1358.799144] ? sysfs_file_kobj+0xbd/0x1e0 [ 1358.804444] ? sysfs_kf_write+0x65/0x160 [ 1358.809643] kernfs_fop_write_iter+0x335/0x510 [ 1358.815427] ? kernfs_ops+0xf0/0xf0 [ 1358.820140] new_sync_write+0x308/0x6f0 [ 1358.825236] ? new_sync_read+0x6f0/0x6f0 [ 1358.830437] ? lock_acquire.part.0+0xbc/0x260 [ 1358.836124] ? ksys_write+0xff/0x200 [ 1358.840938] vfs_write+0x62a/0x980 [ 1358.845552] ksys_write+0xff/0x200 [ 1358.850168] ? __ia32_sys_read+0xc0/0xc0 [ 1358.855369] ? kernfs_fop_llseek+0x160/0x1f0 [ 1358.860961] ? rcu_is_watching+0x15/0xd0 [ 1358.866163] do_syscall_64+0xf3/0x5e0 [ 1358.871070] ? exc_page_fault+0x9e/0xb0 [ 1358.876172] entry_SYSCALL_64_after_hwframe+0x4b/0x53 [ 1358.882640] RIP: 0033:0x7ff2f3035c5e [ 1358.887449] RSP: 002b:00007ffc1e5b8ab0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 1358.896745] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff2f3035c5e [ 1358.905546] RDX: 0000000000000001 RSI: 0000563380564110 RDI: 0000000000000007 [ 1358.914353] RBP: 00007ffc1e5b8ac0 R08: 0000000000000000 R09: 0000000000000000 [ 1358.923163] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff2f2ed0e80 [ 1358.931969] R13: 0000563380564110 R14: 0000000000000007 R15: 00007ff2f2e5e480 [ 1358.940782] [ 1358.944053] \x0aShowing all locks held in the system: [ 1358.952590] 1 lock held by khungtaskd/185: [ 1358.957985] #0: ffffffffadace020 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30 [ 1358.969464] 4 locks held by python3/3145: [ 1358.974759] #0: ff1100012eb243f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1358.984552] #1: ff11000149410888 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1358.995516] #2: ff11000122ba70f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1359.006673] #3: ff11000122b59cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1359.016768] 4 locks held by python3/5078: [ 1359.022065] #0: ff1100012eb243f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200 [ 1359.031858] #1: ff110001a4e7e288 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510 [ 1359.042822] #2: ff11000122ba70f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510 [ 1359.053982] #3: ff11000122b59cb8 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220 [ 1359.064076] [ 1359.066544] =============================================\x0a