[ 765.128631] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 765.131065] nipa-hw-worker: [5/40] Retrying drivers/net:napi_threaded.py
[ 765.296133] kselftest: Running tests in drivers/net
[ 809.522686] kworker/u130:1 (206) used greatest stack depth: 21896 bytes left
[ 989.748347] INFO: task python3:5210 blocked for more than 122 seconds.
[ 989.756572] Not tainted 7.0.0-rc5-yrfy-gbd020e9f07b9 #1
[ 989.763823] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.773413] task:python3 state:D stack:25312 pid:5210 tgid:5210 ppid:5209 task_flags:0x400100 flags:0x00080000
[ 989.786620] Call Trace:
[ 989.790159]
[ 989.793311] __schedule+0xad9/0x1490
[ 989.798127] ? io_schedule_timeout+0x130/0x130
[ 989.803913] ? find_held_lock+0x2b/0x80
[ 989.809020] ? __lock_release.isra.0+0x59/0x170
[ 989.814906] schedule+0xd4/0x260
[ 989.819326] schedule_preempt_disabled+0x18/0x30
[ 989.825296] __mutex_lock+0xd76/0x1f20
[ 989.830303] ? threaded_store+0x132/0x220
[ 989.835604] ? ww_mutex_lock+0x160/0x160
[ 989.840806] ? selinux_inode_free_security+0x310/0x310
[ 989.847375] ? cap_capable+0x1ed/0x3f0
[ 989.852381] ? sysfs_file_ops+0x140/0x140
[ 989.857679] ? threaded_store+0x132/0x220
[ 989.862977] threaded_store+0x132/0x220
[ 989.868072] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.873568] ? sysfs_file_ops+0x140/0x140
[ 989.878863] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.884161] ? sysfs_kf_write+0x65/0x160
[ 989.889362] kernfs_fop_write_iter+0x335/0x510
[ 989.895147] ? kernfs_ops+0xf0/0xf0
[ 989.899851] new_sync_write+0x308/0x6f0
[ 989.904947] ? new_sync_read+0x6f0/0x6f0
[ 989.910147] ? lock_acquire.part.0+0xbc/0x260
[ 989.915824] ? ksys_write+0xff/0x200
[ 989.920637] vfs_write+0x62a/0x980
[ 989.925255] ksys_write+0xff/0x200
[ 989.929868] ? __ia32_sys_read+0xc0/0xc0
[ 989.935067] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.940658] ? rcu_is_watching+0x15/0xd0
[ 989.945858] do_syscall_64+0xf3/0x5e0
[ 989.950757] ? exc_page_fault+0x9e/0xb0
[ 989.955858] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.962327] RIP: 0033:0x7f19ce971c5e
[ 989.967137] RSP: 002b:00007ffe2c1d7450 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.976433] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f19ce971c5e
[ 989.985242] RDX: 0000000000000001 RSI: 000055c819e4c760 RDI: 0000000000000007
[ 989.994050] RBP: 00007ffe2c1d7460 R08: 0000000000000000 R09: 0000000000000000
[ 990.002847] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f19ce80ce80
[ 990.011656] R13: 000055c819e4c760 R14: 0000000000000007 R15: 00007f19cf072b98
[ 990.020470]
[ 990.023742] INFO: task python3:5210 is blocked on a mutex likely owned by task python3:3235.
[ 990.034016] task:python3 state:D stack:25312 pid:3235 tgid:3235 ppid:3234 task_flags:0x400100 flags:0x00080802
[ 990.047219] Call Trace:
[ 990.050757]
[ 990.053907] __schedule+0xad9/0x1490
[ 990.058719] ? io_schedule_timeout+0x130/0x130
[ 990.064502] ? find_held_lock+0x2b/0x80
[ 990.069606] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.075784] schedule+0xd4/0x260
[ 990.080202] schedule_timeout+0x103/0x260
[ 990.085497] ? rcu_is_watching+0x15/0xd0
[ 990.090697] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.097059] ? hrtimers_cpu_dying+0x250/0x250
[ 990.102751] ? ww_mutex_lock+0x160/0x160
[ 990.107950] msleep+0x1b/0x30
[ 990.112075] napi_set_threaded+0x376/0x410
[ 990.117472] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.122969] netif_set_threaded+0x254/0x400
[ 990.128462] ? sysfs_file_ops+0x140/0x140
[ 990.133761] threaded_store+0x1bb/0x220
[ 990.138863] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.144358] ? sysfs_file_ops+0x140/0x140
[ 990.149654] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.154953] ? sysfs_kf_write+0x65/0x160
[ 990.160153] kernfs_fop_write_iter+0x335/0x510
[ 990.165930] ? kernfs_ops+0xf0/0xf0
[ 990.170632] new_sync_write+0x308/0x6f0
[ 990.175737] ? new_sync_read+0x6f0/0x6f0
[ 990.180937] ? lock_acquire.part.0+0xbc/0x260
[ 990.186626] ? ksys_write+0xff/0x200
[ 990.191440] vfs_write+0x62a/0x980
[ 990.196058] ksys_write+0xff/0x200
[ 990.200675] ? __ia32_sys_read+0xc0/0xc0
[ 990.205875] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.211465] ? rcu_is_watching+0x15/0xd0
[ 990.216667] do_syscall_64+0xf3/0x5e0
[ 990.221575] ? irq_exit_rcu+0x39/0xf0
[ 990.226482] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.232949] RIP: 0033:0x7fd432cebc5e
[ 990.237748] RSP: 002b:00007ffdf213ba20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.247047] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd432cebc5e
[ 990.255855] RDX: 0000000000000001 RSI: 00005591f6e0b760 RDI: 0000000000000007
[ 990.264654] RBP: 00007ffdf213ba30 R08: 0000000000000000 R09: 0000000000000000
[ 990.273463] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd432b86e80
[ 990.282269] R13: 00005591f6e0b760 R14: 0000000000000007 R15: 00007fd432b16480
[ 990.291084]
[ 990.294345] \x0aShowing all locks held in the system:
[ 990.302882] 1 lock held by khungtaskd/185:
[ 990.308277] #0: ffffffffafcce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.319761] 4 locks held by python3/3235:
[ 990.325056] #0: ff1100012de493f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.334847] #1: ff1100013d8a2c88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.345810] #2: ff110001211430f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.356967] #3: ff110001211b9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.367064] 4 locks held by python3/5210:
[ 990.372361] #0: ff1100012de493f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.382153] #1: ff1100017fceac88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.393117] #2: ff110001211430f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.404277] #3: ff110001211b9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.414368]
[ 990.416826] =============================================\x0a
[ 1112.630539] INFO: task python3:5210 blocked for more than 245 seconds.
[ 1112.638678] Not tainted 7.0.0-rc5-yrfy-gbd020e9f07b9 #1
[ 1112.645927] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.655513] task:python3 state:D stack:25312 pid:5210 tgid:5210 ppid:5209 task_flags:0x400100 flags:0x00080000
[ 1112.668718] Call Trace:
[ 1112.672256]
[ 1112.675403] __schedule+0xad9/0x1490
[ 1112.680210] ? io_schedule_timeout+0x130/0x130
[ 1112.685995] ? find_held_lock+0x2b/0x80
[ 1112.691101] ? __lock_release.isra.0+0x59/0x170
[ 1112.696985] schedule+0xd4/0x260
[ 1112.701399] schedule_preempt_disabled+0x18/0x30
[ 1112.707379] __mutex_lock+0xd76/0x1f20
[ 1112.712378] ? threaded_store+0x132/0x220
[ 1112.717674] ? ww_mutex_lock+0x160/0x160
[ 1112.722872] ? selinux_inode_free_security+0x310/0x310
[ 1112.729433] ? cap_capable+0x1ed/0x3f0
[ 1112.734439] ? sysfs_file_ops+0x140/0x140
[ 1112.739741] ? threaded_store+0x132/0x220
[ 1112.745036] threaded_store+0x132/0x220
[ 1112.750137] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.755627] ? sysfs_file_ops+0x140/0x140
[ 1112.760924] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.766219] ? sysfs_kf_write+0x65/0x160
[ 1112.771419] kernfs_fop_write_iter+0x335/0x510
[ 1112.777204] ? kernfs_ops+0xf0/0xf0
[ 1112.781916] new_sync_write+0x308/0x6f0
[ 1112.787019] ? new_sync_read+0x6f0/0x6f0
[ 1112.792220] ? lock_acquire.part.0+0xbc/0x260
[ 1112.797905] ? ksys_write+0xff/0x200
[ 1112.802715] vfs_write+0x62a/0x980
[ 1112.807320] ksys_write+0xff/0x200
[ 1112.811933] ? __ia32_sys_read+0xc0/0xc0
[ 1112.817129] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.822716] ? rcu_is_watching+0x15/0xd0
[ 1112.827917] do_syscall_64+0xf3/0x5e0
[ 1112.832813] ? exc_page_fault+0x9e/0xb0
[ 1112.837914] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.844375] RIP: 0033:0x7f19ce971c5e
[ 1112.849185] RSP: 002b:00007ffe2c1d7450 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.858484] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f19ce971c5e
[ 1112.867289] RDX: 0000000000000001 RSI: 000055c819e4c760 RDI: 0000000000000007
[ 1112.876097] RBP: 00007ffe2c1d7460 R08: 0000000000000000 R09: 0000000000000000
[ 1112.884896] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f19ce80ce80
[ 1112.893696] R13: 000055c819e4c760 R14: 0000000000000007 R15: 00007f19cf072b98
[ 1112.902512]
[ 1112.905792] INFO: task python3:5210 is blocked on a mutex likely owned by task python3:3235.
[ 1112.916065] task:python3 state:D stack:25312 pid:3235 tgid:3235 ppid:3234 task_flags:0x400100 flags:0x00080802
[ 1112.929268] Call Trace:
[ 1112.932805]
[ 1112.935952] __schedule+0xad9/0x1490
[ 1112.940766] ? io_schedule_timeout+0x130/0x130
[ 1112.946546] ? find_held_lock+0x2b/0x80
[ 1112.951647] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.957828] schedule+0xd4/0x260
[ 1112.962243] schedule_timeout+0x103/0x260
[ 1112.967541] ? rcu_is_watching+0x15/0xd0
[ 1112.972737] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.979109] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.984798] ? ww_mutex_lock+0x160/0x160
[ 1112.989998] msleep+0x1b/0x30
[ 1112.994124] napi_set_threaded+0x376/0x410
[ 1112.999519] ? napi_resume_irqs+0x1a0/0x1a0
[ 1113.005012] netif_set_threaded+0x254/0x400
[ 1113.010503] ? sysfs_file_ops+0x140/0x140
[ 1113.015798] threaded_store+0x1bb/0x220
[ 1113.020899] ? tx_maxrate_store+0x4e0/0x4e0
[ 1113.026389] ? sysfs_file_ops+0x140/0x140
[ 1113.031683] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.036980] ? sysfs_kf_write+0x65/0x160
[ 1113.042177] kernfs_fop_write_iter+0x335/0x510
[ 1113.047963] ? kernfs_ops+0xf0/0xf0
[ 1113.052672] new_sync_write+0x308/0x6f0
[ 1113.057774] ? new_sync_read+0x6f0/0x6f0
[ 1113.062972] ? lock_acquire.part.0+0xbc/0x260
[ 1113.068655] ? ksys_write+0xff/0x200
[ 1113.073466] vfs_write+0x62a/0x980
[ 1113.078082] ksys_write+0xff/0x200
[ 1113.082696] ? __ia32_sys_read+0xc0/0xc0
[ 1113.087894] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.093482] ? rcu_is_watching+0x15/0xd0
[ 1113.098681] do_syscall_64+0xf3/0x5e0
[ 1113.103586] ? irq_exit_rcu+0x39/0xf0
[ 1113.108491] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.114956] RIP: 0033:0x7fd432cebc5e
[ 1113.119764] RSP: 002b:00007ffdf213ba20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.129061] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd432cebc5e
[ 1113.137857] RDX: 0000000000000001 RSI: 00005591f6e0b760 RDI: 0000000000000007
[ 1113.146663] RBP: 00007ffdf213ba30 R08: 0000000000000000 R09: 0000000000000000
[ 1113.155470] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd432b86e80
[ 1113.164266] R13: 00005591f6e0b760 R14: 0000000000000007 R15: 00007fd432b16480
[ 1113.173079]
[ 1113.176340] \x0aShowing all locks held in the system:
[ 1113.184876] 1 lock held by khungtaskd/185:
[ 1113.190268] #0: ffffffffafcce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.201752] 4 locks held by python3/3235:
[ 1113.207036] #0: ff1100012de493f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.216825] #1: ff1100013d8a2c88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.227787] #2: ff110001211430f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.238942] #3: ff110001211b9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.249039] 4 locks held by python3/5210:
[ 1113.254333] #0: ff1100012de493f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.264124] #1: ff1100017fceac88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.275086] #2: ff110001211430f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.286243] #3: ff110001211b9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.296323]
[ 1113.298789] =============================================\x0a
[ 1235.510731] INFO: task python3:5210 blocked for more than 368 seconds.
[ 1235.518881] Not tainted 7.0.0-rc5-yrfy-gbd020e9f07b9 #1
[ 1235.526131] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.535722] task:python3 state:D stack:25312 pid:5210 tgid:5210 ppid:5209 task_flags:0x400100 flags:0x00080002
[ 1235.548930] Call Trace:
[ 1235.552471]
[ 1235.555624] __schedule+0xad9/0x1490
[ 1235.560442] ? io_schedule_timeout+0x130/0x130
[ 1235.566228] ? find_held_lock+0x2b/0x80
[ 1235.571336] ? __lock_release.isra.0+0x59/0x170
[ 1235.577221] schedule+0xd4/0x260
[ 1235.581642] schedule_preempt_disabled+0x18/0x30
[ 1235.587623] __mutex_lock+0xd76/0x1f20
[ 1235.592629] ? threaded_store+0x132/0x220
[ 1235.597927] ? ww_mutex_lock+0x160/0x160
[ 1235.603127] ? selinux_inode_free_security+0x310/0x310
[ 1235.609700] ? cap_capable+0x1ed/0x3f0
[ 1235.614710] ? sysfs_file_ops+0x140/0x140
[ 1235.620010] ? threaded_store+0x132/0x220
[ 1235.625305] threaded_store+0x132/0x220
[ 1235.630407] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.635902] ? sysfs_file_ops+0x140/0x140
[ 1235.641199] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.646486] ? sysfs_kf_write+0x65/0x160
[ 1235.651687] kernfs_fop_write_iter+0x335/0x510
[ 1235.657475] ? kernfs_ops+0xf0/0xf0
[ 1235.662188] new_sync_write+0x308/0x6f0
[ 1235.667295] ? new_sync_read+0x6f0/0x6f0
[ 1235.672495] ? lock_acquire.part.0+0xbc/0x260
[ 1235.678172] ? ksys_write+0xff/0x200
[ 1235.682987] vfs_write+0x62a/0x980
[ 1235.687601] ksys_write+0xff/0x200
[ 1235.692215] ? __ia32_sys_read+0xc0/0xc0
[ 1235.697415] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.703007] ? rcu_is_watching+0x15/0xd0
[ 1235.708212] do_syscall_64+0xf3/0x5e0
[ 1235.713120] ? exc_page_fault+0x9e/0xb0
[ 1235.718221] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.724690] RIP: 0033:0x7f19ce971c5e
[ 1235.729501] RSP: 002b:00007ffe2c1d7450 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.738801] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f19ce971c5e
[ 1235.747610] RDX: 0000000000000001 RSI: 000055c819e4c760 RDI: 0000000000000007
[ 1235.756418] RBP: 00007ffe2c1d7460 R08: 0000000000000000 R09: 0000000000000000
[ 1235.765226] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f19ce80ce80
[ 1235.774026] R13: 000055c819e4c760 R14: 0000000000000007 R15: 00007f19cf072b98
[ 1235.782840]
[ 1235.786114] INFO: task python3:5210 is blocked on a mutex likely owned by task python3:3235.
[ 1235.796387] task:python3 state:D stack:25312 pid:3235 tgid:3235 ppid:3234 task_flags:0x400100 flags:0x00080802
[ 1235.809589] Call Trace:
[ 1235.813128]
[ 1235.816280] __schedule+0xad9/0x1490
[ 1235.821092] ? io_schedule_timeout+0x130/0x130
[ 1235.826876] ? find_held_lock+0x2b/0x80
[ 1235.831980] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.838158] schedule+0xd4/0x260
[ 1235.842577] schedule_timeout+0x103/0x260
[ 1235.847874] ? rcu_is_watching+0x15/0xd0
[ 1235.853074] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.859444] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.865135] ? ww_mutex_lock+0x160/0x160
[ 1235.870336] msleep+0x1b/0x30
[ 1235.874452] napi_set_threaded+0x376/0x410
[ 1235.879848] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.885344] netif_set_threaded+0x254/0x400
[ 1235.890837] ? sysfs_file_ops+0x140/0x140
[ 1235.896135] threaded_store+0x1bb/0x220
[ 1235.901237] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.906722] ? sysfs_file_ops+0x140/0x140
[ 1235.912018] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.917317] ? sysfs_kf_write+0x65/0x160
[ 1235.922518] kernfs_fop_write_iter+0x335/0x510
[ 1235.928303] ? kernfs_ops+0xf0/0xf0
[ 1235.933015] new_sync_write+0x308/0x6f0
[ 1235.938108] ? new_sync_read+0x6f0/0x6f0
[ 1235.943308] ? lock_acquire.part.0+0xbc/0x260
[ 1235.948995] ? ksys_write+0xff/0x200
[ 1235.953806] vfs_write+0x62a/0x980
[ 1235.958421] ksys_write+0xff/0x200
[ 1235.963028] ? __ia32_sys_read+0xc0/0xc0
[ 1235.968231] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.973821] ? rcu_is_watching+0x15/0xd0
[ 1235.979015] do_syscall_64+0xf3/0x5e0
[ 1235.983923] ? irq_exit_rcu+0x39/0xf0
[ 1235.988831] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.995299] RIP: 0033:0x7fd432cebc5e
[ 1236.000108] RSP: 002b:00007ffdf213ba20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1236.009407] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd432cebc5e
[ 1236.018207] RDX: 0000000000000001 RSI: 00005591f6e0b760 RDI: 0000000000000007
[ 1236.027017] RBP: 00007ffdf213ba30 R08: 0000000000000000 R09: 0000000000000000
[ 1236.035825] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd432b86e80
[ 1236.044633] R13: 00005591f6e0b760 R14: 0000000000000007 R15: 00007fd432b16480
[ 1236.053437]
[ 1236.056705] \x0aShowing all locks held in the system:
[ 1236.065243] 1 lock held by khungtaskd/185:
[ 1236.070639] #0: ffffffffafcce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.082120] 4 locks held by python3/3235:
[ 1236.087416] #0: ff1100012de493f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.097209] #1: ff1100013d8a2c88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.108173] #2: ff110001211430f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.119329] #3: ff110001211b9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.129425] 4 locks held by python3/5210:
[ 1236.134723] #0: ff1100012de493f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.144515] #1: ff1100017fceac88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.155480] #2: ff110001211430f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.166640] #3: ff110001211b9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.176733]
[ 1236.179199] =============================================\x0a
[ 1358.392926] INFO: task python3:5210 blocked for more than 491 seconds.
[ 1358.401071] Not tainted 7.0.0-rc5-yrfy-gbd020e9f07b9 #1
[ 1358.408319] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.417909] task:python3 state:D stack:25312 pid:5210 tgid:5210 ppid:5209 task_flags:0x400100 flags:0x00080002
[ 1358.431116] Call Trace:
[ 1358.434655]
[ 1358.437802] __schedule+0xad9/0x1490
[ 1358.442618] ? io_schedule_timeout+0x130/0x130
[ 1358.448404] ? find_held_lock+0x2b/0x80
[ 1358.453510] ? __lock_release.isra.0+0x59/0x170
[ 1358.459391] schedule+0xd4/0x260
[ 1358.463807] schedule_preempt_disabled+0x18/0x30
[ 1358.469787] __mutex_lock+0xd76/0x1f20
[ 1358.474793] ? threaded_store+0x132/0x220
[ 1358.480094] ? ww_mutex_lock+0x160/0x160
[ 1358.485292] ? selinux_inode_free_security+0x310/0x310
[ 1358.491850] ? cap_capable+0x1ed/0x3f0
[ 1358.496848] ? sysfs_file_ops+0x140/0x140
[ 1358.502146] ? threaded_store+0x132/0x220
[ 1358.507440] threaded_store+0x132/0x220
[ 1358.512545] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.518037] ? sysfs_file_ops+0x140/0x140
[ 1358.523324] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.528617] ? sysfs_kf_write+0x65/0x160
[ 1358.533818] kernfs_fop_write_iter+0x335/0x510
[ 1358.539603] ? kernfs_ops+0xf0/0xf0
[ 1358.544315] new_sync_write+0x308/0x6f0
[ 1358.549418] ? new_sync_read+0x6f0/0x6f0
[ 1358.554618] ? lock_acquire.part.0+0xbc/0x260
[ 1358.560294] ? ksys_write+0xff/0x200
[ 1358.565105] vfs_write+0x62a/0x980
[ 1358.569718] ksys_write+0xff/0x200
[ 1358.574332] ? __ia32_sys_read+0xc0/0xc0
[ 1358.579527] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.585113] ? rcu_is_watching+0x15/0xd0
[ 1358.590312] do_syscall_64+0xf3/0x5e0
[ 1358.595217] ? exc_page_fault+0x9e/0xb0
[ 1358.600319] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.606775] RIP: 0033:0x7f19ce971c5e
[ 1358.611586] RSP: 002b:00007ffe2c1d7450 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.620882] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f19ce971c5e
[ 1358.629689] RDX: 0000000000000001 RSI: 000055c819e4c760 RDI: 0000000000000007
[ 1358.638493] RBP: 00007ffe2c1d7460 R08: 0000000000000000 R09: 0000000000000000
[ 1358.647303] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f19ce80ce80
[ 1358.656111] R13: 000055c819e4c760 R14: 0000000000000007 R15: 00007f19cf072b98
[ 1358.664926]
[ 1358.668191] INFO: task python3:5210 is blocked on a mutex likely owned by task python3:3235.
[ 1358.678461] task:python3 state:D stack:25312 pid:3235 tgid:3235 ppid:3234 task_flags:0x400100 flags:0x00080802
[ 1358.691663] Call Trace:
[ 1358.695201]
[ 1358.698349] __schedule+0xad9/0x1490
[ 1358.703161] ? io_schedule_timeout+0x130/0x130
[ 1358.708945] ? find_held_lock+0x2b/0x80
[ 1358.714046] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.720224] schedule+0xd4/0x260
[ 1358.724643] schedule_timeout+0x103/0x260
[ 1358.729939] ? rcu_is_watching+0x15/0xd0
[ 1358.735135] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.741507] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.747195] ? ww_mutex_lock+0x160/0x160
[ 1358.752391] msleep+0x1b/0x30
[ 1358.756518] napi_set_threaded+0x376/0x410
[ 1358.761912] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.767405] netif_set_threaded+0x254/0x400
[ 1358.772899] ? sysfs_file_ops+0x140/0x140
[ 1358.778194] threaded_store+0x1bb/0x220
[ 1358.783297] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.788787] ? sysfs_file_ops+0x140/0x140
[ 1358.794080] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.799375] ? sysfs_kf_write+0x65/0x160
[ 1358.804577] kernfs_fop_write_iter+0x335/0x510
[ 1358.810362] ? kernfs_ops+0xf0/0xf0
[ 1358.815062] new_sync_write+0x308/0x6f0
[ 1358.820164] ? new_sync_read+0x6f0/0x6f0
[ 1358.825357] ? lock_acquire.part.0+0xbc/0x260
[ 1358.831044] ? ksys_write+0xff/0x200
[ 1358.835856] vfs_write+0x62a/0x980
[ 1358.840471] ksys_write+0xff/0x200
[ 1358.845075] ? __ia32_sys_read+0xc0/0xc0
[ 1358.850273] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.855861] ? rcu_is_watching+0x15/0xd0
[ 1358.861061] do_syscall_64+0xf3/0x5e0
[ 1358.865957] ? irq_exit_rcu+0x39/0xf0
[ 1358.870862] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.877330] RIP: 0033:0x7fd432cebc5e
[ 1358.882135] RSP: 002b:00007ffdf213ba20 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.891432] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd432cebc5e
[ 1358.900237] RDX: 0000000000000001 RSI: 00005591f6e0b760 RDI: 0000000000000007
[ 1358.909043] RBP: 00007ffdf213ba30 R08: 0000000000000000 R09: 0000000000000000
[ 1358.917849] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd432b86e80
[ 1358.926653] R13: 00005591f6e0b760 R14: 0000000000000007 R15: 00007fd432b16480
[ 1358.935467]
[ 1358.938734] \x0aShowing all locks held in the system:
[ 1358.947269] 1 lock held by khungtaskd/185:
[ 1358.952661] #0: ffffffffafcce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.964143] 4 locks held by python3/3235:
[ 1358.969436] #0: ff1100012de493f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.979225] #1: ff1100013d8a2c88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1358.990187] #2: ff110001211430f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.001342] #3: ff110001211b9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.011435] 4 locks held by python3/5210:
[ 1359.016730] #0: ff1100012de493f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.026522] #1: ff1100017fceac88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.037485] #2: ff110001211430f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.048642] #3: ff110001211b9d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.058723]
[ 1359.061189] =============================================\x0a