[ 782.499877] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 782.504476] nipa-hw-worker: [4/39] Retrying drivers/net:napi_threaded.py
[ 782.645957] kselftest: Running tests in drivers/net
[ 989.748709] INFO: task python3:5431 blocked for more than 122 seconds.
[ 989.756932] Not tainted 7.0.0-rc5-xfqj-g0b484fe08e24 #1
[ 989.764183] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.773773] task:python3 state:D stack:25312 pid:5431 tgid:5431 ppid:5430 task_flags:0x400100 flags:0x00080000
[ 989.786979] Call Trace:
[ 989.790515]
[ 989.793663] __schedule+0xad9/0x1490
[ 989.798477] ? io_schedule_timeout+0x130/0x130
[ 989.804253] ? find_held_lock+0x2b/0x80
[ 989.809358] ? __lock_release.isra.0+0x59/0x170
[ 989.815232] schedule+0xd4/0x260
[ 989.819643] schedule_preempt_disabled+0x18/0x30
[ 989.825618] __mutex_lock+0xd76/0x1f20
[ 989.830622] ? threaded_store+0x132/0x220
[ 989.835919] ? ww_mutex_lock+0x160/0x160
[ 989.841118] ? selinux_inode_free_security+0x310/0x310
[ 989.847686] ? cap_capable+0x1ed/0x3f0
[ 989.852692] ? sysfs_file_ops+0x140/0x140
[ 989.857990] ? threaded_store+0x132/0x220
[ 989.863283] threaded_store+0x132/0x220
[ 989.868387] ? tx_maxrate_store+0x4e0/0x4e0
[ 989.873878] ? sysfs_file_ops+0x140/0x140
[ 989.879175] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.884469] ? sysfs_kf_write+0x65/0x160
[ 989.889669] kernfs_fop_write_iter+0x335/0x510
[ 989.895445] ? kernfs_ops+0xf0/0xf0
[ 989.900157] new_sync_write+0x308/0x6f0
[ 989.905259] ? new_sync_read+0x6f0/0x6f0
[ 989.910459] ? lock_acquire.part.0+0xbc/0x260
[ 989.916144] ? ksys_write+0xff/0x200
[ 989.920954] vfs_write+0x62a/0x980
[ 989.925568] ksys_write+0xff/0x200
[ 989.930180] ? __ia32_sys_read+0xc0/0xc0
[ 989.935376] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.940963] ? rcu_is_watching+0x15/0xd0
[ 989.946163] do_syscall_64+0xf3/0x5e0
[ 989.951058] ? irq_exit_rcu+0x39/0xf0
[ 989.955965] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.962434] RIP: 0033:0x7f5c78f1bc5e
[ 989.967244] RSP: 002b:00007ffeab010210 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.976533] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5c78f1bc5e
[ 989.985341] RDX: 0000000000000001 RSI: 000055dbb167b760 RDI: 0000000000000007
[ 989.994149] RBP: 00007ffeab010220 R08: 0000000000000000 R09: 0000000000000000
[ 990.002945] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5c78db6e80
[ 990.011752] R13: 000055dbb167b760 R14: 0000000000000007 R15: 00007f5c7961cb98
[ 990.020563]
[ 990.023857] INFO: task python3:5431 is blocked on a mutex likely owned by task python3:3494.
[ 990.034129] task:python3 state:D stack:25312 pid:3494 tgid:3494 ppid:3493 task_flags:0x400100 flags:0x00080802
[ 990.047332] Call Trace:
[ 990.050869]
[ 990.054016] __schedule+0xad9/0x1490
[ 990.058827] ? io_schedule_timeout+0x130/0x130
[ 990.064612] ? find_held_lock+0x2b/0x80
[ 990.069713] ? __try_to_del_timer_sync+0x154/0x1f0
[ 990.075892] schedule+0xd4/0x260
[ 990.080310] schedule_timeout+0x103/0x260
[ 990.085606] ? rcu_is_watching+0x15/0xd0
[ 990.090803] ? hrtimer_nanosleep_restart+0x180/0x180
[ 990.097174] ? hrtimers_cpu_dying+0x250/0x250
[ 990.102861] ? ww_mutex_lock+0x160/0x160
[ 990.108061] msleep+0x1b/0x30
[ 990.112177] napi_set_threaded+0x376/0x410
[ 990.117571] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.123065] netif_set_threaded+0x254/0x400
[ 990.128559] ? sysfs_file_ops+0x140/0x140
[ 990.133853] threaded_store+0x1bb/0x220
[ 990.138953] ? tx_maxrate_store+0x4e0/0x4e0
[ 990.144445] ? sysfs_file_ops+0x140/0x140
[ 990.149741] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.155036] ? sysfs_kf_write+0x65/0x160
[ 990.160237] kernfs_fop_write_iter+0x335/0x510
[ 990.166022] ? kernfs_ops+0xf0/0xf0
[ 990.170731] new_sync_write+0x308/0x6f0
[ 990.175833] ? new_sync_read+0x6f0/0x6f0
[ 990.181030] ? lock_acquire.part.0+0xbc/0x260
[ 990.186716] ? ksys_write+0xff/0x200
[ 990.191528] vfs_write+0x62a/0x980
[ 990.196143] ksys_write+0xff/0x200
[ 990.200756] ? __ia32_sys_read+0xc0/0xc0
[ 990.205954] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.211546] ? rcu_is_watching+0x15/0xd0
[ 990.216745] do_syscall_64+0xf3/0x5e0
[ 990.221650] ? exc_page_fault+0x9e/0xb0
[ 990.226752] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.233216] RIP: 0033:0x7f3a22c32c5e
[ 990.238025] RSP: 002b:00007ffe1792a010 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.247321] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3a22c32c5e
[ 990.256121] RDX: 0000000000000001 RSI: 0000561a5d66f760 RDI: 0000000000000007
[ 990.264924] RBP: 00007ffe1792a020 R08: 0000000000000000 R09: 0000000000000000
[ 990.273730] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f3a22acde80
[ 990.282535] R13: 0000561a5d66f760 R14: 0000000000000007 R15: 00007f3a22a5e480
[ 990.291350]
[ 990.294606] \x0aShowing all locks held in the system:
[ 990.303142] 1 lock held by khungtaskd/185:
[ 990.308533] #0: ffffffff9f6ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.320018] 4 locks held by python3/3494:
[ 990.325310] #0: ff1100012cb6c3f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.335101] #1: ff11000166153e88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.346063] #2: ff1100011fa950f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.357209] #3: ff1100011fa39d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.367305] 4 locks held by python3/5431:
[ 990.372599] #0: ff1100012cb6c3f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 990.382389] #1: ff110001dd8ea088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 990.393341] #2: ff1100011fa950f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.404499] #3: ff1100011fa39d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.414588]
[ 990.417052] =============================================\x0a
[ 1112.629925] INFO: task python3:5431 blocked for more than 245 seconds.
[ 1112.638074] Not tainted 7.0.0-rc5-xfqj-g0b484fe08e24 #1
[ 1112.645327] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.654915] task:python3 state:D stack:25312 pid:5431 tgid:5431 ppid:5430 task_flags:0x400100 flags:0x00080000
[ 1112.668120] Call Trace:
[ 1112.671661]
[ 1112.674810] __schedule+0xad9/0x1490
[ 1112.679625] ? io_schedule_timeout+0x130/0x130
[ 1112.685413] ? find_held_lock+0x2b/0x80
[ 1112.690520] ? __lock_release.isra.0+0x59/0x170
[ 1112.696404] schedule+0xd4/0x260
[ 1112.700822] schedule_preempt_disabled+0x18/0x30
[ 1112.706799] __mutex_lock+0xd76/0x1f20
[ 1112.711805] ? threaded_store+0x132/0x220
[ 1112.717106] ? ww_mutex_lock+0x160/0x160
[ 1112.722306] ? selinux_inode_free_security+0x310/0x310
[ 1112.728877] ? cap_capable+0x1ed/0x3f0
[ 1112.733883] ? sysfs_file_ops+0x140/0x140
[ 1112.739184] ? threaded_store+0x132/0x220
[ 1112.744482] threaded_store+0x132/0x220
[ 1112.749587] ? tx_maxrate_store+0x4e0/0x4e0
[ 1112.755080] ? sysfs_file_ops+0x140/0x140
[ 1112.760378] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.765675] ? sysfs_kf_write+0x65/0x160
[ 1112.770877] kernfs_fop_write_iter+0x335/0x510
[ 1112.776663] ? kernfs_ops+0xf0/0xf0
[ 1112.781377] new_sync_write+0x308/0x6f0
[ 1112.786481] ? new_sync_read+0x6f0/0x6f0
[ 1112.791674] ? lock_acquire.part.0+0xbc/0x260
[ 1112.797360] ? ksys_write+0xff/0x200
[ 1112.802172] vfs_write+0x62a/0x980
[ 1112.806787] ksys_write+0xff/0x200
[ 1112.811402] ? __ia32_sys_read+0xc0/0xc0
[ 1112.816591] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.822184] ? rcu_is_watching+0x15/0xd0
[ 1112.827385] do_syscall_64+0xf3/0x5e0
[ 1112.832291] ? irq_exit_rcu+0x39/0xf0
[ 1112.837201] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.843672] RIP: 0033:0x7f5c78f1bc5e
[ 1112.848483] RSP: 002b:00007ffeab010210 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.857781] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5c78f1bc5e
[ 1112.866588] RDX: 0000000000000001 RSI: 000055dbb167b760 RDI: 0000000000000007
[ 1112.875397] RBP: 00007ffeab010220 R08: 0000000000000000 R09: 0000000000000000
[ 1112.884206] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5c78db6e80
[ 1112.893016] R13: 000055dbb167b760 R14: 0000000000000007 R15: 00007f5c7961cb98
[ 1112.901833]
[ 1112.905111] INFO: task python3:5431 is blocked on a mutex likely owned by task python3:3494.
[ 1112.915386] task:python3 state:D stack:25312 pid:3494 tgid:3494 ppid:3493 task_flags:0x400100 flags:0x00080802
[ 1112.928662] Call Trace:
[ 1112.932201]
[ 1112.935351] __schedule+0xad9/0x1490
[ 1112.940163] ? io_schedule_timeout+0x130/0x130
[ 1112.945946] ? find_held_lock+0x2b/0x80
[ 1112.951052] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1112.957234] schedule+0xd4/0x260
[ 1112.961655] schedule_timeout+0x103/0x260
[ 1112.966952] ? rcu_is_watching+0x15/0xd0
[ 1112.972154] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1112.978524] ? hrtimers_cpu_dying+0x250/0x250
[ 1112.984206] ? ww_mutex_lock+0x160/0x160
[ 1112.989405] msleep+0x1b/0x30
[ 1112.993530] napi_set_threaded+0x376/0x410
[ 1112.998926] ? napi_resume_irqs+0x1a0/0x1a0
[ 1113.004415] netif_set_threaded+0x254/0x400
[ 1113.009907] ? sysfs_file_ops+0x140/0x140
[ 1113.015207] threaded_store+0x1bb/0x220
[ 1113.020310] ? tx_maxrate_store+0x4e0/0x4e0
[ 1113.025808] ? sysfs_file_ops+0x140/0x140
[ 1113.031107] ? sysfs_file_kobj+0xbd/0x1e0
[ 1113.036406] ? sysfs_kf_write+0x65/0x160
[ 1113.041606] kernfs_fop_write_iter+0x335/0x510
[ 1113.047451] ? kernfs_ops+0xf0/0xf0
[ 1113.052163] new_sync_write+0x308/0x6f0
[ 1113.057267] ? new_sync_read+0x6f0/0x6f0
[ 1113.062469] ? lock_acquire.part.0+0xbc/0x260
[ 1113.068156] ? ksys_write+0xff/0x200
[ 1113.072963] vfs_write+0x62a/0x980
[ 1113.077581] ksys_write+0xff/0x200
[ 1113.082197] ? __ia32_sys_read+0xc0/0xc0
[ 1113.087397] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.092990] ? rcu_is_watching+0x15/0xd0
[ 1113.098190] do_syscall_64+0xf3/0x5e0
[ 1113.103096] ? exc_page_fault+0x9e/0xb0
[ 1113.108199] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.114660] RIP: 0033:0x7f3a22c32c5e
[ 1113.119471] RSP: 002b:00007ffe1792a010 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.128767] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3a22c32c5e
[ 1113.137573] RDX: 0000000000000001 RSI: 0000561a5d66f760 RDI: 0000000000000007
[ 1113.146382] RBP: 00007ffe1792a020 R08: 0000000000000000 R09: 0000000000000000
[ 1113.155190] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f3a22acde80
[ 1113.163999] R13: 0000561a5d66f760 R14: 0000000000000007 R15: 00007f3a22a5e480
[ 1113.172812]
[ 1113.176075] \x0aShowing all locks held in the system:
[ 1113.184614] 1 lock held by khungtaskd/185:
[ 1113.190007] #0: ffffffff9f6ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.201493] 4 locks held by python3/3494:
[ 1113.206789] #0: ff1100012cb6c3f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.216583] #1: ff11000166153e88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.227548] #2: ff1100011fa950f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.238775] #3: ff1100011fa39d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.248870] 4 locks held by python3/5431:
[ 1113.254165] #0: ff1100012cb6c3f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1113.263955] #1: ff110001dd8ea088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1113.274916] #2: ff1100011fa950f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.286077] #3: ff1100011fa39d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.296167] 1 lock held by systemd/6443:
[ 1113.301369]
[ 1113.303835] =============================================\x0a
[ 1235.510188] INFO: task python3:5431 blocked for more than 368 seconds.
[ 1235.518339] Not tainted 7.0.0-rc5-xfqj-g0b484fe08e24 #1
[ 1235.525589] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.535177] task:python3 state:D stack:25312 pid:5431 tgid:5431 ppid:5430 task_flags:0x400100 flags:0x00080002
[ 1235.548382] Call Trace:
[ 1235.551924]
[ 1235.555075] __schedule+0xad9/0x1490
[ 1235.559890] ? io_schedule_timeout+0x130/0x130
[ 1235.565678] ? find_held_lock+0x2b/0x80
[ 1235.570784] ? __lock_release.isra.0+0x59/0x170
[ 1235.576671] schedule+0xd4/0x260
[ 1235.581089] schedule_preempt_disabled+0x18/0x30
[ 1235.587069] __mutex_lock+0xd76/0x1f20
[ 1235.592075] ? threaded_store+0x132/0x220
[ 1235.597374] ? ww_mutex_lock+0x160/0x160
[ 1235.602575] ? selinux_inode_free_security+0x310/0x310
[ 1235.609147] ? cap_capable+0x1ed/0x3f0
[ 1235.614155] ? sysfs_file_ops+0x140/0x140
[ 1235.619458] ? threaded_store+0x132/0x220
[ 1235.624756] threaded_store+0x132/0x220
[ 1235.629859] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.635352] ? sysfs_file_ops+0x140/0x140
[ 1235.640651] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.645946] ? sysfs_kf_write+0x65/0x160
[ 1235.651147] kernfs_fop_write_iter+0x335/0x510
[ 1235.656932] ? kernfs_ops+0xf0/0xf0
[ 1235.661641] new_sync_write+0x308/0x6f0
[ 1235.666744] ? new_sync_read+0x6f0/0x6f0
[ 1235.671946] ? lock_acquire.part.0+0xbc/0x260
[ 1235.677633] ? ksys_write+0xff/0x200
[ 1235.682436] vfs_write+0x62a/0x980
[ 1235.687052] ksys_write+0xff/0x200
[ 1235.691667] ? __ia32_sys_read+0xc0/0xc0
[ 1235.696864] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.702454] ? rcu_is_watching+0x15/0xd0
[ 1235.707656] do_syscall_64+0xf3/0x5e0
[ 1235.712562] ? irq_exit_rcu+0x39/0xf0
[ 1235.717462] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.723931] RIP: 0033:0x7f5c78f1bc5e
[ 1235.728742] RSP: 002b:00007ffeab010210 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.738038] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5c78f1bc5e
[ 1235.746846] RDX: 0000000000000001 RSI: 000055dbb167b760 RDI: 0000000000000007
[ 1235.755653] RBP: 00007ffeab010220 R08: 0000000000000000 R09: 0000000000000000
[ 1235.764460] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5c78db6e80
[ 1235.773270] R13: 000055dbb167b760 R14: 0000000000000007 R15: 00007f5c7961cb98
[ 1235.782086]
[ 1235.785348] INFO: task python3:5431 is blocked on a mutex likely owned by task python3:3494.
[ 1235.795620] task:python3 state:D stack:25312 pid:3494 tgid:3494 ppid:3493 task_flags:0x400100 flags:0x00080802
[ 1235.808825] Call Trace:
[ 1235.812364]
[ 1235.815513] __schedule+0xad9/0x1490
[ 1235.820327] ? io_schedule_timeout+0x130/0x130
[ 1235.826113] ? find_held_lock+0x2b/0x80
[ 1235.831215] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1235.837395] schedule+0xd4/0x260
[ 1235.841815] schedule_timeout+0x103/0x260
[ 1235.847112] ? rcu_is_watching+0x15/0xd0
[ 1235.852310] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1235.858679] ? hrtimers_cpu_dying+0x250/0x250
[ 1235.864368] ? ww_mutex_lock+0x160/0x160
[ 1235.869571] msleep+0x1b/0x30
[ 1235.873698] napi_set_threaded+0x376/0x410
[ 1235.879094] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.884582] netif_set_threaded+0x254/0x400
[ 1235.890078] ? sysfs_file_ops+0x140/0x140
[ 1235.895374] threaded_store+0x1bb/0x220
[ 1235.900480] ? tx_maxrate_store+0x4e0/0x4e0
[ 1235.905971] ? sysfs_file_ops+0x140/0x140
[ 1235.911269] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.916556] ? sysfs_kf_write+0x65/0x160
[ 1235.921758] kernfs_fop_write_iter+0x335/0x510
[ 1235.927545] ? kernfs_ops+0xf0/0xf0
[ 1235.932259] new_sync_write+0x308/0x6f0
[ 1235.937361] ? new_sync_read+0x6f0/0x6f0
[ 1235.942555] ? lock_acquire.part.0+0xbc/0x260
[ 1235.948242] ? ksys_write+0xff/0x200
[ 1235.953045] vfs_write+0x62a/0x980
[ 1235.957662] ksys_write+0xff/0x200
[ 1235.962276] ? __ia32_sys_read+0xc0/0xc0
[ 1235.967476] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.973065] ? rcu_is_watching+0x15/0xd0
[ 1235.978264] do_syscall_64+0xf3/0x5e0
[ 1235.983171] ? exc_page_fault+0x9e/0xb0
[ 1235.988275] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.994742] RIP: 0033:0x7f3a22c32c5e
[ 1235.999551] RSP: 002b:00007ffe1792a010 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1236.008840] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3a22c32c5e
[ 1236.017650] RDX: 0000000000000001 RSI: 0000561a5d66f760 RDI: 0000000000000007
[ 1236.026460] RBP: 00007ffe1792a020 R08: 0000000000000000 R09: 0000000000000000
[ 1236.035268] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f3a22acde80
[ 1236.044064] R13: 0000561a5d66f760 R14: 0000000000000007 R15: 00007f3a22a5e480
[ 1236.052877]
[ 1236.056139] \x0aShowing all locks held in the system:
[ 1236.064674] 1 lock held by khungtaskd/185:
[ 1236.070068] #0: ffffffff9f6ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.081550] 4 locks held by python3/3494:
[ 1236.086847] #0: ff1100012cb6c3f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.096641] #1: ff11000166153e88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.107605] #2: ff1100011fa950f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.118761] #3: ff1100011fa39d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.128856] 4 locks held by python3/5431:
[ 1236.134151] #0: ff1100012cb6c3f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1236.143944] #1: ff110001dd8ea088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1236.154907] #2: ff1100011fa950f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.166066] #3: ff1100011fa39d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.176157]
[ 1236.178622] =============================================\x0a
[ 1358.391592] INFO: task python3:5431 blocked for more than 491 seconds.
[ 1358.399739] Not tainted 7.0.0-rc5-xfqj-g0b484fe08e24 #1
[ 1358.406990] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.416580] task:python3 state:D stack:25312 pid:5431 tgid:5431 ppid:5430 task_flags:0x400100 flags:0x00080002
[ 1358.429786] Call Trace:
[ 1358.433328]
[ 1358.436483] __schedule+0xad9/0x1490
[ 1358.441298] ? io_schedule_timeout+0x130/0x130
[ 1358.447086] ? find_held_lock+0x2b/0x80
[ 1358.452190] ? __lock_release.isra.0+0x59/0x170
[ 1358.458076] schedule+0xd4/0x260
[ 1358.462496] schedule_preempt_disabled+0x18/0x30
[ 1358.468466] __mutex_lock+0xd76/0x1f20
[ 1358.473474] ? threaded_store+0x132/0x220
[ 1358.478774] ? ww_mutex_lock+0x160/0x160
[ 1358.483975] ? selinux_inode_free_security+0x310/0x310
[ 1358.490545] ? cap_capable+0x1ed/0x3f0
[ 1358.495554] ? sysfs_file_ops+0x140/0x140
[ 1358.500856] ? threaded_store+0x132/0x220
[ 1358.506153] threaded_store+0x132/0x220
[ 1358.511257] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.516751] ? sysfs_file_ops+0x140/0x140
[ 1358.522052] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.527349] ? sysfs_kf_write+0x65/0x160
[ 1358.532550] kernfs_fop_write_iter+0x335/0x510
[ 1358.538338] ? kernfs_ops+0xf0/0xf0
[ 1358.543051] new_sync_write+0x308/0x6f0
[ 1358.548155] ? new_sync_read+0x6f0/0x6f0
[ 1358.553357] ? lock_acquire.part.0+0xbc/0x260
[ 1358.559044] ? ksys_write+0xff/0x200
[ 1358.563846] vfs_write+0x62a/0x980
[ 1358.568462] ksys_write+0xff/0x200
[ 1358.573068] ? __ia32_sys_read+0xc0/0xc0
[ 1358.578264] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.583853] ? rcu_is_watching+0x15/0xd0
[ 1358.589054] do_syscall_64+0xf3/0x5e0
[ 1358.593960] ? irq_exit_rcu+0x39/0xf0
[ 1358.598869] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.605337] RIP: 0033:0x7f5c78f1bc5e
[ 1358.610148] RSP: 002b:00007ffeab010210 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.619447] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f5c78f1bc5e
[ 1358.628255] RDX: 0000000000000001 RSI: 000055dbb167b760 RDI: 0000000000000007
[ 1358.637066] RBP: 00007ffeab010220 R08: 0000000000000000 R09: 0000000000000000
[ 1358.645875] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f5c78db6e80
[ 1358.654683] R13: 000055dbb167b760 R14: 0000000000000007 R15: 00007f5c7961cb98
[ 1358.663490]
[ 1358.666759] INFO: task python3:5431 is blocked on a mutex likely owned by task python3:3494.
[ 1358.677031] task:python3 state:D stack:25312 pid:3494 tgid:3494 ppid:3493 task_flags:0x400100 flags:0x00080802
[ 1358.690234] Call Trace:
[ 1358.693772]
[ 1358.696924] __schedule+0xad9/0x1490
[ 1358.701738] ? io_schedule_timeout+0x130/0x130
[ 1358.707524] ? find_held_lock+0x2b/0x80
[ 1358.712626] ? __try_to_del_timer_sync+0x154/0x1f0
[ 1358.718807] schedule+0xd4/0x260
[ 1358.723227] schedule_timeout+0x103/0x260
[ 1358.728515] ? rcu_is_watching+0x15/0xd0
[ 1358.733714] ? hrtimer_nanosleep_restart+0x180/0x180
[ 1358.740086] ? hrtimers_cpu_dying+0x250/0x250
[ 1358.745775] ? ww_mutex_lock+0x160/0x160
[ 1358.750977] msleep+0x1b/0x30
[ 1358.755104] napi_set_threaded+0x376/0x410
[ 1358.760503] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.765998] netif_set_threaded+0x254/0x400
[ 1358.771493] ? sysfs_file_ops+0x140/0x140
[ 1358.776790] threaded_store+0x1bb/0x220
[ 1358.781891] ? tx_maxrate_store+0x4e0/0x4e0
[ 1358.787383] ? sysfs_file_ops+0x140/0x140
[ 1358.792680] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.797970] ? sysfs_kf_write+0x65/0x160
[ 1358.803163] kernfs_fop_write_iter+0x335/0x510
[ 1358.808950] ? kernfs_ops+0xf0/0xf0
[ 1358.813660] new_sync_write+0x308/0x6f0
[ 1358.818762] ? new_sync_read+0x6f0/0x6f0
[ 1358.823966] ? lock_acquire.part.0+0xbc/0x260
[ 1358.829654] ? ksys_write+0xff/0x200
[ 1358.834465] vfs_write+0x62a/0x980
[ 1358.839081] ksys_write+0xff/0x200
[ 1358.843694] ? __ia32_sys_read+0xc0/0xc0
[ 1358.848893] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.854485] ? rcu_is_watching+0x15/0xd0
[ 1358.859684] do_syscall_64+0xf3/0x5e0
[ 1358.864589] ? exc_page_fault+0x9e/0xb0
[ 1358.869694] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.876160] RIP: 0033:0x7f3a22c32c5e
[ 1358.880969] RSP: 002b:00007ffe1792a010 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.890267] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3a22c32c5e
[ 1358.899076] RDX: 0000000000000001 RSI: 0000561a5d66f760 RDI: 0000000000000007
[ 1358.907887] RBP: 00007ffe1792a020 R08: 0000000000000000 R09: 0000000000000000
[ 1358.916696] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f3a22acde80
[ 1358.925502] R13: 0000561a5d66f760 R14: 0000000000000007 R15: 00007f3a22a5e480
[ 1358.934319]
[ 1358.937576] \x0aShowing all locks held in the system:
[ 1358.946107] 1 lock held by khungtaskd/185:
[ 1358.951500] #0: ffffffff9f6ce120 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.962985] 4 locks held by python3/3494:
[ 1358.968281] #0: ff1100012cb6c3f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1358.978076] #1: ff11000166153e88 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1358.989038] #2: ff1100011fa950f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.000197] #3: ff1100011fa39d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.010289] 4 locks held by python3/5431:
[ 1359.015584] #0: ff1100012cb6c3f8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xff/0x200
[ 1359.025373] #1: ff110001dd8ea088 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x247/0x510
[ 1359.036335] #2: ff1100011fa950f0 (kn->active#127){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.047484] #3: ff1100011fa39d00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.057576]
[ 1359.060043] =============================================\x0a