[ 787.446841] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 787.449294] nipa-hw-worker: [5/44] Retrying drivers/net:napi_threaded.py
[ 787.599647] kselftest: Running tests in drivers/net
[ 891.398638] kworker/u129:9 (290) used greatest stack depth: 21936 bytes left
[ 989.703652] INFO: task python3:5786 blocked for more than 122 seconds.
[ 989.711847] Not tainted 7.1.0-rc1-tsgy-gbbb58bca5be8 #1
[ 989.719292] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.728881] task:python3 state:D stack:25672 pid:5786 tgid:5786 ppid:5785 task_flags:0x400100 flags:0x00080000
[ 989.742080] Call Trace:
[ 989.745619]
[ 989.748768] __schedule+0xcba/0x1940
[ 989.753586] ? io_schedule_timeout+0x140/0x140
[ 989.759367] ? find_held_lock+0x2b/0x80
[ 989.764472] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.770358] schedule+0xd4/0x260
[ 989.774775] schedule_preempt_disabled+0x18/0x30
[ 989.780751] __mutex_lock+0x6a3/0x2000
[ 989.785753] ? threaded_store+0x132/0x220
[ 989.791051] ? ww_mutex_lock+0x160/0x160
[ 989.796250] ? selinux_inode_free_security+0x330/0x330
[ 989.802818] ? cap_capable+0x1d6/0x3e0
[ 989.807834] ? threaded_store+0x132/0x220
[ 989.813145] threaded_store+0x132/0x220
[ 989.818248] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.823545] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.828839] ? sysfs_kf_write+0x65/0x160
[ 989.834035] ? sysfs_file_ops+0x140/0x140
[ 989.839330] kernfs_fop_write_iter+0x335/0x510
[ 989.845112] ? kernfs_ops+0xf0/0xf0
[ 989.849821] new_sync_write+0x333/0x750
[ 989.854925] ? __lock_acquire+0x508/0xc10
[ 989.860211] ? new_sync_read+0x740/0x740
[ 989.865412] ? lock_acquire.part.0+0xbc/0x260
[ 989.871097] ? ksys_write+0x116/0x250
[ 989.876006] vfs_write+0x657/0x9b0
[ 989.880623] ksys_write+0x116/0x250
[ 989.885335] ? __ia32_sys_read+0xc0/0xc0
[ 989.890531] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.896117] ? rcu_is_watching+0x15/0xd0
[ 989.901315] do_syscall_64+0xf3/0x650
[ 989.906218] ? irq_exit_rcu+0x1a/0x30
[ 989.911125] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.917590] RIP: 0033:0x7f8b90df622e
[ 989.922400] RSP: 002b:00007ffd53aa6940 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.931698] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f8b90df622e
[ 989.940495] RDX: 0000000000000001 RSI: 0000561ac63ad410 RDI: 0000000000000007
[ 989.949300] RBP: 00007ffd53aa6950 R08: 0000000000000000 R09: 0000000000000000
[ 989.958109] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f8b90c6e390
[ 989.966913] R13: 0000561ac63ad410 R14: 0000000000000007 R15: 0000000000000003
[ 989.975734]
[ 989.979030] INFO: task python3:5786 is blocked on a mutex likely owned by task python3:3701.
[ 989.989301] task:python3 state:D stack:25672 pid:3701 tgid:3701 ppid:3700 task_flags:0x400100 flags:0x00080802
[ 990.002502] Call Trace:
[ 990.006040]
[ 990.009186] __schedule+0xcba/0x1940
[ 990.013998] ? io_schedule_timeout+0x140/0x140
[ 990.019779] ? find_held_lock+0x2b/0x80
[ 990.024883] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.031061] schedule+0xd4/0x260
[ 990.035479] schedule_timeout+0x13f/0x290
[ 990.040782] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.047152] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.052842] ? __free_object+0x2ad/0x490
[ 990.058043] msleep+0x1b/0x30
[ 990.062168] napi_set_threaded+0x3a5/0x440
[ 990.067561] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.073054] netif_set_threaded+0x257/0x400
[ 990.078538] threaded_store+0x1b8/0x220
[ 990.083630] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.088927] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.094222] ? sysfs_kf_write+0x65/0x160
[ 990.099422] ? sysfs_file_ops+0x140/0x140
[ 990.104709] kernfs_fop_write_iter+0x335/0x510
[ 990.110494] ? kernfs_ops+0xf0/0xf0
[ 990.115205] new_sync_write+0x333/0x750
[ 990.120306] ? __lock_acquire+0x508/0xc10
[ 990.125600] ? new_sync_read+0x740/0x740
[ 990.130801] ? lock_acquire.part.0+0xbc/0x260
[ 990.136492] ? ksys_write+0x116/0x250
[ 990.141401] vfs_write+0x657/0x9b0
[ 990.146015] ksys_write+0x116/0x250
[ 990.150719] ? __ia32_sys_read+0xc0/0xc0
[ 990.155916] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.161502] ? rcu_is_watching+0x15/0xd0
[ 990.166701] do_syscall_64+0xf3/0x650
[ 990.171606] ? trace_hardirqs_off+0xd/0x30
[ 990.177001] ? exc_page_fault+0xda/0xf0
[ 990.182104] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.188559] RIP: 0033:0x7f68814ea22e
[ 990.193367] RSP: 002b:00007ffe440cf220 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.202664] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f68814ea22e
[ 990.211470] RDX: 0000000000000001 RSI: 000056088ab3a410 RDI: 0000000000000007
[ 990.220276] RBP: 00007ffe440cf230 R08: 0000000000000000 R09: 0000000000000000
[ 990.229081] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6881362390
[ 990.237887] R13: 000056088ab3a410 R14: 0000000000000007 R15: 0000000000000003
[ 990.246698]
[ 990.249956] \x0aShowing all locks held in the system:
[ 990.258502] 1 lock held by khungtaskd/185:
[ 990.263958] #0: ffffffff8ecfdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.275433] 4 locks held by systemd-journal/804:
[ 990.281411] #0: ff1100180e7b72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 990.291298] #1: ffffffff8ef030a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: nd_jump_root+0x353/0x550
[ 990.303628] #2: ff110001a28df620 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 990.315176] #3: ff1100019e65c058 (&xa->xa_lock#5){-.-.}-{3:3}, at: __folio_mark_dirty+0x9d/0x2c0
[ 990.325954] 4 locks held by python3/3701:
[ 990.331247] #0: ff1100012e9513e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.341133] #1: ff11000162aeb280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.352092] #2: ff11000121380b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.363250] #3: ff11000121459d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.373355] 4 locks held by python3/5786:
[ 990.378647] #0: ff1100012e9513e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.388535] #1: ff110001e5f2ca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.399499] #2: ff11000121380b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.410651] #3: ff11000121459d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.420742]
[ 990.423198] =============================================\x0a
[ 1112.583828] INFO: task python3:5786 blocked for more than 245 seconds.
[ 1112.591974] Not tainted 7.1.0-rc1-tsgy-gbbb58bca5be8 #1
[ 1112.599224] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.608814] task:python3 state:D stack:25672 pid:5786 tgid:5786 ppid:5785 task_flags:0x400100 flags:0x00080000
[ 1112.622020] Call Trace:
[ 1112.625562]
[ 1112.628715] __schedule+0xcba/0x1940
[ 1112.633534] ? io_schedule_timeout+0x140/0x140
[ 1112.639322] ? find_held_lock+0x2b/0x80
[ 1112.644429] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.650314] schedule+0xd4/0x260
[ 1112.654732] schedule_preempt_disabled+0x18/0x30
[ 1112.660710] __mutex_lock+0x6a3/0x2000
[ 1112.665714] ? threaded_store+0x132/0x220
[ 1112.671013] ? ww_mutex_lock+0x160/0x160
[ 1112.676204] ? selinux_inode_free_security+0x330/0x330
[ 1112.682775] ? cap_capable+0x1d6/0x3e0
[ 1112.687785] ? threaded_store+0x132/0x220
[ 1112.693084] threaded_store+0x132/0x220
[ 1112.698186] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.703483] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.708788] ? sysfs_kf_write+0x65/0x160
[ 1112.713992] ? sysfs_file_ops+0x140/0x140
[ 1112.719288] kernfs_fop_write_iter+0x335/0x510
[ 1112.725075] ? kernfs_ops+0xf0/0xf0
[ 1112.729786] new_sync_write+0x333/0x750
[ 1112.734888] ? __lock_acquire+0x508/0xc10
[ 1112.740185] ? new_sync_read+0x740/0x740
[ 1112.745388] ? lock_acquire.part.0+0xbc/0x260
[ 1112.751075] ? ksys_write+0x116/0x250
[ 1112.755976] vfs_write+0x657/0x9b0
[ 1112.760594] ksys_write+0x116/0x250
[ 1112.765307] ? __ia32_sys_read+0xc0/0xc0
[ 1112.770508] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.776098] ? rcu_is_watching+0x15/0xd0
[ 1112.781298] do_syscall_64+0xf3/0x650
[ 1112.786207] ? irq_exit_rcu+0x1a/0x30
[ 1112.791114] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.797575] RIP: 0033:0x7f8b90df622e
[ 1112.802385] RSP: 002b:00007ffd53aa6940 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.811683] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f8b90df622e
[ 1112.820484] RDX: 0000000000000001 RSI: 0000561ac63ad410 RDI: 0000000000000007
[ 1112.829289] RBP: 00007ffd53aa6950 R08: 0000000000000000 R09: 0000000000000000
[ 1112.838100] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f8b90c6e390
[ 1112.846909] R13: 0000561ac63ad410 R14: 0000000000000007 R15: 0000000000000003
[ 1112.855725]
[ 1112.859002] INFO: task python3:5786 is blocked on a mutex likely owned by task python3:3701.
[ 1112.869274] task:python3 state:D stack:25672 pid:3701 tgid:3701 ppid:3700 task_flags:0x400100 flags:0x00080802
[ 1112.882476] Call Trace:
[ 1112.886015]
[ 1112.889166] __schedule+0xcba/0x1940
[ 1112.893979] ? io_schedule_timeout+0x140/0x140
[ 1112.899765] ? find_held_lock+0x2b/0x80
[ 1112.904867] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.911045] schedule+0xd4/0x260
[ 1112.915463] schedule_timeout+0x13f/0x290
[ 1112.920760] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.927132] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.932822] ? __free_object+0x2ad/0x490
[ 1112.938024] msleep+0x1b/0x30
[ 1112.942149] napi_set_threaded+0x3a5/0x440
[ 1112.947544] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.953041] netif_set_threaded+0x257/0x400
[ 1112.958535] threaded_store+0x1b8/0x220
[ 1112.963639] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.968939] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.974238] ? sysfs_kf_write+0x65/0x160
[ 1112.979437] ? sysfs_file_ops+0x140/0x140
[ 1112.984732] kernfs_fop_write_iter+0x335/0x510
[ 1112.990515] ? kernfs_ops+0xf0/0xf0
[ 1112.995229] new_sync_write+0x333/0x750
[ 1113.000332] ? __lock_acquire+0x508/0xc10
[ 1113.005630] ? new_sync_read+0x740/0x740
[ 1113.010832] ? lock_acquire.part.0+0xbc/0x260
[ 1113.016519] ? ksys_write+0x116/0x250
[ 1113.021432] vfs_write+0x657/0x9b0
[ 1113.026047] ksys_write+0x116/0x250
[ 1113.030751] ? __ia32_sys_read+0xc0/0xc0
[ 1113.035940] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.041530] ? rcu_is_watching+0x15/0xd0
[ 1113.046732] do_syscall_64+0xf3/0x650
[ 1113.051640] ? trace_hardirqs_off+0xd/0x30
[ 1113.057036] ? exc_page_fault+0xda/0xf0
[ 1113.062139] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.068609] RIP: 0033:0x7f68814ea22e
[ 1113.073417] RSP: 002b:00007ffe440cf220 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.082713] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f68814ea22e
[ 1113.091522] RDX: 0000000000000001 RSI: 000056088ab3a410 RDI: 0000000000000007
[ 1113.100331] RBP: 00007ffe440cf230 R08: 0000000000000000 R09: 0000000000000000
[ 1113.109139] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6881362390
[ 1113.117948] R13: 000056088ab3a410 R14: 0000000000000007 R15: 0000000000000003
[ 1113.126765]
[ 1113.130029] \x0aShowing all locks held in the system:
[ 1113.138569] 1 lock held by khungtaskd/185:
[ 1113.143963] #0: ffffffff8ecfdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.155445] 4 locks held by python3/3701:
[ 1113.160739] #0: ff1100012e9513e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.170629] #1: ff11000162aeb280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.181582] #2: ff11000121380b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.192737] #3: ff11000121459d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.202833] 4 locks held by python3/5786:
[ 1113.208129] #0: ff1100012e9513e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.218019] #1: ff110001e5f2ca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.228981] #2: ff11000121380b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.240128] #3: ff11000121459d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.250219]
[ 1113.252687] =============================================\x0a
[ 1235.465832] INFO: task python3:5786 blocked for more than 368 seconds.
[ 1235.473982] Not tainted 7.1.0-rc1-tsgy-gbbb58bca5be8 #1
[ 1235.481231] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.490819] task:python3 state:D stack:25672 pid:5786 tgid:5786 ppid:5785 task_flags:0x400100 flags:0x00080002
[ 1235.504025] Call Trace:
[ 1235.507565]
[ 1235.510713] __schedule+0xcba/0x1940
[ 1235.515527] ? io_schedule_timeout+0x140/0x140
[ 1235.521305] ? find_held_lock+0x2b/0x80
[ 1235.526407] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.532291] schedule+0xd4/0x260
[ 1235.536708] schedule_preempt_disabled+0x18/0x30
[ 1235.542685] __mutex_lock+0x6a3/0x2000
[ 1235.547688] ? threaded_store+0x132/0x220
[ 1235.552987] ? ww_mutex_lock+0x160/0x160
[ 1235.558187] ? selinux_inode_free_security+0x330/0x330
[ 1235.564756] ? cap_capable+0x1d6/0x3e0
[ 1235.569765] ? threaded_store+0x132/0x220
[ 1235.575063] threaded_store+0x132/0x220
[ 1235.580154] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.585451] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.590749] ? sysfs_kf_write+0x65/0x160
[ 1235.595947] ? sysfs_file_ops+0x140/0x140
[ 1235.601240] kernfs_fop_write_iter+0x335/0x510
[ 1235.607023] ? kernfs_ops+0xf0/0xf0
[ 1235.611734] new_sync_write+0x333/0x750
[ 1235.616836] ? __lock_acquire+0x508/0xc10
[ 1235.622130] ? new_sync_read+0x740/0x740
[ 1235.627331] ? lock_acquire.part.0+0xbc/0x260
[ 1235.633014] ? ksys_write+0x116/0x250
[ 1235.637925] vfs_write+0x657/0x9b0
[ 1235.642538] ksys_write+0x116/0x250
[ 1235.647250] ? __ia32_sys_read+0xc0/0xc0
[ 1235.652464] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.658056] ? rcu_is_watching+0x15/0xd0
[ 1235.663275] do_syscall_64+0xf3/0x650
[ 1235.668179] ? irq_exit_rcu+0x1a/0x30
[ 1235.673087] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.679552] RIP: 0033:0x7f8b90df622e
[ 1235.684362] RSP: 002b:00007ffd53aa6940 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.693649] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f8b90df622e
[ 1235.702455] RDX: 0000000000000001 RSI: 0000561ac63ad410 RDI: 0000000000000007
[ 1235.711259] RBP: 00007ffd53aa6950 R08: 0000000000000000 R09: 0000000000000000
[ 1235.720064] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f8b90c6e390
[ 1235.728874] R13: 0000561ac63ad410 R14: 0000000000000007 R15: 0000000000000003
[ 1235.737685]
[ 1235.740987] INFO: task python3:5786 is blocked on a mutex likely owned by task python3:3701.
[ 1235.751258] task:python3 state:D stack:25672 pid:3701 tgid:3701 ppid:3700 task_flags:0x400100 flags:0x00080802
[ 1235.764458] Call Trace:
[ 1235.767995]
[ 1235.771145] __schedule+0xcba/0x1940
[ 1235.775956] ? io_schedule_timeout+0x140/0x140
[ 1235.781740] ? find_held_lock+0x2b/0x80
[ 1235.786840] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.793018] schedule+0xd4/0x260
[ 1235.797436] schedule_timeout+0x13f/0x290
[ 1235.802732] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.809102] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.814788] ? __free_object+0x2ad/0x490
[ 1235.819990] msleep+0x1b/0x30
[ 1235.824115] napi_set_threaded+0x3a5/0x440
[ 1235.829508] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.835000] netif_set_threaded+0x257/0x400
[ 1235.840494] threaded_store+0x1b8/0x220
[ 1235.845597] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.850884] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.856179] ? sysfs_kf_write+0x65/0x160
[ 1235.861377] ? sysfs_file_ops+0x140/0x140
[ 1235.866673] kernfs_fop_write_iter+0x335/0x510
[ 1235.872456] ? kernfs_ops+0xf0/0xf0
[ 1235.877165] new_sync_write+0x333/0x750
[ 1235.882266] ? __lock_acquire+0x508/0xc10
[ 1235.887561] ? new_sync_read+0x740/0x740
[ 1235.892761] ? lock_acquire.part.0+0xbc/0x260
[ 1235.898439] ? ksys_write+0x116/0x250
[ 1235.903347] vfs_write+0x657/0x9b0
[ 1235.907959] ksys_write+0x116/0x250
[ 1235.912673] ? __ia32_sys_read+0xc0/0xc0
[ 1235.917871] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.923458] ? rcu_is_watching+0x15/0xd0
[ 1235.928657] do_syscall_64+0xf3/0x650
[ 1235.933560] ? trace_hardirqs_off+0xd/0x30
[ 1235.938955] ? exc_page_fault+0xda/0xf0
[ 1235.944058] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.950518] RIP: 0033:0x7f68814ea22e
[ 1235.955325] RSP: 002b:00007ffe440cf220 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.964618] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f68814ea22e
[ 1235.973423] RDX: 0000000000000001 RSI: 000056088ab3a410 RDI: 0000000000000007
[ 1235.982233] RBP: 00007ffe440cf230 R08: 0000000000000000 R09: 0000000000000000
[ 1235.991039] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6881362390
[ 1235.999843] R13: 000056088ab3a410 R14: 0000000000000007 R15: 0000000000000003
[ 1236.008656]
[ 1236.011921] \x0aShowing all locks held in the system:
[ 1236.020463] 1 lock held by khungtaskd/185:
[ 1236.025853] #0: ffffffff8ecfdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.037331] 4 locks held by python3/3701:
[ 1236.042627] #0: ff1100012e9513e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.052513] #1: ff11000162aeb280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.063475] #2: ff11000121380b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.074629] #3: ff11000121459d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.084725] 4 locks held by python3/5786:
[ 1236.090017] #0: ff1100012e9513e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.099904] #1: ff110001e5f2ca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.110865] #2: ff11000121380b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.122022] #3: ff11000121459d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.132107]
[ 1236.134572] =============================================\x0a
[ 1358.346897] INFO: task python3:5786 blocked for more than 491 seconds.
[ 1358.355047] Not tainted 7.1.0-rc1-tsgy-gbbb58bca5be8 #1
[ 1358.362296] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.371885] task:python3 state:D stack:25672 pid:5786 tgid:5786 ppid:5785 task_flags:0x400100 flags:0x00080002
[ 1358.385087] Call Trace:
[ 1358.388618]
[ 1358.391766] __schedule+0xcba/0x1940
[ 1358.396584] ? io_schedule_timeout+0x140/0x140
[ 1358.402370] ? find_held_lock+0x2b/0x80
[ 1358.407471] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.413356] schedule+0xd4/0x260
[ 1358.417774] schedule_preempt_disabled+0x18/0x30
[ 1358.423752] __mutex_lock+0x6a3/0x2000
[ 1358.428755] ? threaded_store+0x132/0x220
[ 1358.434054] ? ww_mutex_lock+0x160/0x160
[ 1358.439252] ? selinux_inode_free_security+0x330/0x330
[ 1358.445822] ? cap_capable+0x1d6/0x3e0
[ 1358.450830] ? threaded_store+0x132/0x220
[ 1358.456123] threaded_store+0x132/0x220
[ 1358.461224] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.466517] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.471816] ? sysfs_kf_write+0x65/0x160
[ 1358.477015] ? sysfs_file_ops+0x140/0x140
[ 1358.482308] kernfs_fop_write_iter+0x335/0x510
[ 1358.488090] ? kernfs_ops+0xf0/0xf0
[ 1358.492798] new_sync_write+0x333/0x750
[ 1358.497899] ? __lock_acquire+0x508/0xc10
[ 1358.503195] ? new_sync_read+0x740/0x740
[ 1358.508394] ? lock_acquire.part.0+0xbc/0x260
[ 1358.514077] ? ksys_write+0x116/0x250
[ 1358.518987] vfs_write+0x657/0x9b0
[ 1358.523600] ksys_write+0x116/0x250
[ 1358.528309] ? __ia32_sys_read+0xc0/0xc0
[ 1358.533507] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.539093] ? rcu_is_watching+0x15/0xd0
[ 1358.544294] do_syscall_64+0xf3/0x650
[ 1358.549200] ? irq_exit_rcu+0x1a/0x30
[ 1358.554107] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.560565] RIP: 0033:0x7f8b90df622e
[ 1358.565366] RSP: 002b:00007ffd53aa6940 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.574664] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f8b90df622e
[ 1358.583470] RDX: 0000000000000001 RSI: 0000561ac63ad410 RDI: 0000000000000007
[ 1358.592274] RBP: 00007ffd53aa6950 R08: 0000000000000000 R09: 0000000000000000
[ 1358.601079] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f8b90c6e390
[ 1358.609886] R13: 0000561ac63ad410 R14: 0000000000000007 R15: 0000000000000003
[ 1358.618698]
[ 1358.621973] INFO: task python3:5786 is blocked on a mutex likely owned by task python3:3701.
[ 1358.632244] task:python3 state:D stack:25672 pid:3701 tgid:3701 ppid:3700 task_flags:0x400100 flags:0x00080802
[ 1358.645444] Call Trace:
[ 1358.648971]
[ 1358.652120] __schedule+0xcba/0x1940
[ 1358.656930] ? io_schedule_timeout+0x140/0x140
[ 1358.662711] ? find_held_lock+0x2b/0x80
[ 1358.667813] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.673989] schedule+0xd4/0x260
[ 1358.678405] schedule_timeout+0x13f/0x290
[ 1358.683699] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.690068] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.695756] ? __free_object+0x2ad/0x490
[ 1358.700956] msleep+0x1b/0x30
[ 1358.705080] napi_set_threaded+0x3a5/0x440
[ 1358.710475] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.715971] netif_set_threaded+0x257/0x400
[ 1358.721462] threaded_store+0x1b8/0x220
[ 1358.726564] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.731857] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.737155] ? sysfs_kf_write+0x65/0x160
[ 1358.742353] ? sysfs_file_ops+0x140/0x140
[ 1358.747647] kernfs_fop_write_iter+0x335/0x510
[ 1358.753429] ? kernfs_ops+0xf0/0xf0
[ 1358.758141] new_sync_write+0x333/0x750
[ 1358.763242] ? __lock_acquire+0x508/0xc10
[ 1358.768539] ? new_sync_read+0x740/0x740
[ 1358.773741] ? lock_acquire.part.0+0xbc/0x260
[ 1358.779427] ? ksys_write+0x116/0x250
[ 1358.784337] vfs_write+0x657/0x9b0
[ 1358.788950] ksys_write+0x116/0x250
[ 1358.793661] ? __ia32_sys_read+0xc0/0xc0
[ 1358.798858] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.804448] ? rcu_is_watching+0x15/0xd0
[ 1358.809639] do_syscall_64+0xf3/0x650
[ 1358.814544] ? trace_hardirqs_off+0xd/0x30
[ 1358.819937] ? exc_page_fault+0xda/0xf0
[ 1358.825037] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.831504] RIP: 0033:0x7f68814ea22e
[ 1358.836310] RSP: 002b:00007ffe440cf220 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.845605] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f68814ea22e
[ 1358.854411] RDX: 0000000000000001 RSI: 000056088ab3a410 RDI: 0000000000000007
[ 1358.863217] RBP: 00007ffe440cf230 R08: 0000000000000000 R09: 0000000000000000
[ 1358.872025] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f6881362390
[ 1358.880831] R13: 000056088ab3a410 R14: 0000000000000007 R15: 0000000000000003
[ 1358.889646]
[ 1358.892916] \x0aShowing all locks held in the system:
[ 1358.901454] 1 lock held by khungtaskd/185:
[ 1358.906846] #0: ffffffff8ecfdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.918327] 4 locks held by python3/3701:
[ 1358.923621] #0: ff1100012e9513e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.933509] #1: ff11000162aeb280 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.944473] #2: ff11000121380b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.955629] #3: ff11000121459d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.965725] 4 locks held by python3/5786:
[ 1358.971020] #0: ff1100012e9513e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.980907] #1: ff110001e5f2ca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.991865] #2: ff11000121380b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.003019] #3: ff11000121459d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.013112]
[ 1359.015578] =============================================\x0a