[ 773.390621] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 773.393217] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 773.545885] kselftest: Running tests in drivers/net
[ 989.703959] INFO: task python3:5539 blocked for more than 122 seconds.
[ 989.712206] Not tainted 7.1.0-rc7-gzzc-g74e32d746f94 #1
[ 989.719458] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.729776] task:python3 state:D stack:25672 pid:5539 tgid:5539 ppid:5538 task_flags:0x400100 flags:0x00080000
[ 989.742984] Call Trace:
[ 989.746524]
[ 989.749672] __schedule+0xcc3/0x19c0
[ 989.754487] ? io_schedule_timeout+0x140/0x140
[ 989.760271] ? find_held_lock+0x2b/0x80
[ 989.765378] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.771260] schedule+0xd8/0x260
[ 989.775677] schedule_preempt_disabled+0x18/0x30
[ 989.781655] __mutex_lock+0x6af/0x2010
[ 989.786660] ? threaded_store+0x132/0x220
[ 989.791960] ? ww_mutex_lock+0x160/0x160
[ 989.797155] ? selinux_inode_free_security+0x330/0x330
[ 989.803723] ? cap_capable+0x1d6/0x3e0
[ 989.808728] ? threaded_store+0x132/0x220
[ 989.814023] threaded_store+0x132/0x220
[ 989.819126] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.824425] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.829722] ? sysfs_kf_write+0x65/0x160
[ 989.834921] ? sysfs_file_ops+0x140/0x140
[ 989.840217] kernfs_fop_write_iter+0x335/0x510
[ 989.846000] ? kernfs_ops+0xf0/0xf0
[ 989.850710] new_sync_write+0x333/0x750
[ 989.855814] ? __lock_acquire+0x508/0xc10
[ 989.861102] ? new_sync_read+0x740/0x740
[ 989.866303] ? lock_acquire.part.0+0xbc/0x260
[ 989.871987] ? ksys_write+0x116/0x250
[ 989.876897] vfs_write+0x657/0x9b0
[ 989.881510] ksys_write+0x116/0x250
[ 989.886218] ? __ia32_sys_read+0xc0/0xc0
[ 989.891414] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.897001] ? rcu_is_watching+0x15/0xd0
[ 989.902202] do_syscall_64+0xf3/0x650
[ 989.907097] ? irq_exit_rcu+0x1a/0x30
[ 989.912006] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.918471] RIP: 0033:0x7ff4821cd64e
[ 989.923353] RSP: 002b:00007fff28d7e160 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.932741] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff4821cd64e
[ 989.941611] RDX: 0000000000000001 RSI: 000055c4fa1f3bf0 RDI: 0000000000000007
[ 989.950419] RBP: 00007fff28d7e170 R08: 0000000000000000 R09: 0000000000000000
[ 989.959215] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff482046390
[ 989.968020] R13: 000055c4fa1f3bf0 R14: 0000000000000007 R15: 00007ff481fd9120
[ 989.976832]
[ 989.980168] INFO: task python3:5539 is blocked on a mutex likely owned by task python3:3543.
[ 989.990438] task:python3 state:D stack:25672 pid:3543 tgid:3543 ppid:3542 task_flags:0x400100 flags:0x00080802
[ 990.003641] Call Trace:
[ 990.007176]
[ 990.010325] __schedule+0xcc3/0x19c0
[ 990.015137] ? io_schedule_timeout+0x140/0x140
[ 990.020917] ? find_held_lock+0x2b/0x80
[ 990.026017] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.032194] schedule+0xd8/0x260
[ 990.036610] schedule_timeout+0x13f/0x290
[ 990.041907] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.048276] ? trace_contention_end+0xb3/0x1a0
[ 990.054050] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.059740] ? ww_mutex_lock+0x160/0x160
[ 990.064939] msleep+0x1b/0x30
[ 990.069056] napi_set_threaded+0x3a5/0x440
[ 990.074450] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.079943] netif_set_threaded+0x257/0x400
[ 990.085497] threaded_store+0x1b8/0x220
[ 990.090662] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.095958] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.101255] ? sysfs_kf_write+0x65/0x160
[ 990.106452] ? sysfs_file_ops+0x140/0x140
[ 990.111745] kernfs_fop_write_iter+0x335/0x510
[ 990.117531] ? kernfs_ops+0xf0/0xf0
[ 990.122242] new_sync_write+0x333/0x750
[ 990.127341] ? __lock_acquire+0x508/0xc10
[ 990.132639] ? new_sync_read+0x740/0x740
[ 990.137845] ? lock_acquire.part.0+0xbc/0x260
[ 990.143530] ? ksys_write+0x116/0x250
[ 990.148438] vfs_write+0x657/0x9b0
[ 990.153052] ksys_write+0x116/0x250
[ 990.157762] ? __ia32_sys_read+0xc0/0xc0
[ 990.162961] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.168548] ? rcu_is_watching+0x15/0xd0
[ 990.173748] do_syscall_64+0xf3/0x650
[ 990.178719] ? trace_hardirqs_off+0xd/0x30
[ 990.184115] ? exc_page_fault+0xda/0xf0
[ 990.189217] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.195684] RIP: 0033:0x7fb7022f464e
[ 990.200490] RSP: 002b:00007fff80a36c70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.209787] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb7022f464e
[ 990.218583] RDX: 0000000000000001 RSI: 000056419e31ebf0 RDI: 0000000000000007
[ 990.227390] RBP: 00007fff80a36c80 R08: 0000000000000000 R09: 0000000000000000
[ 990.236195] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb70216d390
[ 990.245001] R13: 000056419e31ebf0 R14: 0000000000000007 R15: 00007fb702101120
[ 990.253813]
[ 990.257074] \x0aShowing all locks held in the system:
[ 990.265615] 1 lock held by khungtaskd/185:
[ 990.271069] #0: ffffffff984fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.282544] 2 locks held by systemd-journal/805:
[ 990.288512] #0: ff1100180e9b74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.298401] #1: ffffffff98702ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: link_path_walk+0x9a7/0x1830
[ 990.311033] 4 locks held by python3/3543:
[ 990.316329] #0: ff11000130e6c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.326215] #1: ff1100015d5cca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.337174] #2: ff11000123544b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.348336] #3: ff11000123619e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.358626] 4 locks held by python3/5539:
[ 990.363925] #0: ff11000130e6c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.373806] #1: ff110001868f5c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.384764] #2: ff11000123544b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.395928] #3: ff11000123619e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.406080]
[ 990.408545] =============================================\x0a
[ 1079.816680] kworker/u129:7 (286) used greatest stack depth: 21936 bytes left
[ 1112.584960] INFO: task python3:5539 blocked for more than 245 seconds.
[ 1112.593109] Not tainted 7.1.0-rc7-gzzc-g74e32d746f94 #1
[ 1112.600353] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.609942] task:python3 state:D stack:25672 pid:5539 tgid:5539 ppid:5538 task_flags:0x400100 flags:0x00080000
[ 1112.623139] Call Trace:
[ 1112.626677]
[ 1112.629828] __schedule+0xcc3/0x19c0
[ 1112.634644] ? io_schedule_timeout+0x140/0x140
[ 1112.640429] ? find_held_lock+0x2b/0x80
[ 1112.645532] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.651418] schedule+0xd8/0x260
[ 1112.655837] schedule_preempt_disabled+0x18/0x30
[ 1112.661817] __mutex_lock+0x6af/0x2010
[ 1112.666824] ? threaded_store+0x132/0x220
[ 1112.672124] ? ww_mutex_lock+0x160/0x160
[ 1112.677316] ? selinux_inode_free_security+0x330/0x330
[ 1112.683887] ? cap_capable+0x1d6/0x3e0
[ 1112.688898] ? threaded_store+0x132/0x220
[ 1112.694193] threaded_store+0x132/0x220
[ 1112.699296] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.704583] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.709878] ? sysfs_kf_write+0x65/0x160
[ 1112.715079] ? sysfs_file_ops+0x140/0x140
[ 1112.720373] kernfs_fop_write_iter+0x335/0x510
[ 1112.726159] ? kernfs_ops+0xf0/0xf0
[ 1112.730872] new_sync_write+0x333/0x750
[ 1112.735975] ? __lock_acquire+0x508/0xc10
[ 1112.741274] ? new_sync_read+0x740/0x740
[ 1112.746474] ? lock_acquire.part.0+0xbc/0x260
[ 1112.752151] ? ksys_write+0x116/0x250
[ 1112.757062] vfs_write+0x657/0x9b0
[ 1112.761676] ksys_write+0x116/0x250
[ 1112.766386] ? __ia32_sys_read+0xc0/0xc0
[ 1112.771584] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.777173] ? rcu_is_watching+0x15/0xd0
[ 1112.782367] do_syscall_64+0xf3/0x650
[ 1112.787273] ? irq_exit_rcu+0x1a/0x30
[ 1112.792181] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.798648] RIP: 0033:0x7ff4821cd64e
[ 1112.803450] RSP: 002b:00007fff28d7e160 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.812749] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff4821cd64e
[ 1112.821560] RDX: 0000000000000001 RSI: 000055c4fa1f3bf0 RDI: 0000000000000007
[ 1112.830367] RBP: 00007fff28d7e170 R08: 0000000000000000 R09: 0000000000000000
[ 1112.839175] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff482046390
[ 1112.847981] R13: 000055c4fa1f3bf0 R14: 0000000000000007 R15: 00007ff481fd9120
[ 1112.856799]
[ 1112.860064] INFO: task python3:5539 is blocked on a mutex likely owned by task python3:3543.
[ 1112.870338] task:python3 state:D stack:25672 pid:3543 tgid:3543 ppid:3542 task_flags:0x400100 flags:0x00080802
[ 1112.883540] Call Trace:
[ 1112.887079]
[ 1112.890229] __schedule+0xcc3/0x19c0
[ 1112.895039] ? io_schedule_timeout+0x140/0x140
[ 1112.900812] ? find_held_lock+0x2b/0x80
[ 1112.905916] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.912096] schedule+0xd8/0x260
[ 1112.916516] schedule_timeout+0x13f/0x290
[ 1112.921811] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.928180] ? trace_contention_end+0xb3/0x1a0
[ 1112.933966] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.939646] ? ww_mutex_lock+0x160/0x160
[ 1112.944845] msleep+0x1b/0x30
[ 1112.948970] napi_set_threaded+0x3a5/0x440
[ 1112.954358] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.959854] netif_set_threaded+0x257/0x400
[ 1112.965349] threaded_store+0x1b8/0x220
[ 1112.970454] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.975753] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.981050] ? sysfs_kf_write+0x65/0x160
[ 1112.986240] ? sysfs_file_ops+0x140/0x140
[ 1112.991534] kernfs_fop_write_iter+0x335/0x510
[ 1112.997319] ? kernfs_ops+0xf0/0xf0
[ 1113.002029] new_sync_write+0x333/0x750
[ 1113.007131] ? __lock_acquire+0x508/0xc10
[ 1113.012431] ? new_sync_read+0x740/0x740
[ 1113.017632] ? lock_acquire.part.0+0xbc/0x260
[ 1113.023318] ? ksys_write+0x116/0x250
[ 1113.028228] vfs_write+0x657/0x9b0
[ 1113.032845] ksys_write+0x116/0x250
[ 1113.037556] ? __ia32_sys_read+0xc0/0xc0
[ 1113.042755] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.048335] ? rcu_is_watching+0x15/0xd0
[ 1113.053537] do_syscall_64+0xf3/0x650
[ 1113.058444] ? trace_hardirqs_off+0xd/0x30
[ 1113.063838] ? exc_page_fault+0xda/0xf0
[ 1113.068940] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.075409] RIP: 0033:0x7fb7022f464e
[ 1113.080218] RSP: 002b:00007fff80a36c70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.089515] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb7022f464e
[ 1113.098324] RDX: 0000000000000001 RSI: 000056419e31ebf0 RDI: 0000000000000007
[ 1113.107132] RBP: 00007fff80a36c80 R08: 0000000000000000 R09: 0000000000000000
[ 1113.115940] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb70216d390
[ 1113.124740] R13: 000056419e31ebf0 R14: 0000000000000007 R15: 00007fb702101120
[ 1113.133553]
[ 1113.136819] \x0aShowing all locks held in the system:
[ 1113.145358] 1 lock held by khungtaskd/185:
[ 1113.150753] #0: ffffffff984fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.162218] 2 locks held by systemd-journal/805:
[ 1113.168196] #0: ff1100180e5374e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1113.178084] #1: ffffffff98702ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.190910] 4 locks held by python3/3543:
[ 1113.196208] #0: ff11000130e6c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.206097] #1: ff1100015d5cca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.217061] #2: ff11000123544b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.228223] #3: ff11000123619e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.238319] 4 locks held by python3/5539:
[ 1113.243617] #0: ff11000130e6c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.253504] #1: ff110001868f5c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.264465] #2: ff11000123544b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.275622] #3: ff11000123619e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.285706]
[ 1113.288174] =============================================\x0a
[ 1235.465962] INFO: task python3:5539 blocked for more than 368 seconds.
[ 1235.474107] Not tainted 7.1.0-rc7-gzzc-g74e32d746f94 #1
[ 1235.481351] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.490943] task:python3 state:D stack:25672 pid:5539 tgid:5539 ppid:5538 task_flags:0x400100 flags:0x00080002
[ 1235.504152] Call Trace:
[ 1235.507693]
[ 1235.510847] __schedule+0xcc3/0x19c0
[ 1235.515663] ? io_schedule_timeout+0x140/0x140
[ 1235.521448] ? find_held_lock+0x2b/0x80
[ 1235.526552] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.532440] schedule+0xd8/0x260
[ 1235.536859] schedule_preempt_disabled+0x18/0x30
[ 1235.542838] __mutex_lock+0x6af/0x2010
[ 1235.547845] ? threaded_store+0x132/0x220
[ 1235.553146] ? ww_mutex_lock+0x160/0x160
[ 1235.558347] ? selinux_inode_free_security+0x330/0x330
[ 1235.564916] ? cap_capable+0x1d6/0x3e0
[ 1235.569924] ? threaded_store+0x132/0x220
[ 1235.575220] threaded_store+0x132/0x220
[ 1235.580323] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.585619] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.590921] ? sysfs_kf_write+0x65/0x160
[ 1235.596126] ? sysfs_file_ops+0x140/0x140
[ 1235.601425] kernfs_fop_write_iter+0x335/0x510
[ 1235.607209] ? kernfs_ops+0xf0/0xf0
[ 1235.611923] new_sync_write+0x333/0x750
[ 1235.617026] ? __lock_acquire+0x508/0xc10
[ 1235.622323] ? new_sync_read+0x740/0x740
[ 1235.627524] ? lock_acquire.part.0+0xbc/0x260
[ 1235.633200] ? ksys_write+0x116/0x250
[ 1235.638111] vfs_write+0x657/0x9b0
[ 1235.642726] ksys_write+0x116/0x250
[ 1235.647428] ? __ia32_sys_read+0xc0/0xc0
[ 1235.652626] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.658214] ? rcu_is_watching+0x15/0xd0
[ 1235.663407] do_syscall_64+0xf3/0x650
[ 1235.668314] ? irq_exit_rcu+0x1a/0x30
[ 1235.673212] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.679680] RIP: 0033:0x7ff4821cd64e
[ 1235.684488] RSP: 002b:00007fff28d7e160 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.693785] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff4821cd64e
[ 1235.702585] RDX: 0000000000000001 RSI: 000055c4fa1f3bf0 RDI: 0000000000000007
[ 1235.711382] RBP: 00007fff28d7e170 R08: 0000000000000000 R09: 0000000000000000
[ 1235.720189] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff482046390
[ 1235.728998] R13: 000055c4fa1f3bf0 R14: 0000000000000007 R15: 00007ff481fd9120
[ 1235.737818]
[ 1235.741108] INFO: task python3:5539 is blocked on a mutex likely owned by task python3:3543.
[ 1235.751380] task:python3 state:D stack:25672 pid:3543 tgid:3543 ppid:3542 task_flags:0x400100 flags:0x00080802
[ 1235.764583] Call Trace:
[ 1235.768124]
[ 1235.771274] __schedule+0xcc3/0x19c0
[ 1235.776088] ? io_schedule_timeout+0x140/0x140
[ 1235.781872] ? find_held_lock+0x2b/0x80
[ 1235.786974] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.793157] schedule+0xd8/0x260
[ 1235.797577] schedule_timeout+0x13f/0x290
[ 1235.802876] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.809248] ? trace_contention_end+0xb3/0x1a0
[ 1235.815032] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.820722] ? ww_mutex_lock+0x160/0x160
[ 1235.825921] msleep+0x1b/0x30
[ 1235.830045] napi_set_threaded+0x3a5/0x440
[ 1235.835440] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.840936] netif_set_threaded+0x257/0x400
[ 1235.846430] threaded_store+0x1b8/0x220
[ 1235.851532] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.856828] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.862127] ? sysfs_kf_write+0x65/0x160
[ 1235.867325] ? sysfs_file_ops+0x140/0x140
[ 1235.872623] kernfs_fop_write_iter+0x335/0x510
[ 1235.878406] ? kernfs_ops+0xf0/0xf0
[ 1235.883119] new_sync_write+0x333/0x750
[ 1235.888224] ? __lock_acquire+0x508/0xc10
[ 1235.893521] ? new_sync_read+0x740/0x740
[ 1235.898717] ? lock_acquire.part.0+0xbc/0x260
[ 1235.904405] ? ksys_write+0x116/0x250
[ 1235.909316] vfs_write+0x657/0x9b0
[ 1235.913930] ksys_write+0x116/0x250
[ 1235.918644] ? __ia32_sys_read+0xc0/0xc0
[ 1235.923844] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.929433] ? rcu_is_watching+0x15/0xd0
[ 1235.934634] do_syscall_64+0xf3/0x650
[ 1235.939530] ? trace_hardirqs_off+0xd/0x30
[ 1235.944923] ? exc_page_fault+0xda/0xf0
[ 1235.950029] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.956495] RIP: 0033:0x7fb7022f464e
[ 1235.961306] RSP: 002b:00007fff80a36c70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.970603] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb7022f464e
[ 1235.979411] RDX: 0000000000000001 RSI: 000056419e31ebf0 RDI: 0000000000000007
[ 1235.988221] RBP: 00007fff80a36c80 R08: 0000000000000000 R09: 0000000000000000
[ 1235.997028] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb70216d390
[ 1236.005836] R13: 000056419e31ebf0 R14: 0000000000000007 R15: 00007fb702101120
[ 1236.014650]
[ 1236.017919] \x0aShowing all locks held in the system:
[ 1236.026459] 1 lock held by khungtaskd/185:
[ 1236.031851] #0: ffffffff984fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.043323] 2 locks held by systemd-journal/805:
[ 1236.049311] 4 locks held by python3/3543:
[ 1236.054605] #0: ff11000130e6c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.064495] #1: ff1100015d5cca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.075456] #2: ff11000123544b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.086611] #3: ff11000123619e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.096710] 4 locks held by python3/5539:
[ 1236.102003] #0: ff11000130e6c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.111894] #1: ff110001868f5c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.122858] #2: ff11000123544b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.134015] #3: ff11000123619e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.144098]
[ 1236.146563] =============================================\x0a
[ 1358.346982] INFO: task python3:5539 blocked for more than 491 seconds.
[ 1358.355129] Not tainted 7.1.0-rc7-gzzc-g74e32d746f94 #1
[ 1358.362380] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.371968] task:python3 state:D stack:25672 pid:5539 tgid:5539 ppid:5538 task_flags:0x400100 flags:0x00080002
[ 1358.385170] Call Trace:
[ 1358.388710]
[ 1358.391858] __schedule+0xcc3/0x19c0
[ 1358.396672] ? io_schedule_timeout+0x140/0x140
[ 1358.402446] ? find_held_lock+0x2b/0x80
[ 1358.407548] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.413447] schedule+0xd8/0x260
[ 1358.417868] schedule_preempt_disabled+0x18/0x30
[ 1358.423846] __mutex_lock+0x6af/0x2010
[ 1358.428850] ? threaded_store+0x132/0x220
[ 1358.434148] ? ww_mutex_lock+0x160/0x160
[ 1358.439344] ? selinux_inode_free_security+0x330/0x330
[ 1358.445913] ? cap_capable+0x1d6/0x3e0
[ 1358.450925] ? threaded_store+0x132/0x220
[ 1358.456227] threaded_store+0x132/0x220
[ 1358.461331] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.466625] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.471922] ? sysfs_kf_write+0x65/0x160
[ 1358.477120] ? sysfs_file_ops+0x140/0x140
[ 1358.482414] kernfs_fop_write_iter+0x335/0x510
[ 1358.488198] ? kernfs_ops+0xf0/0xf0
[ 1358.492910] new_sync_write+0x333/0x750
[ 1358.498012] ? __lock_acquire+0x508/0xc10
[ 1358.503308] ? new_sync_read+0x740/0x740
[ 1358.508508] ? lock_acquire.part.0+0xbc/0x260
[ 1358.514195] ? ksys_write+0x116/0x250
[ 1358.519104] vfs_write+0x657/0x9b0
[ 1358.523717] ksys_write+0x116/0x250
[ 1358.528429] ? __ia32_sys_read+0xc0/0xc0
[ 1358.533625] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.539213] ? rcu_is_watching+0x15/0xd0
[ 1358.544415] do_syscall_64+0xf3/0x650
[ 1358.549320] ? irq_exit_rcu+0x1a/0x30
[ 1358.554229] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.560694] RIP: 0033:0x7ff4821cd64e
[ 1358.565505] RSP: 002b:00007fff28d7e160 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.574800] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff4821cd64e
[ 1358.583608] RDX: 0000000000000001 RSI: 000055c4fa1f3bf0 RDI: 0000000000000007
[ 1358.592405] RBP: 00007fff28d7e170 R08: 0000000000000000 R09: 0000000000000000
[ 1358.601211] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff482046390
[ 1358.610020] R13: 000055c4fa1f3bf0 R14: 0000000000000007 R15: 00007ff481fd9120
[ 1358.618833]
[ 1358.622121] INFO: task python3:5539 is blocked on a mutex likely owned by task python3:3543.
[ 1358.632391] task:python3 state:D stack:25672 pid:3543 tgid:3543 ppid:3542 task_flags:0x400100 flags:0x00080802
[ 1358.645592] Call Trace:
[ 1358.649127]
[ 1358.652275] __schedule+0xcc3/0x19c0
[ 1358.657085] ? io_schedule_timeout+0x140/0x140
[ 1358.662868] ? find_held_lock+0x2b/0x80
[ 1358.667968] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.674148] schedule+0xd8/0x260
[ 1358.678562] schedule_timeout+0x13f/0x290
[ 1358.683860] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.690226] ? trace_contention_end+0xb3/0x1a0
[ 1358.696012] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.701700] ? ww_mutex_lock+0x160/0x160
[ 1358.706888] msleep+0x1b/0x30
[ 1358.711010] napi_set_threaded+0x3a5/0x440
[ 1358.716405] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.721900] netif_set_threaded+0x257/0x400
[ 1358.727393] threaded_store+0x1b8/0x220
[ 1358.732486] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.737781] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.743076] ? sysfs_kf_write+0x65/0x160
[ 1358.748274] ? sysfs_file_ops+0x140/0x140
[ 1358.753568] kernfs_fop_write_iter+0x335/0x510
[ 1358.759350] ? kernfs_ops+0xf0/0xf0
[ 1358.764059] new_sync_write+0x333/0x750
[ 1358.769160] ? __lock_acquire+0x508/0xc10
[ 1358.774453] ? new_sync_read+0x740/0x740
[ 1358.779652] ? lock_acquire.part.0+0xbc/0x260
[ 1358.785335] ? ksys_write+0x116/0x250
[ 1358.790242] vfs_write+0x657/0x9b0
[ 1358.794855] ksys_write+0x116/0x250
[ 1358.799565] ? __ia32_sys_read+0xc0/0xc0
[ 1358.804765] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.810353] ? rcu_is_watching+0x15/0xd0
[ 1358.815554] do_syscall_64+0xf3/0x650
[ 1358.820447] ? trace_hardirqs_off+0xd/0x30
[ 1358.825840] ? exc_page_fault+0xda/0xf0
[ 1358.830942] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.837408] RIP: 0033:0x7fb7022f464e
[ 1358.842215] RSP: 002b:00007fff80a36c70 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.851510] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb7022f464e
[ 1358.860318] RDX: 0000000000000001 RSI: 000056419e31ebf0 RDI: 0000000000000007
[ 1358.869126] RBP: 00007fff80a36c80 R08: 0000000000000000 R09: 0000000000000000
[ 1358.877934] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb70216d390
[ 1358.886731] R13: 000056419e31ebf0 R14: 0000000000000007 R15: 00007fb702101120
[ 1358.895543]
[ 1358.898814] \x0aShowing all locks held in the system:
[ 1358.907358] 1 lock held by khungtaskd/185:
[ 1358.912752] #0: ffffffff984fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.924232] 4 locks held by python3/3543:
[ 1358.929527] #0: ff11000130e6c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.939418] #1: ff1100015d5cca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.950378] #2: ff11000123544b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.961535] #3: ff11000123619e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.971634] 4 locks held by python3/5539:
[ 1358.976927] #0: ff11000130e6c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.986812] #1: ff110001868f5c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.997779] #2: ff11000123544b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.008937] #3: ff11000123619e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.019030]
[ 1359.021495] =============================================\x0a