[ 780.434611] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 780.437629] nipa-hw-worker: [5/43] Retrying drivers/net:napi_threaded.py
[ 780.593082] kselftest: Running tests in drivers/net
[ 989.701083] INFO: task python3:5723 blocked for more than 122 seconds.
[ 989.709284] Not tainted 7.1.0-rc1-rhjl-g59d195f5d845 #1
[ 989.716747] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.726339] task:python3 state:D stack:25672 pid:5723 tgid:5723 ppid:5722 task_flags:0x400100 flags:0x00080000
[ 989.739547] Call Trace:
[ 989.743088]
[ 989.746242] __schedule+0xcba/0x1940
[ 989.751054] ? io_schedule_timeout+0x140/0x140
[ 989.756840] ? find_held_lock+0x2b/0x80
[ 989.761948] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.767834] schedule+0xd4/0x260
[ 989.772255] schedule_preempt_disabled+0x18/0x30
[ 989.778236] __mutex_lock+0x6a3/0x2000
[ 989.783242] ? threaded_store+0x132/0x220
[ 989.788543] ? ww_mutex_lock+0x160/0x160
[ 989.793742] ? selinux_inode_free_security+0x330/0x330
[ 989.800315] ? cap_capable+0x1d6/0x3e0
[ 989.805326] ? threaded_store+0x132/0x220
[ 989.810624] threaded_store+0x132/0x220
[ 989.815726] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.821022] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.826323] ? sysfs_kf_write+0x65/0x160
[ 989.831521] ? sysfs_file_ops+0x140/0x140
[ 989.836816] kernfs_fop_write_iter+0x335/0x510
[ 989.842601] ? kernfs_ops+0xf0/0xf0
[ 989.847311] new_sync_write+0x333/0x750
[ 989.852415] ? __lock_acquire+0x508/0xc10
[ 989.857705] ? new_sync_read+0x740/0x740
[ 989.862906] ? lock_acquire.part.0+0xbc/0x260
[ 989.868596] ? ksys_write+0x116/0x250
[ 989.873508] vfs_write+0x657/0x9b0
[ 989.878125] ksys_write+0x116/0x250
[ 989.882837] ? __ia32_sys_read+0xc0/0xc0
[ 989.888038] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.893629] ? rcu_is_watching+0x15/0xd0
[ 989.898833] do_syscall_64+0xf3/0x650
[ 989.903740] ? trace_hardirqs_off+0xd/0x30
[ 989.909126] ? exc_page_fault+0xda/0xf0
[ 989.914229] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.920689] RIP: 0033:0x7fb3c396122e
[ 989.925498] RSP: 002b:00007ffcdfbdde40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.934795] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb3c396122e
[ 989.943603] RDX: 0000000000000001 RSI: 0000556448e6ba10 RDI: 0000000000000007
[ 989.952412] RBP: 00007ffcdfbdde50 R08: 0000000000000000 R09: 0000000000000000
[ 989.961221] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb3c37d9390
[ 989.970029] R13: 0000556448e6ba10 R14: 0000000000000007 R15: 0000000000000003
[ 989.978842]
[ 989.982119] INFO: task python3:5723 is blocked on a mutex likely owned by task python3:3637.
[ 989.992390] task:python3 state:D stack:25672 pid:3637 tgid:3637 ppid:3636 task_flags:0x400100 flags:0x00080802
[ 990.005594] Call Trace:
[ 990.009132]
[ 990.012284] __schedule+0xcba/0x1940
[ 990.017096] ? io_schedule_timeout+0x140/0x140
[ 990.022880] ? find_held_lock+0x2b/0x80
[ 990.027985] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.034162] schedule+0xd4/0x260
[ 990.038580] schedule_timeout+0x13f/0x290
[ 990.043875] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.050244] ? trace_contention_end+0xb3/0x1a0
[ 990.056030] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.061721] ? ww_mutex_lock+0x160/0x160
[ 990.066933] msleep+0x1b/0x30
[ 990.071059] napi_set_threaded+0x3a5/0x440
[ 990.076449] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.081946] netif_set_threaded+0x257/0x400
[ 990.087439] threaded_store+0x1b8/0x220
[ 990.092543] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.097839] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.103136] ? sysfs_kf_write+0x65/0x160
[ 990.108336] ? sysfs_file_ops+0x140/0x140
[ 990.113630] kernfs_fop_write_iter+0x335/0x510
[ 990.119416] ? kernfs_ops+0xf0/0xf0
[ 990.124127] new_sync_write+0x333/0x750
[ 990.129228] ? __lock_acquire+0x508/0xc10
[ 990.134528] ? new_sync_read+0x740/0x740
[ 990.139727] ? lock_acquire.part.0+0xbc/0x260
[ 990.145413] ? ksys_write+0x116/0x250
[ 990.150325] vfs_write+0x657/0x9b0
[ 990.154942] ksys_write+0x116/0x250
[ 990.159655] ? __ia32_sys_read+0xc0/0xc0
[ 990.164855] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.170445] ? rcu_is_watching+0x15/0xd0
[ 990.175646] do_syscall_64+0xf3/0x650
[ 990.180553] ? trace_hardirqs_off+0xd/0x30
[ 990.185947] ? exc_page_fault+0xda/0xf0
[ 990.191048] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.197516] RIP: 0033:0x7f3787a2a22e
[ 990.202324] RSP: 002b:00007ffc6c220470 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.211621] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3787a2a22e
[ 990.220429] RDX: 0000000000000001 RSI: 00005644f7f65a10 RDI: 0000000000000007
[ 990.229237] RBP: 00007ffc6c220480 R08: 0000000000000000 R09: 0000000000000000
[ 990.238036] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f37878a2390
[ 990.246843] R13: 00005644f7f65a10 R14: 0000000000000007 R15: 0000000000000003
[ 990.255657]
[ 990.258923] \x0aShowing all locks held in the system:
[ 990.267459] 1 lock held by khungtaskd/185:
[ 990.272854] #0: ffffffff928fdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.284330] 4 locks held by python3/3637:
[ 990.289624] #0: ff1100012fa743e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.299513] #1: ff110001687fd080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.310473] #2: ff11000122f16b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.321631] #3: ff11000122f71d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.331726] 4 locks held by python3/5723:
[ 990.337021] #0: ff1100012fa743e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.346910] #1: ff110001c7af5c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.357876] #2: ff11000122f16b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.369036] #3: ff11000122f71d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.379127]
[ 990.381594] =============================================\x0a
[ 1112.582349] INFO: task python3:5723 blocked for more than 245 seconds.
[ 1112.590496] Not tainted 7.1.0-rc1-rhjl-g59d195f5d845 #1
[ 1112.597739] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.607327] task:python3 state:D stack:25672 pid:5723 tgid:5723 ppid:5722 task_flags:0x400100 flags:0x00080000
[ 1112.620533] Call Trace:
[ 1112.624075]
[ 1112.627225] __schedule+0xcba/0x1940
[ 1112.632040] ? io_schedule_timeout+0x140/0x140
[ 1112.637829] ? find_held_lock+0x2b/0x80
[ 1112.642932] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.648810] schedule+0xd4/0x260
[ 1112.653228] schedule_preempt_disabled+0x18/0x30
[ 1112.659207] __mutex_lock+0x6a3/0x2000
[ 1112.664212] ? threaded_store+0x132/0x220
[ 1112.669512] ? ww_mutex_lock+0x160/0x160
[ 1112.674713] ? selinux_inode_free_security+0x330/0x330
[ 1112.681283] ? cap_capable+0x1d6/0x3e0
[ 1112.686303] ? threaded_store+0x132/0x220
[ 1112.691612] threaded_store+0x132/0x220
[ 1112.696714] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.702013] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.707310] ? sysfs_kf_write+0x65/0x160
[ 1112.712511] ? sysfs_file_ops+0x140/0x140
[ 1112.717808] kernfs_fop_write_iter+0x335/0x510
[ 1112.723591] ? kernfs_ops+0xf0/0xf0
[ 1112.728303] new_sync_write+0x333/0x750
[ 1112.733407] ? __lock_acquire+0x508/0xc10
[ 1112.738705] ? new_sync_read+0x740/0x740
[ 1112.743909] ? lock_acquire.part.0+0xbc/0x260
[ 1112.749596] ? ksys_write+0x116/0x250
[ 1112.754497] vfs_write+0x657/0x9b0
[ 1112.759115] ksys_write+0x116/0x250
[ 1112.763828] ? __ia32_sys_read+0xc0/0xc0
[ 1112.769021] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.774612] ? rcu_is_watching+0x15/0xd0
[ 1112.779816] do_syscall_64+0xf3/0x650
[ 1112.784726] ? trace_hardirqs_off+0xd/0x30
[ 1112.790122] ? exc_page_fault+0xda/0xf0
[ 1112.795226] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.801692] RIP: 0033:0x7fb3c396122e
[ 1112.806504] RSP: 002b:00007ffcdfbdde40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.815802] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb3c396122e
[ 1112.824612] RDX: 0000000000000001 RSI: 0000556448e6ba10 RDI: 0000000000000007
[ 1112.833421] RBP: 00007ffcdfbdde50 R08: 0000000000000000 R09: 0000000000000000
[ 1112.842229] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb3c37d9390
[ 1112.851040] R13: 0000556448e6ba10 R14: 0000000000000007 R15: 0000000000000003
[ 1112.859854]
[ 1112.863135] INFO: task python3:5723 is blocked on a mutex likely owned by task python3:3637.
[ 1112.873406] task:python3 state:D stack:25672 pid:3637 tgid:3637 ppid:3636 task_flags:0x400100 flags:0x00080802
[ 1112.886611] Call Trace:
[ 1112.890148]
[ 1112.893302] __schedule+0xcba/0x1940
[ 1112.898114] ? io_schedule_timeout+0x140/0x140
[ 1112.903899] ? find_held_lock+0x2b/0x80
[ 1112.908994] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.915178] schedule+0xd4/0x260
[ 1112.919594] schedule_timeout+0x13f/0x290
[ 1112.924890] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.931262] ? trace_contention_end+0xb3/0x1a0
[ 1112.937048] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.942740] ? ww_mutex_lock+0x160/0x160
[ 1112.947940] msleep+0x1b/0x30
[ 1112.952057] napi_set_threaded+0x3a5/0x440
[ 1112.957451] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.962945] netif_set_threaded+0x257/0x400
[ 1112.968439] threaded_store+0x1b8/0x220
[ 1112.973540] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.978836] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.984131] ? sysfs_kf_write+0x65/0x160
[ 1112.989332] ? sysfs_file_ops+0x140/0x140
[ 1112.994630] kernfs_fop_write_iter+0x335/0x510
[ 1113.000413] ? kernfs_ops+0xf0/0xf0
[ 1113.005126] new_sync_write+0x333/0x750
[ 1113.010220] ? __lock_acquire+0x508/0xc10
[ 1113.015517] ? new_sync_read+0x740/0x740
[ 1113.020719] ? lock_acquire.part.0+0xbc/0x260
[ 1113.026407] ? ksys_write+0x116/0x250
[ 1113.031315] vfs_write+0x657/0x9b0
[ 1113.035933] ksys_write+0x116/0x250
[ 1113.040646] ? __ia32_sys_read+0xc0/0xc0
[ 1113.045844] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.051434] ? rcu_is_watching+0x15/0xd0
[ 1113.056624] do_syscall_64+0xf3/0x650
[ 1113.061525] ? trace_hardirqs_off+0xd/0x30
[ 1113.066920] ? exc_page_fault+0xda/0xf0
[ 1113.072022] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.078490] RIP: 0033:0x7f3787a2a22e
[ 1113.083301] RSP: 002b:00007ffc6c220470 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.092598] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3787a2a22e
[ 1113.101407] RDX: 0000000000000001 RSI: 00005644f7f65a10 RDI: 0000000000000007
[ 1113.110214] RBP: 00007ffc6c220480 R08: 0000000000000000 R09: 0000000000000000
[ 1113.119024] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f37878a2390
[ 1113.127834] R13: 00005644f7f65a10 R14: 0000000000000007 R15: 0000000000000003
[ 1113.136648]
[ 1113.139914] \x0aShowing all locks held in the system:
[ 1113.148453] 1 lock held by khungtaskd/185:
[ 1113.153844] #0: ffffffff928fdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.165319] 2 locks held by systemd-journal/804:
[ 1113.171299] #0: ff1100180dd372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1113.181189] #1: ffffffff92b030a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1113.194018] 4 locks held by python3/3637:
[ 1113.199312] #0: ff1100012fa743e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.209204] #1: ff110001687fd080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.220164] #2: ff11000122f16b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.231323] #3: ff11000122f71d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.241417] 4 locks held by python3/5723:
[ 1113.246714] #0: ff1100012fa743e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.256602] #1: ff110001c7af5c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.267565] #2: ff11000122f16b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.278723] #3: ff11000122f71d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.288814]
[ 1113.291280] =============================================\x0a
[ 1235.463525] INFO: task python3:5723 blocked for more than 368 seconds.
[ 1235.471676] Not tainted 7.1.0-rc1-rhjl-g59d195f5d845 #1
[ 1235.478918] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.488504] task:python3 state:D stack:25672 pid:5723 tgid:5723 ppid:5722 task_flags:0x400100 flags:0x00080002
[ 1235.501708] Call Trace:
[ 1235.505251]
[ 1235.508401] __schedule+0xcba/0x1940
[ 1235.513221] ? io_schedule_timeout+0x140/0x140
[ 1235.519008] ? find_held_lock+0x2b/0x80
[ 1235.524115] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.530001] schedule+0xd4/0x260
[ 1235.534419] schedule_preempt_disabled+0x18/0x30
[ 1235.540397] __mutex_lock+0x6a3/0x2000
[ 1235.545401] ? threaded_store+0x132/0x220
[ 1235.550700] ? ww_mutex_lock+0x160/0x160
[ 1235.555900] ? selinux_inode_free_security+0x330/0x330
[ 1235.562471] ? cap_capable+0x1d6/0x3e0
[ 1235.567482] ? threaded_store+0x132/0x220
[ 1235.572782] threaded_store+0x132/0x220
[ 1235.577882] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.583177] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.588474] ? sysfs_kf_write+0x65/0x160
[ 1235.593672] ? sysfs_file_ops+0x140/0x140
[ 1235.598966] kernfs_fop_write_iter+0x335/0x510
[ 1235.604753] ? kernfs_ops+0xf0/0xf0
[ 1235.609466] new_sync_write+0x333/0x750
[ 1235.614572] ? __lock_acquire+0x508/0xc10
[ 1235.619869] ? new_sync_read+0x740/0x740
[ 1235.625073] ? lock_acquire.part.0+0xbc/0x260
[ 1235.630759] ? ksys_write+0x116/0x250
[ 1235.635669] vfs_write+0x657/0x9b0
[ 1235.640282] ksys_write+0x116/0x250
[ 1235.644995] ? __ia32_sys_read+0xc0/0xc0
[ 1235.650196] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.655787] ? rcu_is_watching+0x15/0xd0
[ 1235.660987] do_syscall_64+0xf3/0x650
[ 1235.665897] ? trace_hardirqs_off+0xd/0x30
[ 1235.671293] ? exc_page_fault+0xda/0xf0
[ 1235.676398] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.682868] RIP: 0033:0x7fb3c396122e
[ 1235.687679] RSP: 002b:00007ffcdfbdde40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.696978] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb3c396122e
[ 1235.705788] RDX: 0000000000000001 RSI: 0000556448e6ba10 RDI: 0000000000000007
[ 1235.714598] RBP: 00007ffcdfbdde50 R08: 0000000000000000 R09: 0000000000000000
[ 1235.723405] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb3c37d9390
[ 1235.732206] R13: 0000556448e6ba10 R14: 0000000000000007 R15: 0000000000000003
[ 1235.741020]
[ 1235.744296] INFO: task python3:5723 is blocked on a mutex likely owned by task python3:3637.
[ 1235.754567] task:python3 state:D stack:25672 pid:3637 tgid:3637 ppid:3636 task_flags:0x400100 flags:0x00080802
[ 1235.767770] Call Trace:
[ 1235.771309]
[ 1235.774457] __schedule+0xcba/0x1940
[ 1235.779270] ? io_schedule_timeout+0x140/0x140
[ 1235.785056] ? find_held_lock+0x2b/0x80
[ 1235.790160] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.796337] schedule+0xd4/0x260
[ 1235.800757] schedule_timeout+0x13f/0x290
[ 1235.806053] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.812423] ? trace_contention_end+0xb3/0x1a0
[ 1235.818210] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.823901] ? ww_mutex_lock+0x160/0x160
[ 1235.829102] msleep+0x1b/0x30
[ 1235.833218] napi_set_threaded+0x3a5/0x440
[ 1235.838613] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.844107] netif_set_threaded+0x257/0x400
[ 1235.849600] threaded_store+0x1b8/0x220
[ 1235.854702] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.860000] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.865295] ? sysfs_kf_write+0x65/0x160
[ 1235.870495] ? sysfs_file_ops+0x140/0x140
[ 1235.875792] kernfs_fop_write_iter+0x335/0x510
[ 1235.881575] ? kernfs_ops+0xf0/0xf0
[ 1235.886287] new_sync_write+0x333/0x750
[ 1235.891390] ? __lock_acquire+0x508/0xc10
[ 1235.896687] ? new_sync_read+0x740/0x740
[ 1235.901888] ? lock_acquire.part.0+0xbc/0x260
[ 1235.907576] ? ksys_write+0x116/0x250
[ 1235.912476] vfs_write+0x657/0x9b0
[ 1235.917090] ksys_write+0x116/0x250
[ 1235.921804] ? __ia32_sys_read+0xc0/0xc0
[ 1235.927001] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.932588] ? rcu_is_watching+0x15/0xd0
[ 1235.937787] do_syscall_64+0xf3/0x650
[ 1235.942684] ? trace_hardirqs_off+0xd/0x30
[ 1235.948079] ? exc_page_fault+0xda/0xf0
[ 1235.953182] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.959642] RIP: 0033:0x7f3787a2a22e
[ 1235.964448] RSP: 002b:00007ffc6c220470 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.973744] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3787a2a22e
[ 1235.982554] RDX: 0000000000000001 RSI: 00005644f7f65a10 RDI: 0000000000000007
[ 1235.991364] RBP: 00007ffc6c220480 R08: 0000000000000000 R09: 0000000000000000
[ 1236.000174] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f37878a2390
[ 1236.008983] R13: 00005644f7f65a10 R14: 0000000000000007 R15: 0000000000000003
[ 1236.017798]
[ 1236.021062] \x0aShowing all locks held in the system:
[ 1236.029600] 1 lock held by khungtaskd/185:
[ 1236.034993] #0: ffffffff928fdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.046464] 2 locks held by systemd-journal/804:
[ 1236.052446] #0: ff1100180e5372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1236.062327] #1: ffffffff92b030a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1236.075154] 4 locks held by python3/3637:
[ 1236.080449] #0: ff1100012fa743e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.090340] #1: ff110001687fd080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.101308] #2: ff11000122f16b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.112463] #3: ff11000122f71d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.122556] 4 locks held by python3/5723:
[ 1236.127852] #0: ff1100012fa743e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.137743] #1: ff110001c7af5c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.148702] #2: ff11000122f16b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.159860] #3: ff11000122f71d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.169950]
[ 1236.172416] =============================================\x0a
[ 1358.344473] INFO: task python3:5723 blocked for more than 491 seconds.
[ 1358.352623] Not tainted 7.1.0-rc1-rhjl-g59d195f5d845 #1
[ 1358.359874] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.369461] task:python3 state:D stack:25672 pid:5723 tgid:5723 ppid:5722 task_flags:0x400100 flags:0x00080002
[ 1358.382665] Call Trace:
[ 1358.386205]
[ 1358.389353] __schedule+0xcba/0x1940
[ 1358.394167] ? io_schedule_timeout+0x140/0x140
[ 1358.399954] ? find_held_lock+0x2b/0x80
[ 1358.405056] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.410941] schedule+0xd4/0x260
[ 1358.415358] schedule_preempt_disabled+0x18/0x30
[ 1358.421336] __mutex_lock+0x6a3/0x2000
[ 1358.426339] ? threaded_store+0x132/0x220
[ 1358.431637] ? ww_mutex_lock+0x160/0x160
[ 1358.436833] ? selinux_inode_free_security+0x330/0x330
[ 1358.443404] ? cap_capable+0x1d6/0x3e0
[ 1358.448409] ? threaded_store+0x132/0x220
[ 1358.453703] threaded_store+0x132/0x220
[ 1358.458804] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.464097] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.469396] ? sysfs_kf_write+0x65/0x160
[ 1358.474595] ? sysfs_file_ops+0x140/0x140
[ 1358.479891] kernfs_fop_write_iter+0x335/0x510
[ 1358.485674] ? kernfs_ops+0xf0/0xf0
[ 1358.490386] new_sync_write+0x333/0x750
[ 1358.495487] ? __lock_acquire+0x508/0xc10
[ 1358.500785] ? new_sync_read+0x740/0x740
[ 1358.505987] ? lock_acquire.part.0+0xbc/0x260
[ 1358.511674] ? ksys_write+0x116/0x250
[ 1358.516575] vfs_write+0x657/0x9b0
[ 1358.521189] ksys_write+0x116/0x250
[ 1358.525897] ? __ia32_sys_read+0xc0/0xc0
[ 1358.531096] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.536683] ? rcu_is_watching+0x15/0xd0
[ 1358.541884] do_syscall_64+0xf3/0x650
[ 1358.546791] ? trace_hardirqs_off+0xd/0x30
[ 1358.552183] ? exc_page_fault+0xda/0xf0
[ 1358.557286] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.563743] RIP: 0033:0x7fb3c396122e
[ 1358.568550] RSP: 002b:00007ffcdfbdde40 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.577846] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fb3c396122e
[ 1358.586653] RDX: 0000000000000001 RSI: 0000556448e6ba10 RDI: 0000000000000007
[ 1358.595459] RBP: 00007ffcdfbdde50 R08: 0000000000000000 R09: 0000000000000000
[ 1358.604267] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fb3c37d9390
[ 1358.613065] R13: 0000556448e6ba10 R14: 0000000000000007 R15: 0000000000000003
[ 1358.621867]
[ 1358.625157] INFO: task python3:5723 is blocked on a mutex likely owned by task python3:3637.
[ 1358.635429] task:python3 state:D stack:25672 pid:3637 tgid:3637 ppid:3636 task_flags:0x400100 flags:0x00080802
[ 1358.648630] Call Trace:
[ 1358.652165]
[ 1358.655313] __schedule+0xcba/0x1940
[ 1358.660348] ? io_schedule_timeout+0x140/0x140
[ 1358.666132] ? find_held_lock+0x2b/0x80
[ 1358.671233] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.677413] schedule+0xd4/0x260
[ 1358.681829] schedule_timeout+0x13f/0x290
[ 1358.687122] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.693488] ? trace_contention_end+0xb3/0x1a0
[ 1358.699271] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.704960] ? ww_mutex_lock+0x160/0x160
[ 1358.710157] msleep+0x1b/0x30
[ 1358.714279] napi_set_threaded+0x3a5/0x440
[ 1358.719676] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.725171] netif_set_threaded+0x257/0x400
[ 1358.730662] threaded_store+0x1b8/0x220
[ 1358.735763] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.741058] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.746355] ? sysfs_kf_write+0x65/0x160
[ 1358.751552] ? sysfs_file_ops+0x140/0x140
[ 1358.756847] kernfs_fop_write_iter+0x335/0x510
[ 1358.762629] ? kernfs_ops+0xf0/0xf0
[ 1358.767340] new_sync_write+0x333/0x750
[ 1358.772442] ? __lock_acquire+0x508/0xc10
[ 1358.777736] ? new_sync_read+0x740/0x740
[ 1358.782934] ? lock_acquire.part.0+0xbc/0x260
[ 1358.788618] ? ksys_write+0x116/0x250
[ 1358.793528] vfs_write+0x657/0x9b0
[ 1358.798142] ksys_write+0x116/0x250
[ 1358.802852] ? __ia32_sys_read+0xc0/0xc0
[ 1358.808051] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.813638] ? rcu_is_watching+0x15/0xd0
[ 1358.818839] do_syscall_64+0xf3/0x650
[ 1358.823746] ? trace_hardirqs_off+0xd/0x30
[ 1358.829137] ? exc_page_fault+0xda/0xf0
[ 1358.834237] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.840702] RIP: 0033:0x7f3787a2a22e
[ 1358.845510] RSP: 002b:00007ffc6c220470 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.854806] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f3787a2a22e
[ 1358.863613] RDX: 0000000000000001 RSI: 00005644f7f65a10 RDI: 0000000000000007
[ 1358.872420] RBP: 00007ffc6c220480 R08: 0000000000000000 R09: 0000000000000000
[ 1358.881218] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f37878a2390
[ 1358.890024] R13: 00005644f7f65a10 R14: 0000000000000007 R15: 0000000000000003
[ 1358.898836]
[ 1358.902106] \x0aShowing all locks held in the system:
[ 1358.910645] 1 lock held by khungtaskd/185:
[ 1358.916039] #0: ffffffff928fdee0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.927516] 3 locks held by systemd-journal/804:
[ 1358.933493] #0: ff1100180df372e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1358.943381] #1: ffffffff92b030a0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.956187] #2: ff110001cf7e5360 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.967747] 4 locks held by python3/3637:
[ 1358.973040] #0: ff1100012fa743e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.982927] #1: ff110001687fd080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.993886] #2: ff11000122f16b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.005038] #3: ff11000122f71d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.015133] 4 locks held by python3/5723:
[ 1359.020428] #0: ff1100012fa743e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.030317] #1: ff110001c7af5c80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.041284] #2: ff11000122f16b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.052439] #3: ff11000122f71d60 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.062530] 7 locks held by sshd-auth/7579:
[ 1359.068022] #0: ff1100180dbb72e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x1940
[ 1359.077910] #1: ffffffff928fdee0 (rcu_read_lock){....}-{1:3}, at: __update_idle_core+0x5a/0x700
[ 1359.088578] #2: ffffffff928fdee0 (rcu_read_lock){....}-{1:3}, at: filemap_map_pages+0x1fc/0xf50
[ 1359.099244] #3: ffffffff928fdee0 (rcu_read_lock){....}-{1:3}, at: __pte_offset_map+0x2e/0x220
[ 1359.109719] #4: ff1100013c507d00 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1359.121264] #5: ff1100013494a228 (dev->qdisc_tx_busylock ?: &qdisc_tx_busylock){+...}-{3:3}, at: __dev_xmit_skb+0xcba/0x1c20
[ 1359.134761] #6: ff11000122ae4698 (_xmit_ETHER){+.-.}-{3:3}, at: sch_direct_xmit+0x2c1/0xaa0
[ 1359.145046]
[ 1359.147514] =============================================\x0a