[ 767.645194] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 767.647751] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 767.800037] kselftest: Running tests in drivers/net
[ 989.693105] INFO: task python3:5544 blocked for more than 122 seconds.
[ 989.701338] Not tainted 7.1.0-rc7-yera-gaac1facfbfd5 #1
[ 989.708591] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.718836] task:python3 state:D stack:25672 pid:5544 tgid:5544 ppid:5543 task_flags:0x400100 flags:0x00080000
[ 989.732044] Call Trace:
[ 989.735583]
[ 989.738734] __schedule+0xcc3/0x19c0
[ 989.743552] ? io_schedule_timeout+0x140/0x140
[ 989.749336] ? find_held_lock+0x2b/0x80
[ 989.754442] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.760326] schedule+0xd8/0x260
[ 989.764744] schedule_preempt_disabled+0x18/0x30
[ 989.770796] __mutex_lock+0x6af/0x2010
[ 989.775804] ? threaded_store+0x132/0x220
[ 989.781103] ? ww_mutex_lock+0x160/0x160
[ 989.786292] ? selinux_inode_free_security+0x330/0x330
[ 989.792864] ? cap_capable+0x1d6/0x3e0
[ 989.797874] ? threaded_store+0x132/0x220
[ 989.803172] threaded_store+0x132/0x220
[ 989.808270] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.813568] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.818866] ? sysfs_kf_write+0x65/0x160
[ 989.824065] ? sysfs_file_ops+0x140/0x140
[ 989.829363] kernfs_fop_write_iter+0x335/0x510
[ 989.835214] ? kernfs_ops+0xf0/0xf0
[ 989.839930] new_sync_write+0x333/0x750
[ 989.845033] ? __lock_acquire+0x508/0xc10
[ 989.850331] ? new_sync_read+0x740/0x740
[ 989.855534] ? lock_acquire.part.0+0xbc/0x260
[ 989.861223] ? ksys_write+0x116/0x250
[ 989.866136] vfs_write+0x657/0x9b0
[ 989.870753] ksys_write+0x116/0x250
[ 989.875465] ? __ia32_sys_read+0xc0/0xc0
[ 989.880664] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.886322] ? rcu_is_watching+0x15/0xd0
[ 989.891526] do_syscall_64+0xf3/0x650
[ 989.896432] ? trace_hardirqs_off+0xd/0x30
[ 989.901828] ? exc_page_fault+0xda/0xf0
[ 989.906932] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.913401] RIP: 0033:0x7faa799c564e
[ 989.918212] RSP: 002b:00007ffdd3e0a9a0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.927512] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007faa799c564e
[ 989.936320] RDX: 0000000000000001 RSI: 0000560f498cfbf0 RDI: 0000000000000007
[ 989.945119] RBP: 00007ffdd3e0a9b0 R08: 0000000000000000 R09: 0000000000000000
[ 989.953925] R10: 0000000000000000 R11: 0000000000000202 R12: 00007faa7983e390
[ 989.962733] R13: 0000560f498cfbf0 R14: 0000000000000007 R15: 00007faa797d1120
[ 989.971554]
[ 989.974865] INFO: task python3:5544 is blocked on a mutex likely owned by task python3:3549.
[ 989.985140] task:python3 state:D stack:25632 pid:3549 tgid:3549 ppid:3548 task_flags:0x400100 flags:0x00080802
[ 989.998335] Call Trace:
[ 990.001875]
[ 990.005025] __schedule+0xcc3/0x19c0
[ 990.009840] ? io_schedule_timeout+0x140/0x140
[ 990.015623] ? find_held_lock+0x2b/0x80
[ 990.020727] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.026908] schedule+0xd8/0x260
[ 990.031329] schedule_timeout+0x13f/0x290
[ 990.036625] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.042995] ? trace_contention_end+0xb3/0x1a0
[ 990.048783] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.054474] ? ww_mutex_lock+0x160/0x160
[ 990.059675] msleep+0x1b/0x30
[ 990.063793] napi_set_threaded+0x3a5/0x440
[ 990.069190] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.074687] netif_set_threaded+0x257/0x400
[ 990.080184] threaded_store+0x1b8/0x220
[ 990.085290] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.090587] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.095885] ? sysfs_kf_write+0x65/0x160
[ 990.101087] ? sysfs_file_ops+0x140/0x140
[ 990.106384] kernfs_fop_write_iter+0x335/0x510
[ 990.112171] ? kernfs_ops+0xf0/0xf0
[ 990.116882] new_sync_write+0x333/0x750
[ 990.121983] ? __lock_acquire+0x508/0xc10
[ 990.127278] ? new_sync_read+0x740/0x740
[ 990.132479] ? lock_acquire.part.0+0xbc/0x260
[ 990.138163] ? ksys_write+0x116/0x250
[ 990.143072] vfs_write+0x657/0x9b0
[ 990.147686] ksys_write+0x116/0x250
[ 990.152398] ? __ia32_sys_read+0xc0/0xc0
[ 990.157598] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.163189] ? rcu_is_watching+0x15/0xd0
[ 990.168389] do_syscall_64+0xf3/0x650
[ 990.173297] ? trace_hardirqs_off+0xd/0x30
[ 990.178690] ? exc_page_fault+0xda/0xf0
[ 990.183793] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.190251] RIP: 0033:0x7f673fd6764e
[ 990.195058] RSP: 002b:00007fff14fa4a90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.204356] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f673fd6764e
[ 990.213164] RDX: 0000000000000001 RSI: 000055fe42cdfbf0 RDI: 0000000000000007
[ 990.221977] RBP: 00007fff14fa4aa0 R08: 0000000000000000 R09: 0000000000000000
[ 990.230785] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f673fbe0390
[ 990.239593] R13: 000055fe42cdfbf0 R14: 0000000000000007 R15: 00007f673fb75120
[ 990.248398]
[ 990.251661] \x0aShowing all locks held in the system:
[ 990.260202] 1 lock held by khungtaskd/185:
[ 990.265597] #0: ffffffffac4fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.277066] 2 locks held by systemd-journal/800:
[ 990.283047] #0: ff1100180dd374e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 990.292937] #1: ffffffffac702ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: link_path_walk+0x9a7/0x1830
[ 990.305567] 4 locks held by python3/3549:
[ 990.310864] #0: ff1100011d61c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.320752] #1: ff11000116243880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.331715] #2: ff11000121e78b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.342875] #3: ff11000121f09e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.352978] 4 locks held by python3/5544:
[ 990.358275] #0: ff1100011d61c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.368165] #1: ff110001bb3d2080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.379128] #2: ff11000121e78b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.390284] #3: ff11000121f09e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.400372] 5 locks held by systemd/6218:
[ 990.405668] #0: ff1100012fa6c3e8 (sb_writers#6){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.415554] #1: ff110001a42bca80 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.426518] #2: ffffffffac569c20 (cgroup_mutex){+.+.}-{4:4}, at: cgroup_lock_and_drain_offline+0xa6/0x640
[ 990.438165] #3: ffffffffac3687b0 (cpu_hotplug_lock){++++}-{0:0}, at: cgroup_update_dfl_csses+0x2a3/0x5c0
[ 990.449715] #4: ffffffffac569990 (cgroup_threadgroup_rwsem){++++}-{0:0}, at: cgroup_update_dfl_csses+0x2af/0x5c0
[ 990.462044]
[ 990.464512] =============================================\x0a
[ 1112.575241] INFO: task python3:5544 blocked for more than 245 seconds.
[ 1112.583391] Not tainted 7.1.0-rc7-yera-gaac1facfbfd5 #1
[ 1112.590642] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.600231] task:python3 state:D stack:25672 pid:5544 tgid:5544 ppid:5543 task_flags:0x400100 flags:0x00080000
[ 1112.613429] Call Trace:
[ 1112.616966]
[ 1112.620113] __schedule+0xcc3/0x19c0
[ 1112.624929] ? io_schedule_timeout+0x140/0x140
[ 1112.630716] ? find_held_lock+0x2b/0x80
[ 1112.635820] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.641703] schedule+0xd8/0x260
[ 1112.646109] schedule_preempt_disabled+0x18/0x30
[ 1112.652089] __mutex_lock+0x6af/0x2010
[ 1112.657083] ? threaded_store+0x132/0x220
[ 1112.662382] ? ww_mutex_lock+0x160/0x160
[ 1112.667580] ? selinux_inode_free_security+0x330/0x330
[ 1112.674150] ? cap_capable+0x1d6/0x3e0
[ 1112.679157] ? threaded_store+0x132/0x220
[ 1112.684454] threaded_store+0x132/0x220
[ 1112.689547] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.694844] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.700139] ? sysfs_kf_write+0x65/0x160
[ 1112.705339] ? sysfs_file_ops+0x140/0x140
[ 1112.710636] kernfs_fop_write_iter+0x335/0x510
[ 1112.716419] ? kernfs_ops+0xf0/0xf0
[ 1112.721131] new_sync_write+0x333/0x750
[ 1112.726236] ? __lock_acquire+0x508/0xc10
[ 1112.731532] ? new_sync_read+0x740/0x740
[ 1112.736733] ? lock_acquire.part.0+0xbc/0x260
[ 1112.742418] ? ksys_write+0x116/0x250
[ 1112.747327] vfs_write+0x657/0x9b0
[ 1112.751942] ksys_write+0x116/0x250
[ 1112.756641] ? __ia32_sys_read+0xc0/0xc0
[ 1112.761838] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.767425] ? rcu_is_watching+0x15/0xd0
[ 1112.772626] do_syscall_64+0xf3/0x650
[ 1112.777530] ? trace_hardirqs_off+0xd/0x30
[ 1112.782925] ? exc_page_fault+0xda/0xf0
[ 1112.788028] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.794493] RIP: 0033:0x7faa799c564e
[ 1112.799300] RSP: 002b:00007ffdd3e0a9a0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.808598] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007faa799c564e
[ 1112.817405] RDX: 0000000000000001 RSI: 0000560f498cfbf0 RDI: 0000000000000007
[ 1112.826211] RBP: 00007ffdd3e0a9b0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.835016] R10: 0000000000000000 R11: 0000000000000202 R12: 00007faa7983e390
[ 1112.843824] R13: 0000560f498cfbf0 R14: 0000000000000007 R15: 00007faa797d1120
[ 1112.852635]
[ 1112.855908] INFO: task python3:5544 is blocked on a mutex likely owned by task python3:3549.
[ 1112.866181] task:python3 state:D stack:25632 pid:3549 tgid:3549 ppid:3548 task_flags:0x400100 flags:0x00080802
[ 1112.879381] Call Trace:
[ 1112.882909]
[ 1112.886058] __schedule+0xcc3/0x19c0
[ 1112.890870] ? io_schedule_timeout+0x140/0x140
[ 1112.896652] ? find_held_lock+0x2b/0x80
[ 1112.901752] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.907928] schedule+0xd8/0x260
[ 1112.912346] schedule_timeout+0x13f/0x290
[ 1112.917644] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.924014] ? trace_contention_end+0xb3/0x1a0
[ 1112.929797] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.935485] ? ww_mutex_lock+0x160/0x160
[ 1112.940682] msleep+0x1b/0x30
[ 1112.944808] napi_set_threaded+0x3a5/0x440
[ 1112.950203] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.955699] netif_set_threaded+0x257/0x400
[ 1112.961189] threaded_store+0x1b8/0x220
[ 1112.966291] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.971590] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.976886] ? sysfs_kf_write+0x65/0x160
[ 1112.982086] ? sysfs_file_ops+0x140/0x140
[ 1112.987379] kernfs_fop_write_iter+0x335/0x510
[ 1112.993162] ? kernfs_ops+0xf0/0xf0
[ 1112.997871] new_sync_write+0x333/0x750
[ 1113.002972] ? __lock_acquire+0x508/0xc10
[ 1113.008265] ? new_sync_read+0x740/0x740
[ 1113.013463] ? lock_acquire.part.0+0xbc/0x260
[ 1113.019145] ? ksys_write+0x116/0x250
[ 1113.024056] vfs_write+0x657/0x9b0
[ 1113.028668] ksys_write+0x116/0x250
[ 1113.033377] ? __ia32_sys_read+0xc0/0xc0
[ 1113.038575] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.044162] ? rcu_is_watching+0x15/0xd0
[ 1113.049363] do_syscall_64+0xf3/0x650
[ 1113.054267] ? trace_hardirqs_off+0xd/0x30
[ 1113.059659] ? exc_page_fault+0xda/0xf0
[ 1113.064758] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.071223] RIP: 0033:0x7f673fd6764e
[ 1113.076028] RSP: 002b:00007fff14fa4a90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.085324] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f673fd6764e
[ 1113.094132] RDX: 0000000000000001 RSI: 000055fe42cdfbf0 RDI: 0000000000000007
[ 1113.102940] RBP: 00007fff14fa4aa0 R08: 0000000000000000 R09: 0000000000000000
[ 1113.111748] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f673fbe0390
[ 1113.120553] R13: 000055fe42cdfbf0 R14: 0000000000000007 R15: 00007f673fb75120
[ 1113.129366]
[ 1113.132629] \x0aShowing all locks held in the system:
[ 1113.141164] 1 lock held by khungtaskd/185:
[ 1113.146557] #0: ffffffffac4fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.158037] 4 locks held by python3/3549:
[ 1113.163333] #0: ff1100011d61c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.173224] #1: ff11000116243880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.184186] #2: ff11000121e78b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.195343] #3: ff11000121f09e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.205429] 4 locks held by python3/5544:
[ 1113.210721] #0: ff1100011d61c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.220610] #1: ff110001bb3d2080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.231560] #2: ff11000121e78b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.242718] #3: ff11000121f09e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.252809]
[ 1113.255273] =============================================\x0a
[ 1235.456383] INFO: task python3:5544 blocked for more than 368 seconds.
[ 1235.464531] Not tainted 7.1.0-rc7-yera-gaac1facfbfd5 #1
[ 1235.471782] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.481372] task:python3 state:D stack:25672 pid:5544 tgid:5544 ppid:5543 task_flags:0x400100 flags:0x00080002
[ 1235.494579] Call Trace:
[ 1235.498118]
[ 1235.501268] __schedule+0xcc3/0x19c0
[ 1235.506084] ? io_schedule_timeout+0x140/0x140
[ 1235.511868] ? find_held_lock+0x2b/0x80
[ 1235.516973] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.522857] schedule+0xd8/0x260
[ 1235.527266] schedule_preempt_disabled+0x18/0x30
[ 1235.533246] __mutex_lock+0x6af/0x2010
[ 1235.538250] ? threaded_store+0x132/0x220
[ 1235.543542] ? ww_mutex_lock+0x160/0x160
[ 1235.548742] ? selinux_inode_free_security+0x330/0x330
[ 1235.555310] ? cap_capable+0x1d6/0x3e0
[ 1235.560316] ? threaded_store+0x132/0x220
[ 1235.565613] threaded_store+0x132/0x220
[ 1235.570713] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.576010] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.581310] ? sysfs_kf_write+0x65/0x160
[ 1235.586512] ? sysfs_file_ops+0x140/0x140
[ 1235.591808] kernfs_fop_write_iter+0x335/0x510
[ 1235.597591] ? kernfs_ops+0xf0/0xf0
[ 1235.602301] new_sync_write+0x333/0x750
[ 1235.607402] ? __lock_acquire+0x508/0xc10
[ 1235.612696] ? new_sync_read+0x740/0x740
[ 1235.617898] ? lock_acquire.part.0+0xbc/0x260
[ 1235.623584] ? ksys_write+0x116/0x250
[ 1235.628492] vfs_write+0x657/0x9b0
[ 1235.633098] ksys_write+0x116/0x250
[ 1235.637801] ? __ia32_sys_read+0xc0/0xc0
[ 1235.642997] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.648585] ? rcu_is_watching+0x15/0xd0
[ 1235.653785] do_syscall_64+0xf3/0x650
[ 1235.658689] ? trace_hardirqs_off+0xd/0x30
[ 1235.664082] ? exc_page_fault+0xda/0xf0
[ 1235.669184] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.675649] RIP: 0033:0x7faa799c564e
[ 1235.680460] RSP: 002b:00007ffdd3e0a9a0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.689756] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007faa799c564e
[ 1235.698562] RDX: 0000000000000001 RSI: 0000560f498cfbf0 RDI: 0000000000000007
[ 1235.707368] RBP: 00007ffdd3e0a9b0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.716177] R10: 0000000000000000 R11: 0000000000000202 R12: 00007faa7983e390
[ 1235.724983] R13: 0000560f498cfbf0 R14: 0000000000000007 R15: 00007faa797d1120
[ 1235.733794]
[ 1235.737098] INFO: task python3:5544 is blocked on a mutex likely owned by task python3:3549.
[ 1235.747368] task:python3 state:D stack:25632 pid:3549 tgid:3549 ppid:3548 task_flags:0x400100 flags:0x00080802
[ 1235.760568] Call Trace:
[ 1235.764104]
[ 1235.767253] __schedule+0xcc3/0x19c0
[ 1235.772063] ? io_schedule_timeout+0x140/0x140
[ 1235.777838] ? find_held_lock+0x2b/0x80
[ 1235.782941] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.789118] schedule+0xd8/0x260
[ 1235.793535] schedule_timeout+0x13f/0x290
[ 1235.798832] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.805202] ? trace_contention_end+0xb3/0x1a0
[ 1235.810984] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.816672] ? ww_mutex_lock+0x160/0x160
[ 1235.821872] msleep+0x1b/0x30
[ 1235.825989] napi_set_threaded+0x3a5/0x440
[ 1235.831376] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.836870] netif_set_threaded+0x257/0x400
[ 1235.842363] threaded_store+0x1b8/0x220
[ 1235.847465] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.852760] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.858055] ? sysfs_kf_write+0x65/0x160
[ 1235.863251] ? sysfs_file_ops+0x140/0x140
[ 1235.868545] kernfs_fop_write_iter+0x335/0x510
[ 1235.874328] ? kernfs_ops+0xf0/0xf0
[ 1235.879039] new_sync_write+0x333/0x750
[ 1235.884142] ? __lock_acquire+0x508/0xc10
[ 1235.889435] ? new_sync_read+0x740/0x740
[ 1235.894633] ? lock_acquire.part.0+0xbc/0x260
[ 1235.900318] ? ksys_write+0x116/0x250
[ 1235.905233] vfs_write+0x657/0x9b0
[ 1235.909837] ksys_write+0x116/0x250
[ 1235.914548] ? __ia32_sys_read+0xc0/0xc0
[ 1235.919745] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.925332] ? rcu_is_watching+0x15/0xd0
[ 1235.930532] do_syscall_64+0xf3/0x650
[ 1235.935437] ? trace_hardirqs_off+0xd/0x30
[ 1235.940829] ? exc_page_fault+0xda/0xf0
[ 1235.945931] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.952396] RIP: 0033:0x7f673fd6764e
[ 1235.957201] RSP: 002b:00007fff14fa4a90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.966496] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f673fd6764e
[ 1235.975309] RDX: 0000000000000001 RSI: 000055fe42cdfbf0 RDI: 0000000000000007
[ 1235.984117] RBP: 00007fff14fa4aa0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.992924] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f673fbe0390
[ 1236.001729] R13: 000055fe42cdfbf0 R14: 0000000000000007 R15: 00007f673fb75120
[ 1236.010541]
[ 1236.013809] \x0aShowing all locks held in the system:
[ 1236.022348] 1 lock held by khungtaskd/185:
[ 1236.027742] #0: ffffffffac4fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.039212] 2 locks held by systemd-journal/800:
[ 1236.045189] #0: ff1100180e8b74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.055076] #1: ffffffffac702ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: dput.part.0+0x2fd/0x410
[ 1236.067305] 4 locks held by python3/3549:
[ 1236.072597] #0: ff1100011d61c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.082483] #1: ff11000116243880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.093444] #2: ff11000121e78b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.104600] #3: ff11000121f09e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.114697] 4 locks held by python3/5544:
[ 1236.119995] #0: ff1100011d61c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.129885] #1: ff110001bb3d2080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.140844] #2: ff11000121e78b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.152001] #3: ff11000121f09e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.162093]
[ 1236.164559] =============================================\x0a
[ 1292.800925] kworker/u129:10 (288) used greatest stack depth: 21936 bytes left
[ 1358.337528] INFO: task python3:5544 blocked for more than 491 seconds.
[ 1358.345679] Not tainted 7.1.0-rc7-yera-gaac1facfbfd5 #1
[ 1358.352931] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.362521] task:python3 state:D stack:25672 pid:5544 tgid:5544 ppid:5543 task_flags:0x400100 flags:0x00080002
[ 1358.375729] Call Trace:
[ 1358.379268]
[ 1358.382419] __schedule+0xcc3/0x19c0
[ 1358.387237] ? io_schedule_timeout+0x140/0x140
[ 1358.393023] ? find_held_lock+0x2b/0x80
[ 1358.398128] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.404014] schedule+0xd8/0x260
[ 1358.408434] schedule_preempt_disabled+0x18/0x30
[ 1358.414406] __mutex_lock+0x6af/0x2010
[ 1358.419411] ? threaded_store+0x132/0x220
[ 1358.424710] ? ww_mutex_lock+0x160/0x160
[ 1358.429910] ? selinux_inode_free_security+0x330/0x330
[ 1358.436479] ? cap_capable+0x1d6/0x3e0
[ 1358.441489] ? threaded_store+0x132/0x220
[ 1358.446791] threaded_store+0x132/0x220
[ 1358.451896] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.457193] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.462490] ? sysfs_kf_write+0x65/0x160
[ 1358.467692] ? sysfs_file_ops+0x140/0x140
[ 1358.472986] kernfs_fop_write_iter+0x335/0x510
[ 1358.478774] ? kernfs_ops+0xf0/0xf0
[ 1358.483489] new_sync_write+0x333/0x750
[ 1358.488595] ? __lock_acquire+0x508/0xc10
[ 1358.493891] ? new_sync_read+0x740/0x740
[ 1358.499086] ? lock_acquire.part.0+0xbc/0x260
[ 1358.504772] ? ksys_write+0x116/0x250
[ 1358.509684] vfs_write+0x657/0x9b0
[ 1358.514298] ksys_write+0x116/0x250
[ 1358.519008] ? __ia32_sys_read+0xc0/0xc0
[ 1358.524209] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.529800] ? rcu_is_watching+0x15/0xd0
[ 1358.535001] do_syscall_64+0xf3/0x650
[ 1358.539909] ? trace_hardirqs_off+0xd/0x30
[ 1358.545305] ? exc_page_fault+0xda/0xf0
[ 1358.550399] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.556861] RIP: 0033:0x7faa799c564e
[ 1358.561672] RSP: 002b:00007ffdd3e0a9a0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.570969] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007faa799c564e
[ 1358.579771] RDX: 0000000000000001 RSI: 0000560f498cfbf0 RDI: 0000000000000007
[ 1358.588577] RBP: 00007ffdd3e0a9b0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.597388] R10: 0000000000000000 R11: 0000000000000202 R12: 00007faa7983e390
[ 1358.606199] R13: 0000560f498cfbf0 R14: 0000000000000007 R15: 00007faa797d1120
[ 1358.615014]
[ 1358.618301] INFO: task python3:5544 is blocked on a mutex likely owned by task python3:3549.
[ 1358.628575] task:python3 state:D stack:25632 pid:3549 tgid:3549 ppid:3548 task_flags:0x400100 flags:0x00080802
[ 1358.641782] Call Trace:
[ 1358.645320]
[ 1358.648470] __schedule+0xcc3/0x19c0
[ 1358.653279] ? io_schedule_timeout+0x140/0x140
[ 1358.659062] ? find_held_lock+0x2b/0x80
[ 1358.664166] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.670345] schedule+0xd8/0x260
[ 1358.674764] schedule_timeout+0x13f/0x290
[ 1358.680060] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.686429] ? trace_contention_end+0xb3/0x1a0
[ 1358.692207] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.697896] ? ww_mutex_lock+0x160/0x160
[ 1358.703096] msleep+0x1b/0x30
[ 1358.707222] napi_set_threaded+0x3a5/0x440
[ 1358.712620] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.718114] netif_set_threaded+0x257/0x400
[ 1358.723608] threaded_store+0x1b8/0x220
[ 1358.728713] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.734012] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.739308] ? sysfs_kf_write+0x65/0x160
[ 1358.744510] ? sysfs_file_ops+0x140/0x140
[ 1358.749807] kernfs_fop_write_iter+0x335/0x510
[ 1358.755590] ? kernfs_ops+0xf0/0xf0
[ 1358.760303] new_sync_write+0x333/0x750
[ 1358.765407] ? __lock_acquire+0x508/0xc10
[ 1358.770702] ? new_sync_read+0x740/0x740
[ 1358.775904] ? lock_acquire.part.0+0xbc/0x260
[ 1358.781589] ? ksys_write+0x116/0x250
[ 1358.786502] vfs_write+0x657/0x9b0
[ 1358.791118] ksys_write+0x116/0x250
[ 1358.795831] ? __ia32_sys_read+0xc0/0xc0
[ 1358.801030] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.806621] ? rcu_is_watching+0x15/0xd0
[ 1358.811823] do_syscall_64+0xf3/0x650
[ 1358.816731] ? trace_hardirqs_off+0xd/0x30
[ 1358.822124] ? exc_page_fault+0xda/0xf0
[ 1358.827226] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.833693] RIP: 0033:0x7f673fd6764e
[ 1358.838504] RSP: 002b:00007fff14fa4a90 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.847801] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007f673fd6764e
[ 1358.856611] RDX: 0000000000000001 RSI: 000055fe42cdfbf0 RDI: 0000000000000007
[ 1358.865420] RBP: 00007fff14fa4aa0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.874231] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f673fbe0390
[ 1358.883039] R13: 000055fe42cdfbf0 R14: 0000000000000007 R15: 00007f673fb75120
[ 1358.891845]
[ 1358.895117] \x0aShowing all locks held in the system:
[ 1358.903656] 1 lock held by khungtaskd/185:
[ 1358.909052] #0: ffffffffac4fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.920526] 3 locks held by systemd-journal/800:
[ 1358.926505] #0: ff1100180ddb74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.936395] #1: ffffffffac702ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.949212] #2: ff110001a449e9c0 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.960773] 4 locks held by python3/3549:
[ 1358.966068] #0: ff1100011d61c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.975957] #1: ff11000116243880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.986919] #2: ff11000121e78b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.998076] #3: ff11000121f09e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.008171] 4 locks held by python3/5544:
[ 1359.013466] #0: ff1100011d61c3e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.023357] #1: ff110001bb3d2080 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.034319] #2: ff11000121e78b50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.045477] #3: ff11000121f09e00 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.055569]
[ 1359.058034] =============================================\x0a