[ 759.447467] nipa-hw-worker: drivers/net:napi_threaded.py: timed out
[ 759.450208] nipa-hw-worker: [5/47] Retrying drivers/net:napi_threaded.py
[ 759.604413] kselftest: Running tests in drivers/net
[ 810.683690] block nvme0n1: No UUID available providing old NGUID
SUBSYSTEM=block
DEVICE=b259:0
[ 989.687660] INFO: task python3:5317 blocked for more than 122 seconds.
[ 989.695884] Not tainted 7.1.0-rc6-qpbe-gb23588628fda #1
[ 989.703688] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 989.713278] task:python3 state:D stack:25672 pid:5317 tgid:5317 ppid:5316 task_flags:0x400100 flags:0x00080000
[ 989.726490] Call Trace:
[ 989.730028]
[ 989.733179] __schedule+0xcc3/0x19c0
[ 989.737994] ? io_schedule_timeout+0x140/0x140
[ 989.743776] ? find_held_lock+0x2b/0x80
[ 989.748882] ? __lock_release.isra.0+0x6b/0x1a0
[ 989.754764] schedule+0xd8/0x260
[ 989.759175] schedule_preempt_disabled+0x18/0x30
[ 989.765153] __mutex_lock+0x6af/0x2010
[ 989.770155] ? threaded_store+0x132/0x220
[ 989.775455] ? ww_mutex_lock+0x160/0x160
[ 989.780652] ? selinux_inode_free_security+0x330/0x330
[ 989.787221] ? cap_capable+0x1d6/0x3e0
[ 989.792236] ? threaded_store+0x132/0x220
[ 989.797572] threaded_store+0x132/0x220
[ 989.802674] ? xps_rxqs_store+0x2e0/0x2e0
[ 989.807970] ? sysfs_file_kobj+0xbd/0x1e0
[ 989.813257] ? sysfs_kf_write+0x65/0x160
[ 989.818455] ? sysfs_file_ops+0x140/0x140
[ 989.823750] kernfs_fop_write_iter+0x335/0x510
[ 989.829536] ? kernfs_ops+0xf0/0xf0
[ 989.834237] new_sync_write+0x333/0x750
[ 989.839339] ? __lock_acquire+0x508/0xc10
[ 989.844637] ? new_sync_read+0x740/0x740
[ 989.849836] ? lock_acquire.part.0+0xbc/0x260
[ 989.855522] ? ksys_write+0x116/0x250
[ 989.860429] vfs_write+0x657/0x9b0
[ 989.865044] ksys_write+0x116/0x250
[ 989.869755] ? __ia32_sys_read+0xc0/0xc0
[ 989.874952] ? kernfs_fop_llseek+0x160/0x1f0
[ 989.880542] ? rcu_is_watching+0x15/0xd0
[ 989.885740] do_syscall_64+0xf3/0x650
[ 989.890644] ? trace_hardirqs_off+0xd/0x30
[ 989.896040] ? exc_page_fault+0xda/0xf0
[ 989.901140] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 989.907605] RIP: 0033:0x7fd92a62a64e
[ 989.912414] RSP: 002b:00007ffee4aaa7d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 989.921712] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd92a62a64e
[ 989.930520] RDX: 0000000000000001 RSI: 000055fcdf6fc420 RDI: 0000000000000007
[ 989.939319] RBP: 00007ffee4aaa7e0 R08: 0000000000000000 R09: 0000000000000000
[ 989.948120] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd92a4a3390
[ 989.956928] R13: 000055fcdf6fc420 R14: 0000000000000007 R15: 00007fd92a439120
[ 989.965749]
[ 989.969061] INFO: task python3:5317 is blocked on a mutex likely owned by task python3:3338.
[ 989.979331] task:python3 state:D stack:25672 pid:3338 tgid:3338 ppid:3337 task_flags:0x400100 flags:0x00080802
[ 989.992533] Call Trace:
[ 989.996069]
[ 989.999219] __schedule+0xcc3/0x19c0
[ 990.004030] ? io_schedule_timeout+0x140/0x140
[ 990.009814] ? find_held_lock+0x2b/0x80
[ 990.014916] ? __try_to_del_timer_sync+0x155/0x1f0
[ 990.021092] schedule+0xd8/0x260
[ 990.025509] schedule_timeout+0x13f/0x290
[ 990.030805] ? hrtimer_nanosleep_restart+0x270/0x270
[ 990.037174] ? trace_contention_end+0xb3/0x1a0
[ 990.042959] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 990.048649] ? ww_mutex_lock+0x160/0x160
[ 990.053848] msleep+0x1b/0x30
[ 990.057973] napi_set_threaded+0x3a5/0x440
[ 990.063366] ? napi_resume_irqs+0x1a0/0x1a0
[ 990.068857] netif_set_threaded+0x257/0x400
[ 990.074350] threaded_store+0x1b8/0x220
[ 990.079449] ? xps_rxqs_store+0x2e0/0x2e0
[ 990.084746] ? sysfs_file_kobj+0xbd/0x1e0
[ 990.090040] ? sysfs_kf_write+0x65/0x160
[ 990.095238] ? sysfs_file_ops+0x140/0x140
[ 990.100533] kernfs_fop_write_iter+0x335/0x510
[ 990.106316] ? kernfs_ops+0xf0/0xf0
[ 990.111025] new_sync_write+0x333/0x750
[ 990.116126] ? __lock_acquire+0x508/0xc10
[ 990.121420] ? new_sync_read+0x740/0x740
[ 990.126620] ? lock_acquire.part.0+0xbc/0x260
[ 990.132305] ? ksys_write+0x116/0x250
[ 990.137216] vfs_write+0x657/0x9b0
[ 990.141832] ksys_write+0x116/0x250
[ 990.146545] ? __ia32_sys_read+0xc0/0xc0
[ 990.151741] ? kernfs_fop_llseek+0x160/0x1f0
[ 990.157331] ? rcu_is_watching+0x15/0xd0
[ 990.162528] do_syscall_64+0xf3/0x650
[ 990.167431] ? irq_exit_rcu+0x1a/0x30
[ 990.172337] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 990.178804] RIP: 0033:0x7ff322d2464e
[ 990.183612] RSP: 002b:00007ffc95494840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 990.192908] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff322d2464e
[ 990.201712] RDX: 0000000000000001 RSI: 000055fe2cbbf420 RDI: 0000000000000007
[ 990.210522] RBP: 00007ffc95494850 R08: 0000000000000000 R09: 0000000000000000
[ 990.219327] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff322b9d390
[ 990.228134] R13: 000055fe2cbbf420 R14: 0000000000000007 R15: 00007ff322b31120
[ 990.236946]
[ 990.240203] \x0aShowing all locks held in the system:
[ 990.248741] 1 lock held by khungtaskd/184:
[ 990.254131] #0: ffffffff884fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 990.265613] 4 locks held by python3/3338:
[ 990.270907] #0: ff1100012fb543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.280794] #1: ff1100013d256880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.291754] #2: ff110001237acb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.302910] #3: ff11000123819d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.313012] 4 locks held by python3/5317:
[ 990.318305] #0: ff1100012fb543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 990.328192] #1: ff110001a9e66880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 990.339150] #2: ff110001237acb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 990.350307] #3: ff11000123819d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 990.360396]
[ 990.362863] =============================================\x0a
[ 1074.168588] kworker/u130:4 (529) used greatest stack depth: 21568 bytes left
[ 1074.177321] kworker/u130:5 (530) used greatest stack depth: 21456 bytes left
[ 1112.569078] INFO: task python3:5317 blocked for more than 245 seconds.
[ 1112.577228] Not tainted 7.1.0-rc6-qpbe-gb23588628fda #1
[ 1112.584476] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1112.594066] task:python3 state:D stack:25672 pid:5317 tgid:5317 ppid:5316 task_flags:0x400100 flags:0x00080000
[ 1112.607263] Call Trace:
[ 1112.610804]
[ 1112.613954] __schedule+0xcc3/0x19c0
[ 1112.618770] ? io_schedule_timeout+0x140/0x140
[ 1112.624555] ? find_held_lock+0x2b/0x80
[ 1112.629659] ? __lock_release.isra.0+0x6b/0x1a0
[ 1112.635544] schedule+0xd8/0x260
[ 1112.639963] schedule_preempt_disabled+0x18/0x30
[ 1112.645941] __mutex_lock+0x6af/0x2010
[ 1112.650947] ? threaded_store+0x132/0x220
[ 1112.656247] ? ww_mutex_lock+0x160/0x160
[ 1112.661448] ? selinux_inode_free_security+0x330/0x330
[ 1112.668020] ? cap_capable+0x1d6/0x3e0
[ 1112.673026] ? threaded_store+0x132/0x220
[ 1112.678324] threaded_store+0x132/0x220
[ 1112.683427] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.688727] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.694025] ? sysfs_kf_write+0x65/0x160
[ 1112.699224] ? sysfs_file_ops+0x140/0x140
[ 1112.704520] kernfs_fop_write_iter+0x335/0x510
[ 1112.710303] ? kernfs_ops+0xf0/0xf0
[ 1112.715018] new_sync_write+0x333/0x750
[ 1112.720120] ? __lock_acquire+0x508/0xc10
[ 1112.725417] ? new_sync_read+0x740/0x740
[ 1112.730612] ? lock_acquire.part.0+0xbc/0x260
[ 1112.736300] ? ksys_write+0x116/0x250
[ 1112.741210] vfs_write+0x657/0x9b0
[ 1112.745826] ksys_write+0x116/0x250
[ 1112.750539] ? __ia32_sys_read+0xc0/0xc0
[ 1112.755738] ? kernfs_fop_llseek+0x160/0x1f0
[ 1112.761326] ? rcu_is_watching+0x15/0xd0
[ 1112.766529] do_syscall_64+0xf3/0x650
[ 1112.771435] ? trace_hardirqs_off+0xd/0x30
[ 1112.776832] ? exc_page_fault+0xda/0xf0
[ 1112.781927] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1112.788394] RIP: 0033:0x7fd92a62a64e
[ 1112.793206] RSP: 002b:00007ffee4aaa7d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1112.802506] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd92a62a64e
[ 1112.811316] RDX: 0000000000000001 RSI: 000055fcdf6fc420 RDI: 0000000000000007
[ 1112.820125] RBP: 00007ffee4aaa7e0 R08: 0000000000000000 R09: 0000000000000000
[ 1112.828935] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd92a4a3390
[ 1112.837742] R13: 000055fcdf6fc420 R14: 0000000000000007 R15: 00007fd92a439120
[ 1112.846558]
[ 1112.849844] INFO: task python3:5317 is blocked on a mutex likely owned by task python3:3338.
[ 1112.860116] task:python3 state:D stack:25672 pid:3338 tgid:3338 ppid:3337 task_flags:0x400100 flags:0x00080802
[ 1112.873319] Call Trace:
[ 1112.876858]
[ 1112.880010] __schedule+0xcc3/0x19c0
[ 1112.884822] ? io_schedule_timeout+0x140/0x140
[ 1112.890609] ? find_held_lock+0x2b/0x80
[ 1112.895715] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1112.901894] schedule+0xd8/0x260
[ 1112.906314] schedule_timeout+0x13f/0x290
[ 1112.911611] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1112.917972] ? trace_contention_end+0xb3/0x1a0
[ 1112.923756] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1112.929445] ? ww_mutex_lock+0x160/0x160
[ 1112.934647] msleep+0x1b/0x30
[ 1112.938773] napi_set_threaded+0x3a5/0x440
[ 1112.944169] ? napi_resume_irqs+0x1a0/0x1a0
[ 1112.949654] netif_set_threaded+0x257/0x400
[ 1112.955147] threaded_store+0x1b8/0x220
[ 1112.960249] ? xps_rxqs_store+0x2e0/0x2e0
[ 1112.965548] ? sysfs_file_kobj+0xbd/0x1e0
[ 1112.970844] ? sysfs_kf_write+0x65/0x160
[ 1112.976045] ? sysfs_file_ops+0x140/0x140
[ 1112.981342] kernfs_fop_write_iter+0x335/0x510
[ 1112.987126] ? kernfs_ops+0xf0/0xf0
[ 1112.991838] new_sync_write+0x333/0x750
[ 1112.996943] ? __lock_acquire+0x508/0xc10
[ 1113.002240] ? new_sync_read+0x740/0x740
[ 1113.007434] ? lock_acquire.part.0+0xbc/0x260
[ 1113.013121] ? ksys_write+0x116/0x250
[ 1113.018031] vfs_write+0x657/0x9b0
[ 1113.022649] ksys_write+0x116/0x250
[ 1113.027364] ? __ia32_sys_read+0xc0/0xc0
[ 1113.032562] ? kernfs_fop_llseek+0x160/0x1f0
[ 1113.038153] ? rcu_is_watching+0x15/0xd0
[ 1113.043353] do_syscall_64+0xf3/0x650
[ 1113.048250] ? irq_exit_rcu+0x1a/0x30
[ 1113.053151] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1113.059608] RIP: 0033:0x7ff322d2464e
[ 1113.064417] RSP: 002b:00007ffc95494840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1113.073708] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff322d2464e
[ 1113.082516] RDX: 0000000000000001 RSI: 000055fe2cbbf420 RDI: 0000000000000007
[ 1113.091319] RBP: 00007ffc95494850 R08: 0000000000000000 R09: 0000000000000000
[ 1113.100125] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff322b9d390
[ 1113.108932] R13: 000055fe2cbbf420 R14: 0000000000000007 R15: 00007ff322b31120
[ 1113.117744]
[ 1113.121011] \x0aShowing all locks held in the system:
[ 1113.129553] 1 lock held by khungtaskd/184:
[ 1113.134937] #0: ffffffff884fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1113.146423] 4 locks held by python3/3338:
[ 1113.151719] #0: ff1100012fb543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.161612] #1: ff1100013d256880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.172574] #2: ff110001237acb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.183732] #3: ff11000123819d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.193828] 4 locks held by python3/5317:
[ 1113.199125] #0: ff1100012fb543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1113.209015] #1: ff110001a9e66880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1113.219977] #2: ff110001237acb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1113.231137] #3: ff11000123819d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1113.241230]
[ 1113.243699] =============================================\x0a
[ 1235.450498] INFO: task python3:5317 blocked for more than 368 seconds.
[ 1235.458648] Not tainted 7.1.0-rc6-qpbe-gb23588628fda #1
[ 1235.465902] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1235.475498] task:python3 state:D stack:25672 pid:5317 tgid:5317 ppid:5316 task_flags:0x400100 flags:0x00080002
[ 1235.488714] Call Trace:
[ 1235.492256]
[ 1235.495413] __schedule+0xcc3/0x19c0
[ 1235.500231] ? io_schedule_timeout+0x140/0x140
[ 1235.506020] ? find_held_lock+0x2b/0x80
[ 1235.511131] ? __lock_release.isra.0+0x6b/0x1a0
[ 1235.517024] schedule+0xd8/0x260
[ 1235.521448] schedule_preempt_disabled+0x18/0x30
[ 1235.527433] __mutex_lock+0x6af/0x2010
[ 1235.532440] ? threaded_store+0x132/0x220
[ 1235.537747] ? ww_mutex_lock+0x160/0x160
[ 1235.542941] ? selinux_inode_free_security+0x330/0x330
[ 1235.549518] ? cap_capable+0x1d6/0x3e0
[ 1235.554529] ? threaded_store+0x132/0x220
[ 1235.559829] threaded_store+0x132/0x220
[ 1235.564936] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.570238] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.575544] ? sysfs_kf_write+0x65/0x160
[ 1235.580753] ? sysfs_file_ops+0x140/0x140
[ 1235.586053] kernfs_fop_write_iter+0x335/0x510
[ 1235.591844] ? kernfs_ops+0xf0/0xf0
[ 1235.596559] new_sync_write+0x333/0x750
[ 1235.601666] ? __lock_acquire+0x508/0xc10
[ 1235.606968] ? new_sync_read+0x740/0x740
[ 1235.612174] ? lock_acquire.part.0+0xbc/0x260
[ 1235.617861] ? ksys_write+0x116/0x250
[ 1235.622781] vfs_write+0x657/0x9b0
[ 1235.627399] ksys_write+0x116/0x250
[ 1235.632114] ? __ia32_sys_read+0xc0/0xc0
[ 1235.637316] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.642910] ? rcu_is_watching+0x15/0xd0
[ 1235.648117] do_syscall_64+0xf3/0x650
[ 1235.653027] ? trace_hardirqs_off+0xd/0x30
[ 1235.658428] ? exc_page_fault+0xda/0xf0
[ 1235.663536] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.670007] RIP: 0033:0x7fd92a62a64e
[ 1235.674822] RSP: 002b:00007ffee4aaa7d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.684126] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd92a62a64e
[ 1235.692938] RDX: 0000000000000001 RSI: 000055fcdf6fc420 RDI: 0000000000000007
[ 1235.701749] RBP: 00007ffee4aaa7e0 R08: 0000000000000000 R09: 0000000000000000
[ 1235.710563] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd92a4a3390
[ 1235.719372] R13: 000055fcdf6fc420 R14: 0000000000000007 R15: 00007fd92a439120
[ 1235.728186]
[ 1235.731491] INFO: task python3:5317 is blocked on a mutex likely owned by task python3:3338.
[ 1235.741771] task:python3 state:D stack:25672 pid:3338 tgid:3338 ppid:3337 task_flags:0x400100 flags:0x00080802
[ 1235.754980] Call Trace:
[ 1235.758520]
[ 1235.761674] __schedule+0xcc3/0x19c0
[ 1235.766491] ? io_schedule_timeout+0x140/0x140
[ 1235.772278] ? find_held_lock+0x2b/0x80
[ 1235.777391] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1235.783573] schedule+0xd8/0x260
[ 1235.787994] schedule_timeout+0x13f/0x290
[ 1235.793293] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1235.799667] ? trace_contention_end+0xb3/0x1a0
[ 1235.805457] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1235.811155] ? ww_mutex_lock+0x160/0x160
[ 1235.816357] msleep+0x1b/0x30
[ 1235.820487] napi_set_threaded+0x3a5/0x440
[ 1235.825887] ? napi_resume_irqs+0x1a0/0x1a0
[ 1235.831390] netif_set_threaded+0x257/0x400
[ 1235.836888] threaded_store+0x1b8/0x220
[ 1235.841996] ? xps_rxqs_store+0x2e0/0x2e0
[ 1235.847299] ? sysfs_file_kobj+0xbd/0x1e0
[ 1235.852600] ? sysfs_kf_write+0x65/0x160
[ 1235.857802] ? sysfs_file_ops+0x140/0x140
[ 1235.863101] kernfs_fop_write_iter+0x335/0x510
[ 1235.868889] ? kernfs_ops+0xf0/0xf0
[ 1235.873605] new_sync_write+0x333/0x750
[ 1235.878711] ? __lock_acquire+0x508/0xc10
[ 1235.884004] ? new_sync_read+0x740/0x740
[ 1235.889210] ? lock_acquire.part.0+0xbc/0x260
[ 1235.894903] ? ksys_write+0x116/0x250
[ 1235.899816] vfs_write+0x657/0x9b0
[ 1235.904436] ksys_write+0x116/0x250
[ 1235.909150] ? __ia32_sys_read+0xc0/0xc0
[ 1235.914354] ? kernfs_fop_llseek+0x160/0x1f0
[ 1235.919948] ? rcu_is_watching+0x15/0xd0
[ 1235.925153] do_syscall_64+0xf3/0x650
[ 1235.930061] ? irq_exit_rcu+0x1a/0x30
[ 1235.934968] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1235.941441] RIP: 0033:0x7ff322d2464e
[ 1235.946253] RSP: 002b:00007ffc95494840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1235.955553] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff322d2464e
[ 1235.964368] RDX: 0000000000000001 RSI: 000055fe2cbbf420 RDI: 0000000000000007
[ 1235.973172] RBP: 00007ffc95494850 R08: 0000000000000000 R09: 0000000000000000
[ 1235.981986] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff322b9d390
[ 1235.990798] R13: 000055fe2cbbf420 R14: 0000000000000007 R15: 00007ff322b31120
[ 1235.999623]
[ 1236.002895] \x0aShowing all locks held in the system:
[ 1236.011448] 1 lock held by khungtaskd/184:
[ 1236.016844] #0: ffffffff884fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1236.028330] 2 locks held by systemd-journal/803:
[ 1236.034313] #0: ff1100180e1b74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1236.044199] #1: ffffffff88702ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: dput.part.0+0x2fd/0x410
[ 1236.056448] 4 locks held by python3/3338:
[ 1236.061746] #0: ff1100012fb543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.071647] #1: ff1100013d256880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.082615] #2: ff110001237acb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.093780] #3: ff11000123819d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.103890] 4 locks held by python3/5317:
[ 1236.109190] #0: ff1100012fb543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1236.119086] #1: ff110001a9e66880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1236.130055] #2: ff110001237acb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1236.141217] #3: ff11000123819d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1236.151318]
[ 1236.153786] =============================================\x0a
[ 1358.331914] INFO: task python3:5317 blocked for more than 491 seconds.
[ 1358.340062] Not tainted 7.1.0-rc6-qpbe-gb23588628fda #1
[ 1358.347312] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 1358.356901] task:python3 state:D stack:25672 pid:5317 tgid:5317 ppid:5316 task_flags:0x400100 flags:0x00080002
[ 1358.370098] Call Trace:
[ 1358.373639]
[ 1358.376790] __schedule+0xcc3/0x19c0
[ 1358.381605] ? io_schedule_timeout+0x140/0x140
[ 1358.387393] ? find_held_lock+0x2b/0x80
[ 1358.392497] ? __lock_release.isra.0+0x6b/0x1a0
[ 1358.398380] schedule+0xd8/0x260
[ 1358.402798] schedule_preempt_disabled+0x18/0x30
[ 1358.408776] __mutex_lock+0x6af/0x2010
[ 1358.413780] ? threaded_store+0x132/0x220
[ 1358.419080] ? ww_mutex_lock+0x160/0x160
[ 1358.424272] ? selinux_inode_free_security+0x330/0x330
[ 1358.430843] ? cap_capable+0x1d6/0x3e0
[ 1358.435850] ? threaded_store+0x132/0x220
[ 1358.441148] threaded_store+0x132/0x220
[ 1358.446250] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.451545] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.456848] ? sysfs_kf_write+0x65/0x160
[ 1358.462047] ? sysfs_file_ops+0x140/0x140
[ 1358.467337] kernfs_fop_write_iter+0x335/0x510
[ 1358.473112] ? kernfs_ops+0xf0/0xf0
[ 1358.477826] new_sync_write+0x333/0x750
[ 1358.482928] ? __lock_acquire+0x508/0xc10
[ 1358.488226] ? new_sync_read+0x740/0x740
[ 1358.493417] ? lock_acquire.part.0+0xbc/0x260
[ 1358.499103] ? ksys_write+0x116/0x250
[ 1358.504015] vfs_write+0x657/0x9b0
[ 1358.508634] ksys_write+0x116/0x250
[ 1358.513344] ? __ia32_sys_read+0xc0/0xc0
[ 1358.518544] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.524127] ? rcu_is_watching+0x15/0xd0
[ 1358.529329] do_syscall_64+0xf3/0x650
[ 1358.534236] ? trace_hardirqs_off+0xd/0x30
[ 1358.539623] ? exc_page_fault+0xda/0xf0
[ 1358.544728] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.551196] RIP: 0033:0x7fd92a62a64e
[ 1358.556004] RSP: 002b:00007ffee4aaa7d0 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.565301] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd92a62a64e
[ 1358.574100] RDX: 0000000000000001 RSI: 000055fcdf6fc420 RDI: 0000000000000007
[ 1358.582907] RBP: 00007ffee4aaa7e0 R08: 0000000000000000 R09: 0000000000000000
[ 1358.591716] R10: 0000000000000000 R11: 0000000000000202 R12: 00007fd92a4a3390
[ 1358.600523] R13: 000055fcdf6fc420 R14: 0000000000000007 R15: 00007fd92a439120
[ 1358.609335]
[ 1358.612615] INFO: task python3:5317 is blocked on a mutex likely owned by task python3:3338.
[ 1358.622889] task:python3 state:D stack:25672 pid:3338 tgid:3338 ppid:3337 task_flags:0x400100 flags:0x00080802
[ 1358.636092] Call Trace:
[ 1358.639632]
[ 1358.642782] __schedule+0xcc3/0x19c0
[ 1358.647593] ? io_schedule_timeout+0x140/0x140
[ 1358.653381] ? find_held_lock+0x2b/0x80
[ 1358.658482] ? __try_to_del_timer_sync+0x155/0x1f0
[ 1358.664662] schedule+0xd8/0x260
[ 1358.669082] schedule_timeout+0x13f/0x290
[ 1358.674381] ? hrtimer_nanosleep_restart+0x270/0x270
[ 1358.680757] ? trace_contention_end+0xb3/0x1a0
[ 1358.686543] ? hrtimers_cpu_dying+0x3e0/0x3e0
[ 1358.692232] ? ww_mutex_lock+0x160/0x160
[ 1358.697434] msleep+0x1b/0x30
[ 1358.701562] napi_set_threaded+0x3a5/0x440
[ 1358.706958] ? napi_resume_irqs+0x1a0/0x1a0
[ 1358.712452] netif_set_threaded+0x257/0x400
[ 1358.717945] threaded_store+0x1b8/0x220
[ 1358.723047] ? xps_rxqs_store+0x2e0/0x2e0
[ 1358.728343] ? sysfs_file_kobj+0xbd/0x1e0
[ 1358.733640] ? sysfs_kf_write+0x65/0x160
[ 1358.738842] ? sysfs_file_ops+0x140/0x140
[ 1358.744137] kernfs_fop_write_iter+0x335/0x510
[ 1358.749923] ? kernfs_ops+0xf0/0xf0
[ 1358.754636] new_sync_write+0x333/0x750
[ 1358.759741] ? __lock_acquire+0x508/0xc10
[ 1358.765038] ? new_sync_read+0x740/0x740
[ 1358.770228] ? lock_acquire.part.0+0xbc/0x260
[ 1358.775916] ? ksys_write+0x116/0x250
[ 1358.780827] vfs_write+0x657/0x9b0
[ 1358.785443] ksys_write+0x116/0x250
[ 1358.790153] ? __ia32_sys_read+0xc0/0xc0
[ 1358.795352] ? kernfs_fop_llseek+0x160/0x1f0
[ 1358.800940] ? rcu_is_watching+0x15/0xd0
[ 1358.806141] do_syscall_64+0xf3/0x650
[ 1358.811038] ? irq_exit_rcu+0x1a/0x30
[ 1358.815936] entry_SYSCALL_64_after_hwframe+0x4b/0x53
[ 1358.822402] RIP: 0033:0x7ff322d2464e
[ 1358.827213] RSP: 002b:00007ffc95494840 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1358.836508] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007ff322d2464e
[ 1358.845316] RDX: 0000000000000001 RSI: 000055fe2cbbf420 RDI: 0000000000000007
[ 1358.854115] RBP: 00007ffc95494850 R08: 0000000000000000 R09: 0000000000000000
[ 1358.862913] R10: 0000000000000000 R11: 0000000000000202 R12: 00007ff322b9d390
[ 1358.871720] R13: 000055fe2cbbf420 R14: 0000000000000007 R15: 00007ff322b31120
[ 1358.880531]
[ 1358.883804] \x0aShowing all locks held in the system:
[ 1358.892343] 1 lock held by khungtaskd/184:
[ 1358.897738] #0: ffffffff884fdea0 (rcu_read_lock){....}-{1:3}, at: rcu_lock_acquire.constprop.0+0x7/0x30
[ 1358.909213] 3 locks held by systemd-journal/803:
[ 1358.915192] #0: ff1100180e5b74e0 (&rq->__lock){-.-.}-{2:2}, at: __schedule+0x18d/0x19c0
[ 1358.925081] #1: ffffffff88702ce0 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}, at: fs_reclaim_acquire+0x7b/0x120
[ 1358.937896] #2: ff110001a58b7200 (ptlock_ptr(ptdesc)#2){+.+.}-{3:3}, at: pte_offset_map_lock+0x114/0x2d0
[ 1358.949451] 4 locks held by python3/3338:
[ 1358.954740] #0: ff1100012fb543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1358.964632] #1: ff1100013d256880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1358.975597] #2: ff110001237acb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1358.986759] #3: ff11000123819d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1358.996854] 4 locks held by python3/5317:
[ 1359.002153] #0: ff1100012fb543e8 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0x116/0x250
[ 1359.012040] #1: ff110001a9e66880 (&of->mutex#2){+.+.}-{4:4}, at: kernfs_fop_write_iter+0x248/0x510
[ 1359.023003] #2: ff110001237acb50 (kn->active#118){.+.+}-{0:0}, at: kernfs_fop_write_iter+0x29e/0x510
[ 1359.034160] #3: ff11000123819d68 (&dev->lock){+.+.}-{4:4}, at: threaded_store+0x132/0x220
[ 1359.044252]
[ 1359.046720] =============================================\x0a